Method, apparatus, and system for adjusting display content of head-up display

Abstract

A method, apparatus, and system for adjusting display content of a head-up display. The method comprises: (S201) acquiring environment information of a current driving road, and driving information, wherein the driving information comprises eyeball areas of a driver and pupil areas of the driver; (S202) determining a display position of current display content and a display background of the current display content on the basis of the environment information and the driving information, wherein the current display content is display content of a head-up display at a current moment; (S203) determining whether the display background interferes with the display effect of the current display content; and (S204) if the display background interferes with the display effect of the current display content, adjusting the current display content to obtain adjusted display content, wherein the display effect of the adjusted display content is not affected by the display background.

Description

Method, device and system for adjusting display content of head-up display

[0001] This application claims priority to Chinese Patent Application No. 2024104618026, filed on April 17, 2024, entitled "Method, device and system for adjusting display content of head-up display" and Chinese Patent Application No. 2024111955626, filed on August 28, 2024, entitled "Method for adjusting display content of head-up display and related device", the contents of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] The present application relates to the field of automotive electronics, and in particular to a method, device and system for adjusting display content of a head-up display. BACKGROUND

[0003] Head-up displays (HUD) are gradually applied in more and more vehicles, which mainly use optical reflection principle to project vehicle speed, navigation and road environment objects and other related information in the form of images on the display screen. However, the head-up displays in the prior art are easily affected by environmental factors, resulting in poor display effect of the head-up display. Therefore, how to improve the display effect of the head-up display has become a technical problem to be solved at present.

[0004] SUMMARY

[0005] In view of the above problems, the present application provides a method, device and system for adjusting display content of a head-up display to achieve the purpose of improving the display effect of the head-up display. The specific scheme is as follows:

[0006] In a first aspect, the embodiments of the present application provide a method for adjusting display content of a head-up display, the method comprising:

[0007] obtaining environmental information of a current driving road and driving information; the driving information includes a region where an eyeball of a driver is located and a region where a pupil of the driver is located;

[0008] determining a display position of current display content and a display background of the current display content according to the environmental information and the driving information; the current display content is display content of the head-up display at a current time;

[0009] judging whether the display background causes interference to a display effect of the current display content;

[0010] If the display background interferes with the display effect of the current display content, the current display content is adjusted to obtain adjusted display content; the display effect of the adjusted display content is not interfered by the display background.

[0011] In a second aspect, an apparatus for adjusting display content of a head-up display is provided, and the apparatus comprises:

[0012] The acquisition module is configured to acquire environment information of a current driving road and driving information, wherein the driving information comprises a region where eyeballs of a driver are located and a region where pupils of the driver are located.

[0013] The determination module is configured to determine, according to the environment information and the driving information, a display position of current display content and a display background of the current display content, wherein the current display content is display content of the head-up display at a current time.

[0014] The judgment module is configured to judge whether the display background interferes with the display effect of the current display content.

[0015] The adjustment module is configured to, if the display background interferes with the display effect of the current display content, adjust the current display content to obtain adjusted display content, wherein the display effect of the adjusted display content is not interfered by the display background.

[0016] In a third aspect, an apparatus for adjusting display content of a head-up display is provided, and the apparatus is configured to switch a display mode of the head-up display to adjust the display content of the head-up display, wherein the display mode comprises a manual mode and an automatic mode, and the manual mode comprises a daily mode, a sports mode and a snow mode.

[0017] When the display mode is the daily mode, a color of the display content of the head-up display is white.

[0018] When the display mode is the sports mode, the display content of the head-up display flashes at a fixed frequency.

[0019] When the display mode is the snow mode, a color of the display content of the head-up display is blue.

[0020] When the display mode is the automatic mode, the apparatus executes any one of the above methods for adjusting display content of a head-up display. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] FIG. 1 is a schematic diagram of an assisted driving scene to which the present application is applicable;

[0023] FIG. 2 is a flowchart of a method for adjusting the display content of a head-up display according to an embodiment of the present application;

[0024] FIG. 3 is a schematic diagram of the correspondence between the display position of the current display content and the display background according to an embodiment of the present application;

[0025] FIG. 4 is another schematic diagram of the correspondence between the display position of the current display content and the display background according to an embodiment of the present application;

[0026] FIG. 5 is a flowchart of obtaining a preset convolutional neural network weight file according to an embodiment of the present application;

[0027] FIG. 6 is a schematic diagram of IoU calculation according to an embodiment of the present application;

[0028] FIG. 7 is a schematic diagram of DIoU calculation according to an embodiment of the present application;

[0029] FIG. 8 is a schematic diagram of actual IoU calculation according to an embodiment of the present application;

[0030] FIG. 9 is a schematic diagram of single background adjustment according to an embodiment of the present application;

[0031] FIG. 10 is a schematic diagram of complex background adjustment according to an embodiment of the present application;

[0032] FIG. 11 is another schematic diagram of complex background adjustment according to an embodiment of the present application;

[0033] FIG. 12 is a detailed flowchart of a method for adjusting the display content of a head-up display according to an embodiment of the present application;

[0034] FIG. 13 is a flowchart of judging interference in a method for adjusting the display content of a head-up display according to an embodiment of the present application;

[0035] FIG. 14 is an example diagram of an image grid divided in an embodiment of the present application;

[0036] FIG. 15 is a system block diagram applicable in the vehicle assisted driving scene according to the present application;

[0037] FIG. 16 is a control flow logic block diagram of each module applicable in the vehicle assisted driving scene according to the present application;

[0038] FIG. 17 is a schematic diagram of a calibration of a HUD projection area in a vehicle assisted driving scenario according to the present application;

[0039] FIG. 18 is a flowchart of an AR-HUD brightness self-adaptive adjustment method based on H3A statistical data in a vehicle assisted driving scenario according to the present application;

[0040] FIG. 19 is a schematic diagram of a device for adjusting the display content of a head-up display according to an embodiment of the present application;

[0041] FIG. 20 is another schematic diagram of a device for adjusting the display content of a head-up display according to an embodiment of the present application;

[0042] FIG. 21 is another schematic diagram of a device for adjusting the display content of a head-up display according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] The embodiments of the present application are described below in conjunction with the accompanying drawings. The terms used in the implementation part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0044] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art can know that, as technology develops and new scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0045] The terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a way of distinguishing the objects with the same attributes in the description of the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or equipment containing a series of units do not necessarily limit to those units, but can include other units not clearly listed or inherent to these processes, methods, products or equipment.

[0046] The present application can be applied in the field of assisted driving, and the application scenarios of the present application will be introduced below taking vehicles as an example.

[0047] First, the application scenario of the present application is introduced.

[0048] The prior art is analyzed, and it is found that the prior art adjusts the display content of the HUD (head-up display) in a manual adjustment manner, which cannot adapt the display information of the HUD to the complex and changeable road environment background, and it is difficult to provide the user with smoother and clearer HUD information without interference, and the display effect is poor. Moreover, due to the complexity and variability of the road environment, manual adjustment cannot meet the real-time requirement, which will reduce the experience of the driver. At the same time, the HUD information that is not clear and smooth will interfere with the judgment of the driver and affect the driving mood.

[0049] When the background of the HUD display content is a road, the display content of the same color such as white cannot meet the use of all scenes. For example, when the vehicle is driving in the snow, the white display content will be integrated with the snow background, making it difficult to identify. When the vehicle is driving in the sand and is reflected by strong sunlight, the yellow display content will be difficult to identify. When the background of the HUD display content is a complex road environment object, the display content with unchanged color and form will also be difficult to identify due to interference. For example, when the white display content coincides with the white car in front, the white lane line or the zebra crossing, and is further affected by strong environmental light, it will be difficult to distinguish the color of the display content and the background object, which is not conducive to the driver to make correct judgments. At the same time, in this case, the driver is difficult to manually adjust the color and form of the HUD display content to improve the distinguishability of the road environment background and the display information. The driver needs to obtain clear and smooth information and maintain concentration while driving, and the effort to distinguish the difference between the HUD display content and the background will distract the driver's attention and affect normal driving, which will increase the risk of driving in the long run.

[0050] As shown in FIG. 1, the vehicle is driving on a highway, and the vehicle is deployed with at least an image acquisition device and a head-up display. The image acquisition device refers to a device capable of acquiring images, such as a front-view camera of an advanced driving assistance system ADAS (Advanced Driving Assistance System). The head-up display refers to a device capable of outputting light to realize image projection, such as an augmented reality head-up display device AR-HUD (Augmented Reality-head-up display).

[0051] The head-up display comprises at least an eye nucleus module and a brightness module. The eye nucleus module is used to obtain the position information of the eye nucleus and the pupil, i.e., the position information of the human eye. The eye nucleus position is used to determine the display position (i.e., the projection area). For example, the display position is the intersection area of the visual field range corresponding to the eye nucleus position and the windshield of the vehicle. The brightness module is used to provide the projection brightness, so that the head-up display projects the image at the display position according to the projection brightness. For example, the head-up display further comprises a light source. The light source projects the image to the front windshield and reflects into the human eye through internal multiple light paths, so that the driver can see a virtual image at a long distance and in a large size in front.

[0052] The image acquisition device comprises at least an image signal processing module (ISP). The ISP is used to process the image collected by the image acquisition device to obtain the relevant statistical data of the image. For example, the H3A (Histogram 3A) statistical data of the image comprises the pixel height and the pixel width of the image, and the pixel statistical data such as the average pixel value of the image.

[0053] Based on this, the image acquisition device collects the image of the driving field of view of the vehicle, processes the image, and sends the image processing result to the head-up display. The head-up display projects the image on the front windshield of the vehicle based on the image processing result of the image acquisition device. The image at least contains the navigation prompt information to assist the driving of the vehicle.

[0054] To solve the above problems, the embodiment of the present application provides a method, device and system for adjusting the display content of the head-up display, which are as follows.

[0055] As shown in FIG. 2, the embodiment of the present application provides a method for adjusting the display content of the head-up display. The method can be applied to an electronic device capable of data processing. The electronic device can be an image acquisition device deployed on a transportation tool such as a vehicle, an image projection device deployed on the vehicle, or a vehicle terminal connected with the image acquisition device and the image projection device. The technical solution in the embodiment is mainly used to improve the display effect of the head-up display to improve the driving experience of the transportation tool.

[0056] Specifically, the method in the embodiment can comprise the following steps:

[0057] S201: Obtain the environmental information of the current driving road and the driving information. The driving information comprises the eye nucleus area of the driver and the pupil area of the driver.

[0058] The display is a head-up display, and the display mode of the HUD includes a manual mode and an automatic mode. When the display mode of the HUD is the manual adjustment mode, if the manual adjustment mode is the daily mode, the color of the current display content is set to white. If the manual adjustment mode is the sports mode, the current display content is made to flick at a fixed frequency to reduce the occlusion of the current display content to the road. If the manual adjustment mode is the snow mode, the color of the current display content is set to blue to distinguish the color of the display content from the white snow.

[0059] When the display mode of the HUD is the automatic adjustment mode, a driver monitoring coordinate system O d -X d Y d Z d is constructed based on a driver monitoring camera. The origin O d , the X d direction is the camera optical axis direction, the Z d direction is the direction perpendicular to the upward direction of the camera, and the Y d direction is perpendicular to the X d O d Z d plane. The installation position of the driver monitoring camera can be set at a position in the vehicle that can observe the binocular region image of the driver, such as the windshield of the vehicle. The region S1 where the driver's eyeball is located and the region S2 where the pupil is located are obtained through the driver monitoring camera.

[0060] According to the light reflection intensity of the images corresponding to the region S1 where the driver's eyeball is located and the region S2 where the pupil is located, the region S d -X d Y d Z d where the driver's eyeball is located and the region S d,1 where the pupil is located in the O d,2 coordinate system are calculated. The region S d,1 where the driver's eyeball is located and the region S d,2 where the pupil is located are taken as the driving information.

[0061] A front-looking coordinate system O c -X c Y c Z c is constructed with the centroid of the front-looking camera on the vehicle as the origin, the X c direction is the camera optical axis direction, the Z c direction is the direction perpendicular to the upward direction of the camera, and the Y c direction is perpendicular to the X c O c Z cplane. The environmental brightness information, color information, and position information of the color information and brightness information in the road in front of the vehicle in the O c X c Y c Z c coordinate system are collected by the front-view camera, and are used as the environmental information of the current driving road. c X c Y c Z c coordinate system are collected by the front-view camera, and are used as the environmental information of the current driving road.

[0062] S202: determining the display position of the current display content and the display background of the current display content according to the environmental information and the driving information; the current display content is the display content of the head-up display at the current time.

[0063] The environmental information and the driving information are analyzed to determine the display position of the current display content and the environmental information observed from the driver's perspective and overlapping with the current display content, and the environmental information observed from the driver's perspective and overlapping with the current display content is used as the display background.

[0064] When the image of the road in front of the vehicle observed from the driver's perspective is as shown in FIG. 3, if the display position of the current display content is located at the first position, the display background corresponding to the first position is the sky and the road image with the lane boundary; if the display position of the current display content is located at the second position, the display background corresponding to the second position is the road image with the lane boundary; and if the display position of the current display content is located at the third position, the display background corresponding to the third position is the road image with the lane boundary.

[0065] In an optional embodiment, determining the display position of the current display content and the display background of the current display content according to the environmental information and the driving information comprises:

[0066] determining the display position of the current display content according to the driving information;

[0067] using the overlapping area of the environmental information and the display position of the current display content in the vehicle coordinate system as the display background of the current display content; the vehicle coordinate system is a coordinate system constructed with the vehicle center as the origin.

[0068] Based on this, in the embodiment, the overlapping area of the current display content and the environmental information in the coordinate system constructed with the vehicle center as the origin is used as the display background, so that the determined display background is more accurate.

[0069] the vehicle coordinate system O is constructed with the vehicle center as the originv -X v Y v Z v , X v direction is the direction of vehicle forward movement, Z v direction is the direction upward perpendicular to the direction of vehicle forward movement, Y v direction is perpendicular to X v O v Z v plane.

[0070] The coordinates of the driver monitoring coordinate system O d -X d Y d Z d are converted to the vehicle coordinate system O v -X v Y v Z v , then the coordinates of the point O v -X v Y v Z v in the O d coordinates are (x v,d , y v,d , z v,d ), X v rotates around X d by an angle of a d , Y v rotates around Y d by an angle of b d , and Z v rotates around Z d by an angle of g d .

[0071] The rotation matrix R d is shown in formula (1):

[0072] The translation matrix T d is shown in formula (2):

[0073]

[0074] When the point in the O d -X d Y d Z d coordinate system is converted to the O v -X v Y v Z v coordinate system, the rotation matrix R d and the translation matrix T d are used for the following conversion, as shown in formula (3):

[0075] The region S of the driver's eyeball in the vehicle coordinate system v,1 The region S of the pupil in the vehicle coordinate system v,2 The region S of the display position of the current display content in the vehicle coordinate system v,HUD The content region of the current display content is S HUD .

[0076] After determining the display position of the current display content, the overlapping region of the environmental information and the display position of the current display content in the angle of the driver's observation is determined in combination with the environmental information, and the determined overlapping region is taken as the display background of the current display content.

[0077] The coordinates of the front view coordinate system O c -X c Y c Z c are converted to the vehicle coordinate system O v -X v Y v Z v , so that the coordinates of the point O v -X v Y v Z v in the O c -X v,c Y v,c Z v,c coordinate system are (x v , y c , z c ), the X v rotates by an angle of α c , the Y c rotates by an angle of β v , and the Z c rotates by an angle of γ c .

[0078] The rotation matrix R c is shown in formula (4):

[0079] The translation matrix T c is shown in formula (5):

[0080] The point in the O c -X c Y c Z c coordinate system is converted to the O v -X v Y v Z v coordinate system, and the rotation matrix R cTranslation matrix T c The following conversion is made, as shown in equation (6):

[0081] According to the O v -X v Y v Z v The region S in which the driver's eyeball is located under the coordinate system v,1 , the region S in which the pupil is located v,2 , and the region S of the current display content v,HUD , the region S of the road environment corresponding to the current display content can be calculated v,4 .

[0082] As shown in FIG. 4, the current display content and the road environment background corresponding to the current display content are overlapped at the virtual image imaging surface. The overlapping part of the current display content and the environment background is the final image obtained by the human eye. The part of the environment background that overlaps with the current display content is the display background.

[0083] S203: Determine whether the display background interferes with the display effect of the current display content.

[0084] According to the environment information, the color and brightness of each position in the display background are determined. The color and brightness of each position in the display background are compared with the color and brightness of the same position in the current display content to determine whether the display background interferes with the display effect of the current display content. When the color and / or brightness of a position in the display background is similar to the color and / or brightness of the same position in the current display content, the display background interferes with the display effect of the current display content.

[0085] In an optional embodiment, determining whether the display background interferes with the display effect of the current display content comprises:

[0086] According to a preset convolutional neural network, the type of the display background is determined. The type of the display background includes a single background and a complex background. The single background is a display background that includes only one brightness, one color, and one texture. The complex background is a display background that includes multiple colors, multiple brightnesses, or multiple textures.

[0087] When the type of the display background is the single background, whether the display background interferes with the display effect of the current display content is determined according to a first interference degree determination formula.

[0088] When the type of the display background is the complex background, whether the display background interferes with the display effect of the current display content is determined according to a second interference degree determination formula.

[0089] Based on this, in the embodiment, for single background and complex background, whether the display background interferes with the display effect of the current display content can be determined according to different formulas, so as to avoid the problem of low accuracy of interference determination when the same method is used to determine the interference of different types of backgrounds, and the accuracy of interference determination can be improved.

[0090] As shown in FIG. 5, the training process of the preset convolutional neural network in the embodiment includes: obtaining a frame image through a front-view camera, determining a display background of display content generated by the HUD from the frame image, dividing the display background into a single background and a complex background, adjusting the brightness of the single background and the complex background to simulate roads at different times such as daytime, nighttime and dusk, and adding weather models such as rain, snow and sandstorm to enhance the road data. The single background and the complex background are labeled respectively, and parameters such as loss function, training times and learning rate change are set before training the convolutional neural network. The labeled complex background and single background are input into the convolutional neural network for training to obtain a network weight file and generate a preset convolutional neural network.

[0091] The objects in the display background are subjected to convolution feature extraction by the preset convolutional neural network, and then the background objects are determined by segmenting the display background according to the features. The type of the object is determined according to the obtained feature information, and the type of the display background is determined according to the type of the object.

[0092] When the display background only includes one brightness, one color and one texture, the type of the current display background is divided into a single background by the preset convolutional neural network. When the display background includes multiple colors, multiple brightnesses or multiple textures, the type of the current display background is divided into a complex background by the convolutional neural network.

[0093] In an optional embodiment, determining whether the display background interferes with the display effect of the current display content according to the first interference degree determination formula includes:

[0094] obtaining the brightness and color of the single background, the brightness and color of the current display content, a first comparison threshold, and a first brightness difference coefficient and a first color difference coefficient;

[0095] determining the difference between the brightness of the single background and the brightness of the current display content according to the brightness of the single background and the brightness of the current display content, to obtain a first brightness difference;

[0096] determining the difference between the color of the single background and the color of the current display content according to the color of the single background and the color of the current display content, to obtain a first color difference;

[0097] The first brightness difference, the first color difference, the first brightness difference coefficient, the first color difference coefficient, and the first comparison threshold are input into a first interference degree judgment formula to obtain a judgment result; the judgment result is used to indicate whether the display background interferes with the display effect of the currently displayed content.

[0098] Based on this, in this embodiment, for a single background, the differences in brightness and color between the single background and the current display content can be used to determine interference, thereby improving the accuracy of determining interference for the single background.

[0099] Get the brightness L1 and color E1 of each part of the single background, as well as the brightness L of the current display content h and color E h According to the brightness L1 of each part in the single background and the brightness L of each part in the current display content h , calculate the brightness difference between the single background and the current display content to obtain the first brightness difference ΔL1. h , calculate the color difference between the single background and the current display content, and get the first color difference ΔE h The first brightness difference ΔL1 and the first color difference ΔE h Input into the first interference degree judgment formula to judge whether the display background interferes with the display effect of the current display content. The first interference degree judgment formula is shown in formula (7):

[0100] Wherein λ1 is the first brightness difference coefficient, λ2 is the first brightness difference coefficient, and k1 is the set first comparison threshold. When the sum of the first brightness difference ΔL1 and the first color difference ΔE1 added by a certain coefficient is greater than the set first comparison threshold k1, the judgment result is that the single background causes interference to the current display content. If the sum of the first brightness difference ΔL1 and the first color difference ΔE1 added by a certain coefficient is less than the first comparison threshold k1, the judgment result is that the single background does not cause interference to the current display content.

[0101] In an optional embodiment, judging whether the display background interferes with the display effect of the currently displayed content according to the second interference degree judgment formula includes:

[0102] Obtaining the brightness and color of the complex background, the brightness and color of the currently displayed content, a second brightness difference coefficient, a second color difference coefficient, a second comparison threshold, and an intersection-over-union ratio between the currently displayed content and objects in the display background;

[0103] According to the brightness of the complex background and the brightness of the current display content, a difference between the brightness of the complex background and the brightness of the current display content is determined to obtain a second brightness difference;

[0104] According to the color of the complex background and the color of the current display content, a difference between the color of the complex background and the color of the current display content is determined to obtain a second color difference;

[0105] The second brightness difference, the second color difference, the second brightness difference coefficient, the second color difference coefficient, the second comparison threshold, and the intersection-over-union ratio between the current display content and the object in the display background are input into a second interference degree judgment formula to obtain a judgment result. The judgment result is used to indicate whether the display background interferes with the display effect of the current display content.

[0106] Based on this, in the embodiment, for the complex background, the difference between the complex background and the current display content in brightness and color can be used to judge the interference, thereby improving the accuracy of judging the interference of the complex background.

[0107] The brightness L2 and the color E2 of each part of the complex background, and the brightness L h and the color E h of each part of the current display content are obtained. The second brightness difference ΔL2 and the second color difference ΔE2 of the complex background and the current display content are calculated.

[0108] Since the type of the display background is determined according to the preset convolutional neural network, the background object is segmented, and therefore the intersection-over-union (IoU) between the current display content and the background object can be calculated. In the embodiment of the application, DIoU (Distance-IoU) is selected as the intersection-over-union, which considers the distance, overlap rate and scale between the current display content and the background object.

[0109] As shown in FIG. 6, the dark gray box and the white box partially overlap, wherein the area of the dark gray region is A, the area of the white region is B, and the area of the overlapping light gray region is C. The calculation formula of the IoU is shown in formula (8):

[0110] As shown in FIG. 7, the frame 1 and the frame 2 partially overlap, wherein point a is the center point of the frame 1, point b is the center point of the frame 2, the diagonal length c of the minimum circumscribed rectangle containing the frame 1 and the frame 2, and the distance d between the center points of the frame 1 and the frame 2. The calculation formula of the DIoU is shown in formula (9):

[0111] wherein, ρ 2(a, b) represents the square of the distance between the center point a and the center point b. In a specific calculation, as shown in FIG. 8, the gray is the ground, the right white arrow is the current display content of the HUD, and the white quadrilateral is the ground mark in the display background. The calculation of the IoU should be the area of the shaded area / the area of the white area. In the calculation of the center point of the current display content of the HUD and the center point of the white quadrilateral, the center point of the minimum circumscribed rectangle is replaced.

[0112] The second luminance difference ΔL2, the second color difference ΔE2 and the DIoU are input into the second interference degree judgment formula to judge whether the display background causes interference to the display effect of the current display content. The second interference degree judgment formula is shown in formula (10):

[0113] Wherein, λ3 is the second luminance difference coefficient, λ4 is the second color difference coefficient, and k2 is the second comparison threshold set. When the product of the sum of the second luminance difference ΔL2 and the second color difference ΔE2 added with a certain coefficient and the DIoU is greater than the second comparison threshold k2 set, the judgment result indicates that the complex background causes interference to the current display content. If the product of the sum of the second luminance difference ΔL2 and the second color difference ΔE2 added with a certain coefficient and the DIoU is less than the second comparison threshold k2, the judgment result indicates that the complex background does not cause interference to the current display content.

[0114] S204: If the display background causes interference to the display effect of the current display content, the current display content is adjusted to obtain an adjusted display content. The display effect of the adjusted display content is not interfered by the display background.

[0115] When the display background causes interference to the display effect of the current display content, it means that the color and / or luminance of at least part of the area in the display background and the current display content are relatively close. The color and / or luminance of the part of the current display content is adjusted to obtain an adjusted display content.

[0116] In an optional embodiment, if the display background causes interference to the display effect of the current display content, the current display content is adjusted to obtain an adjusted display content, including:

[0117] When the type of the display background is a single background, if the display background causes interference to the display effect of the current display content, the color and display form of the current display content are adjusted according to the display background to obtain an adjusted display content.

[0118] When the type of the display background is the complex background, if the display background interferes with the display effect of the current display content, the display background of the current frame is predicted according to the display backgrounds of the previous n frames to obtain a predicted display background; n is an integer greater than or equal to 2; the color and display form of the current display content are adjusted according to the predicted display background to obtain adjusted display content.

[0119] When the type of the display background is the single background, it indicates that the color, brightness and texture in the current display background are relatively stable. When the single background interferes with the display effect of the current display content, since the color, brightness and texture in the single background are relatively stable, the single background currently obtained is highly probably not changed in a short time, and the color and / or display form of the current display content can be directly adjusted based on the single background currently obtained, so that the adjusted display content will not be interfered by the single background currently obtained.

[0120] When the type of the display background is the complex background, it indicates that the transformation frequency of the color, brightness and texture in the current display background is relatively large. When the complex background interferes with the display effect of the current display content, if the current display content is directly adjusted based on the complex background collected, since the transformation frequency of the color, brightness and texture in the complex background is relatively large, after the adjustment of the display content is completed, the complex background may have been changed, so that the display effect is still interfered by the complex background. Therefore, after it is determined that the complex background interferes with the display effect of the current display content, the display background of the current frame is predicted based on the n frames of display backgrounds before the complex background is collected to obtain a predicted display background. After the predicted display background is determined, the color and / or display form of the current display content are adjusted based on the predicted display background, so that the adjusted display content will not be interfered by the predicted display background.

[0121] It should be noted that the generation manner of the predicted display background is specifically: the positions of each object in the current frame of the complex background are predicted by Kalman filtering based on the positions of each object in the n frames of the complex background collected before.

[0122] Based on this, in the embodiment, the current display content can be adjusted according to the display background in different manners for the single background and the complex background respectively, so as to avoid the problem of poor accuracy when the current display content is adjusted in the same manner for the display background of different types, and the accuracy of the adjustment of the display content can be improved in the embodiment.

[0123] In an optional embodiment, the color and display form of the current display content are adjusted according to the display background to obtain adjusted display content, including:

[0124] When the display background is a single dark background or a low-brightness background, the color of the current display content is modified to be light, or the display form of the current display content is modified to add a light background color under the display content, to obtain adjusted display content.

[0125] When the display background is a single light background or a high-brightness background, the color of the current display content is modified to be dark, or the display form of the current display content is modified to add a dark background color under the display content, to obtain adjusted display content.

[0126] When the type of the display background is a single background, if the display background causes interference to the display effect of the current display content, the interference reason is determined according to the display background and the current display content, and the interference reason includes color interference and brightness interference. When the interference reason is that the color of the single background causes interference to the current display content, if the display background is a single dark background, the color of the current display content is modified to be light, and if the display background is a first light background, the color of the current display content is modified to be dark. When the interference reason is that the brightness of the single background causes interference to the current display content, if the display background is a high-brightness background, the display form of the current display content is modified to add a background color under the display content, and the background color is dark. If the display background is a low-brightness background, the display form of the current display content is modified to add a dark background color under the display content.

[0127] Based on this, in the embodiment, the color and display form of the current display content can be adjusted according to the background depth or brightness of the single background, so that the background no longer interferes with the current display content, thereby improving the user viewing experience.

[0128] For ease of understanding, the following examples are given:

[0129] As shown in FIG. 9, when the color of the single background is white and the value in the current display content is white, the interference reason is that the color of the single background causes interference to the current display content, and at this time, the color of the value 35 in the current display content is adjusted to be dark blue. When the brightness of the single background is too high, the value in the current display content is white, and the brightness of the current display content is similar to that of the single background, the interference reason is that the brightness of the single background causes interference to the current display content, and at this time, the value 35 in the current display content is adjusted to be dark, or the display form of the value in the current display content is adjusted, such as adding a background color. It should be noted that in FIG. 9, the single background of white is replaced by gray.

[0130] In an alternative embodiment, the color and display form of the current display content are adjusted according to the predicted display background, to obtain adjusted display content, including:

[0131] When the overlapping area of the predicted display background and the current display content includes multiple similar colors, the display form of the current display content is modified to add a preset background color under the display content; the similar colors are colors in the predicted display background that have a color difference less than a color difference threshold from the color of the current display content; the preset background color has a color difference greater than the color difference threshold from the multiple similar colors.

[0132] When the overlapping area of the predicted display background and the current display content includes one similar color, the color of the current display content is modified to a preset color; the preset color has a color difference greater than the color difference threshold from the one similar color.

[0133] When the complex background affects the display effect of the current display content, it indicates that the overlapping part of the complex background and the current display content affects the display effect of the current display content. If the overlapping part of the complex background and the current display content includes multiple similar colors, to avoid the multiple similar colors in the predicted display background corresponding to the current frame from continuing to affect the display effect of the current display content, the display form of the current display content needs to be modified to add a preset background color under the display content, so that the driver can easily distinguish the current display content from the predicted display background.

[0134] If the overlapping part of the complex background and the current display content includes only one similar color, the driver can easily distinguish the current display content from the predicted display background by modifying the color of the current display content to a preset color. As shown in FIG. 10, when the color of the predicted display background includes black, white, and yellow, and the value in the current display content is white, since the white background in the predicted display background overlaps the value, and the yellow in the predicted display background is similar to the white, the multiple colors in the overlapping area of the predicted display background and the current display content interfere with the current display content, so the display form of the current display content needs to be adjusted to add a deep blue background color under the value 35 in the current display content.

[0135] Based on this, in this embodiment, the color of the current display content can be adjusted according to the number of similar colors in the overlapping area of the predicted display background and the current display content for the complex background, so that the complex background does not interfere with the current display content because of the similar colors, thereby improving the user viewing experience.

[0136] As shown in FIG. 11, when the white quadrilateral ground mark on the ground in the predicted display background overlaps the right-turn white arrow in the current display content, one color in the overlapping area of the predicted display background and the current display content interferes with the current display content. To distinguish the right-turn arrow from the white quadrilateral ground mark in the predicted display background, the color of the white arrow in the current display content is adjusted to a deep blue color.

[0137] In an optional embodiment, after judging whether the display background interferes with the display effect of the current display content, the method further comprises:

[0138] If the display background does not interfere with the display effect of the current display content, the color and display form of the current display content are restored to the preset values.

[0139] If the display background does not interfere with the display effect of the current display content, if the display background is a single background, the color and display form of the current display content are restored to the preset values corresponding to the single background. If the display background is a complex background, the color and display form of the current display content are restored to the preset values corresponding to the complex background.

[0140] For ease of understanding, the embodiments of the present application are illustrated as follows:

[0141] As shown in FIG. 12, first, the display mode of the HUD is judged. When the display mode is a manual mode, it is further judged which mode the manual mode selected by the driver is, i.e., a daily mode, a sports mode or a snow mode, and the corresponding display form and color are set based on the selected mode. When the display mode is an automatic mode, the display position of the current display content is determined according to the driving information and the environmental information, and the display background corresponding to the display position of the current display content is determined. The type of the display background is judged by a preset convolutional neural network. When the type of the display background is a single background, it is judged whether the single background interferes with the display effect of the current display content according to the brightness and color of the display background and the brightness and color of the current display content, and the display form and / or color of the current display content are adjusted when interference occurs, and the preset form and color are restored if no interference occurs. When the type of the display background is a complex background, the DIoU of the current display content and the objects in the display background is calculated. It is judged whether the complex background interferes with the current display content according to the position, brightness and color of each object in the complex background, the DIoU of the current display content and the objects in the display background, and the brightness and color of the current display content. If interference occurs, the complex background of the previous n frames is predicted by Kalman filtering to obtain a predicted display background. The display form and color of the current display content are adjusted based on the predicted display background. If no interference occurs, the preset display form and color are restored. After the adjustment is completed, it is continuously judged whether the HUD stops displaying. If yes, the process is ended. If no, the process returns to the step of determining the display position.

[0142] From the above scheme, the method for adjusting the display content of the head-up display provided by the embodiment of the present application can obtain the environment information of the current driving road and the driving information including the region where the eyeball of the driver is located and the region where the pupil of the driver is located. According to the environment information and the driving information, the display position of the current display content and the display background of the current display content are determined. It is judged whether the display background causes interference to the display effect of the current display content. If the display background causes interference to the display effect of the current display content, the current display content is adjusted to obtain the adjusted display content. Since the current display content is adjusted when the display effect of the current display content is interfered by the display background in the present application, the display background cannot interfere with the adjusted display content, so that the influence of the display background on the display effect of the current display content is avoided, and the display effect of the head-up display is improved.

[0143] In an optional embodiment, when it is judged in S203 whether the display background causes interference to the display effect of the current display content, the judgment can be realized by the following method, as shown in FIG. 13.

[0144] S1301: Obtain the pixel statistical data of the background image output by the image acquisition device.

[0145] The background image is the image corresponding to the display background collected by the image acquisition device at the display position of the current display content.

[0146] The pixel statistical data can be H3A statistical data, and the pixel statistical data is the pixel statistical data obtained by the ISP in the image acquisition device.

[0147] Specifically, the pixel statistical data includes the average pixel value of the background image.

[0148] In an implementation manner, the average pixel value of the background image can be obtained by the following method.

[0149] First, the background image is divided into a plurality of image grids.

[0150] Then, the average pixel value of each image grid is calculated, for example, the average pixel value of all pixel points in the image grid is calculated to obtain the average pixel value corresponding to each image grid.

[0151] Finally, the average pixel values corresponding to all image grids are calculated to obtain the average pixel value of the background image.

[0152] In another implementation manner, the average pixel value of the background image can be obtained by the following method.

[0153] Firstly, the background image is divided into image grids, so that the background image is divided into a plurality of image grids;

[0154] Then, the average pixel value corresponding to each image grid in the background image is obtained; for example, the average pixel value of all pixel points in the image grid is obtained to obtain the average pixel value corresponding to each image grid;

[0155] Finally, the average pixel value is weighted according to the weight coefficient corresponding to each image grid to obtain the average pixel value of the background image.

[0156] The weight coefficient includes a weight value corresponding to each image grid, and the weight values corresponding to different image grids can be the same or different.

[0157] It should be noted that in the case where the weight values corresponding to different image grids are different, the weight value corresponding to the image grid close to the middle position of the background image is greater than the weight value corresponding to the image grid close to the edge position of the background image. For example, as shown in FIG. 14, in the background image, the farther away from the middle position, the smaller the weight value corresponding to the image grid, and the closer to the middle position, the greater the weight value corresponding to the image grid. Thus, in this embodiment, different weight values are set for image grids at different positions to improve the accuracy of the average pixel value of the background image.

[0158] For example, the weight coefficient corresponding to each image grid is a natural number greater than 0 and less than 1, and based on this, in this embodiment, the average pixel value of the background image, i.e., the weighted average pixel value, can be obtained by weighted sum of the average pixel value according to the weight coefficient corresponding to each image grid.

[0159] It should be noted that when the image acquisition device acquires the background image, the image acquisition device acquires the image according to the coordinate data of the display background in the coordinate system of the image acquisition device.

[0160] Specifically, in this embodiment, before obtaining the pixel statistical data of the background image output by the image acquisition device, the first coordinate data of the display position in the head-up display is obtained first, then the first coordinate data is converted into second coordinate data in the front view coordinate system according to the conversion relationship between the vehicle coordinate system and the front view coordinate system, and then the second coordinate data is provided to the image acquisition device, so that the image acquisition device acquires the background image according to the second coordinate data and outputs the pixel statistical data of the background image, thereby obtaining the pixel statistical data of the background image output by the image acquisition device in S1301.

[0161] The vehicle coordinate system is the coordinate system of the head-up display, i.e., the O v -X v Y v Zv The front view coordinate system is the coordinate system of the image acquisition device, i.e., the O c X c Y c Z c Based on this, in the embodiment, the coordinate data is converted, so that the image acquisition device can output more accurate pixel statistical data, thereby improving the accuracy of subsequent interference judgment.

[0162] Specifically, in the embodiment, when converting the first coordinate data into the second coordinate data, the first coordinate data can be converted into world coordinate data according to the conversion relationship between the vehicle coordinate system and the world coordinate system, and then the world coordinate data can be converted into the second coordinate data according to the conversion relationship between the world coordinate system and the front view coordinate system. Thus, after the second coordinate data is sent to the image acquisition device, the image acquisition device can perform image acquisition according to the second coordinate data, i.e., obtain the background image of the display position and output the pixel statistical data of the background image. Based on this, in the embodiment, the world coordinate system can be used as a coordinate conversion medium between different coordinate systems to realize coordinate conversion between different coordinate systems.

[0163] It should be noted that in the vehicle, the head-up display and the image acquisition device are fixed in position, and their respective coordinate systems are fixed, and the conversion relationship between the corresponding coordinate systems is fixed.

[0164] Step S1302: Obtain the current ambient brightness of the display background according to the pixel statistical data.

[0165] Specifically, the current ambient brightness obtained in the embodiment is the estimated brightness obtained by brightness estimation according to the pixel statistical data.

[0166] In one implementation mode, the pixel statistical data is processed based on a mapping relationship between the pixel statistical data and the current ambient brightness to obtain the current ambient brightness of the display background.

[0167] For example, in the embodiment, the pixel statistical data can be processed according to the proportional coefficient to obtain the current ambient brightness of the display background.

[0168] The proportional coefficient is related to at least one acquisition parameter of the image acquisition device. The proportional coefficient represents that the pixel statistical data and the current ambient brightness are in a linear relationship, i.e., a direct proportional function relationship. The acquisition parameter can include at least one of the following:

[0169] The pixel size of the image acquisition device for acquiring the background image, the quantum efficiency of the image acquisition device, the aperture number of the image acquisition device, the exposure time of the image acquisition device for acquiring the background image, and the gain parameter of the image acquisition device for acquiring the background image.

[0170] wherein the quantum efficiency refers to a percentage of incident photons that can be converted into electrons in the image acquisition device. The gain parameter is composed of three parts, namely, an analog gain, a digital gain, and an image signal processing (ISP) gain.

[0171] For example, the proportional coefficient can be denoted as K, and the current ambient brightness can be obtained by formula (11) in this embodiment: L = K x S (11)

[0172] wherein L represents the current ambient brightness, S represents pixel statistical data such as an average pixel value, and K is determined based on a pixel size pixel size of the image acquisition device for capturing the background image, a quantum efficiency QE of the image acquisition device, an aperture number F of the image acquisition device, an exposure time t of the image acquisition device for capturing the background image, and a gain parameter gain of the image acquisition device for capturing the background image. The gain parameter gain is a product of an analog gain Anolog gain, a digital gain Digital gain, and an image signal processing (ISP) gain ISP gain.

[0173] Based on this, the ambient brightness is estimated in combination with the proportional coefficient corresponding to the image acquisition device in this embodiment, so that the deviation caused by the image acquisition device can be considered, and the estimated ambient brightness is more accurate.

[0174] S1303: Obtain a brightness contrast between the current ambient brightness and a current display brightness of the current display content.

[0175] wherein the brightness contrast is a ratio of the current display brightness to the current ambient brightness. The current display brightness can be obtained from display parameters used by the head-up display to output the current display content.

[0176] S1304: Determine whether the display background causes interference to the display effect of the current display content according to the brightness contrast.

[0177] wherein the brightness contrast can be compared with a first threshold value and a second threshold value in this embodiment, and the display background is determined to cause interference to the display effect of the current display content according to a comparison result.

[0178] wherein the second threshold value is less than the first threshold value.

[0179] Specifically, S1304 can be implemented in the following manner:

[0180] In a case where the brightness contrast is greater than or equal to the first threshold value, it is determined that the display background causes interference to the display effect of the current display content.

[0181] In a case where the luminance contrast is less than or equal to the second threshold value, it is determined that the display background causes interference to the display effect of the current display content;

[0182] In a case where the luminance contrast is less than the first threshold value and greater than the second threshold value, it is determined that the display background does not cause interference to the display effect of the current display content.

[0183] Based on this, in the embodiment, the luminance contrast is divided into three range levels by the two threshold values, and then it is judged whether the display background causes interference to the current display according to the range where the luminance contrast is located, so that the interference judgment is realized by the background luminance.

[0184] Based on the above implementation, in step S204, when the current display content is adjusted to obtain the adjusted display content, the following manner can be used:

[0185] First, the target luminance is obtained according to the luminance contrast; then, the display luminance of the current display content is adjusted according to the target luminance to obtain the adjusted display content.

[0186] Specifically, in the embodiment, the luminance contrast can be compared with the first threshold value and the second threshold value. In a case where the luminance contrast is greater than or equal to the first threshold value, the display luminance of the current display content is adjusted according to the target luminance corresponding to the first threshold value to obtain the adjusted display content; in a case where the luminance contrast is less than or equal to the second threshold value, the display luminance of the current display content is adjusted according to the target luminance corresponding to the second threshold value to obtain the adjusted display content. For example, in a case where the luminance contrast is greater than or equal to the first threshold value, the first luminance corresponding to the first threshold value is determined as the target luminance; in a case where the luminance contrast is less than or equal to the second threshold value, the second luminance corresponding to the second threshold value is determined as the target luminance; in a case where the luminance contrast is less than the first threshold value and greater than the second threshold value, the current display luminance is determined as the target luminance. Then, the target luminance is added to the luminance adjustment instruction, the luminance adjustment instruction is used to instruct the head-up display to adjust the display luminance of the current display content to the target luminance to obtain the adjusted display content, and the head-up display outputs the adjusted display content to the display position. Based on this, in the embodiment, the current display content is adjusted to the target luminance according to the range where the luminance contrast is located, so that the adjustment of the display content is realized.

[0187] It can be seen that, in the embodiment, the pixel statistical data of the background image output by the image acquisition device is obtained, and based on this, the current environment brightness is obtained according to the pixel statistical data. Then, whether the display background interferes with the display effect of the current display content is determined according to the brightness contrast between the current environment brightness and the current display brightness. When it is determined that the display background interferes with the display effect of the current display content, the target brightness is obtained according to the brightness contrast, and the display brightness of the current display content is adjusted according to the target brightness to obtain the adjusted display content. Therefore, the embodiment can automatically adjust the display brightness based on the estimated environment brightness when the head-up display outputs the current display content at the display position, avoid the interference of the background brightness on the display content, and avoid the situation that the display brightness is too high or too low, causing the driver's eyes to be tired or uncomfortable. Therefore, the display effect of the display content is improved, and the experience of the driver watching the display content during driving the transport tool is improved.

[0188] Moreover, in the embodiment, the pixel statistical data provided by the image acquisition device is used to estimate the environment brightness and adjust the brightness accordingly, without image processing by a specific signal processor. Therefore, high-complexity algorithms are not required, and additional devices do not need to be deployed. Therefore, the complexity of hardware implementation and the complexity of algorithm implementation can be reduced.

[0189] Taking the scene of deploying AR-HUD devices and ADAS on a car as an example, the technical solutions of the present application are described in detail as follows:

[0190] Firstly, in the scheme of manually adjusting the projection brightness, the driver needs to adjust it according to his own needs and feelings. This method is simple and easy to implement, but it is easily affected by the subjective factors of the driver, and adjusting the brightness during driving may distract the driver's attention, which may pose a safety hazard.

[0191] Secondly, the automatic adjustment based on the external sensor can sense the brightness information of the surrounding environment in real time by adding an ambient light sensor to the AR-HUD device, and automatically adjust the screen brightness according to the information. The advantage of this scheme is that it can sense the change of ambient light in real time and make corresponding adjustments to improve the adaptability and comfort of the brightness. However, the introduction of external sensors may increase the manufacturing cost and complexity of the AR-HUD device, and the accuracy and stability of the sensor may also affect the brightness adjustment effect.

[0192] In addition, the brightness self-adaptive adjustment based on image analysis can perceive the current image environment brightness information by acquiring the scene image in front of the windshield, and adaptively change the HUD brightness. However, the collected scene image generally needs to be processed by an ISP pipeline for better visual effect, but it cannot truly reflect the original information of the scene, and the complexity of the general algorithm is high, and the processing calculation time is long.

[0193] Therefore, the application proposes an AR-HUD brightness self-adaptive adjustment scheme which is more accurate in ambient light perception and more cost-effective, accurately calculates ambient light brightness (i.e. the current ambient brightness in the foregoing) through H3A statistical data in the ISP processing of the ADAS forward-looking camera, and uses the microcontroller of the HUD sent by the host module to adaptively adjust the projection brightness of the HUD in real time, thereby solving the deficiencies of the prior art.

[0194] The application is described as follows:

[0195] Firstly, the system block diagram of the application is shown in FIG. 15, mainly including modules of: an ISP module (a module deployed in the ADAS forward-looking camera), a HUD module (a module deployed in the HUD device) and a host module (which can be deployed in the ADAS forward-looking camera or in the HUD device). The ISP module is mainly responsible for realizing hardware statistics of image brightness information, and transmitting the statistical results, i.e. H3A statistical data (pixel statistical data in the foregoing), to the host module. The HUD module is mainly responsible for sending and self-adaptively adjusting the eye nucleus position, including: sending the eye nucleus position to the host module and receiving the brightness adjustment command sent by the host module. The host module contains calculation, storage, logical judgment, communication and other functional modules, and is mainly responsible for receiving the H3A statistical data transmitted by the ISP module and the eye nucleus position output by the HUD module, calculating the target brightness according to the input data, and transmitting the brightness adjustment command to the HUD controller through IIC communication, so as to realize self-adaptive adjustment of the HUD brightness.

[0196] The control flow logical block diagram of each module in the application is shown in FIG. 16, and the operations performed by the ISP module, the HUD module and the host module in the HUD brightness self-adaptive adjustment and the logic between the modules are as follows:

[0197] 1. The eye nucleus module in the HUD module sends the eye nucleus position to the host module;

[0198] 2. The host module calibrates the HUD projection area, and performs the following coordinate space calibration conversion: divides the eye nucleus position→human eye coordinate system→world coordinate system→ADAS forward-looking camera coordinate system, and sends the calibrated ADAS forward-looking camera coordinate to the ISP module as the light measurement area (i.e. the projection area) of the AE statistics of the ISP, and performs H3A statistics;

[0199] 3. The ISP module receives the light measurement area, performs H3A statistics, outputs an 8-bit pixel value (i.e. the average value of the pixels) of the statistical value to the host module;

[0200] 4. The master control module receives H3A statistical data, estimates the brightness, takes the brightness estimate value (i.e. the current ambient brightness in the foregoing) as the background brightness, calculates the ratio of the current projection brightness and the background brightness, and calculates the target brightness based on the contrast threshold (i.e. the first threshold and the second threshold in the foregoing), and sends the brightness adjustment command containing the target brightness to the HUD module;

[0201] 5. The HUD module receives the brightness adjustment command sent by the master control module and performs brightness self-adaptive adjustment.

[0202] The HUD projection area calibration diagram of the present application is shown in Figure 17. The light source in the HUD module returns through multiple internal light paths, projects the driving assistance information to the front windshield glass and reflects into the human eye, and the driver can see a virtual image in the front at a long distance and magnification. In the human eye coordinate system x-y-z (i.e. vehicle coordinate system), the virtual image position coordinates are ABCD. The virtual image ABCD seen by the human eye has a corresponding position in the world coordinate system x'-y'-z' of the road, which can be converted to A'B'C'D' through calibration. The ADAS front-view camera above the windshield glass captures the front road conditions in real time, and can convert the position A'B'C'D' of the HUD image in the world coordinate system to A”B”C”D” in the ADAS front-view coordinate system x”-y”-z” (i.e. front-view coordinate system). As described above, the conversion and calibration of the human eye coordinate system x-y-z→world coordinate system x'-y'-z'→ADAS front-view coordinate system x”-y”-z” are completed, and the accurate position of the HUD projection area in the image capture picture of the ADAS front-view camera in the field of view angle FOV can be obtained.

[0203] The AR-HUD brightness self-adaptive adjustment method based on H3A statistical data proposed in the present application is shown in Figure 18, and the specific steps are as follows:

[0204] S1801: The ADAS front-view camera acquires the front road image in real time;

[0205] S1802: HUD projection area (i.e. display position) calibration. The HUD imaging position is calibrated as follows: first, adjust the projection to the best eye nucleus position to ensure the clarity and readability of the projected information, and determine the coordinates A, B, C, D in the human eye coordinate system. Secondly, these coordinates are converted to the world coordinate system and ensured to match the actual position of the vehicle, obtaining A', B', C', D'. Finally, they are further converted to the ADAS imaging coordinate system, obtaining A”, B”, C”, D”. Calibrating the projection area of the HUD in the ADAS image ensures the correctness of the projection background brightness calculation.

[0206] S1803: H3A statistical data of the ROI (Region of interest) area (i.e. projection area) for ISP is analyzed. The calibration coordinates A", B", C", D" of the HUD projection in the ADAS image obtained in S1802 will be used as the coordinates of the light measurement area of the ADAS image for H3A statistical data. H3A (Histogram 3A) is based on the histogram information of the image, and the brightness and color distribution are statistically analyzed to realize the optimization of the automatic focusing, automatic exposure and automatic white balance algorithm parameters. The light measurement area A", B", C", D" is divided into MxN image grids, and the statistical value and average value of each grid are calculated to obtain the average pixel value of each image grid; each image grid corresponds to a weight, and the higher the weight, the greater the influence of the projection area on the light measurement. Based on this, the weighted average pixel value of the projection area image (i.e. the average pixel value of the background image mentioned above) is quickly calculated according to the weight, and the calculation of the average value and the weight of the image grid is greatly reduced compared to the pixel-by-pixel traversal calculation. It can be seen that the advantage of using the H3A data of the image for brightness statistics is that:

[0207] (1) No additional calculation unit is needed, only the existing ADAS front-view camera AE statistical information (i.e. H3A statistical data) is called.

[0208] (2) H3A statistical data is more original and has not been processed by ISP Pipeline, which can more accurately reflect the actual environmental brightness.

[0209] (3) The use of image grid and weighting greatly reduces the calculation amount, and the brightness statistics is rapid.

[0210] S1804: The microcontroller in the main control model estimates the environmental brightness. The microcontroller receives the H3A statistical data of the ADAS front-view camera for the front picture, and estimates the environmental illumination. The specific conversion relationship between the average pixel value of the image and the environmental illumination value is as shown in formula (12) and formula (13): -2 Average pixel value of image ∝ L x F Gain = Anolog_gain x Digital_gain x ISP_gain (13)

[0211] wherein, L is the ambient illuminance, F is the aperture number of the front-view camera, t is the exposure time of the front-view camera, QE is the quantum efficiency of the front-view camera, pixel_size is the pixel size of the front-view camera, and gain is the total gain, which is composed of three parts of analog gain, digital gain and ISP gain, as shown in formula (13). Further, formula (12) can be arranged into formula (14): Average pixel value of image = k x L x F -2 x t x Gain (14)

[0212] wherein, k can be obtained by calibration, that is, QE x pixel_size, and then the current ambient brightness L is accurately estimated by H3A average pixel value. The current ambient brightness and the average pixel value are in a proportional function relationship. Only the relationship between the average pixel value and the ambient brightness in several groups of test data needs to be measured, and the proportional coefficient K can be obtained by linear fitting, as K in formula (11).

[0213] After the current ambient brightness is calculated, the optimal projection brightness (i.e., target brightness) calculation is performed in the main control module, as follows.

[0214] S1805: Optimal projection brightness calculation. After the main control module obtains the current ambient brightness L, the optimal projection brightness is calculated using the contrast threshold, and the specific calculation method is as follows:

[0215] 1) Obtain the current projection brightness of the AR-HUD;

[0216] 2) Calculate the brightness contrast of the current projection brightness and the current ambient brightness, and determine whether it is greater than or equal to the upper threshold (first threshold) or less than or equal to the lower threshold (i.e., second threshold), as formula (15);

[0217] If the brightness contrast is greater than or equal to the upper threshold, it can be determined that the background environment interferes with the current display content of the projection area. At this time, the brightness under the upper threshold condition (i.e., the first brightness) is determined as the target brightness, which is used to indicate the adjustment of the current projection brightness until the target brightness is reached.

[0218] If the brightness contrast is between the upper threshold and the lower threshold, it can be determined that the background environment does not interfere with the current display content of the projection area. At this time, the current projection brightness is determined as the target brightness, and no adjustment is made to the brightness.

[0219] If the brightness contrast is less than or equal to the lower threshold, it can be determined that the background environment interferes with the current display content of the projection area. At this time, the brightness under the lower threshold condition (i.e., the second brightness) is determined as the target brightness, which is used to indicate the adjustment of the current brightness until the target brightness is reached.

[0220] Wherein, the contrast threshold is obtained by calibration measurement, the upper contrast threshold is suggested to be set between 5 and 7, and the lower contrast threshold is suggested to be set between 1 and 2.

[0221] S1806: HUD brightness adaptive projection. The controller in the HUD module receives the brightness adjustment command sent by the master module, and performs brightness adaptive projection, i.e., adjusts the display brightness of the current display content to the target brightness.

[0222] It can be seen that the application provides an AR-HUD brightness adaptive adjustment scheme based on H3A data. In the scheme, the front image is captured in real time by using the ADAS front-view camera, the current environmental brightness is calculated by using the H3A statistical data in the ISP of the front-view camera, and the HUD display brightness is adaptively adjusted according to different environmental brightness, so as to provide appropriate contrast and enhance the driving experience.

[0223] As shown in FIG. 19, the application provides an apparatus for adjusting the display content of the head-up display. The apparatus can include the following modules:

[0224] The acquisition module 1901 is configured to acquire the environmental information of the current driving road and the driving information. The driving information includes the region where the eyeball of the driver is located and the region where the pupil of the driver is located.

[0225] The determination module 1902 is configured to determine the display position of the current display content and the display background of the current display content according to the environmental information and the driving information. The current display content is the display content of the head-up display at the current time.

[0226] The judgment module 1903 is configured to judge whether the display background causes interference to the display effect of the current display content.

[0227] The adjustment module 1904 is configured to adjust the current display content to obtain the adjusted display content if the display background causes interference to the display effect of the current display content. The display effect of the adjusted display content is not interfered by the display background.

[0228] It can be learned from the above scheme that the device for adjusting display content of a head-up display provided in the embodiment of the application obtains environment information of a current driving road and driving information including a region where eyeballs of a driver are located and a region where pupils of the driver are located. According to the environment information and the driving information, a display position of current display content and a display background of the current display content are determined. It is judged whether the display background causes interference to a display effect of the current display content. If the display background causes interference to the display effect of the current display content, the current display content is adjusted to obtain adjusted display content. Since the current display content is adjusted when the display effect of the current display content is interfered by the display background in the application, the display background cannot interfere with the adjusted display content, so that the display effect of the head-up display is improved.

[0229] In an optional embodiment, when judging whether the display background causes interference to the display effect of the current display content according to the first interference degree judgment formula, the judging module 1903 is specifically used for:

[0230] judging a type of the display background according to a preset convolutional neural network; the type of the display background includes a single background and a complex background; the single background is a display background including only one brightness, one color and one texture; the complex background is a display background including multiple colors, multiple brightnesses or multiple textures;

[0231] when the type of the display background is the single background, judging whether the display background causes interference to the display effect of the current display content according to the first interference degree judgment formula;

[0232] when the type of the display background is the complex background, judging whether the display background causes interference to the display effect of the current display content according to a second interference degree judgment formula.

[0233] In an optional embodiment, the judging module 1903 is specifically used for:

[0234] obtaining the brightness and the color of the single background, the brightness and the color of the current display content, a first comparison threshold, and a first brightness difference coefficient and a first color difference coefficient;

[0235] determining a difference between the brightness of the single background and the brightness of the current display content according to the brightness of the single background and the brightness of the current display content, to obtain a first brightness difference;

[0236] determining a difference between the color of the single background and the color of the current display content according to the color of the single background and the color of the current display content, to obtain a first color difference;

[0237] The first luminance difference, the first color difference, the first luminance difference coefficient, the first color difference coefficient, and the first comparison threshold are input into a first interference degree judgment formula to obtain a judgment result; the judgment result is used to indicate whether the display background interferes with the display effect of the current display content.

[0238] In an optional embodiment, the judgment module 1903 is specifically used for judging whether the display background interferes with the display effect of the current display content according to a second interference degree judgment formula.

[0239] The luminance and color of the complex background, the luminance and color of the current display content, a second luminance difference coefficient, a second color difference coefficient, a second comparison threshold, and an intersection-over-union ratio between the current display content and objects in the display background are obtained.

[0240] A second luminance difference between the luminance of the complex background and the luminance of the current display content is determined according to the luminance of the complex background and the luminance of the current display content.

[0241] A second color difference between the color of the complex background and the color of the current display content is determined according to the color of the complex background and the color of the current display content.

[0242] The second luminance difference, the second color difference, the second luminance difference coefficient, the second color difference coefficient, the second comparison threshold, and the intersection-over-union ratio between the current display content and objects in the display background are input into the second interference degree judgment formula to obtain a judgment result; the judgment result is used to indicate whether the display background interferes with the display effect of the current display content.

[0243] In an optional embodiment, the adjustment module 1904 is specifically used for:

[0244] When the type of the display background is a single background, if the display background interferes with the display effect of the current display content, the color and display form of the current display content are adjusted according to the display background to obtain the adjusted display content.

[0245] When the type of the display background is a complex background, if the display background interferes with the display effect of the current display content, the display background of the current frame is predicted according to the display backgrounds of the previous n frames to obtain a predicted display background; n is an integer greater than or equal to 2; the color and display form of the current display content are adjusted according to the predicted display background to obtain the adjusted display content.

[0246] In an optional embodiment, the adjusting module 1904 is specifically configured to adjust the color and display form of the current display content according to the display background, to obtain the adjusted display content.

[0247] When the display background is a single dark background or a low-brightness background, the color of the current display content is modified to be light, or the display form of the current display content is modified to add a light background color under the display content, to obtain the adjusted display content.

[0248] When the display background is a single light background or a high-brightness background, the color of the current display content is modified to be dark, or the display form of the current display content is modified to add a dark background color under the display content, to obtain the adjusted display content.

[0249] In an optional embodiment, the adjusting module 1904 is specifically configured to adjust the color and display form of the current display content according to the predicted display background, to obtain the adjusted display content.

[0250] When the overlapping area of the predicted display background and the current display content includes multiple similar colors, the display form of the current display content is modified to add a preset background color under the display content; the similar colors are colors in the predicted display background that have a color difference from the color of the current display content less than a color difference threshold; the preset background color has a color difference from the multiple similar colors greater than the color difference threshold.

[0251] When the overlapping area of the predicted display background and the current display content includes one similar color, the color of the current display content is modified to be a preset color; the preset color has a color difference from the one similar color greater than the color difference threshold.

[0252] In an optional embodiment, the determining module 1902 is specifically configured to:

[0253] determine a display position of the current display content according to the driving information;

[0254] take an overlapping area of the environmental information and the display position of the current display content in a vehicle coordinate system as the display background of the current display content; the vehicle coordinate system is a coordinate system constructed with a vehicle center as an origin

[0255] In an optional embodiment, the apparatus further includes the following modules, as shown in FIG. 20:

[0256] The restoring module 1905 is configured to restore the color and display form of the current display content to preset values if the display background does not interfere with the display effect of the current display content.

[0257] In an optional embodiment, the apparatus further comprises the following modules, as shown in FIG. 21:

[0258] a mode selection module 1906, configured to select a manual mode or an automatic mode; the manual mode comprises a daily mode, a sports mode and a snow mode;

[0259] wherein the acquisition module 1901 is configured to acquire environment information of a current driving road and driving information after the mode selection module 1906 selects the automatic mode; the driving information comprises a region where an eyeball of a driver is located and a region where a pupil of the driver is located.

[0260] In an optional embodiment, the judgment module 1903 is specifically configured to: obtain pixel statistical data of a background image output by an image acquisition device; the background image is a background image corresponding to the display background obtained by the image acquisition device collecting the display position; obtain a current environment brightness of the display background according to the pixel statistical data; obtain a brightness contrast between the current environment brightness and a current display brightness of the current display content; and determine whether the display background interferes with the display effect of the current display content according to the brightness contrast.

[0261] In an optional embodiment, when determining whether the display background interferes with the display effect of the current display content according to the brightness contrast, the judgment module 1903 is specifically configured to: in a case where the brightness contrast is greater than or equal to a first threshold value, or in a case where the brightness contrast is less than or equal to a second threshold value, determine that the display background interferes with the display effect of the current display content; and in a case where the brightness contrast is less than the first threshold value and greater than the second threshold value, determine that the display background does not interfere with the display effect of the current display content. The pixel statistical data comprises: an average pixel value of the background image; the average pixel value of the background image is obtained by the following manner:

[0262] obtaining a pixel average value corresponding to each image grid in the background image; and performing weighted processing on the pixel average value according to a weight coefficient corresponding to each image grid to obtain the average pixel value of the background image.

[0263] In an optional embodiment, when obtaining the current environment brightness of the display background according to the pixel statistical data, the judgment module 1903 is specifically configured to: process the pixel statistical data according to a proportion coefficient to obtain the current environment brightness of the display background; wherein the proportion coefficient is related to at least one acquisition parameter of the image acquisition device, and the acquisition parameter comprises at least one of the following:

[0264] The image acquisition device acquires a pixel size of the background image, a quantum efficiency of the image acquisition device, an aperture number of the image acquisition device, an exposure time of the image acquisition device for acquiring the background image, and a gain parameter of the image acquisition device for acquiring the background image.

[0265] In an optional embodiment, the adjusting module 1904 is specifically configured to: in a case where the luminance contrast is greater than or equal to a first threshold value, adjust the display luminance of the current display content according to a target luminance corresponding to the first threshold value, to obtain adjusted display content; and in a case where the luminance contrast is less than or equal to a second threshold value, adjust the display luminance of the current display content according to a target luminance corresponding to the second threshold value, to obtain adjusted display content.

[0266] In an optional embodiment, before the judging module 1903 obtains the pixel statistical data of the background image output by the image acquisition device, the system is further configured to: obtain first coordinate data of the display position in the head-up display; convert the first coordinate data into second coordinate data on a front view coordinate system according to a conversion relationship between a vehicle coordinate system and the front view coordinate system; wherein the vehicle coordinate system is a coordinate system of the head-up display, and the front view coordinate system is a coordinate system of the image acquisition device; and provide the second coordinate data to the image acquisition device, so that the image acquisition device acquires the background image according to the second coordinate data and outputs the pixel statistical data of the background image.

[0267] It should be noted that the specific implementation of each module in this embodiment can refer to the corresponding content in the foregoing, which will not be described in detail here.

[0268] In addition, the embodiment of the present application further provides a system for adjusting display content of a head-up display, the system being configured to switch a display mode of the head-up display to adjust the display content of the head-up display; the display mode including a manual mode and an automatic mode; the manual mode including a daily mode, a sports mode, and a snow mode;

[0269] When the display mode is the daily mode, the color of the display content of the head-up display is white;

[0270] When the display mode is the sports mode, the display content of the head-up display flashes at a fixed frequency;

[0271] When the display mode is the snow mode, the color of the display content of the head-up display is blue;

[0272] When the display mode is the automatic mode, the system executes any one of the above methods for adjusting display content of a head-up display.

[0273] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, any of the above-mentioned methods for adjusting the display content of a head-up display is implemented.

[0274] An embodiment of the present application further provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the above-mentioned methods for adjusting the display content of a head-up display.

[0275] An embodiment of the present application further provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0276] The memory is used to store computer programs;

[0277] The processor is configured to execute the computer program so that the electronic device can implement any one of the above methods for adjusting the display content of a head-up display.

[0278] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0279] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0280] In the embodiments described above, the entire or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, the entire or part of the embodiments can be implemented in the form of a computer program product.

[0281] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the entire or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, training device or data center to another website site, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer readable storage medium can be any available medium that can be stored by the computer or data storage device such as training device, data center, etc. integrated with one or more available media sets. The available media can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, solid state disk (SSD)) and the like.

Claims

1. A method of adjusting a head-up display content, characterized by, The method comprises: obtaining environment information of a current driving road and driving information; the driving information comprises a region where eyeballs of a driver are located and a region where pupils of the driver are located; determining a display position of current display content and a display background of the current display content according to the environment information and the driving information; the current display content is display content of a head-up display at a current time; judging whether the display background interferes with display effect of the current display content; if the display background interferes with the display effect of the current display content, adjusting the current display content to obtain adjusted display content; the display effect of the adjusted display content is not interfered with by the display background.

2. The method of claim 1, wherein, The judgment of whether the display background interferes with the display effect of the current display content comprises: judging a type of the display background according to a preset convolutional neural network; the type of the display background comprises a single background and a complex background; the single background is a display background comprising only one brightness, one color and one texture; the complex background is a display background comprising multiple colors, multiple brightnesses or multiple textures; when the type of the display background is the single background, judging whether the display background interferes with the display effect of the current display content according to a first interference degree judgment formula; when the type of the display background is the complex background, judging whether the display background interferes with the display effect of the current display content according to a second interference degree judgment formula.

3. The method of claim 2, wherein, The judgment of whether the display background interferes with the display effect of the current display content according to the first interference degree judgment formula comprises: obtaining brightness and color of the single background, brightness and color of the current display content, a first comparison threshold, and a first brightness difference coefficient and a first color difference coefficient; determining a difference between the brightness of the single background and the brightness of the current display content according to the brightness of the single background and the brightness of the current display content to obtain a first brightness difference; determining a difference between the color of the single background and the color of the current display content according to the color of the single background and the color of the current display content to obtain a first color difference; inputting the first brightness difference, the first color difference, the first brightness difference coefficient, the first color difference coefficient and the first comparison threshold into the first interference degree judgment formula to obtain a judgment result; the judgment result is used to indicate whether the display background interferes with the display effect of the current display content.

4. The method of claim 2, wherein, The judgment of whether the display background interferes with the display effect of the current display content according to the second interference degree judgment formula comprises: obtaining brightness and color of the complex background, brightness and color of the current display content, a second brightness difference coefficient, a second color difference coefficient, a second comparison threshold, and an intersection-over-union ratio between the current display content and objects in the display background. determining a difference between the brightness of the complex background and the brightness of the current display content according to the brightness of the complex background and the brightness of the current display content, to obtain a second brightness difference; determining a difference between the color of the complex background and the color of the current display content according to the color of the complex background and the color of the current display content, to obtain a second color difference; inputting the second brightness difference, the second color difference, the second brightness difference coefficient, the second color difference coefficient, the second comparison threshold, and the intersection-over-union ratio between the current display content and the object in the display background into the second interference degree judgment formula to obtain a judgment result; the judgment result is used to indicate whether the display background interferes with the display effect of the current display content. if the display background interferes with the display effect of the current display content, adjusting the current display content to obtain an adjusted display content, including:

5. The method of claim 2, wherein, if the type of the display background is a single background, and the display background interferes with the display effect of the current display content, adjusting the color and display form of the current display content according to the display background to obtain the adjusted display content; if the type of the display background is a complex background, and the display background interferes with the display effect of the current display content, predicting the display background of the current frame according to the display background of the previous n frames to obtain a predicted display background; n is an integer greater than or equal to 2; adjusting the color and display form of the current display content according to the predicted display background to obtain the adjusted display content. adjusting the color and display form of the current display content according to the display background to obtain the adjusted display content, including:

6. The method of claim 5, wherein, if the display background is a single dark background or a low-brightness background, modifying the color of the current display content to a light color, or modifying the display form of the current display content to add a light background color under the display content to obtain the adjusted display content; if the display background is a single light background or a high-brightness background, modifying the color of the current display content to a dark color, or modifying the display form of the current display content to add a dark background color under the display content to obtain the adjusted display content. adjusting the color and display form of the current display content according to the predicted display background to obtain the adjusted display content, including:

7. The method of claim 5, wherein, if the overlapping area between the predicted display background and the current display content includes multiple similar colors, modifying the display form of the current display content to add a preset background color under the display content; the similar colors are colors in the predicted display background that have a color difference less than a color difference threshold from the color of the current display content; the preset background color has a color difference greater than the color difference threshold from the multiple similar colors; ​ When a similar color is included in the overlapping area of the predicted display background and the current display content, the color of the current display content is modified to a preset color; the preset color has a color difference with the similar color greater than the color difference threshold.

8. The method of claim 1, 2, 3, 4, 5, 6, or 7, wherein, The display position of the current display content and the display background of the current display content are determined according to the environment information and the driving information, and the display position of the current display content is determined according to the driving information. The display position of the current display content is determined according to the driving information. The overlapping area of the environment information and the display position of the current display content in a vehicle coordinate system is taken as the display background of the current display content; the vehicle coordinate system is a coordinate system with a vehicle center as an origin.

9. The method of claim 1, 2, 3, 4, 5, 6, 7, or 8, wherein, It is determined whether the display background interferes with the display effect of the current display content, including: Pixel statistical data of a background image output by an image acquisition device is obtained; the background image is a background image corresponding to the display background obtained by the image acquisition device performing image acquisition on the display position; The current environment brightness of the display background is obtained according to the pixel statistical data; The brightness contrast between the current environment brightness and the current display brightness of the current display content is obtained; It is determined whether the display background interferes with the display effect of the current display content according to the brightness contrast.

10. The method of claim 9, wherein, It is determined whether the display background interferes with the display effect of the current display content according to the brightness contrast, including: In a case where the brightness contrast is greater than or equal to a first threshold value, or in a case where the brightness contrast is less than or equal to a second threshold value, it is determined that the display background interferes with the display effect of the current display content; In a case where the brightness contrast is less than the first threshold value and greater than the second threshold value, it is determined that the display background does not interfere with the display effect of the current display content.

11. The method of claim 10, wherein, The current display content is adjusted to obtain adjusted display content, including: In a case where the brightness contrast is greater than or equal to a first threshold value, the display brightness of the current display content is adjusted according to a target brightness corresponding to the first threshold value to obtain adjusted display content; In a case where the brightness contrast is less than or equal to a second threshold value, the display brightness of the current display content is adjusted according to a target brightness corresponding to the second threshold value to obtain adjusted display content.

12. The method of claim 9, 10 or 11, characterized in that, The pixel statistical data includes an average pixel value of the background image. The average pixel value of the background image is obtained by the following method: An average pixel value corresponding to each image grid in the background image is obtained; The average pixel value is weighted according to a weight coefficient corresponding to each image grid to obtain the average pixel value of the background image.

13. The method of claim 12, wherein, The weight coefficient includes a weight value corresponding to each image grid, and the weight values corresponding to different image grids are the same or different.

14. The method of claim 13, wherein, In the case that the weight values corresponding to different image grids are different, the weight value corresponding to an image grid close to the middle position of the background image is greater than the weight value corresponding to an image grid close to the edge position of the background image.

15. The method of claim 9, 10, or 11, wherein, The pixel statistical data comprises an average pixel value of the background image. The average pixel value of the background image is obtained by the following manner: An average pixel value corresponding to each image grid in the background image is obtained. An average value calculation is performed on the average pixel values corresponding to all the image grids to obtain the average pixel value of the background image.

16. The method of claim 9, 10, 11, or 12, wherein, According to the pixel statistical data, the current ambient brightness of the display background is obtained, comprising: The pixel statistical data is processed according to a proportion coefficient to obtain the current ambient brightness of the display background; The proportion coefficient is related to at least one acquisition parameter of the image acquisition device, and the acquisition parameter comprises at least one of the following: The pixel size of the background image acquired by the image acquisition device, the quantum efficiency of the image acquisition device, the aperture number of the image acquisition device, the exposure time of the background image acquired by the image acquisition device, and the gain parameter of the background image acquired by the image acquisition device.

17. The method of claim 9, 10, 11, or 12, wherein, Before obtaining the pixel statistical data of the background image output by the image acquisition device, the method further comprises: Obtaining first coordinate data of the display position in the head-up display; Converting the first coordinate data into second coordinate data on the front view coordinate according to the conversion relationship between the vehicle coordinate system and the front view coordinate system; The vehicle coordinate system is the coordinate system of the head-up display, and the front view coordinate system is the coordinate system of the image acquisition device; Providing the second coordinate data to the image acquisition device, so that the image acquisition device acquires the background image according to the second coordinate data and outputs the pixel statistical data of the background image.

18. The method of claim 17, wherein, Converting the first coordinate data into second coordinate data on the front view coordinate according to the conversion relationship between the vehicle coordinate system and the front view coordinate system, comprising: Converting the first coordinate data into world coordinate data according to the conversion relationship between the vehicle coordinate system and the world coordinate system; Converting the world coordinate data into second coordinate data on the front view coordinate according to the conversion relationship between the world coordinate system and the front view coordinate system.

19. An apparatus for adjusting a heads-up display content, the apparatus comprising: The device comprises: An acquisition module configured to acquire environmental information of a current driving road and driving information, wherein the driving information comprises a region where an eyeball of a driver is located and a region where a pupil of the driver is located; A determination module configured to determine a display position of current display content and a display background of the current display content according to the environmental information and the driving information, wherein the current display content is display content of a head-up display at a current time; A judgment module configured to judge whether the display background causes interference to a display effect of the current display content. An adjusting module is configured to adjust the current display content to obtain adjusted display content if the display background interferes with the display effect of the current display content.

20. A system for adjusting the display content of a heads-up display, the system comprising: The system is configured to switch a display mode of the head-up display to adjust display content of the head-up display; the display mode includes a manual mode and an automatic mode; the manual mode includes a daily mode, a sports mode and a snow mode; When the display mode is the daily mode, the color of the display content of the head-up display is white; When the display mode is the sports mode, the display content of the head-up display flashes at a fixed frequency; When the display mode is the snow mode, the color of the display content of the head-up display is blue; When the display mode is the automatic mode, the system executes the method for adjusting the display content of the head-up display in any one of claims 1 to 18.

Images