Calibration method and apparatus for head up display, and electronic device and storage medium

Abstract

Provided are a calibration method and apparatus for a head up display, and an electronic device and a storage medium. The calibration method for a head up display comprises: acquiring a virtual image to be processed corresponding to each to be processed among a plurality of continuously distributed eye box partitions to be processed (S1301); for said virtual image corresponding to each eye box partition, identifying the coverage range of a preset visible area corresponding to said eye box partition in a display screen, which coverage range is in said virtual image (S1303); converting the coverage range from a coordinate system of said virtual image into a coordinate system of the display screen, so as to obtain a mapping area of the preset visible area corresponding to said eye box partition in the display screen (S1305); and on the basis of a mapping area corresponding to an adjacent eye box partition for said eye box partition, adjusting the mapping area corresponding to said eye box partition in the display screen, so as to obtain a target visible area corresponding to said eye box partition in the display screen (S1307). The calibration method can optimize the display effect of a head up display, thereby improving the watching experience of a user.

Description

Head-up display device calibration method, device, electronic device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 26, 2024, with application number 202410352780.X. The entire contents of the above application are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of head-up display technology, and in particular to a calibration method, apparatus, electronic device, and storage medium for a head-up display device. Background Art

[0003] Naked-eye 3D (3-dimension) display technology refers to a type of 3D display technology that allows users to view three-dimensional images directly with the naked eye without the need for dedicated 3D glasses, creating a 3D effect. For example, by placing a slit grating or a cylindrical lens grating in front of the display screen, the light emitted by the display screen is split by the slit grating or cylindrical lens grating, and then refracted by the imaging structure before entering the driver's left and right eyes respectively. The driver's left and right eyes see different images, forming binocular parallax, which synthesizes a three-dimensional image with a sense of depth in the user's brain, achieving a 3D viewing experience. In the process of realizing naked-eye 3D display technology, the quality of the image display directly affects the user's ultimate viewing experience.

[0004] However, the head-up display (HUD) itself is a non-uniform system with distortion or processing tolerances. Even if the pixel pitch and grating design are uniform, there will still be certain differences, causing the image seen by the human eye to shift, tilt, distort, and other problems. For example, when a user is watching, the left eye may see the image intended for the right eye, or the right eye may see the image intended for the left eye, that is, crosstalk occurs between the images seen by the user's left eye and right eye, resulting in a poor user experience. Summary of the Invention

[0005] In order to solve the problems of related technologies, the embodiments of the present application provide a calibration method, apparatus, electronic device and storage medium for a head-up display device. The technical solution is as follows:

[0006] In one aspect, a calibration method for a head-up display device is provided, wherein the head-up display device includes a display screen and an imaging structure; the method includes:

[0007] Acquire a virtual image to be processed corresponding to each of a plurality of continuously distributed eye box partitions to be processed; the virtual image to be processed is formed by light emitted from a real visible area corresponding to the eye box partition to be processed on the display screen and projected onto the eye box partition to be processed through the imaging structure;

[0008] For each virtual image to be processed corresponding to the eye box partition to be processed, identifying the coverage of the preset visible area corresponding to the eye box partition to be processed on the display screen in the virtual image to be processed;

[0009] Converting the coverage range from the coordinate system corresponding to the to-be-processed virtual image to the coordinate system corresponding to the display screen, and obtaining a mapping area of ​​the preset visible area corresponding to the to-be-processed eye box partition on the display screen;

[0010] Based on the mapping area corresponding to the adjacent eye box partition of the eye box partition to be processed, the mapping area corresponding to the eye box partition to be processed in the display screen is adjusted to obtain the target visible area corresponding to the eye box partition to be processed in the display screen; the adjacent eye box partition is the eye box partition to be processed adjacent to the eye box partition to be processed among the multiple eye box partitions to be processed.

[0011] In another aspect, a calibration device for a head-up display device is provided. The head-up display device includes a display screen and an imaging structure. The device includes:

[0012] a virtual image acquisition module, configured to acquire a virtual image to be processed corresponding to each of a plurality of continuously distributed eye box partitions to be processed; the virtual image to be processed is formed by light emitted from a real visible area corresponding to the eye box partition to be processed on the display screen, passing through the imaging structure and projected onto the eye box partition to be processed;

[0013] an area recognition module, configured to identify, for each virtual image to be processed corresponding to the eye box partition to be processed, a coverage range of a preset visible area corresponding to the eye box partition to be processed on the display screen in the virtual image to be processed;

[0014] an area mapping module, configured to convert the coverage area from the coordinate system corresponding to the virtual image to be processed to the coordinate system corresponding to the display screen, and obtain a mapping area of ​​the preset visible area corresponding to the eye box partition to be processed on the display screen;

[0015] An area adjustment module is used to adjust the mapping area corresponding to the eye box partition to be processed in the display screen based on the mapping area corresponding to the adjacent eye box partition of the eye box partition to be processed, so as to obtain the target visible area corresponding to the eye box partition to be processed in the display screen; the adjacent eye box partition is the eye box partition to be processed adjacent to the eye box partition to be processed among the multiple eye box partitions to be processed.

[0016] In an exemplary embodiment, the region identification module includes:

[0017] A first adjacent determination module is configured to determine, for each of the eye box partitions to be processed, a virtual image to be processed corresponding to an adjacent eye box partition of the eye box partition to be processed, to obtain at least one adjacent virtual image;

[0018] a virtual image superposition module, configured to superimpose the virtual image to be processed corresponding to the eye box partition to be processed with the at least one adjacent virtual image to obtain at least one image to be recognized;

[0019] A first boundary determination module is configured to determine, for each pixel point in the at least one image to be identified, that the pixel point is a boundary pixel point corresponding to at least one side boundary of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed, when the brightness information of the pixel point matches the first brightness information; the first brightness information indicates the brightness of the middle line of the preset visible area in the image to be identified; and the multiple boundary pixel points corresponding to at least one side boundary in the virtual image to be processed indicate the coverage range.

[0020] In an exemplary embodiment, the apparatus further includes a boundary brightness module for determining whether the boundary can be determined based on the boundary line brightness before the boundary of the preset visible area corresponding to the eye box partition to be processed is determined based on the midline brightness in the virtual image to be processed. The boundary brightness module includes:

[0021] a brightness matching module configured to determine, for each pixel in the virtual image to be processed and corresponding to each eye box partition to be processed, whether the brightness information of the pixel matches second brightness information, wherein the second brightness information indicates the brightness of a boundary line of a preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed;

[0022] The second adjacent determination module is configured to, if there is no pixel whose brightness information matches the second brightness information in the virtual image to be processed, perform the step of determining, for each eye box partition to be processed, a virtual image to be processed corresponding to an adjacent eye box partition of the eye box partition to be processed, to obtain at least one adjacent virtual image.

[0023] In an exemplary embodiment, the region identification module includes:

[0024] A second boundary determination module is used to determine that the pixel point is a boundary pixel point of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed when the brightness information of the pixel point matches the second brightness information; multiple boundary pixel points in the virtual image to be processed indicate the coverage range.

[0025] In an exemplary embodiment, the region adjustment module includes:

[0026] The area splicing module is used to splice the mapping area corresponding to the adjacent eye box partition of the eye box partition to be processed with the mapping area corresponding to the eye box partition to be processed to obtain the target visible area corresponding to the eye box partition to be processed in the display screen.

[0027] In an exemplary embodiment, the apparatus further includes a brightness processing module for eliminating backlighting of the virtual image to be processed before identifying the coverage of the preset visible area corresponding to the eye box partition to be processed in the display screen in the virtual image to be processed. The brightness processing module includes:

[0028] An image acquisition module, configured to acquire an original image displayed in a preset visible area corresponding to the eye box partition to be processed in the display screen;

[0029] The brightness normalization module is used to normalize the brightness information of each pixel in the to-be-processed virtual image corresponding to the to-be-processed eye box partition based on the brightness information of each pixel in the original image.

[0030] In an exemplary embodiment, the apparatus further includes a head-up display module for displaying an image based on the calibrated head-up display device, the head-up display module including:

[0031] An information acquisition module is configured to acquire an image to be displayed, a left-eye eyebox partition, and a right-eye eyebox partition; the left-eye eyebox partition is a to-be-processed eyebox partition corresponding to the left eye of the target object among the plurality of to-be-processed eyebox partitions; the right-eye eyebox partition is a to-be-processed eyebox partition corresponding to the right eye of the target object among the plurality of to-be-processed eyebox partitions;

[0032] The image display module is configured to display the image to be displayed based on the target visible area corresponding to the left eye box partition; and to display the image to be displayed based on the target visible area corresponding to the right eye box partition.

[0033] On the other hand, an electronic device is provided, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the calibration method of the head-up display device of any of the above aspects.

[0034] On the other hand, a computer-readable storage medium is provided, in which at least one instruction or at least one program is stored. The at least one instruction or the at least one program is loaded and executed by a processor to implement a calibration method for a head-up display device as described in any of the above aspects.

[0035] In another aspect, a computer program product or computer program is provided. The computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the head-up display device calibration method according to any of the above aspects.

[0036] The embodiment of the present application obtains a virtual image to be processed corresponding to each of a plurality of continuously distributed eye box partitions to be processed; for each virtual image to be processed corresponding to the eye box partition to be processed, identifies the coverage of the preset visible area corresponding to the eye box partition to be processed on the display screen in the virtual image to be processed; converts the coverage from the coordinate system of the virtual image to be processed to the coordinate system of the display screen to obtain the mapping area of ​​the preset visible area corresponding to the eye box partition to be processed on the display screen; and adjusts the mapping area corresponding to the eye box partition to be processed on the display screen based on the mapping area corresponding to the adjacent eye box partition to be processed to obtain the target visible area corresponding to the eye box partition to be processed on the display screen. The present application adjusts the preset visible area corresponding to each eye box partition to be processed on the display screen using the preset visible area corresponding to the adjacent eye box partition to be processed on the display screen, thereby optimizing the display effect of the head-up display device, so that the user's eyes can achieve the effect of the left eye seeing the image intended for the left eye and the right eye seeing the image intended for the right eye at any position of the eye box, thereby improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] FIG1 is a schematic diagram of an application environment of a head-up display device provided in an embodiment of the present application;

[0039] FIG2 is a schematic diagram of the principle of a head-up display device provided in an embodiment of the present application;

[0040] FIG3 is a schematic diagram of a naked-eye 3D principle provided by an embodiment of the present application;

[0041] FIG4 is a schematic diagram showing a principle of a crosstalk phenomenon provided by an embodiment of the present application;

[0042] FIG5 is a schematic diagram of an eye box partition provided in an embodiment of the present application;

[0043] FIG6 is a schematic diagram of a first pixel distribution provided in an embodiment of the present application;

[0044] FIG7 is a schematic diagram of a second pixel distribution provided in an embodiment of the present application;

[0045] FIG8 is a schematic diagram of a third pixel distribution provided in an embodiment of the present application;

[0046] FIG9 is a schematic diagram of a fourth pixel distribution provided in an embodiment of the present application;

[0047] FIG10 is a schematic diagram of an implementation environment of a calibration method for a head-up display device provided in an embodiment of the present application;

[0048] FIG11 is a schematic diagram of an actual photographing effect at the eye box position provided by an embodiment of the present application;

[0049] FIG12 is a schematic diagram of an eye box position distribution provided by an embodiment of the present application;

[0050] FIG13 is a flow chart of a calibration method for a head-up display device provided in an embodiment of the present application;

[0051] FIG14 is a schematic diagram of virtual images to be processed corresponding to a plurality of eye box partitions to be processed provided by an embodiment of the present application;

[0052] FIG15 is a flow chart of a boundary recognition method provided in an embodiment of the present application;

[0053] FIG16 is a schematic diagram of a calibration method for a head-up display device provided in an embodiment of the present application;

[0054] FIG17 is a flow chart of another boundary recognition method provided in an embodiment of the present application;

[0055] FIG18 is a schematic diagram of another calibration method for a head-up display device provided in an embodiment of the present application;

[0056] FIG19 is a schematic diagram of a target visible area corresponding to an eye box partition to be processed on a display screen provided by an embodiment of the present application;

[0057] FIG20 is a structural block diagram of a calibration device for a head-up display device provided in an embodiment of the present application;

[0058] Figure 21 is a hardware structure block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0061] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0062] Please refer to Figure 1, which shows a schematic diagram of the application environment of a head-up display device provided by an embodiment of the present application. The head-up display device can be used in automobiles as a driving assistance instrument, and can also be applied to other vehicles, such as airplanes, high-speed trains, etc. The technical solution of the present application will be introduced below using the vehicle-mounted HUD as an example. Through the image display method of the embodiment of the present application, the HUD can clearly and accurately project vehicle speed, navigation information, warning information, etc. in the form of images and characters through optical components in front of the driver, forming a virtual image with a three-dimensional effect in front of the driver's line of sight, and has a good imaging effect.

[0063] Please refer to Figure 2, which is a schematic diagram of the principle of a head-up display device provided in an embodiment of the present application. The 3D HUD includes an image generation unit, a cylindrical lens grating (or a slit grating) attached to the image generation unit, and a reflector assembly. The image generation unit is composed of an LCD (Liquid Crystal Display) and a backlight. The reflector assembly may include a plurality of reflectors, as shown in Figure 2, including reflector 1 and reflector 2, wherein reflector 1 may be a plane mirror or a free-surface curved mirror, and reflector 2 may be a free-surface curved mirror. When implementing a 3D HUD, the outgoing light of the image generation unit is restricted by the cylindrical lens grating (or a slit grating), and the outgoing angles are different, thereby achieving spectroscopic separation; the light emitted by the cylindrical lens grating (or the slit grating) is refracted by the reflector assembly and the windshield of the vehicle in turn, and then enters the left eye and right eye of the user respectively, and the images seen by the left eye and the right eye of the user are different.

[0064] Please refer to Figure 3, which is a schematic diagram of the principle of a naked-eye 3D principle provided by an embodiment of the present application. By optically designing the HUD, the user sees image P1 with his left eye and image P2 with his right eye. Image P1 and image P2 can be combined in the user's brain to form a three-dimensional picture with a sense of depth; by changing the position between the two images and adjusting the binocular parallax, the virtual image distance subjectively perceived by the user can be changed (in fact, the virtual image distance remains unchanged). The closer the two images are, the closer the virtual image distance subjectively perceived by the user; conversely, the farther the two images are, the farther the virtual image distance subjectively perceived by the user.

[0065] Please refer to FIG4 , which illustrates a schematic diagram illustrating the crosstalk phenomenon according to an embodiment of the present application. Assuming the eyebox is divided into four eyebox partitions, e1, e2, e3, and e4, and the distance between a person's two eyes is the distance between the eyebox partitions, and by default, when the user's left eye is detected within e1 and the right eye within e3, the left eye sees image P1 and the right eye sees image P2. Since the eyebox partitions are not strictly demarcated as shown in the figure above, it is possible that if the user moves their head slightly, for example, to the left, the user's left eye may move beyond eyebox partition e1. Since the eyebox is not strictly divided into partitions, the user may still see images even beyond e1. Furthermore, based on the optical design of the HUD, the eyebox partitions can be assumed to repeat periodically. Therefore, the left eye may actually see image P2 intended for the right eye. If the right eye moves from e3 to e2, the right eye sees image P1 intended for the left eye, thus causing crosstalk.

[0066] Please refer to Figure 5, which is a schematic diagram of an eye box partition provided in an embodiment of the present application. Taking a lenticular lens as an example, assuming that a lenticular lens covers 8 sub-pixels, the eye box is divided into 8 eye box partitions. When the user views a virtual image, two images are given to the user's left and right eyes: the left eye is a pure white image L, and the right eye is a pure black image R, and the two images are two images with parallax. As shown in Figure 5, of the eight sub-pixels covered by a lenticular lens, four correspond to the white image seen by the left eye, and the other four correspond to the black image seen by the right eye. Furthermore, only in a specific eyebox partition can the left eye see a complete white image, and vice versa for the black image seen by the right eye. That is, when the human eye views a virtual image, only one eyebox partition in the eyebox area meets the left eye's requirement for consistent brightness. If the eye moves right or left, centered on the eyebox partition that meets the left eye's brightness requirement, only three, two, or even one white sub-pixel light can be seen in the remaining eyebox partitions. This significantly reduces brightness, preventing the human eye from integrating the 3D effect.

[0067] Please refer to Figure 6, which is a schematic diagram of the first pixel distribution provided by an embodiment of the present application. As shown in Figure 6-1, it is the sub-pixel distribution of the image itself. The four sub-pixels A1a-A1d correspond to the white image (L) viewed by the left eye, and the other four sub-pixels correspond to the black image (R) viewed by the right eye. The light emitted from the four sub-pixels only corresponds to one eye box partition, which can meet the requirement of consistent brightness required by the left eye; as shown in Figure 6-2, the situation when the cylindrical lens grating (or slit grating) and the display screen are relatively tilted is explained. The tilt angle of the grating is arctan (1 / 3), so that the tilted grating matches the aspect ratio of its sub-pixels. The diagonal line of a certain sub-pixel is used for comparison. Tilt, determine the distribution position of pixels when tilted relative to the grating, and determine the image displayed by the image generation unit of the HUD based on the distribution position. However, because the entire grating area is tilted to the right, if the integrity of the image on the display screen viewed by the user's eyes is required, it can be seen that the sub-pixels in row A2 of Figure 6-2 are shifted to the right by one sub-pixel relative to row A1. That is, when the spatial position relative to the display screen changes by one line, the area viewed by the left eye moves by one sub-pixel on the display screen. In other words, the tilted pixels need to be shifted to the right by one sub-pixel for each row up, and by one sub-pixel for each row down, so that the left eye always sees the left image and the right eye always sees the right image when the human eye views.

[0068] Please refer to Figures 7-9, which illustrate schematic diagrams of the second, third, and fourth pixel distributions provided in embodiments of the present application. Figure 7 illustrates the sub-pixel distribution and position of the image itself. To ensure that the left eye consistently sees the complete left image and the right eye consistently sees the complete right image, this effect is achieved by varying the sub-pixel distribution on the display screen. Specifically, due to the different positions of the eight sub-pixels, the light rays entering the eyebox from the sub-pixels correspond to different eyebox partitions, resulting in different image content information perceived by the human eye. The sub-pixel distribution corresponds one-to-one with the image displayed on the HUD, resulting in eight images with different sub-pixel distributions. These eight images result in the user's left and right eyes seeing a pure white or pure black image at different positions within the eyebox, with each image corresponding to a different sub-pixel distribution. If these eight sub-pixels correspond to positions S1-S8, the sub-pixel distribution at each of these eight positions (i.e., the positions on the display screen where the light emitted by the sub-pixels is located) will be periodically arranged around the starting point L1 at the top of the image. With starting point L1 at the top of the image as the reference center, the sub-pixel distribution will differ when starting point L1 is at any of positions S1-S8 on the LCD screen. For example, when starting point L1 is at position S1, the sub-pixel distribution is L1, L2, L3, L4, R1, R2, R3, R4, which is the first sub-pixel distribution. When starting point L1 is at position S2, the sub-pixel distribution is R4, L1, L2, L3, L4, R1, R2, R3, which is the second sub-pixel distribution. The sub-pixel distributions for the other six positions (S3-S8) are similar. That is, the starting point position of a certain sub-pixel is changed from the pixel perspective. With respect to the lenticular lens (or slit grating) attached to the display screen, the corresponding light distribution will also shift after the relative position of the sub-pixel changes. This results in different white / black areas visible to the left and right eyes at different positions in the eye box. From an image perspective, this is equivalent to shifting the pixels on these eight different image frames one sub-pixel to the left for each row up and one sub-pixel to the right for each row down. Because the left and right eyes can only see a pure white / pure black image effect at a certain position in the eye box, if the human eye moves left or right, the image needs to be switched, which in turn changes the distribution of the remaining sub-pixels to ensure that the left eye sees a white image and the right eye sees a black image. Specifically, refer to Figure 9, which shows a schematic diagram of the specific image arrangement for the eight states, where L is the white image area viewed by the left eye and R is the black image area viewed by the right eye.

[0069] Please refer to Figures 10-12, which are schematic diagrams of the implementation environment of a calibration method for a head-up display device provided by an embodiment of the present application, a schematic diagram of the effect of actual shooting at the eye box position, and a schematic diagram of the camera position distribution at the eye box position. If a camera is placed at the center of the eye box partition, the actual performance of the distribution state image of 8 sub-pixels is captured. Taking the sub-pixel distribution of state 4 as an example, a white image is given to the left eye. In theory, the center position of this eye box partition is the best observation position for the user's human eye. The user's left eye can see a completely white image with uniform image brightness. Taking the pupil distance interval of 60-70mm as an example, the corresponding right eye sees a completely black image with uniform image brightness in the corresponding complementary area (state 8). The actual image captured represents the virtual image seen by the human eye in the eye box partition through the HUD imaging structure. However, as shown in Figure 11, when the human eye is positioned at the center of the eyebox partition, it cannot fully perceive pure black or pure white. Instead, the lower left and upper right sides are tinged with white, black, or even a gradient from black to gray and then white. This effect is caused by HUD distortion or manufacturing tolerances, resulting in an incomplete black and white image display. This means that the left eye cannot fully perceive the pure white image intended for the left eye, but rather a tinge of the pure black image intended for the right eye. This phenomenon is known as crosstalk. HUD eyebox partition testing typically involves nine camera positions. This example uses the camera position at the center of the eyebox partition as an example, and the image captured by this camera position only represents the virtual image that the human eye would see within the eyebox partition if captured at this position. This crosstalk phenomenon occurs not only at the center position 5 of the HUD eyebox partition, but also at the remaining positions in the eyebox partition.

[0070] In view of this, if the eyes are to see black / white in any eye box partition, it is necessary to capture the actual performance images of the sub-pixel distribution status images of the remaining eye box partitions of the HUD, so that the state changes of the black and white bands in the remaining eye box partitions are linked with the spatial coordinates, and pieced together into a complete black image / white image. By linking with the DMS (Driver Monitor System), the human eye can see the ideal image effect at any position in the eye box, that is, the user's left eye sees the image intended for the left eye, and the user's right eye sees the image intended for the right eye.

[0071] Please refer to Figure 13, which is a flow chart of a calibration method for a head-up display device provided in an embodiment of the present application. It should be noted that this specification provides method operation steps such as the embodiments or flow charts, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many steps and does not represent the only execution order. When the actual system or product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, a parallel processor or a multi-threaded processing environment). Specifically, as shown in Figure 13, the head-up display device includes a display screen and an imaging structure, and the method may include:

[0072] S1301, obtaining a virtual image to be processed corresponding to each of a plurality of continuously distributed eye box partitions to be processed.

[0073] The multiple continuously distributed eye box partitions to be processed form the eye box of the head-up display device. Specifically, the eye box of the head-up display device is divided into multiple eye box partitions to be processed, which serve as areas where the user's eyes can move.

[0074] The virtual image to be processed is formed by light emitted from the real visible area of ​​the display screen corresponding to the eyebox partition to be processed, which is projected onto the eyebox partition to be processed through an imaging structure. The imaging structure includes a lenticular lens (or slit grating) and a reflector assembly attached to the display screen. Specifically, the imaging structure is used to split the light emitted from the display screen and project it onto the eyebox.

[0075] The display screen is used to display images in different regions. Specifically, because a lenticular lens grating (or slit grating) is attached to the display screen, each untreated eye box section can only see a portion of the display screen due to the grating's light splitting effect. Each untreated eye box section can see a different area of ​​the display screen. In specific implementations, a liquid crystal display screen is typically used.

[0076] Specifically, the image displayed on the display screen is split by a cylindrical mirror grating (or a slit grating) to form a left-eye virtual image and a right-eye virtual image with parallax. The left-eye virtual image and the right-eye virtual image are then projected onto different eye box partitions to be processed through a reflector assembly and a transparent medium (such as a vehicle's windshield), forming virtual images to be processed corresponding to the eye box partitions to be processed.

[0077] The real visible area is the area on the display screen that can be seen by the eye box partition to be processed.

[0078] Specifically, when the head-up display device is set up, each eye box partition to be processed in the eye box corresponds to a preset visible area in the display screen. However, due to the distortion of the head-up display device or processing tolerances, the actual visible area corresponding to each eye box partition to be processed in the display screen has a certain gap from the corresponding preset visible area. As a result, a certain eye box partition to be processed can see more than just the image displayed in the preset visible area, so that when the head-up display device is applied to naked-eye 3D, crosstalk phenomenon as shown in Figure 11 will occur.

[0079] In a specific implementation, a pure black image is displayed in the preset visible area corresponding to the eye box partition to be processed in the display screen, and a pure white image is displayed in other areas of the display screen. A camera is placed at the eye box partition to be processed to obtain the virtual image to be processed corresponding to the eye box partition to be processed, so as to further identify the coverage range of the preset visible area corresponding to the eye box partition to be processed in the display screen in the virtual image to be processed.

[0080] S1303: For each virtual image to be processed corresponding to the eye box partition to be processed, identifying the coverage of the preset visible area corresponding to the eye box partition to be processed on the display screen in the virtual image to be processed.

[0081] The preset visible area is the visible area corresponding to the eye box partition to be processed set when the head-up display device leaves the factory.

[0082] The coverage range is the corresponding preset visible area seen at the eye box partition to be processed, and the coverage range in the corresponding virtual image to be processed reflects the difference between it and the corresponding real visible area.

[0083] Specifically, due to distortion or manufacturing tolerances in the head-up display device, the actual visible area corresponding to each unprocessed eyebox partition on the display screen differs from the corresponding preset visible area. The virtual image to be processed is formed by light emitted from the actual visible area corresponding to the unprocessed eyebox partition on the display screen, projected onto the unprocessed eyebox partition through the imaging structure. Therefore, the preset visible area cannot completely cover the virtual image to be processed. For example, FIG14 is a schematic diagram of the virtual images to be processed corresponding to multiple unprocessed eyebox partitions provided in an embodiment of the present application, corresponding to states 1, 3, 4, and 5 in FIG9 , respectively. The slashed areas represent the preset visible areas corresponding to the corresponding unprocessed eyebox partitions. It can be seen that the preset visible areas cannot completely cover the virtual image to be processed. The coverage range is identified so that further adjustments can be made to minimize the difference between the virtual image to be processed and the actual visible area.

[0084] In an exemplary embodiment, before step S1303, the following steps may be included:

[0085] For the eye box partition to be processed, which corresponds to a preset visible area on the display screen, obtaining the original image displayed in the preset visible area;

[0086] Based on the brightness information of each pixel in the original image, the brightness information of each pixel in the to-be-processed virtual image corresponding to the to-be-processed eye box partition is normalized.

[0087] The eye box partition to be processed is formed by dividing the eye box of the head-up display device, and serves as one of the areas where the user's eyes can move.

[0088] The preset visible area is the visible area corresponding to the eye box partition to be processed set when the head-up display device leaves the factory.

[0089] The original image is the image displayed on the display screen corresponding to the preset visible area of ​​the eye box partition to be processed. In specific implementations, a pure white image is usually displayed on the display screen corresponding to the preset visible area as the original image. Specifically, because the identification of the boundary of the preset visible area in the virtual image to be processed relies on the judgment of brightness, and the brightness of the backlight will affect this judgment basis, the virtual images to be processed are all shot under the same lighting and will have the brightness influence of the backlight on the basis of the light and dark distribution caused by the imaging structure. Placing a pure white image on the display screen and shooting another one from the same camera position as the original image is equivalent to recording the distribution of the influence of the backlight brightness at that camera position, which is used to eliminate the backlight for the virtual image to be processed.

[0090] The normalization process is specifically to divide the brightness of each pixel in the virtual image to be processed by the brightness of the pixel at the corresponding position in the original image.

[0091] Specifically, since the backlight behind the display screen may have poor uniformity, it is impossible to identify the boundary of the preset visible area in the virtual image to be processed based on the brightness information of the pixel points. By normalizing the brightness of the pixel points in the virtual image to be processed, the influence of the uneven backlight brightness is eliminated, and the light and dark distribution caused by the imaging structure is obtained more realistically.

[0092] It can be seen from the above technical solutions of the embodiment of the present application that the embodiment of the present application normalizes the brightness information of each pixel in the virtual image to be processed corresponding to the eye box partition to be processed through the brightness information of each pixel in the original image, thereby eliminating the influence of uneven backlight brightness on the light and dark distribution of the virtual image to be processed, so as to facilitate the subsequent identification of the boundaries of the preset visible area in the virtual image to be processed based on the brightness information of the pixel points.

[0093] In an exemplary embodiment, as shown in FIG15 , which is a flowchart of a boundary recognition method provided in an embodiment of the present application, step S1303 may include:

[0094] S1501 , for each eye box partition to be processed, determining the virtual images to be processed corresponding to the adjacent eye box partitions of the eye box partition to be processed, and obtaining at least one adjacent virtual image.

[0095] The eye box partition to be processed is formed by dividing the eye box of the head-up display device, and serves as one of the areas where the user's eyes can move.

[0096] The adjacent eye box partition is an eye box partition to be processed that is adjacent to the eye box partition to be processed among the multiple eye box partitions to be processed.

[0097] The virtual image to be processed is formed by light emitted from the real visible area of ​​the display screen corresponding to the eyebox partition to be processed, which is projected onto the eyebox partition to be processed through an imaging structure. The imaging structure includes a lenticular lens (or slit grating) and a reflector assembly attached to the display screen. Specifically, the imaging structure is used to split the light emitted from the display screen and project it onto the eyebox.

[0098] The adjacent virtual images are used to assist in identifying the coverage of the corresponding preset visible area in the to-be-processed virtual image corresponding to the to-be-processed eye box partition.

[0099] In a specific implementation, the number of adjacent eye box partitions to the eye box partition to be processed may be one or more, and accordingly, the number of adjacent virtual images may be one or more.

[0100] S1503 , for the virtual image to be processed corresponding to the eye box partition to be processed, superimpose the virtual image to be processed with at least one adjacent virtual image to obtain at least one image to be recognized.

[0101] The brightness of each pixel in the image to be identified is the larger value between the brightness of the corresponding pixel in the virtual image to be processed and the brightness of the corresponding pixel in the adjacent virtual image.

[0102] For example, as shown in Figure 16, which is a schematic diagram of a calibration method for a head-up display device provided in an embodiment of the present application, for the virtual image to be processed shown in state 4, it is superimposed with the virtual image to be processed shown in state 3 to determine one side boundary of the corresponding preset visible area, and it is superimposed with the virtual image to be processed shown in state 5 to determine the other side boundary of the corresponding preset visible area.

[0103] S1505, for each pixel point in at least one image to be identified, when the brightness information of the pixel point matches the first brightness information, determine that the pixel point is a boundary pixel point corresponding to at least one side boundary of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed.

[0104] The first brightness information indicates the brightness of a center line of a preset visible area in the image to be recognized.

[0105] The plurality of boundary pixel points corresponding to at least one side boundary of the virtual image to be processed indicate the coverage range.

[0106] The preset visible area is the visible area corresponding to the eye box partition to be processed set when the head-up display device leaves the factory.

[0107] The boundary pixel points are pixel points on the boundary of the coverage range of the preset visual area corresponding to the eye box partition to be processed in the corresponding virtual image to be processed.

[0108] The midline brightness is the brightness of the pixels on the midline of a preset visible area in the image to be recognized. Specifically, the preset visible area includes the preset visible area in the virtual image to be processed corresponding to the eye box partition to be processed and the preset visible area in the adjacent virtual image.

[0109] The coverage range is the corresponding preset visible area seen at the eye box partition to be processed, and the coverage range in the corresponding virtual image to be processed reflects the difference between it and the corresponding real visible area.

[0110] For example, as shown in FIG16 , the virtual image to be processed shown in state 3 is superimposed with the virtual image to be processed shown in state 4, and the middle line of the slashed area in the superimposed image is taken as one side boundary line of the slashed area in the virtual image to be processed shown in state 4. The virtual image to be processed shown in state 5 is superimposed with the virtual image to be processed shown in state 4, and the middle line of the slashed area in the superimposed image is taken as the other side boundary line of the slashed area in the virtual image to be processed shown in state 4, thereby obtaining the coverage of the slashed area in the virtual image to be processed shown in state 4. In a specific implementation, the optimal brightness of the middle line is approximately 85%.

[0111] It can be seen from the above technical solutions of the embodiment of the present application that the embodiment of the present application determines the boundary of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed based on the brightness of the middle line of the preset visible area in the image to be identified, so as to further adjust the preset visible area corresponding to the eye box partition to be processed in the display screen.

[0112] In an exemplary embodiment, as shown in FIG17 , which is a flowchart of another boundary recognition method provided in an embodiment of the present application, before step S1501, the following steps may be included:

[0113] S1701 , for each to-be-processed virtual image corresponding to each to-be-processed eye box partition, and for each pixel in the to-be-processed virtual image, determining whether brightness information of the pixel matches second brightness information.

[0114] Specifically, if the result of the judgment for each pixel in the virtual image to be processed is no, that is, there is no pixel in the virtual image to be processed whose brightness information matches the second brightness information, step S1501 can be executed; conversely, if the result of the judgment is yes, that is, the brightness information of the pixel matches the second brightness information, step S1703 can be executed.

[0115] The eye box partition to be processed is formed by dividing the eye box of the head-up display device, and serves as one of the areas where the user's eyes can move.

[0116] The virtual image to be processed is formed by light emitted from the real visible area of ​​the display screen corresponding to the eyebox partition to be processed, which is projected onto the eyebox partition to be processed through an imaging structure. The imaging structure includes a lenticular lens (or slit grating) and a reflector assembly attached to the display screen. Specifically, the imaging structure is used to split the light emitted from the display screen and project it onto the eyebox.

[0117] The second brightness information indicates the brightness of the boundary line of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed. In a specific implementation, the second brightness information can be a brightness range or a fixed brightness value. For example, if the preset visible area on the display screen displays a pure white image, the second brightness information can be set to 80% to 100%, or to any fixed value between 80% and 100%; if the preset visible area on the display screen displays a pure black image, the second brightness information can be set to 0% to 20%, or to any fixed value between 0% and 20%.

[0118] The preset visible area is the visible area corresponding to the eye box partition to be processed set when the head-up display device leaves the factory.

[0119] The boundary line brightness is the brightness of the pixel points on the boundary line of the corresponding preset visible area in the virtual image to be processed corresponding to the eye box partition to be processed.

[0120] Specifically, if there are no pixels in the virtual image to be processed whose brightness information matches the second brightness information, it is considered that the preset visible area in the virtual image to be processed overlaps with the preset visible area in the corresponding adjacent virtual image. For example, as shown in FIG16 , taking the virtual images to be processed shown in states 3 and 4 as an example, if the preset visible areas corresponding to the different eye box partitions to be processed on the display screen display a pure white image, the second brightness information is 80% to 100%, and the brightness information of the pixels in the virtual image to be processed is less than 80%, then the preset visible areas in the virtual image to be processed shown in state 3 and the virtual image to be processed shown in state 4 overlap. Since the size of the virtual image to be processed is fixed, that is, the size of the real visible area on the display screen is fixed, if the real boundary lines are followed, the subsequent mapping area adjustment method is not applicable. Therefore, instead of taking the boundary of the preset visible area in the virtual image to be processed as its boundary, the middle line of the preset visible area in the image formed by superimposing the virtual image to be processed and the adjacent virtual image is taken as one side boundary of the preset visible area in the virtual image to be processed, which is also one side boundary of the preset visible area in the adjacent virtual image.

[0121] Specifically, as shown in Figure 18, which is a schematic diagram of another calibration method for a head-up display device provided in an embodiment of the present application, when the brightness information of a pixel point matches the second brightness information, the pixel point is determined as a boundary pixel point of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed, and multiple boundary pixel points form the boundary of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed.

[0122] S1703: Determine that the pixel point is a boundary pixel point of a preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed.

[0123] The boundary pixel points are pixel points on the boundary line of the corresponding preset visible area in the virtual image to be processed corresponding to the eye box partition to be processed. Specifically, multiple boundary pixel points form the boundary of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed.

[0124] The plurality of boundary pixel points in the virtual image to be processed indicate the coverage range.

[0125] It can be seen from the above technical solutions of the embodiment of the present application that the embodiment of the present application determines the boundary of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed according to the brightness information of the pixel points in the virtual image to be processed corresponding to each eye box partition to be processed, so as to further adjust the preset visible area corresponding to the eye box partition to be processed in the display screen.

[0126] S1305 , converting the coverage range from the coordinate system corresponding to the virtual image to be processed to the coordinate system corresponding to the display screen, and obtaining a mapping area of ​​the preset visible area corresponding to the eye box partition to be processed in the display screen.

[0127] The mapping area is a mapping of the coverage of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed on the display screen. Specifically, the mapping area is used to adjust the gap between the preset visible area and the real visible area on the display screen.

[0128] S1307, based on the mapping areas corresponding to the adjacent eye box partitions of the eye box partition to be processed, adjusting the mapping area corresponding to the eye box partition to be processed in the display screen to obtain the target visible area corresponding to the eye box partition to be processed in the display screen.

[0129] The adjacent eye box partition is an eye box partition to be processed that is adjacent to the eye box partition to be processed among the multiple eye box partitions to be processed.

[0130] The target visible area corresponding to the eye box partition to be processed in the display screen is used to display an image that one wants to see in the eye box partition to be processed.

[0131] Specifically, the spatial position relationship between the eye box partition to be processed and the adjacent eye box partition is mapped to the spatial position relationship between the mapping area corresponding to the eye box partition to be processed in the display screen and the mapping area corresponding to the adjacent eye box partition in the display screen. Based on the spatial position relationship, the mapping area corresponding to the adjacent eye box partition in the display screen is used to make up for the gap between the mapping area corresponding to the eye box partition to be processed in the display screen and the corresponding real visible area, thereby minimizing the gap, completing the calibration of the preset visible area corresponding to the eye box partition to be processed, and the obtained target visible area is almost identical to the real visible area, so that the image displayed in the target visible area corresponding to the eye box partition to be processed can completely cover the virtual image seen in the eye box partition to be processed.

[0132] For example, as shown in FIG14 , for the virtual image to be processed shown in state 4, and the virtual images to be processed shown in states 2, 3, and 5 adjacent to state 4, the slash areas therein are mapped to the display screen to obtain corresponding mapping areas, and the mapping areas of states 2, 3, and 5 are used to adjust the mapping area corresponding to state 4.

[0133] It can be seen from the above technical solutions of the embodiment of the present application that the embodiment of the present application optimizes the display effect of the head-up display device by adjusting the preset visible area corresponding to each eye box partition to be processed in the display screen using the preset visible area corresponding to the adjacent eye box partition to be processed in the display screen, so that the user's eyes can see the image intended for the left eye and the right eye can see the image intended for the right eye at any position of the eye box, thereby improving the user's viewing experience.

[0134] In an exemplary embodiment, the above step S1307 may include the following steps:

[0135] The mapping area corresponding to the adjacent eye box partition of the eye box partition to be processed is spliced ​​with the mapping area corresponding to the eye box partition to be processed to obtain the target visible area corresponding to the eye box partition to be processed on the display screen.

[0136] The eye box partition to be processed is formed by dividing the eye box of the head-up display device, and serves as one of the areas where the user's eyes can move.

[0137] The adjacent eye box partition is an eye box partition to be processed that is adjacent to the eye box partition to be processed among the multiple eye box partitions to be processed.

[0138] The mapping area is a mapping of the coverage of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed on the display screen. Specifically, the mapping area is used to adjust the gap between the preset visible area and the real visible area on the display screen.

[0139] The target visible area corresponding to the eye box partition to be processed in the display screen is used to display an image that one wants to see in the eye box partition to be processed.

[0140] Specifically, the spatial positional relationship between the eye box partition to be processed and the adjacent eye box partition is mapped to the spatial positional relationship between the mapping area corresponding to the eye box partition to be processed on the display screen and the mapping area corresponding to the adjacent eye box partition on the display screen. Based on this spatial positional relationship, the mapping area corresponding to the adjacent eye box partition on the display screen is spliced ​​with the mapping area corresponding to the eye box partition to be processed on the display screen. For example, as shown in FIG19 , a schematic diagram of a target visible area corresponding to the eye box partition to be processed on the display screen provided in an embodiment of the present application is provided. In order to make up for the gap between the mapping area corresponding to the eye box partition to be processed on the display screen and the corresponding real visible area, thereby minimizing the gap, the calibration of the preset visible area corresponding to the eye box partition to be processed is completed, and the obtained target visible area and the real visible area can almost completely overlap, so that the image displayed in the target visible area corresponding to the eye box partition to be processed can completely cover the virtual image seen in the eye box partition to be processed.

[0141] It can be seen from the above technical solutions of the embodiment of the present application that the embodiment of the present application splices the mapping areas corresponding to the adjacent eye box partitions in the display screen with the mapping areas corresponding to the eye box partition to be processed in the display screen, and uses the mapping areas corresponding to the adjacent eye box partitions in the display screen to make up for the gap between the mapping areas corresponding to the eye box partition to be processed in the display screen and the corresponding real visible areas, thereby minimizing the gap, completing the calibration of the preset visible areas corresponding to the eye box partition to be processed, optimizing the display effect of the head-up display device, and improving the user viewing experience.

[0142] The following describes the process of displaying an image based on a calibrated head-up display device, which may include:

[0143] Acquire an image to be displayed, a left-eye eyebox partition, and a right-eye eyebox partition; the left-eye eyebox partition is an eyebox partition to be processed corresponding to the left eye of the target object among the multiple eyebox partitions to be processed; the right-eye eyebox partition is an eyebox partition to be processed corresponding to the right eye of the target object among the multiple eyebox partitions to be processed;

[0144] The image to be displayed is displayed based on the target visible area corresponding to the left eye box partition; and the image to be displayed is displayed based on the target visible area corresponding to the right eye box partition.

[0145] The image to be displayed is an image that needs to be shown to the target object.

[0146] The plurality of to-be-processed eye box partitions form an eye box of the head-up display device. Specifically, the eye box of the head-up display device is divided into a plurality of to-be-processed eye box partitions, which serve as areas where the user's eyes can move.

[0147] The target objects are users who use head-up display devices.

[0148] The target visible area is the area of ​​the display screen visible from the corresponding untreated eyebox partition. After calibration of the head-up display, the target visible area corresponding to each untreated eyebox partition nearly overlaps with the corresponding real visible area, so that only the image displayed in the target visible area corresponding to the untreated eyebox partition can be seen from the untreated eyebox partition.

[0149] Specifically, the image to be displayed is displayed in the target visual area corresponding to the left eye box partition, and the image to be displayed is displayed in the target visual area corresponding to the right eye box partition, so that the left eye of the target object can only see the image to be displayed in the target visual area corresponding to the left eye box partition in the left eye box partition, and the right eye of the target object can only see the image to be displayed in the target visual area corresponding to the right eye box partition, avoiding crosstalk and forming binocular parallax, so that the picture seen by the target object has a sense of depth and space, and a three-dimensional picture with a sense of depth is synthesized in the target object's brain to achieve the effect of naked-eye 3D.

[0150] It can be seen from the above technical solutions of the embodiment of the present application that the embodiment of the present application displays the image to be displayed through the calibrated head-up display device, so that the left eye of the target object can only see the image displayed in the target visual area corresponding to the left eye box partition in the left eye eye box partition, and the right eye of the target object can only see the image displayed in the target visual area corresponding to the right eye box partition, avoiding crosstalk, so that the user can experience the naked-eye 3D effect without crosstalk at any position of the eye box, thereby improving the user's viewing experience.

[0151] Corresponding to the calibration methods of the head-up display device provided in the above-mentioned embodiments, an embodiment of the present application also provides a calibration device for a head-up display device. Since the calibration device for the head-up display device provided in the embodiment of the present application corresponds to the calibration methods for the head-up display device provided in the above-mentioned embodiments, the implementation methods of the aforementioned calibration methods for the head-up display device are also applicable to the calibration device for the head-up display device provided in this embodiment, and will not be described in detail in this embodiment.

[0152] Please refer to Figure 20, which shows a schematic diagram of the structure of a calibration device for a head-up display device provided in an embodiment of the present application. This device has the function of implementing the calibration method for the head-up display device in the above method embodiment. This function can be implemented by hardware or by hardware executing corresponding software. As shown in Figure 20, the head-up display device includes a display screen and an imaging structure. The device may include:

[0153] The virtual image acquisition module 210 is used to acquire a virtual image to be processed corresponding to each of the plurality of continuously distributed eye box partitions to be processed; the virtual image to be processed is formed by light emitted from the real visible area corresponding to the eye box partition to be processed on the display screen through the imaging structure and projected onto the eye box partition to be processed;

[0154] The region identification module 220 is configured to identify, for each virtual image to be processed corresponding to the eye box partition to be processed, a coverage range of the preset visible area corresponding to the eye box partition to be processed on the display screen in the virtual image to be processed;

[0155] The area mapping module 2030 is used to convert the coverage area from the coordinate system corresponding to the virtual image to be processed to the coordinate system corresponding to the display screen, and obtain the mapping area of ​​the preset visible area corresponding to the eye box partition to be processed on the display screen;

[0156] The area adjustment module 2040 is used to adjust the mapping area corresponding to the eye box partition to be processed in the display screen based on the mapping area corresponding to the adjacent eye box partition of the eye box partition to be processed, so as to obtain the target visible area corresponding to the eye box partition to be processed in the display screen; the adjacent eye box partition is the eye box partition to be processed adjacent to the eye box partition to be processed among the multiple eye box partitions to be processed.

[0157] In an exemplary embodiment, the region identification module includes:

[0158] A first adjacent determination module is configured to determine, for each eye box partition to be processed, a virtual image to be processed corresponding to an adjacent eye box partition of the eye box partition to be processed, and obtain at least one adjacent virtual image;

[0159] a virtual image superposition module for superimposing the virtual image to be processed corresponding to the eye box partition to be processed with at least one adjacent virtual image to obtain at least one image to be recognized;

[0160] A first boundary determination module is configured to determine, for each pixel point in at least one image to be identified, that the pixel point is a boundary pixel point corresponding to at least one side boundary of a preset visible area corresponding to an eye box partition to be processed in a virtual image to be processed, when the brightness information of the pixel point matches the first brightness information; the first brightness information indicates the brightness of a center line of the preset visible area in the image to be identified; and the multiple boundary pixel points corresponding to at least one side boundary in the virtual image to be processed indicate a coverage range.

[0161] In an exemplary embodiment, the apparatus further includes a boundary brightness module for determining whether the boundary can be determined based on the boundary line brightness before the boundary of the preset visible area corresponding to the eye box partition to be processed is determined based on the midline brightness in the virtual image to be processed. The boundary brightness module includes:

[0162] a brightness matching module for determining, for each pixel in the virtual image to be processed corresponding to each eye box partition to be processed, whether the brightness information of the pixel matches second brightness information; the second brightness information indicates the brightness of a boundary line of a preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed;

[0163] The second adjacent determination module is configured to, if there is no pixel whose brightness information matches the second brightness information in the virtual image to be processed, perform the step of determining, for each eye box partition to be processed, the virtual image to be processed corresponding to the adjacent eye box partition to be processed, to obtain at least one adjacent virtual image.

[0164] In an exemplary embodiment, the region identification module includes:

[0165] The second boundary determination module is used to determine that the pixel point is a boundary pixel point of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed when the brightness information of the pixel point matches the second brightness information; multiple boundary pixel points in the virtual image to be processed indicate the coverage range.

[0166] In an exemplary embodiment, the region adjustment module includes:

[0167] The area splicing module is used to splice the mapping area corresponding to the adjacent eye box partition of the eye box partition to be processed with the mapping area corresponding to the eye box partition to be processed to obtain the target visible area corresponding to the eye box partition to be processed on the display screen.

[0168] In an exemplary embodiment, the apparatus further includes a brightness processing module for eliminating backlighting of the virtual image to be processed before identifying the coverage of the preset visible area corresponding to the eye box partition to be processed in the display screen in the virtual image to be processed. The brightness processing module includes:

[0169] An image acquisition module, configured to obtain an original image displayed in a preset visible area corresponding to the eye box partition to be processed on the display screen;

[0170] The brightness normalization module is used to normalize the brightness information of each pixel in the to-be-processed virtual image corresponding to the to-be-processed eye box partition based on the brightness information of each pixel in the original image.

[0171] In an exemplary embodiment, the apparatus further includes a head-up display module for displaying an image based on the calibrated head-up display device, the head-up display module including:

[0172] An information acquisition module is used to acquire an image to be displayed, a left eye box partition, and a right eye box partition; the left eye box partition is an eye box partition to be processed corresponding to the left eye of the target object among the multiple eye box partitions to be processed; the right eye box partition is an eye box partition to be processed corresponding to the right eye of the target object among the multiple eye box partitions to be processed;

[0173] The image display module is used to display the image to be displayed based on the target visual area corresponding to the left eye box partition; and to display the image to be displayed based on the target visual area corresponding to the right eye box partition.

[0174] It should be noted that the apparatus provided in the above embodiments, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0175] An embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement any one of the head-up display device calibration methods provided in the above method embodiments.

[0176] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for functions, etc.; the data storage area can store data created based on the use of the device, etc. In addition, the memory can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0177] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal, a server, or a similar computing device, that is, the above-mentioned electronic device may include a computer terminal, a server, or a similar computing device. Figure 21 is a hardware structure block diagram of a computer device that runs a calibration method for a head-up display device provided in an embodiment of the present invention. As shown in Figure 21, the internal structure of the computer device may include but is not limited to: a processor, a network interface, and a memory. Among them, the processor, network interface, and memory in the computer device can be connected via a bus or other means. In Figure 21 shown in the embodiments of this specification, the connection via a bus is used as an example.

[0178] The processor (also known as the CPU (Central Processing Unit)) is the computing and control core of a computer device. Network interfaces may optionally include standard wired interfaces and wireless interfaces (such as Wi-Fi and mobile communication interfaces). Memory is a storage device within a computer device used to store programs and data. It is understood that the memory herein may be a high-speed RAM storage device or a non-volatile memory device, such as at least one disk storage device; optionally, it may be at least one storage device located remotely from the processor. The memory provides storage space that stores the operating system of the electronic device, including but not limited to Windows (an operating system), Linux (an operating system), Android (a mobile operating system), and iOS (a mobile operating system), etc., though this is not limited in the present invention. Furthermore, the memory space also stores one or more instructions suitable for being loaded and executed by the processor. These instructions may be one or more computer programs (including program code). In the embodiments of this specification, the processor loads and executes one or more instructions stored in the memory to implement the calibration method for a head-up display device provided in the above-described method embodiments.

[0179] An embodiment of the present application also provides a computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a calibration method for a head-up display device. The at least one instruction or the at least one program is loaded and executed by the processor to implement any one of the calibration methods for a head-up display device provided in the above method embodiments.

[0180] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0181] It should be noted that the order of the embodiments of the present application described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. The above description is of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0182] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0183] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program instructing the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk, or an optical disk, etc.

Claims

1. A method for calibrating a head-up display device, the head-up display device comprising a display screen and an imaging structure; the method comprising: Acquire a virtual image to be processed corresponding to each of a plurality of continuously distributed eye box partitions to be processed; The virtual image to be processed is formed by the outgoing light from the real visible area corresponding to the eye box partition to be processed in the display screen being projected onto the eye box partition to be processed through the imaging structure; For each virtual image to be processed corresponding to the eye box partition to be processed, identifying the coverage of the preset visible area corresponding to the eye box partition to be processed on the display screen in the virtual image to be processed; Converting the coverage range from the coordinate system corresponding to the to-be-processed virtual image to the coordinate system corresponding to the display screen, and obtaining a mapping area of ​​the preset visible area corresponding to the to-be-processed eye box partition on the display screen; Based on the mapping areas corresponding to the adjacent eye box partitions of the eye box partition to be processed, adjusting the mapping area corresponding to the eye box partition to be processed in the display screen to obtain the target visible area corresponding to the eye box partition to be processed in the display screen; The adjacent eye box partition is an eye box partition to be processed that is adjacent to the eye box partition to be processed among the plurality of eye box partitions to be processed.

2. The calibration method of the head-up display device according to claim 1, wherein: For each virtual image to be processed corresponding to the eye box partition to be processed, identifying a coverage range of a preset visible area corresponding to the eye box partition to be processed on the display screen in the virtual image to be processed includes: For each of the eye box partitions to be processed, determining the virtual images to be processed corresponding to the adjacent eye box partitions of the eye box partition to be processed, and obtaining at least one adjacent virtual image; For the virtual image to be processed corresponding to the eye box partition to be processed, superimposing the virtual image to be processed with the at least one adjacent virtual image to obtain at least one image to be recognized; For each pixel point in the at least one image to be identified, when the brightness information of the pixel point matches the first brightness information, the pixel point is determined to be a boundary pixel point corresponding to at least one side boundary of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed; the first brightness information indicates the center line brightness of the preset visible area in the image to be identified; and multiple boundary pixel points corresponding to at least one side boundary in the virtual image to be processed indicate the coverage range.

3. The method for calibrating a head-up display device according to claim 2, further comprising: For each virtual image to be processed corresponding to each eye box partition to be processed, and for each pixel in the virtual image to be processed, determining whether the brightness information of the pixel matches the second brightness information; The second brightness information indicates the brightness of a boundary line of a preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed; If there is no pixel point whose brightness information matches the second brightness information in the to-be-processed virtual image, the step of determining, for each of the to-be-processed eye box partitions, the to-be-processed virtual image corresponding to the adjacent eye box partition of the to-be-processed eye box partition is performed to obtain at least one adjacent virtual image.

4. The calibration method of the head-up display device according to claim 3, wherein: The identifying the coverage of the preset visible area corresponding to the eye box partition to be processed in the display screen in the virtual image to be processed includes: When the brightness information of the pixel point matches the second brightness information, the pixel point is determined to be a boundary pixel point of the preset visible area corresponding to the eye box partition to be processed in the virtual image to be processed; multiple boundary pixel points in the virtual image to be processed indicate the coverage range.

5. The calibration method for a head-up display device according to any one of claims 2 to 4, wherein: The step of adjusting the mapping area corresponding to the eye box partition to be processed on the display screen based on the mapping area corresponding to the adjacent eye box partition of the eye box partition to be processed to obtain the target visible area corresponding to the eye box partition to be processed on the display screen includes: The mapping area corresponding to the adjacent eye box partition of the eye box partition to be processed is spliced ​​with the mapping area corresponding to the eye box partition to be processed to obtain the target visible area corresponding to the eye box partition to be processed in the display screen.

6. The method for calibrating a head-up display device according to claim 1, before identifying the coverage of the preset visible area corresponding to the eye box partition to be processed in the display screen in the virtual image to be processed, further comprising: For the eye box partition to be processed, which corresponds to a preset visible area in the display screen, obtaining an original image displayed in the preset visible area; Based on the brightness information of each pixel in the original image, the brightness information of each pixel in the to-be-processed virtual image corresponding to the to-be-processed eye box partition is normalized.

7. The calibration method of a head-up display device according to claim 1, further comprising: Obtaining an image to be displayed, a left-eye eyebox partition, and a right-eye eyebox partition; The left eye box partition is an eye box partition to be processed corresponding to the left eye of the target object among the plurality of eye box partitions to be processed; The right eye box partition is an eye box partition to be processed corresponding to the right eye of the target object among the plurality of eye box partitions to be processed; displaying the image to be displayed based on the target visible area corresponding to the left-eye eye box partition; And based on the target visible area corresponding to the right eye box partition, the image to be displayed is displayed.

8. A calibration device for a head-up display device, the head-up display device comprising a display screen and an imaging structure; the device comprising: A virtual image acquisition module, configured to acquire a virtual image to be processed corresponding to each of a plurality of continuously distributed eye box partitions to be processed; The virtual image to be processed is formed by the outgoing light from the real visible area corresponding to the eye box partition to be processed in the display screen being projected onto the eye box partition to be processed through the imaging structure; an area recognition module, configured to identify, for each virtual image to be processed corresponding to the eye box partition to be processed, a coverage range of a preset visible area corresponding to the eye box partition to be processed on the display screen in the virtual image to be processed; an area mapping module, configured to convert the coverage area from the coordinate system corresponding to the virtual image to be processed to the coordinate system corresponding to the display screen, and obtain a mapping area of ​​the preset visible area corresponding to the eye box partition to be processed on the display screen; an area adjustment module, configured to adjust the mapping area corresponding to the eye box partition to be processed in the display screen based on the mapping area corresponding to the adjacent eye box partition of the eye box partition to be processed, so as to obtain a target visible area corresponding to the eye box partition to be processed in the display screen; The adjacent eye box partition is an eye box partition to be processed that is adjacent to the eye box partition to be processed among the plurality of eye box partitions to be processed.

9. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the calibration method of a head-up display device according to any one of claims 1 to 7.

10. A computer-readable storage medium, wherein the storage medium stores at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the calibration method of a head-up display device according to any one of claims 1 to 7.

Images