Image acquisition method and device, head-up display and storage medium

Abstract

The invention discloses an image acquisition method and device, a head-up display and a storage medium, and relates to the technical field of auxiliary driving, the method is applied to the head-up display, and the method comprises the following steps: acquiring state combination modes respectively corresponding to different camera positions in a three-dimensional eye box; determining a matched target camera position according to the current human eye coordinate, and obtaining a target state combination mode corresponding to the target camera position; and configuring the sub-pixel distribution state corresponding to each block image in the target state combination mode to the matched screen partition, so as to obtain a fused image through the screen partition with the sub-pixel distribution state configuration completed. According to the technical scheme provided by the embodiment of the invention, fused image acquisition based on the distribution state of the plurality of sub-pixels is realized, and the image fusion efficiency of the head-up display is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of driver assistance technology, and in particular to an image acquisition method, apparatus, head-up display, and storage medium. Background Technology

[0002] With the continuous development of intelligent assisted driving technology, head-up displays (HUDs) are widely used in various vehicle interiors, and among them, HUDs with naked-eye 3D function (i.e., 3DHUDs) have become the focus of industry attention.

[0003] Because the driver's movements are highly dynamic and multi-dimensional, the user's eye position changes frequently. This dynamic change makes it difficult to accurately maintain the correspondence between the pre-set eye box partitions and the preset visual area. To solve the above problem, the existing technology regenerates the image based on a specific image generation algorithm after each driver movement to adapt to the driver's eye position.

[0004] However, image reconstruction based on image generation algorithms not only requires a long processing time, but also suffers from serious crosstalk problems, which reduces the clarity of the displayed image, affects the driver's accurate judgment of road conditions and reaction speed, and thus brings driving safety risks. Summary of the Invention

[0005] This invention provides an image acquisition method, apparatus, head-up display, and storage medium to solve the crosstalk problem of images displayed on a head-up display when the position of the human eye changes.

[0006] According to one aspect of the present invention, an image acquisition method is provided, applied to a head-up display, comprising:

[0007] Obtain the state combination method corresponding to different camera positions in the 3D eye box; wherein, the state combination method includes the sub-pixel distribution state corresponding to multiple block images respectively;

[0008] The target camera position is determined based on the current human eye coordinates, and the target state combination method corresponding to the target camera position is obtained;

[0009] The sub-pixel distribution states corresponding to each block image in the target state combination method are configured to the matching screen partitions, so as to obtain the fused image through the screen partitions configured with the sub-pixel distribution states; wherein, the block images are matched one by one with the screen partitions.

[0010] The method for obtaining the state combination method corresponding to different camera positions in the 3D eye box specifically includes: obtaining the image segmentation method and state combination method corresponding to different camera positions in the 3D eye box.

[0011] In the head-up display, each lenticular lens grating covers a number of sub-pixels greater than or equal to 4 and less than or equal to 10.

[0012] The number of screen partitions is greater than or equal to 2 and less than or equal to 6.

[0013] Adjacent screen partitions are separated by straight line boundaries.

[0014] The step of determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: obtaining the center points of each position between the first camera position and a plurality of adjacent second camera positions; wherein, the first camera position is the camera position closest to the current human eye coordinates; and obtaining the corresponding target state combination method based on the current human eye coordinates and each of the center points.

[0015] The step of determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: if it is determined that the current human eye coordinates are located at the center point between the third camera position and the fourth camera position, then the human eye movement direction is obtained; if it is determined that the human eye movement direction is from the third camera position to the fourth camera position, then the state combination method corresponding to the fourth camera position is used as the target state combination method; if it is determined that the human eye movement direction is from the fourth camera position to the third camera position, then the state combination method corresponding to the third camera position is used as the target state combination method.

[0016] The step of determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: obtaining the target location point based on the current human eye coordinates through a location prediction rule; obtaining the target camera position corresponding to the target location point and obtaining the target state combination method corresponding to the target camera position.

[0017] The step of obtaining the target location point based on the current human eye coordinates and using position prediction rules includes: obtaining the target location point based on the current human eye coordinates and the human eye movement boundary using position prediction rules.

[0018] The step of determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: obtaining the switching boundary of different state combination methods, and when the current human eye coordinates reach the first switching boundary, taking the first state combination method corresponding to the first switching boundary as the target state combination method corresponding to the current human eye coordinates.

[0019] According to another aspect of the present invention, an image calibration method is provided, applied to in-vehicle infotainment equipment, comprising:

[0020] Acquire virtual image images corresponding to the current camera position in the 3D eyebox under different sub-pixel distribution states;

[0021] Configure the corresponding sub-pixel distribution state for each block of image at the current camera position, and display the corresponding images according to the configured sub-pixel distribution state to obtain the fused image;

[0022] If it is determined that the current fused image conforms to the image fusion rules, the sub-pixel distribution state corresponding to each block image is used as the state combination method of the current camera position;

[0023] Sequentially obtain the state combination methods corresponding to different camera positions in the 3D eye box.

[0024] After obtaining the fused image based on the display images corresponding to the configured sub-pixel distribution states, the process further includes: if it is determined that the current fused image does not conform to the image fusion rules, reconfiguring the sub-pixel distribution states for at least one block image of the current camera position, and obtaining the fused image based on the display images corresponding to the reconfigured sub-pixel distribution states, until the current fused image conforms to the image fusion rules, and using the sub-pixel distribution states corresponding to each block image in the current fused image as the state combination method of the current camera position.

[0025] According to one aspect of the present invention, an image acquisition device is provided for use in a head-up display, comprising:

[0026] The state combination mode acquisition module is used to acquire the state combination mode corresponding to different camera positions in the 3D eye box; wherein, the state combination mode includes the sub-pixel distribution state corresponding to multiple block images respectively;

[0027] The target camera position determination module is used to determine the matching target camera position based on the current human eye coordinates, and to obtain the target state combination method corresponding to the target camera position;

[0028] The fused image acquisition module is used to configure the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, so as to acquire the fused image through the screen partition configured with the sub-pixel distribution state; wherein, the block image is matched one by one with the screen partition.

[0029] According to one aspect of the present invention, an image calibration apparatus is provided for use in vehicle-mounted equipment, comprising:

[0030] The virtual image acquisition module is used to acquire virtual image images corresponding to the current camera position in the 3D eye box under different sub-pixel distribution states;

[0031] The fused image acquisition module is used to configure the corresponding sub-pixel distribution state for each block of image at the current camera position, and to acquire the fused image based on the corresponding display image according to the configured sub-pixel distribution state.

[0032] The state combination acquisition module is used to determine the state combination method of the current camera position if it is determined that the current fused image conforms to the image fusion rules, and to take the sub-pixel distribution state corresponding to each block image as the state combination method of the current camera position.

[0033] The combination mode acquisition module is used to sequentially acquire the state combination modes corresponding to different camera positions in the 3D eye box.

[0034] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image acquisition method described in Embodiments 1 to 3 of the present invention, or to perform the image calibration method described in Embodiment 4 of the present invention.

[0035] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute the image acquisition method described in Embodiments 1 to 3 of the present invention, or to execute the image calibration method described in Embodiment 4 of the present invention.

[0036] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the image acquisition method described in Embodiments 1 to 3 of the present invention, or performs the image calibration method described in Embodiment 4 of the present invention.

[0037] The technical solution of this invention involves obtaining the state combination methods corresponding to different camera positions in a 3D eye-box; determining the matching target camera position based on the current human eye coordinates, and obtaining the target state combination method corresponding to the target camera position; configuring the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, so as to obtain the fused image through the screen partition configured with the sub-pixel distribution state; thereby realizing the acquisition of fused images based on multiple sub-pixel distribution states, which greatly improves the image fusion efficiency of the head-up display.

[0038] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of an image acquisition method provided according to Embodiment 1 of the present invention;

[0041] Figure 2 This is a schematic diagram of a sub-pixel distribution state provided in Embodiment 1 of the present invention;

[0042] Figure 3 This is a schematic diagram of the diagonal and vertical sub-pixel distributions provided in Embodiment 1 of the present invention;

[0043] Figure 4 This is a schematic diagram of different sub-pixel distribution states provided in Embodiment 1 of the present invention.

[0044] Figure 5 This is a schematic diagram of the positions of different camera positions in a three-dimensional eye box provided according to Embodiment 1 of the present invention;

[0045] Figure 6 This is a schematic diagram showing the positions of the camera, windshield, and virtual image according to Embodiment 1 of the present invention.

[0046] Figure 7 This is a schematic diagram of the fusion process of the sub-pixel distribution states of different image blocks according to Embodiment 1 of the present invention;

[0047] Figure 8 This is a schematic diagram of the fusion process of sub-pixel distribution states under different camera positions according to Embodiment 1 of the present invention;

[0048] Figure 9 This is a schematic diagram of the fusion method for sub-pixel distribution states under different camera positions according to Embodiment 1 of the present invention;

[0049] Figure 10 This is a schematic diagram of the fusion method for the sub-pixel distribution state at the position center point between two adjacent camera positions according to Embodiment 1 of the present invention;

[0050] Figure 11 This is a flowchart of an image acquisition method provided according to Embodiment 2 of the present invention;

[0051] Figure 12 This is a flowchart of an image acquisition method provided according to Embodiment 3 of the present invention;

[0052] Figure 13 This is a flowchart of an image calibration method provided according to Embodiment 4 of the present invention.

[0053] Figure 14 This is a schematic diagram of the structure of an image acquisition device according to Embodiment 5 of the present invention;

[0054] Figure 15 This is a schematic diagram of the structure of an image calibration device according to Embodiment Six of the present invention;

[0055] Figure 16 This is a schematic diagram of the structure of an electronic device that implements the image acquisition method of this invention. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0057] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0058] Example 1

[0059] Figure 1 This is a flowchart of an image acquisition method provided in Embodiment 1 of the present invention. This method is applicable to head-up displays (HUDs) that show fused images under different state combinations based on human eye coordinates. This method can be executed by the image acquisition device in Embodiment 5 of the present invention. The image acquisition device can be implemented in hardware and / or software, and can be configured in a head-up display. Figure 1 As shown, the method includes:

[0060] S101. Obtain the state combination method corresponding to different camera positions in the three-dimensional eye box; wherein, the state combination method includes the sub-pixel distribution state corresponding to multiple block images respectively.

[0061] like Figure 2 As shown, taking the width of the cylindrical lens covering 8 sub-pixels as an example, each pixel in the LCD screen is composed of 8 sub-pixels; S1 to S8 represent the arrangement sequence number of each sub-pixel in a pixel, L represents the sub-pixel seen by the left eye, and R represents the sub-pixel seen by the right eye. By periodically moving the starting point sub-pixel (i.e., L1) of the pixel and continuously changing the position of the sub-pixels in sequence, the pixel can obtain a total of 8 sub-pixel distribution states, i.e., state 1 to state 8.

[0062] like Figure 3 As shown, A1 to A8 represent different pixels. When the lenticular lens grating is placed at an angle relative to the liquid crystal display screen, the sub-pixels in the displayed image are distributed in a diagonal pattern. When the lenticular lens grating is placed parallel to the liquid crystal display screen, the sub-pixels in the displayed image are distributed in a vertical pattern. In particular, regardless of whether the distribution is diagonal or vertical, the steps of the image acquisition method disclosed in this application are the same for the HUD. In this embodiment of the invention, the lenticular lens grating is placed at an angle relative to the liquid crystal display screen, and the sub-pixels in the displayed image are distributed in a diagonal pattern. At the same time, in order to facilitate the distinction between the different images seen by the left and right eyes, the left eye (i.e., "L") sees a pure white image, and the right eye (i.e., "R") sees a pure black image, as an example.

[0063] like Figure 4 As shown, each viewing area of ​​the eye box corresponds to a different display image. In the first image, starting from state 1, the starting pixel of the image is state 1, and the states of each pixel are arranged sequentially from state 1 to state 8. In the second image, starting from state 2, the states of each pixel are arranged sequentially from state 2 to state 8 and then back to state 1. In the third image, starting from state 3, the states of each pixel are arranged sequentially from state 3 to state 8 and then back to state 1 and state 2. The other images are arranged in the same order, thus obtaining a total of 8 seed pixel distribution states corresponding to the display images.

[0064] An eyebox refers to the area of ​​human vision that a user can see in a complete image, usually represented as a two-dimensional rectangular area (width × height). In this embodiment of the invention, a three-dimensional eyebox refers to a three-dimensional area that the human eye can move in space. It is a three-dimensional extension based on the traditional eyebox, specifically by adding a new coordinate axis (i.e., the z-axis) to the traditional eyebox. This coordinate axis is used to define the direction of forward and backward movement of the human eye (i.e., the depth direction). For example, taking a traditional eyebox with a length × width of 130mm × 50mm (millimeters) as an example, the corresponding three-dimensional eyebox is a length × width × depth of 130mm × 50mm × z.

[0065] like Figure 5 As shown, a 130mm×50mm×z 3D eyebox is used as the shooting reference to simulate the position of the human eye. Nine camera positions are arranged on the central plane of the 3D eyebox, which are also the positions of the cameras; at the same time, nine camera positions are also arranged at the same positions on each of the two end faces in the z-axis direction. At this time, a total of 27 camera positions are arranged on the 3D eyebox; Figure 6 As shown, the camera is placed at the camera position of the 3D eye box, and the actual performance of the sub-pixel distribution state is captured through the windshield, that is, the sub-pixel distribution state image is acquired. The above-acquired image represents the virtual image seen by the human eye through the eye box of the head-up display. Thus, the actual performance of the sub-pixel distribution state is captured at each camera position, and a total of 27×8=216 sub-pixel distribution state images are obtained.

[0066] The 3D image is pre-divided into different blocks according to the screen partitions of the LCD screen, thus forming multiple block images. The number of block images can be pre-planned, for example, using... Figure 7 For example, the number of image blocks can be configured to 4, that is... Figure 7 The image consists of block images 1, 2, 3, and 4. Within each block image, the arrangement of sub-pixel distribution states is determined to obtain a pure color image. Taking the above technical solution as an example, where the left eye (i.e., "L") sees a pure white image and the right eye (i.e., "R") sees a pure black image, in a certain block image, when arranged in one of the above 8 sub-pixel distribution states, the left eye obtains a pure white image and the right eye obtains a pure black image, which is the most ideal image display effect.

[0067] Therefore, for block image 1, under the above 8 sub-pixel distribution states, the display image under each sub-pixel distribution state is obtained, that is, the sub-pixel distribution state image. The distribution state in which the left eye obtains a pure white image and the right eye obtains a pure black image is taken as the sub-pixel distribution state corresponding to block image 1. For example, assuming that block image 1 is distributed in sub-pixel distribution state 1, the left eye image is a pure white image and the right eye image is a pure black image. At this time, the sub-pixel distribution state corresponding to block image 1 is sub-pixel distribution state 1.

[0068] Similarly, continuing to acquire block images 2 through 4, under what sub-pixel distribution state can the aforementioned ideal image be obtained? Assuming that block image 2 matches sub-pixel distribution state 2, block image 3 matches sub-pixel distribution state 3, and block image 4 matches sub-pixel distribution state 4, then for the image displayed at the current camera position, the sub-pixel distribution states 1, 2, 3, and 4 corresponding to block images 1, 2, 3, and 4 respectively constitute the state combination method corresponding to the current camera position. In particular, in a state combination method, different block images may also correspond to the same sub-pixel distribution state.

[0069] Because the viewing angle differs from different camera positions, the way states are combined often differs from those from different camera positions; Figure 5 Taking camera positions C5 and C6 as examples; Figure 8 As shown, taking the image seen by the right eye as an example, the state combination method corresponding to the C5 camera position is the state combination method composed of sub-pixel distribution state 1, sub-pixel distribution state 2, sub-pixel distribution state 3, and sub-pixel distribution state 4. When the camera moves from the C5 camera position to the C6 camera position (equivalent to the user's right eye moving from the C5 position to the C6 position), if the above combination method is still used at the C6 camera position, the right eye will not see a pure black image, but a black and white image with white areas. At this time, the image display effect is poor.

[0070] This is because as the viewer's eye moves to the right, the black and white stripes in the four images from the C6 camera position move rhythmically with the eye. At a certain point, the black stripes exceed the extraction boundary. If the same fusion method as the C5 camera position is used at this point, the images captured from the C6 camera position, based on the fixed extraction boundary, cannot be stitched together into a single, pure black image. In fact... Figure 9As shown, under the C6 camera position, block images 1, 2, 3, and 4 are matched with sub-pixel distribution states 2, 3, 4, and 5, respectively. Based on this, the sub-pixel distribution state combination under the C6 camera position is obtained, which is the state combination method composed of sub-pixel distribution state 2, sub-pixel distribution state 3, sub-pixel distribution state 4, and sub-pixel distribution state 5.

[0071] S102. Determine the target camera position based on the current human eye coordinates, and obtain the target state combination method corresponding to the target camera position.

[0072] Human eye coordinates can be obtained through a driver monitoring system (DMS) installed in the driver's cab. The DMS obtains the driver's eye coordinates by detecting the direction of the driver's eye movement and gaze. The camera position closest to the human eye coordinates can be used as the target camera position, and the target state combination corresponding to the target camera position can be used as the state combination corresponding to the current human eye coordinates.

[0073] In addition, the actual image acquired by the camera can also be used as the basis for judgment. That is, after the camera obtains a pure black (or pure white) image at a certain position through image fusion, it is translated in the eye box until a white (or black) area is detected in the image acquired by the camera. The camera coordinates at this time are recorded, and the coordinates are used as the basis for switching the state combination mode. This indicates that the current position needs to switch the state combination mode, and the state combination mode that can cover the entire black (or white) area under this coordinate is taken as the target state combination mode.

[0074] Furthermore, based on optical computing principles, the most suitable position for image switching between different eye positions can be pre-calculated. This determines the required coordinate movement for image switching and serves as the basis for switching state combination methods. The image fusion method for the current position can be determined by selecting the state combination method of the closest camera position based on the distance between the current position and other camera positions. Therefore, based on the current eye coordinates, when the eye moves to the aforementioned switching position, different state combination methods can be switched to perform image fusion using the target state combination method, thereby improving the clarity of the displayed image.

[0075] S103. Configure the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, so as to obtain the fused image through the screen partition configured with the sub-pixel distribution state; wherein, the block image is matched one by one with the screen partition.

[0076] Screen partitioning refers to dividing the LCD screen into multiple independent light control units, each of which can independently adjust brightness and color. As described in the above technical solution, since the 3D image is pre-divided into different blocks according to the screen partitioning of the LCD screen, the block image is actually matched one-to-one with the screen partition. After configuring the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, the fused image under different sub-pixel distribution states can be presented on the LCD screen through screen partitioning, thereby completing the acquisition and display of the fused image.

[0077] Optionally, in this embodiment of the invention, determining the matching target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: obtaining the center points of each position between the first camera position and a plurality of adjacent second camera positions; wherein, the first camera position is the camera position closest to the current human eye coordinates; and obtaining the corresponding target state combination method based on the current human eye coordinates and each of the center points.

[0078] Specifically, the entire eye box can be divided into different areas based on the center point of each camera position and other adjacent camera positions. When the human eye coordinates are located in the corresponding area of ​​a certain camera position, the image is fused using the state combination method of that camera position. As long as the human eye coordinates do not exceed the area, the current state combination method is used for fusion, thereby continuously monitoring the human eye coordinates. The first camera position is the camera position closest to the current human eye coordinates.

[0079] When the human eye's coordinates exceed the corresponding area of ​​the first camera position, that is, after exceeding a certain position center point, it enters the corresponding area of ​​one of the second camera positions. At this time, the state combination mode of the new area is used as the target state combination mode to complete the switching of the state combination mode. Thus, the spatial area is divided based on each camera position, so that when the human eye enters different position coordinates, the state combination mode can be quickly switched. This ensures both the clarity of the displayed image and a fast image fusion speed, avoiding lag in the image configuration process.

[0080] Optionally, in this embodiment of the invention, determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: if it is determined that the current human eye coordinates are located at the center point between the third camera position and the fourth camera position, then obtaining the human eye movement direction; if it is determined that the human eye movement direction is from the third camera position to the fourth camera position, then using the state combination method corresponding to the fourth camera position as the target state combination method; if it is determined that the human eye movement direction is from the fourth camera position to the third camera position, then using the state combination method corresponding to the third camera position as the target state combination method.

[0081] Specifically, such as Figure 10As shown, when the human eye is located at the center point between different camera positions, due to the brightness difference at the image boundary, the extracted boundary may always have a gray area. At this time, no matter how you switch, you cannot guarantee that a pure color image will be displayed. At this time, based on the direction of human eye movement, the state combination mode corresponding to the position where the human eye will reach in the next moment is switched in advance. That is, when it is determined that the direction of human eye movement is from the third camera position to the fourth camera position, it is predicted that the position where the human eye will reach in the next moment will be closer to the fourth camera position. At this time, the state combination mode corresponding to the fourth camera position is used as the target state combination mode.

[0082] When the direction of human eye movement is determined to be from the fourth camera position to the third camera position, the system predicts that the eye will reach a position closer to the third camera position in the next moment. At this point, the state combination corresponding to the third camera position is used as the target state combination. This allows the system to pre-switch to the state combination indicating that the eye is closer to the camera position when the eye is at the center point between the third and fourth cameras. This achieves the prediction and acquisition of the final eye movement position. Furthermore, because the eye moves relatively quickly, it does not affect the clarity of the image viewed during the movement, ensuring that the fused image is acquired by the time the eye movement is complete, thus improving the efficiency of acquiring the fused image at the endpoint of the movement.

[0083] The technical solution of this invention involves obtaining the state combination methods corresponding to different camera positions in a 3D eye-box; determining the matching target camera position based on the current human eye coordinates, and obtaining the target state combination method corresponding to the target camera position; configuring the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, so as to obtain the fused image through the screen partition configured with the sub-pixel distribution state; thereby realizing the acquisition of fused images based on multiple sub-pixel distribution states, which greatly improves the image fusion efficiency of the head-up display.

[0084] Example 2

[0085] Figure 11 This is a flowchart of an image acquisition method provided in Embodiment 2 of the present invention. The relationship between this embodiment and the above embodiments is that different camera positions may correspond to different image segmentation methods. For example... Figure 11 As shown, the method includes:

[0086] S201. Obtain the image segmentation method and state combination method corresponding to different camera positions in the 3D eye box.

[0087] To improve the clarity of the merged image, different image segmentation methods can be set for different camera positions based on empirical values. The number, shape, area, and position of each segmented image can vary to obtain the corresponding state combination method for each segmented image. This avoids situations where some segments cannot obtain a matching state combination method under a single image segmentation method, resulting in poor image display of that segmented image. Alternatively, an initial matching image segmentation method can be set for each camera position based on empirical values, and then the image segmentation method for that camera position can be adjusted according to the actual image fusion effect to determine the final state combination method.

[0088] S202. Determine the target camera position based on the current human eye coordinates, and obtain the target state combination method corresponding to the target camera position.

[0089] S203. Configure the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, so as to obtain the fused image through the screen partition configured with the sub-pixel distribution state; wherein, the block image is matched one by one with the screen partition.

[0090] Optionally, in this embodiment of the invention, the number of sub-pixels covered by each lenticular lens in the head-up display is greater than or equal to 4 and less than or equal to 10. Specifically, in a head-up display with naked-eye 3D functionality, if the number of sub-pixels covered by a lenticular lens is small, problems such as low resolution, insufficient number of viewpoints, and sensitivity to crosstalk will occur; if the number of sub-pixels covered by a lenticular lens is large, problems such as weak stereoscopic effect, decreased brightness, and increased crosstalk will occur. Therefore, the number of sub-pixels covered by each lenticular lens can be set to greater than or equal to 4 and less than or equal to 10, so that the head-up display can balance the requirements of resolution, number of viewpoints, and crosstalk, and achieve a clear, crosstalk-free, and stereoscopic 3D display effect.

[0091] Optionally, in this embodiment of the invention, the number of screen partitions is greater than or equal to 2 and less than or equal to 6. A larger number of screen partitions can improve the display effect of the merged image, but it will also increase the complexity of image fusion and reduce the efficiency of image fusion. Therefore, the number of screen partitions can be set to greater than or equal to 2 and less than or equal to 6 to keep the number of screen partitions moderate, so that the merged image has both a good display effect and a high image fusion efficiency.

[0092] Optionally, in this embodiment of the invention, adjacent screen partitions are divided by straight line boundaries. By using straight lines as the dividing boundaries between adjacent screen partitions, not only is the optical mapping process from the screen to the virtual image simplified, but the clarity of information presentation is also improved, and the stability of the head-up display imaging process is enhanced.

[0093] The technical solution of this invention, when obtaining the image segmentation method corresponding to different camera positions in the 3D eye box, if all the state combinations corresponding to the image segmentation method under the current camera position do not conform to the image fusion rules, the current image segmentation method is changed; if at least one state combination corresponding to the image segmentation method under the current camera position conforms to the image fusion rules, the current state combination method and image segmentation method are selected, thereby ensuring a better display effect of the fused image.

[0094] Example 3

[0095] Figure 12 This is a flowchart of an image acquisition method provided in Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments is that this embodiment predicts the target location that the human eye is about to reach. Figure 12 As shown, the method includes:

[0096] S301. Obtain the state combination method corresponding to different camera positions in the three-dimensional eye box; wherein, the state combination method includes the sub-pixel distribution state corresponding to multiple block images.

[0097] S302. Based on the current human eye coordinates, obtain the target location point through the location prediction rule.

[0098] The aforementioned position prediction rules may include at least one of the following: polynomial fitting rules, discrete Kalman filtering rules, sampling point cleaning rules, and gaze prediction rules.

[0099] S303. Obtain the target camera position corresponding to the target location point, and obtain the target state combination mode corresponding to the target camera position.

[0100] Based on the current eye coordinates, when predicting the target location that the eye is about to reach, a multinomial fitting rule can be used; whereby the multinomial fitting rule is based on historical sampling points. The position of the target point to which the human eye is to move is predicted using the following polynomial equation:

[0101] (Equation 1);

[0102] Among them, the interval time of sampling points It is a fixed value, and the number of sampling points is also a fixed value. It is to All coefficients are polynomials, and the weight matrix is ​​defined as follows:

[0103] ;

[0104] in, ; This is a parameter that controls the attenuation width. Sampling points with a longer time distance from the prediction point have less weight. It is obtained by solving the following equation using the least squares method based on historical data. (Equation 2);

[0105] ;

[0106] in, Substituting Equation 2 into Equation 1 above, we can calculate and obtain the target position that the human eye is about to reach.

[0107] Based on the current eye coordinates, when predicting the target location to which the human eye will move using the discrete Kalman filter rule, each time step includes two steps: prediction and update. In the prediction step, the predicted state and prediction covariance are calculated using the following equations:

[0108] ;

[0109] ;

[0110] Simultaneously, the Kalman gain is updated using the following equation, and the optimal estimate of the human eye position is obtained. and its uncertainty :

[0111] ;

[0112] ;

[0113] .

[0114] in For the observation matrix, Here is the state transition matrix. Let be the covariance matrix of the process noise. It is the covariance matrix of the observation noise; the above optimal estimate This refers to the target location that the human eye is about to reach.

[0115] When predicting the target location to which the human eye will move using sampling point cleaning rules, the density of sampling points is first dynamically adjusted based on the time interval between the current time and the target time point to increase the importance of sampling points with shorter time intervals; the distance from the target time point... The closer the points are, the higher the retention density; that is, sparse sampling is performed on more distant data points. Sparse sampling can be performed using distance weighting or hierarchical sampling.

[0116] Furthermore, since some of the vehicle's movements (such as sudden braking, cornering, and going over speed bumps) can cause significant changes in the speed of human eyes, some sampling points can be corrected based on the vehicle's movement. Since these movements are usually short-lived, the time window in which the movement occurred can be determined based on the vehicle's movement data, and the sampling point data in that part can be deleted. Alternatively, the sampling point data within the time window can be filled using the interpolation algorithm corresponding to the polynomial fitting rule based on the sampling point data before and after the time window, or the sampling point data can be corrected based on the vehicle's movement data.

[0117] Specifically, traditional methods assume that eye position is independent data with temporal characteristics. However, in reality, the driver's eye position is strongly correlated with their visual needs, such as checking the dashboard or observing road conditions. Therefore, an external camera can be used to capture road images in front of or to the side of the vehicle, forming a road video stream. Simultaneously, DMS data is used to capture a video stream of the driver's face, and the two video streams are time-aligned. Then, the "SwinTransformer" model is used to extract latent road scene features from the road video frames. These road scene features are then input into an LSTM (Long Short-Term Memory) model to obtain temporal road scene features.

[0118] Then, driver gaze features corresponding to video frames are extracted from the driver's facial video stream. These driver gaze features are input into an LSTM model to obtain temporal driver gaze features. A fully connected layer is used to concatenate the feature outputs of the two streams. An attention layer is used to achieve mutual attention between in-vehicle and out-of-vehicle features, and finally, the prediction result of the gaze point is output, thus obtaining the prediction result of the human eye position and the target location. In particular, supervised training is used to train the road scene feature extraction branch and the driver gaze feature extraction branch separately. The loss function of the output layer adopts the mean squared error (MSE) equation, so only the driver's eye position needs to be obtained, and other information does not need to be considered.

[0119] Therefore, based on one or more of the following rules—polynomial fitting, discrete Kalman filtering, sampling point cleaning, and gaze prediction—the target position to which the human eye is to move is predicted and obtained. Then, according to the camera position corresponding to the target position, the state combination method corresponding to that camera position is used as the state combination method of the displayed image, thereby ensuring that the pre-configuration of the displayed image is completed and that the human eye observes a display image with a high degree of clarity.

[0120] S304. Configure the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, so as to obtain the fused image through the screen partition configured with the sub-pixel distribution state; wherein, the block image is matched one by one with the screen partition.

[0121] Optionally, in this embodiment of the invention, obtaining the target location point based on the current human eye coordinates using a location prediction rule includes: obtaining the target location point based on the current human eye coordinates and the human eye movement boundary using a location prediction rule. Specifically, with time intervals... As the range of motion increases, the prediction error will also increase. Therefore, the driver's habitual eye movement range can be determined based on historical sampling data. When the range of motion of the human eye is near its limit, the driver's tendency to steer is greater, thus reducing the predictability. The numerical values ​​are used to further reduce crosstalk and improve the clarity of the displayed image.

[0122] Optionally, in this embodiment of the invention, determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: obtaining the switching boundary of different state combination methods, and when the current human eye coordinates reach the first switching boundary, taking the first state combination method corresponding to the first switching boundary as the target state combination method corresponding to the current human eye coordinates.

[0123] Specifically, for image fusion tasks, since it is necessary to use DMS to calculate when the human eye reaches a certain boundary or the boundary is about to turn white, it is necessary to immediately switch to the image that the human eye can see in the best fusion state. That is, although the content seen by the human eye is only related to the position of the human eye, whether to switch can actually be composed of many factors such as the position and speed of the human eye. Therefore, instead of predicting the specific position of the human eye, it is necessary to predict whether the image needs to switch the fusion strategy, thus transforming an exact regression task into a classification task.

[0124] For example, whether to switch the image fusion method depends on the current location and current speed, and the corresponding classification probability can be expressed as:

[0125] ;

[0126] In addition, other features can be added to the classifier, such as interpupillary distance, human eye motion acceleration, etc. Other feature engineering can also be used, such as transforming position features into distances from preset boundaries and transforming velocity features into normal components along the boundaries, to improve prediction accuracy.

[0127] Therefore, when the current human eye coordinates reach the first switching boundary, the first state combination corresponding to the first switching boundary is used as the target state combination corresponding to the current human eye coordinates. This transforms the precise regression task into a classification task, simplifies the computational complexity of the target state combination, improves the accuracy of the prediction results of the target location, and further ensures the clarity of the displayed image observed by the human eye.

[0128] The technical solution of this invention obtains the target location point based on the current human eye coordinates using at least one of the following: polynomial fitting rule, discrete Kalman filter rule, sampling point cleaning rule, and gaze prediction rule. Then, based on the target camera position corresponding to the target location point, the target state combination mode corresponding to the target camera position is obtained. This achieves the prediction and acquisition of the target location point to which the human eye will move, and the state combination mode of the target location point corresponding to the camera position is used as the state combination mode of the displayed image. This ensures that the pre-configuration of the displayed image is completed, and that the human eye observes a display image with a high degree of clarity.

[0129] Example 4

[0130] Figure 13 This is a flowchart of an image calibration method provided in Embodiment 4 of the present invention. This method is applicable to situations where a vehicle-mounted system calibrates the sub-pixel distribution state of segmented images from different camera positions within a 3D camera module. This method can be executed by the image calibration device described in Embodiment 6 of the present invention. This image calibration device can be implemented in hardware and / or software and can be configured within the vehicle-mounted system. Figure 13 As shown, the method includes:

[0131] S401. Obtain the virtual image images corresponding to the current camera position in the 3D eye box under different sub-pixel distribution states.

[0132] At the current position (with) Figure 5 Taking camera position C1 as an example, obtain the distribution state of different sub-pixels (using...). Figure 4 Taking the distribution of 8 seed pixels as an example, the corresponding virtual images are obtained, and 8 different imaging results can be obtained accordingly.

[0133] S402. Configure the corresponding sub-pixel distribution state for each block image of the current camera position, and display the corresponding images according to the configured sub-pixel distribution state to obtain the fused image.

[0134] by Figure 7Taking a medium with 4 image blocks as an example, for image block 1, image block 2, image block 3, and image block 4, sub-pixel distribution state 1, sub-pixel distribution state 2, sub-pixel distribution state 3, and sub-pixel distribution state 4 are selected respectively as the corresponding sub-pixel distribution states. Based on this, image block 1, image block 2, image block 3, and image block 4 are imaged with different sub-pixel distribution states.

[0135] S403. If it is determined that the current fused image conforms to the image fusion rules, the sub-pixel distribution state corresponding to each block image is used as the state combination method of the current camera position.

[0136] The image fusion rule defines that when the solid color ratio of the fused image is greater than or equal to the solid color ratio threshold, the currently displayed image presents a clearer display effect. The solid color ratio threshold includes the black ratio threshold and / or the white ratio threshold. If the current fused image conforms to the image fusion rule, that is, the solid color ratio of the current fused image is greater than or equal to the above solid color ratio threshold, the display effect of the current fused image is relatively clear. At this time, the sub-pixel distribution state corresponding to each block image is used as the state combination method of the current camera position.

[0137] Optionally, in this embodiment of the invention, after obtaining the fused image based on the display images corresponding to the configured sub-pixel distribution states, the method further includes: if it is determined that the current fused image does not conform to the image fusion rules, reconfiguring the sub-pixel distribution states for at least one block image of the current camera position, and obtaining the fused image based on the display images corresponding to the reconfigured sub-pixel distribution states, until the current fused image conforms to the image fusion rules, and using the sub-pixel distribution states corresponding to each block image in the current fused image as the state combination method of the current camera position.

[0138] Specifically, if it is determined that the current fused image does not conform to the image fusion rules, the sub-pixel distribution state of one or more block images of the current camera position is reconfigured to change the imaging mode of the above one or more block images. Based on the display images corresponding to the reconfigured sub-pixel distribution state, a new fused image is re-acquired, and it is further determined whether the new fused image conforms to the image fusion rules.

[0139] If the newly fused image still does not conform to the image fusion rules, repeat the above fusion process, that is, continue to change the sub-pixel distribution state corresponding to one or more block images, and judge again whether the new fused image conforms to the image fusion rules, until the current fused image conforms to the image fusion rules. Then, take the sub-pixel distribution state corresponding to each block image in the current fused image as the state combination method of the current camera position, so as to complete the calibration of the fused image corresponding to the current camera position.

[0140] S404. Sequentially obtain the state combination methods corresponding to different camera positions in the 3D eye box.

[0141] by Figure 5 Taking the 27 camera positions in the 3D eye box as an example, the state combination methods corresponding to the 27 camera positions in the 3D eye box are obtained in turn. After the calibration of the fused images of all the camera positions is completed, the fused image calibration process of the 3D eye box is completed.

[0142] The technical solution of this invention obtains the virtual image corresponding to each camera position in the 3D eye box under different sub-pixel distribution states, and configures the corresponding sub-pixel distribution states for each block image of each camera position. Based on the display images corresponding to the configured sub-pixel distribution states, a fused image is obtained. When the current fused image conforms to the image fusion rules, the sub-pixel distribution states corresponding to each block image are used as the state combination method of the current camera position. This achieves the calibration and acquisition of the state combination methods corresponding to different camera positions in the 3D eye box, ensuring the clarity of the displayed image under the state combination method.

[0143] Example 5

[0144] Figure 14 This is a structural block diagram of an image acquisition device provided in Embodiment 5 of the present invention. The image acquisition device specifically includes:

[0145] The state combination mode acquisition module 501 is used to acquire the state combination mode corresponding to different camera positions in the 3D eye box; wherein, the state combination mode includes the sub-pixel distribution state corresponding to multiple block images respectively;

[0146] The target camera position determination module 502 is used to determine the matching target camera position based on the current human eye coordinates, and to obtain the target state combination method corresponding to the target camera position;

[0147] The fused image acquisition module 503 is used to configure the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, so as to acquire the fused image through the screen partition configured with the sub-pixel distribution state; wherein, the block image is matched one by one with the screen partition.

[0148] The technical solution of this invention involves obtaining the state combination methods corresponding to different camera positions in a 3D eye-box; determining the matching target camera position based on the current human eye coordinates, and obtaining the target state combination method corresponding to the target camera position; configuring the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, so as to obtain the fused image through the screen partition configured with the sub-pixel distribution state; thereby realizing the acquisition of fused images based on multiple sub-pixel distribution states, which greatly improves the image fusion efficiency of the head-up display.

[0149] Optionally, the state combination method acquisition module 501 is specifically used to acquire the image block method and state combination method corresponding to different camera positions in the 3D eye box.

[0150] Optionally, in the head-up display, each lenticular lens grating covers a number of sub-pixels greater than or equal to 4 and less than or equal to 10.

[0151] Optionally, the number of screen partitions is greater than or equal to 2 and less than or equal to 6.

[0152] Optionally, adjacent screen partitions can be divided by straight line boundaries.

[0153] Optionally, the target camera position determination module 502 is specifically used to obtain the center points of each position between the first camera position and a plurality of adjacent second camera positions; wherein, the first camera position is the camera position closest to the current human eye coordinates; and obtain the corresponding target state combination method according to the current human eye coordinates and each of the center points.

[0154] Optionally, the target camera position determination module 502 is specifically used to: if it is determined that the current eye coordinates are located at the center point between the third camera position and the fourth camera position, then obtain the direction of eye movement; if it is determined that the direction of eye movement is from the third camera position to the fourth camera position, then use the state combination mode corresponding to the fourth camera position as the target state combination mode; if it is determined that the direction of eye movement is from the fourth camera position to the third camera position, then use the state combination mode corresponding to the third camera position as the target state combination mode.

[0155] Optionally, the target camera position determination module 502 is specifically used to obtain the target position point based on the current human eye coordinates and through position prediction rules; obtain the target camera position corresponding to the target position point; and obtain the target state combination method corresponding to the target camera position.

[0156] Optionally, the target position determination module 502 is specifically used to obtain the target position point based on the current human eye coordinates and the human eye movement boundary through position prediction rules.

[0157] Optionally, the target camera position determination module 502 is specifically used to obtain the switching boundaries of different state combinations, and when the current human eye coordinates reach the first switching boundary, the first state combination corresponding to the first switching boundary is used as the target state combination corresponding to the current human eye coordinates.

[0158] The image acquisition device provided by this invention can execute the image acquisition methods provided in Embodiments 1 to 3 of this invention, and has the corresponding functional modules and beneficial effects for executing the methods. Technical details not described in detail in this embodiment can be found in the image acquisition methods provided in Embodiments 1 to 3 of this invention.

[0159] Example 6

[0160] Figure 15 This is a structural block diagram of an image calibration device provided in Embodiment Six of the present invention. The image calibration device specifically includes:

[0161] The virtual image acquisition module 601 is used to acquire virtual image images corresponding to the current camera position in the 3D eye box under different sub-pixel distribution states;

[0162] The fused image acquisition module 602 is used to configure the corresponding sub-pixel distribution state for each block image of the current camera position, and to acquire the fused image according to the corresponding display image based on the configured sub-pixel distribution state.

[0163] The state combination acquisition module 603 is used to determine, if the current fused image conforms to the image fusion rules, to take the sub-pixel distribution state corresponding to each block image as the state combination method of the current camera position.

[0164] The combination mode acquisition module 604 is used to sequentially acquire the state combination modes corresponding to different camera positions in the 3D eye box.

[0165] The technical solution of this invention obtains the virtual image corresponding to each camera position in the 3D eye box under different sub-pixel distribution states, and configures the corresponding sub-pixel distribution states for each block image of each camera position. Based on the display images corresponding to the configured sub-pixel distribution states, a fused image is obtained. When the current fused image conforms to the image fusion rules, the sub-pixel distribution states corresponding to each block image are used as the state combination method of the current camera position. This achieves the calibration and acquisition of the state combination methods corresponding to different camera positions in the 3D eye box, ensuring the clarity of the displayed image under the state combination method.

[0166] Optionally, the image calibration device is further configured to, if it is determined that the current fused image does not conform to the image fusion rules, reconfigure the sub-pixel distribution state for at least one block image of the current camera position, and obtain the fused image according to the display images corresponding to the reconfigured sub-pixel distribution states, until the current fused image conforms to the image fusion rules, and use the sub-pixel distribution states corresponding to each block image in the current fused image as the state combination method of the current camera position.

[0167] The image calibration device provided by this invention can execute the image acquisition method provided in Embodiment 4 of this invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the image acquisition method provided in Embodiment 4 of this invention.

[0168] Example 7

[0169] Figure 16 A schematic diagram of a head-up display 10, which can be used to implement embodiments of the present invention, is shown. The head-up display is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The head-up display can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0170] like Figure 16 As shown, the head-up display 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the head-up display 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0171] Multiple components in the head-up display 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, optical disk, etc.; and a communication unit 19, such as a network card, modem, wireless transceiver, etc. The communication unit 19 allows the head-up display 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0172] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image acquisition methods or image calibration methods.

[0173] In some embodiments, the image acquisition method or image calibration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the image acquisition method or image calibration method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the image acquisition method or image calibration method by any other suitable means (e.g., by means of firmware).

[0174] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0175] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable image acquisition device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0176] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0177] To provide user interaction, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).

[0178] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0179] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0180] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0181] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image acquisition method, characterized in that, Applications in head-up displays include: Obtain the state combination method corresponding to different camera positions in the 3D eye box; wherein, the state combination method includes the sub-pixel distribution state corresponding to multiple block images respectively; The target camera position is determined based on the current human eye coordinates, and the target state combination method corresponding to the target camera position is obtained; The sub-pixel distribution states corresponding to each block image in the target state combination method are configured to the matching screen partitions, so as to obtain the fused image through the screen partitions configured with the sub-pixel distribution states; wherein, the block images are matched one by one with the screen partitions.

2. The method according to claim 1, characterized in that, The method for obtaining the state combinations corresponding to different camera positions in the 3D eye-box specifically includes: Obtain the image segmentation method and state combination method corresponding to different camera positions in the 3D eye box.

3. The method according to claim 1, characterized in that, In the head-up display, each lenticular lens grating covers a number of sub-pixels greater than or equal to 4 and less than or equal to 10.

4. The method according to claim 1, characterized in that, The number of screen partitions is greater than or equal to 2 and less than or equal to 6.

5. The method according to claim 1, characterized in that, Adjacent screen partitions are separated by straight line boundaries.

6. The method according to claim 1, characterized in that, The step of determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: Obtain the center points of each position between the first camera position and multiple adjacent second camera positions; wherein, the first camera position is the camera position closest to the current human eye coordinates; Based on the current human eye coordinates and the center points of each location, obtain the corresponding target state combination method.

7. The method according to claim 1, characterized in that, The step of determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: If the current eye coordinates are determined to be at the center point between the third and fourth camera positions, then the direction of eye movement is obtained. If it is determined that the direction of the human eye movement is from the third camera position to the fourth camera position, then the state combination method corresponding to the fourth camera position is taken as the target state combination method. If it is determined that the direction of the human eye movement is from the fourth camera position to the third camera position, then the state combination mode corresponding to the third camera position is taken as the target state combination mode.

8. The method according to claim 1, characterized in that, The step of determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: Based on the current human eye coordinates, the target location is obtained through location prediction rules; Obtain the target camera position corresponding to the target location, and obtain the target state combination method corresponding to the target camera position.

9. The method according to claim 8, characterized in that, The step of obtaining the target location point based on the current human eye coordinates and using location prediction rules includes: The target location is obtained based on the current eye coordinates and eye movement boundaries using location prediction rules.

10. The method according to claim 1, characterized in that, The step of determining the target camera position based on the current human eye coordinates and obtaining the target state combination method corresponding to the target camera position includes: Obtain the switching boundaries of different state combinations, and when the current human eye coordinates reach the first switching boundary, take the first state combination corresponding to the first switching boundary as the target state combination corresponding to the current human eye coordinates.

11. An image calibration method, characterized in that, Applications in in-vehicle infotainment systems include: Acquire virtual image images corresponding to the current camera position in the 3D eyebox under different sub-pixel distribution states; Configure the corresponding sub-pixel distribution state for each block of image at the current camera position, and display the corresponding images according to the configured sub-pixel distribution state to obtain the fused image; If it is determined that the current fused image conforms to the image fusion rules, the sub-pixel distribution state corresponding to each block image is used as the state combination method of the current camera position; Sequentially obtain the state combination methods corresponding to different camera positions in the 3D eye box.

12. The method according to claim 11, characterized in that, After obtaining the fused image from the display images corresponding to the configured sub-pixel distribution states, the process also includes: If it is determined that the current fused image does not conform to the image fusion rules, the sub-pixel distribution state of at least one block image of the current camera position is reconfigured, and the fused image is obtained according to the display image corresponding to the reconfigured sub-pixel distribution state. This process continues until the current fused image conforms to the image fusion rules. Then, the sub-pixel distribution state corresponding to each block image in the current fused image is used as the state combination method of the current camera position.

13. An image acquisition device, characterized in that, Applications in head-up displays include: The state combination mode acquisition module is used to acquire the state combination mode corresponding to different camera positions in the 3D eye box; wherein, the state combination mode includes the sub-pixel distribution state corresponding to multiple block images respectively; The target camera position determination module is used to determine the matching target camera position based on the current human eye coordinates, and to obtain the target state combination method corresponding to the target camera position; The fused image acquisition module is used to configure the sub-pixel distribution state corresponding to each block image in the target state combination method to the matching screen partition, so as to acquire the fused image through the screen partition configured with the sub-pixel distribution state; wherein, the block image is matched one by one with the screen partition.

14. An image calibration device, characterized in that, Applications in in-vehicle infotainment systems include: The virtual image acquisition module is used to acquire virtual image images corresponding to the current camera position in the 3D eye box under different sub-pixel distribution states; The fused image acquisition module is used to configure the corresponding sub-pixel distribution state for each block of image at the current camera position, and to acquire the fused image based on the corresponding display image according to the configured sub-pixel distribution state. The state combination acquisition module is used to determine the state combination method of the current camera position if it is determined that the current fused image conforms to the image fusion rules, and to take the sub-pixel distribution state corresponding to each block image as the state combination method of the current camera position. The combination mode acquisition module is used to sequentially acquire the state combination modes corresponding to different camera positions in the 3D eye box.

15. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image acquisition method of any one of claims 1-10, or to perform the image calibration method of claim 11 or 12.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the image acquisition method of any one of claims 1-10, or to perform the image calibration method of claim 11 or 12.

17. A computer program product comprising a computer program that, when executed by a processor, implements the image acquisition method of any one of claims 1-10, or the image calibration method of claim 11 or 12.

Images