Image processing method and apparatus, head-up display and storage medium

Abstract

An image processing method and apparatus, a head-up display, and a storage medium, relating to the technical field of assisted driving. The method is applied to the head-up display, and comprises: acquiring viewpoint position coordinates of a target user in real time, and determining whether a viewpoint position of the target user is offset (S101); if it is determined that the viewpoint position of the target user is offset, performing pixel grayscale adjustment on a 3D composite image on the basis of the current viewpoint position coordinates (S102); and on the basis of the current viewpoint position coordinates, performing pixel brightness adjustment on the 3D composite image subjected to pixel grayscale adjustment, and displaying the 3D composite image subjected to pixel brightness adjustment (S103).

Description

Image processing methods, apparatus, heads-up displays, and storage media

[0001] This application claims priority to Chinese Patent Application No. 202411972859.9, filed with the Chinese Patent Office on December 30, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of driver assistance technology, such as an image processing method, apparatus, head-up display, and storage medium. Background Technology

[0003] With the continuous development of intelligent driver assistance technology, head-up displays (HUDs) are widely used in various vehicle interiors. They can project information such as vehicle speed, navigation, and warnings into the driver's front using optical components in the form of 3D synthesized images.

[0004] Head-up displays (HUDs) are inherently non-uniform systems, making them highly susceptible to crosstalk. This crosstalk can cause images from the left and right eyes to overlap, thereby compromising the display of 3D composite images. The crosstalk in HUDs mainly originates from the curved edges of the lenticular lens grating. In related technologies, the crosstalk problem is usually solved by periodically darkening the curved edges of the lenticular lens grating or by globally reducing the number of illuminated pixels.

[0005] However, both of the above improvement methods will lower the display parameters such as resolution, contrast and brightness of the 3D synthesized image, which will greatly affect the display effect of the 3D synthesized image and result in a poor user experience. Summary of the Invention

[0006] This application provides an image processing method, apparatus, head-up display, and storage medium to solve the crosstalk problem of naked-eye 3D images.

[0007] According to one aspect of this application, an image processing method is provided for use in a head-up display, comprising:

[0008] The viewpoint coordinates of the target user are acquired in real time, and it is determined whether the viewpoint position of the target user has shifted.

[0009] If it is determined that the viewpoint position of the target user has shifted, the pixel grayscale of the 3D composite image is adjusted according to the current viewpoint position coordinates;

[0010] Based on the current viewpoint coordinates, the pixel brightness of the 3D composite image after pixel grayscale adjustment is adjusted, and the 3D composite image after pixel brightness adjustment is displayed.

[0011] The step of determining whether the target user's viewpoint position has shifted includes: obtaining the grid area where the target user's viewpoint is located based on the target user's viewpoint position coordinates, and determining whether the target user's viewpoint position has shifted based on whether the grid area where the target user's viewpoint is located has changed; wherein, if it is determined that the grid area where the target user's viewpoint is located has changed, it is determined that the target user's viewpoint position has shifted; if it is determined that the grid area where the target user's viewpoint is located has not changed, it is determined that the target user's viewpoint position has not shifted.

[0012] The step of adjusting the pixel grayscale of the 3D composite image according to the current viewpoint position coordinates includes: obtaining a matching target grayscale adjustment rule through a grayscale rule mapping table according to the current viewpoint position coordinates, and adjusting the pixel grayscale of the 3D composite image according to the target grayscale adjustment rule; wherein, the target grayscale adjustment rule includes the pixel identifier of the grayscale to be adjusted and the grayscale source identifier of the adjusted grayscale.

[0013] The step of obtaining a matching target grayscale adjustment rule based on the current viewpoint position coordinates through a grayscale rule mapping table specifically includes: if no matching target grayscale adjustment rule is obtained based on the current viewpoint position coordinates through the grayscale rule mapping table, obtaining the first viewpoint position coordinates that are closest to the current viewpoint position coordinates in the grayscale rule mapping table; and using the grayscale adjustment rule corresponding to the first viewpoint position coordinates as the target grayscale adjustment rule that matches the current viewpoint position coordinates.

[0014] The step of adjusting the pixel brightness of the 3D composite image after pixel grayscale adjustment based on the current viewpoint position coordinates includes: obtaining a matching target brightness adjustment rule through a brightness rule mapping table based on the current viewpoint position coordinates, and adjusting the pixel brightness of the 3D composite image after pixel grayscale adjustment according to the target brightness adjustment rule; wherein, the target brightness adjustment rule includes the pixel identifier of the brightness to be adjusted and the adjusted brightness value; the brightness of each pixel in the target brightness adjustment rule is greater than or equal to a minimum brightness threshold.

[0015] The step of adjusting the pixel brightness of the 3D composite image after pixel grayscale adjustment based on the current viewpoint position coordinates includes: reducing the brightness of crosstalk pixels and increasing the brightness of non-crosstalk similar pixels, so that the total pixel brightness of the 3D composite image after pixel brightness adjustment is equal to the total pixel brightness of the 3D composite image before pixel brightness adjustment.

[0016] According to one aspect of this application, an image processing method is provided for use in a head-up display, comprising:

[0017] The position offset detection module is configured to acquire the viewpoint position coordinates of the target user in real time and determine whether the viewpoint position of the target user has shifted.

[0018] The grayscale adjustment execution module is configured to adjust the pixel grayscale of the 3D composite image according to the current viewpoint position coordinates if it is determined that the viewpoint position of the target user has shifted.

[0019] The brightness adjustment execution module is configured to adjust the pixel brightness of the 3D composite image after pixel grayscale adjustment based on the current viewpoint position coordinates, and then display the 3D composite image after pixel brightness adjustment.

[0020] The position offset detection module is configured to obtain the grid area where the target user's viewpoint is located based on the target user's viewpoint position coordinates, and determine whether the target user's viewpoint position has shifted based on whether the grid area where the target user's viewpoint is located has changed; wherein, if it is determined that the grid area where the target user's viewpoint is located has changed, it is determined that the target user's viewpoint position has shifted; if it is determined that the grid area where the target user's viewpoint is located has not changed, it is determined that the target user's viewpoint position has not shifted.

[0021] The grayscale adjustment execution module is configured to obtain a matching target grayscale adjustment rule through a grayscale rule mapping table based on the current viewpoint position coordinates, and perform pixel grayscale adjustment on the 3D composite image according to the target grayscale adjustment rule; wherein, the target grayscale adjustment rule includes the pixel identifier of the grayscale to be adjusted and the grayscale source identifier of the adjusted grayscale.

[0022] The grayscale adjustment execution module is further configured to, if no matching target grayscale adjustment rule is obtained from the grayscale rule mapping table based on the current viewpoint position coordinates, obtain the first viewpoint position coordinates that are closest to the current viewpoint position coordinates from the grayscale rule mapping table; and use the grayscale adjustment rule corresponding to the first viewpoint position coordinates as the target grayscale adjustment rule that matches the current viewpoint position coordinates.

[0023] The brightness adjustment execution module is configured to obtain a matching target brightness adjustment rule through a brightness rule mapping table based on the current viewpoint position coordinates, and adjust the pixel brightness of the 3D composite image after pixel grayscale adjustment according to the target brightness adjustment rule; wherein, the target brightness adjustment rule includes the pixel identifier of the brightness to be adjusted and the adjusted brightness value; the brightness of each pixel in the target brightness adjustment rule is greater than or equal to the minimum brightness threshold.

[0024] The brightness adjustment execution module is further configured to reduce the brightness of crosstalk pixels and increase the brightness of non-crosstalk similar pixels, so that the total pixel brightness of the 3D composite image after pixel brightness adjustment is equal to the total pixel brightness of the 3D composite image before pixel brightness adjustment.

[0025] According to another aspect of this application, a head-up display is provided, the head-up display comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image processing method described in any embodiment of this application.

[0026] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the image processing method described in any embodiment of this application.

[0027] According to another aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the image processing method described in any embodiment of this application.

[0028] The technical solution of this application embodiment involves a head-up display that determines whether the target user's viewpoint position has shifted based on the real-time acquired viewpoint position coordinates. When it is determined that the target user's viewpoint position has shifted, the display adjusts the pixel grayscale of the 3D composite image according to the current viewpoint position coordinates. Then, based on the current viewpoint position coordinates, it adjusts the pixel brightness of the 3D composite image after the pixel grayscale adjustment to display the 3D composite image after the pixel brightness adjustment. This solves the crosstalk problem of naked-eye 3D images, avoids the overlap of left and right eye images in the viewer's eyes, and ensures that the naked-eye 3D image maintains high display parameters, thereby improving the display effect of naked-eye 3D images. Attached Figure Description

[0029] Figure 1 is a flowchart of an image processing method provided according to Embodiment 1 of this application;

[0030] Figure 2 is a schematic diagram of the grayscale value matrix before and after pixel grayscale adjustment according to Embodiment 1 of this application;

[0031] Figure 3 is a schematic diagram of the 3D composite image before and after pixel grayscale adjustment according to Embodiment 1 of this application;

[0032] Figure 4 is a schematic diagram of pixel light before pixel brightness adjustment according to Embodiment 1 of this application;

[0033] Figure 5 is a schematic diagram of pixel light after pixel brightness adjustment according to Embodiment 1 of this application;

[0034] Figure 6 is a schematic diagram of the brightness value matrix before and after pixel brightness adjustment according to Embodiment 1 of this application;

[0035] Figure 7 is a schematic diagram of the 3D composite image before and after pixel brightness adjustment according to Embodiment 1 of this application;

[0036] Figure 8 is a flowchart of an image processing method according to Embodiment 2 of this application;

[0037] Figure 9 is a flowchart of an image processing method according to Embodiment 3 of this application;

[0038] Figure 10 is a schematic diagram of an image processing apparatus according to Embodiment 4 of this application;

[0039] Figure 11 is a schematic diagram of the structure of a head-up display that implements the image processing method of the present application embodiment. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. The described embodiments are only a part of, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort should fall within the scope of protection of this application.

[0041] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0042] Example 1

[0043] Figure 1 is a flowchart of an image processing method provided in Embodiment 1 of this application. This method is applicable to situations where a head-up display (HUD) solves the crosstalk problem of 3D composite images by adjusting the pixel grayscale source and pixel brightness. This method can be executed by the image processing device in any embodiment of this application. The image processing device can be implemented in hardware and / or software and can be configured in the head-up display. As shown in Figure 1, the method includes:

[0044] S101. Obtain the viewpoint coordinates of the target user in real time, and determine whether the viewpoint position of the target user has shifted.

[0045] The head-up display can acquire the driver's (i.e., the observer's) viewpoint position in real time through a viewpoint detection device installed inside the driver's cab. The viewpoint detection device may include an eye tracker or a driver monitoring system (DMS). The viewpoint position actually refers to the position of the driver's pupils. The viewpoint detection device can acquire the driver's viewpoint position information by detecting the direction of the driver's eye movements and the direction of his gaze.

[0046] In particular, for some driving scenarios, the 3D composite images provided by the head-up display are used to provide auxiliary information for the front passenger or rear passenger. In this case, the actual user of the head-up display is the front passenger or rear passenger. Accordingly, the target user at this time specifically refers to the front passenger or rear passenger. Obviously, the viewpoint position coordinates obtained are also the viewpoint position coordinates of the front passenger or rear passenger.

[0047] After acquiring the viewpoint coordinates of the target user in real time, if the viewpoint coordinates are different between two consecutive sampling times, it indicates that the target user's viewpoint position has moved, meaning that the viewpoint position at the later sampling time has shifted compared to the viewpoint position at the earlier sampling time. If the viewpoint coordinates are the same between two consecutive sampling times, it indicates that the target user's viewpoint position has not moved, meaning that the viewpoint position at the later sampling time has not shifted compared to the viewpoint position at the earlier sampling time.

[0048] Specifically, if the viewpoint position shifts a small distance, it indicates that the user has only made a slight posture adjustment and is actually still in the same viewpoint position. In this case, there is no need to adjust the pixel grayscale and pixel brightness of the 3D composite image, meaning that the target user's viewpoint position has not shifted. The user's viewpoint position can be determined by using a pre-set viewpoint position shift distance threshold. That is, if the shift distance between the sampling times is less than the shift distance threshold, it indicates that the target user's viewpoint position has not shifted.

[0049] Similarly, if the viewpoint position moves a large distance, it means that the user has changed their viewpoint position and is located at a different viewpoint position. In this case, it is necessary to adjust the pixel grayscale and pixel brightness of the 3D composite image. That is, it is determined that the viewpoint position of the target user has shifted. In other words, if the movement distance between the sampling time points is greater than or equal to the movement distance threshold, it indicates that the viewpoint position of the target user has shifted.

[0050] Optionally, in this embodiment of the application, determining whether the viewpoint position of the target user has shifted includes: obtaining the grid area where the viewpoint of the target user is located based on the viewpoint position coordinates of the target user, and determining whether the viewpoint position of the target user has shifted based on whether the grid area where the viewpoint of the target user has changed; wherein, if it is determined that the grid area where the viewpoint of the target user has changed, it is determined that the viewpoint position of the target user has shifted; if it is determined that the grid area where the viewpoint of the target user has not changed, it is determined that the viewpoint position of the target user has not shifted.

[0051] Specifically, the interior space can be pre-divided into multiple different grid areas. Based on the viewpoint coordinates of the target user, the grid area where the current viewpoint is located can be determined. If the viewpoint coordinates obtained at two adjacent sampling times are located in the same grid area, it means that the grid area where the target user's viewpoint is located has not changed. In this case, it is determined that the viewpoint position of the target user has not shifted, and there is no need to adjust the pixel grayscale and pixel brightness of the 3D synthesized image.

[0052] If the viewpoint coordinates obtained at two adjacent sampling times are located in different grid regions, it indicates that the grid region where the target user's viewpoint is located has changed. In this case, it is determined that the target user's viewpoint position has not shifted, and there is no need to adjust the pixel grayscale and pixel brightness of the 3D composite image. Thus, by pre-dividing the vehicle interior space into multiple different grid regions, and judging whether the target user's viewpoint position has shifted based on the grid region where the target user's viewpoint is located, the coordinate point comparison is transformed into the point location region comparison, thereby improving the efficiency of viewpoint position shift detection.

[0053] S102. If it is determined that the viewpoint position of the target user has shifted, the pixel grayscale of the 3D composite image is adjusted according to the current viewpoint position coordinates.

[0054] In practical applications, due to the tolerances in the design, manufacturing and assembly of head-up displays, the naked-eye 3D images generated by image synthesis technology still have pixel crosstalk problems, requiring adjustments to the pixel arrangement.

[0055] The parallax image of the head-up display has the same resolution as the final generated 3D composite image (assuming the resolution is represented by H×V); and each pixel is actually composed of three types of pixels: R (red), G (green), and B (blue). Therefore, the pixel matrix of the parallax image and the 3D composite image can actually be represented as 3H×V, that is, each pixel is unfolded into a row vector composed of "R value", "B value", and "G value" in sequence. Taking the gray value matrix A in Figure 2 as an example, if the resolution of the current 3D composite image is "5×4", then when it is converted into the 3H×V form, it can actually form a "15×4" gray value matrix A.

[0056] In grayscale matrix A, "1", "2", "3", and "4" represent the numbers of the parallax images, indicating which parallax image the grayscale value of the current pixel originates from. The parallax images of the head-up display are pre-configured when the head-up display is assembled. Figure 2 shows an example with four parallax images for the head-up display. If it is determined that the target user's viewpoint position has shifted, for example, the user changes from viewpoint position A to viewpoint position B (viewpoint position B is the current viewpoint position coordinates), viewpoint position B can be sent to the auxiliary computing device in the vehicle, and the auxiliary computing device can calculate and obtain the corresponding pixel grayscale adjustment rules.

[0057] For example, the specific rules for adjusting the grayscale of the pixels corresponding to viewpoint position B can be as follows: change the grayscale source of the pixels in the sixth, seventh, and eighth columns of the first row to "3", "4", and "2" respectively; change the grayscale source of the pixels in the seventh, eighth, and ninth columns of the second row to "3", "4", and "2" respectively; change the grayscale source of the pixels in the eighth, ninth, and tenth columns of the third row to "3", "4", and "2" respectively; and change the grayscale source of the pixels in the ninth, tenth, and eleventh columns of the fourth row to "3", "4", and "2" respectively.

[0058] The head-up display adjusts the source of pixel grayscale values ​​in grayscale matrix A according to the aforementioned pixel grayscale adjustment rules to obtain grayscale matrix B. Grayscale matrix B is the 3D composite image after pixel grayscale adjustment. Figure 3 shows the display effect of grayscale matrix A (i.e., the 3D composite image before pixel grayscale adjustment) and the display effect of grayscale matrix B (i.e., the 3D composite image after pixel grayscale adjustment).

[0059] S103. Based on the current viewpoint position coordinates, adjust the pixel brightness of the 3D composite image after pixel grayscale adjustment, and display the 3D composite image after pixel brightness adjustment.

[0060] Due to the differences in the position of different pixels in the cylindrical grating and the existence of brightness tolerance, the energy contributed by different pixels at a given observation position varies. As shown in Figure 4, taking a two-viewpoint grating as an example, the solid arrows represent the light rays that actually need to be split by the grating, and the dashed arrows represent crosstalk rays. The light rays emitted by a pixel can reach both observation point 1 and observation point 2, but the energy is different, and the energy ratio of each pixel at observation point 1 and observation point 2 is different.

[0061] Assuming the brightness energy value of each pixel is 1, observation point 1 can observe 10% of the energy of pixel 2, 20% of the energy of pixel 3, 80% of the energy of pixel 4, 90% of the energy of pixel 5, and 100% of the energy of pixel 6. The total brightness value that observation point 1 can receive is 3; among them, the brightness of 0.3 is crosstalk, and the brightness of 2.7 is non-crosstalk, with the crosstalk accounting for 10%.

[0062] Similarly, observation point 2 can observe 100% of the energy of pixel 1, 90% of the energy of pixel 2, 80% of the energy of pixel 3, 20% of the energy of pixel 4, and 10% of the energy of pixel 5. The total brightness value that observation point 2 can receive is 3; among them, 0.3 brightness is crosstalk, and 2.7 brightness is non-crosstalk, with crosstalk accounting for 10%.

[0063] By reducing the brightness of crosstalk pixels, the crosstalk ratio at the observation point can be reduced. Taking Figure 5 as an example, the energy of pixels 2, 3, 4, and 5 is adjusted to 0.8, 0.6, 0.6, and 0.8, respectively. At this time, the total brightness value that observation point 1 can receive is 2.4. Among them, 0.2 brightness is crosstalk, and 2.2 brightness is non-crosstalk, with a crosstalk ratio of 8.3%. The total brightness value that observation point 2 can receive is 2.4. Among them, 0.2 brightness is crosstalk, and 2.2 brightness is non-crosstalk, with a crosstalk ratio of 8.3%. Obviously, by reducing the brightness of crosstalk pixels, the crosstalk of the 3D composite image is reduced.

[0064] The head-up display (HUD) can send the current viewpoint coordinates to an auxiliary computing device inside the vehicle, which then calculates and obtains the corresponding pixel brightness adjustment rules. As shown in Figure 6, the HUD adjusts the pixel brightness in the brightness value matrix C according to the aforementioned pixel grayscale adjustment rules, thereby obtaining the brightness value matrix D. The brightness value matrix D is the 3D composite image after pixel brightness adjustment. Figure 7 shows the display effect of brightness value matrix A (i.e., the 3D composite image before pixel brightness adjustment) and the display effect of brightness value matrix B (i.e., the 3D composite image after pixel brightness adjustment), respectively.

[0065] Optionally, in this embodiment of the application, adjusting the pixel brightness of the 3D composite image after pixel grayscale adjustment according to the current viewpoint position coordinates includes: reducing the brightness of crosstalk pixels and increasing the brightness of non-crosstalk similar pixels, so that the total pixel brightness of the 3D composite image after pixel brightness adjustment is equal to the total pixel brightness of the 3D composite image before pixel brightness adjustment.

[0066] Specifically, the pixel brightness adjustment rule reduces the brightness of pixels causing crosstalk while increasing the brightness of pixels not causing crosstalk, ensuring that the total pixel brightness of the 3D composite image remains unchanged before and after the adjustment. Taking the above technical solution as an example, after adjusting the energy of pixels 2, 3, 4, and 5 to 0.8, 0.6, 0.6, and 0.8 respectively, the energy of pixels 1 and 3 is simultaneously adjusted to 1.6. At this point, the total brightness value that observation point 1 can receive is still 3; of which 0.2 brightness is crosstalk and 2.8 brightness is non-crosstalk, with a crosstalk percentage of 6.7%. Observation point 2 can also receive a total brightness value of 3; of which 0.2 brightness is crosstalk and 2.8 brightness is non-crosstalk, with a crosstalk percentage of 6.7%. Therefore, while reducing crosstalk in the 3D composite image, the brightness of the image observed by the user from different viewing angles remains consistent, improving the display effect of the naked-eye 3D image.

[0067] The technical solution of this application embodiment involves a head-up display that determines whether the target user's viewpoint position has shifted based on the real-time acquired viewpoint position coordinates. When it is determined that the target user's viewpoint position has shifted, the display adjusts the pixel grayscale of the 3D composite image according to the current viewpoint position coordinates. Then, based on the current viewpoint position coordinates, it adjusts the pixel brightness of the 3D composite image after the pixel grayscale adjustment to display the 3D composite image after the pixel brightness adjustment. This solves the crosstalk problem of naked-eye 3D images, avoids the overlap of left and right eye images in the viewer's eyes, and ensures that the naked-eye 3D image maintains high display parameters, thereby improving the display effect of naked-eye 3D images.

[0068] Example 2

[0069] Figure 8 is a flowchart of an image processing method provided in Embodiment 2 of this application. The relationship between this embodiment and the above embodiments is that the head-up display records the grayscale adjustment rules for each viewpoint position through a grayscale rule mapping table. As shown in Figure 4, the method includes:

[0070] S201. Obtain the viewpoint coordinates of the target user in real time, and determine whether the viewpoint position of the target user has shifted.

[0071] S202. If it is determined that the viewpoint position of the target user has shifted, the target grayscale adjustment rule is obtained through the grayscale rule mapping table according to the current viewpoint position coordinates, and the pixel grayscale of the 3D composite image is adjusted according to the target grayscale adjustment rule; wherein, the target grayscale adjustment rule includes the pixel identifier of the grayscale to be adjusted and the grayscale source identifier after adjustment.

[0072] The grayscale rule mapping table is pre-built using a large amount of test data. It records the pixel identifiers of the grayscale to be adjusted and the source identifiers of the adjusted grayscale at different viewpoints. In other words, it specifies which pixels need to have their grayscale values ​​adjusted and from which parallax image the adjusted grayscale value should originate. Therefore, after obtaining the viewpoint coordinates at the current moment, the matching target grayscale adjustment rule can be obtained based on these coordinates, thereby completing the pixel grayscale adjustment of the 3D composite image. This improves the efficiency of obtaining pixel grayscale adjustment results while adjusting the pixel arrangement of the 3D composite image, ensuring the real-time display of the 3D composite image.

[0073] Optionally, in this embodiment of the application, the step of obtaining a matching target grayscale adjustment rule through a grayscale rule mapping table based on the current viewpoint position coordinates specifically includes: if no matching target grayscale adjustment rule is obtained through the grayscale rule mapping table based on the current viewpoint position coordinates, obtaining the first viewpoint position coordinates that are closest to the current viewpoint position coordinates in the grayscale rule mapping table; and using the grayscale adjustment rule corresponding to the first viewpoint position coordinates as the target grayscale adjustment rule that matches the current viewpoint position coordinates.

[0074] Specifically, when constructing the grayscale rule mapping table, it may not be possible to plan grayscale adjustment rules for all viewpoint positions, or the user's actual viewpoint position may exceed the viewpoint range recorded in the grayscale rule mapping table. In this case, by comparing the current viewpoint position coordinates with each viewpoint position recorded in the grayscale rule mapping table, the coordinates of the nearest first viewpoint position are obtained, and the grayscale adjustment rule corresponding to the first viewpoint position coordinates is used as the target grayscale adjustment rule to match the current viewpoint position coordinates. This allows for the acquisition of similar grayscale adjustment rules by means of proximity when an accurate grayscale adjustment rule cannot be obtained through the grayscale rule mapping table. This minimizes the crosstalk effect of naked-eye 3D images and improves the display effect of naked-eye 3D images.

[0075] Furthermore, as described in the above technical solution, if the interior space has been pre-divided into multiple different grid areas, then the grayscale rule mapping table records the different grid areas and their corresponding grayscale adjustment rules. At this time, after determining the matching grid area based on the current viewpoint position coordinates, the grayscale adjustment rule corresponding to that grid area is used as the target grayscale adjustment rule that matches the current viewpoint position coordinates.

[0076] S203. Based on the current viewpoint position coordinates, adjust the pixel brightness of the 3D composite image after pixel grayscale adjustment, and display the 3D composite image after pixel brightness adjustment.

[0077] The technical solution of this application embodiment is that when the head-up display determines that the viewpoint position of the target user has shifted, it obtains the matching target grayscale adjustment rule through the grayscale rule mapping table according to the current viewpoint position coordinates, and performs pixel grayscale adjustment on the 3D composite image according to the target grayscale adjustment rule. In this way, while realizing the pixel arrangement adjustment of the 3D composite image, the efficiency of obtaining the pixel grayscale adjustment result is improved, ensuring the real-time display of the 3D composite image.

[0078] Example 3

[0079] Figure 9 is a flowchart of an image processing method provided in Embodiment 3 of this application. The relationship between this embodiment and the above embodiments is that the head-up display records the brightness adjustment rules for each viewpoint position through a brightness rule mapping table. As shown in Figure 9, the method includes:

[0080] S301. Obtain the viewpoint coordinates of the target user in real time, and determine whether the viewpoint position of the target user has shifted.

[0081] S302. If it is determined that the viewpoint position of the target user has shifted, the pixel grayscale of the 3D composite image is adjusted according to the current viewpoint position coordinates.

[0082] S303. Based on the current viewpoint position coordinates, obtain the matching target brightness adjustment rule through the brightness rule mapping table, and adjust the pixel brightness of the 3D composite image after pixel grayscale adjustment according to the target brightness adjustment rule; wherein, the target brightness adjustment rule includes the pixel identifier of the brightness to be adjusted and the adjusted brightness value; the brightness of each pixel in the target brightness adjustment rule is greater than or equal to the minimum brightness threshold.

[0083] The brightness rule mapping table is also pre-built using a large amount of test data. It records the pixel identifiers of the pixels whose brightness needs to be adjusted and the adjusted brightness values ​​at different viewpoint positions. In other words, it specifies which pixels' grayscale values ​​need to be adjusted and what the adjusted brightness value of each pixel should be. Therefore, after obtaining the viewpoint position coordinates at the current moment, the matching target brightness adjustment rules can be obtained based on these viewpoint position coordinates, and then pixel brightness adjustment can be performed on the 3D composite image after pixel grayscale adjustment. This improves the efficiency of obtaining pixel brightness adjustment results while realizing pixel brightness adjustment of the 3D composite image, ensuring the real-time display of the 3D composite image.

[0084] Specifically, a minimum brightness threshold can be preset. For various brightness adjustment rules, the brightness of each pixel must be greater than or equal to the minimum brightness threshold to avoid some pixels on the display screen being too dim, affecting the viewing effect of the 3D composite image. At the same time, a maximum brightness threshold can also be preset. For various brightness adjustment rules, the brightness of each pixel must be less than or equal to the maximum brightness threshold to avoid some pixels on the display screen being too bright, thereby improving the display uniformity of the 3D composite image.

[0085] Optionally, in this embodiment of the application, the step of obtaining a matching target brightness adjustment rule through a brightness rule mapping table based on the current viewpoint position coordinates specifically includes: if no matching target brightness adjustment rule is obtained through the brightness rule mapping table based on the current viewpoint position coordinates, obtaining the second viewpoint position coordinates that are closest to the current viewpoint position coordinates in the brightness rule mapping table; and using the brightness adjustment rule corresponding to the second viewpoint position coordinates as the target brightness adjustment rule that matches the current viewpoint position coordinates.

[0086] Specifically, when constructing the brightness rule mapping table, it may not be possible to plan brightness adjustment rules for all viewpoint positions, or the user's actual viewpoint position may exceed the viewpoint range recorded in the brightness rule mapping table. In this case, by comparing the current viewpoint position coordinates with the various viewpoint positions recorded in the brightness rule mapping table, the coordinates of the nearest second viewpoint position are obtained, and the brightness adjustment rule corresponding to the second viewpoint position coordinates is used as the target brightness adjustment rule to match the current viewpoint position coordinates. This allows for the acquisition of similar brightness adjustment rules by means of proximity when an accurate brightness adjustment rule cannot be obtained through the brightness rule mapping table, thereby minimizing the crosstalk effect of naked-eye 3D images and improving the display effect of naked-eye 3D images.

[0087] Furthermore, as described in the above technical solution, if the interior space has been pre-divided into multiple different grid areas, then the brightness rule mapping table records the different grid areas and their corresponding brightness adjustment rules. At this time, after determining the matching grid area based on the current viewpoint position coordinates, the brightness adjustment rule corresponding to that grid area is used as the target brightness adjustment rule that matches the current viewpoint position coordinates.

[0088] S304. Display the 3D composite image after pixel brightness adjustment.

[0089] The technical solution of this application embodiment is that when the head-up display determines that the viewpoint position of the target user has shifted, it obtains the matching target brightness adjustment rule through the brightness rule mapping table according to the current viewpoint position coordinates, and adjusts the pixel brightness of the 3D composite image according to the target brightness adjustment rule. Thus, while realizing the pixel brightness adjustment of the 3D composite image, it improves the efficiency of obtaining the pixel brightness adjustment result and ensures the real-time display of the 3D composite image.

[0090] Example 4

[0091] Figure 10 is a structural block diagram of an image processing apparatus provided in Embodiment 4 of this application. The image processing apparatus specifically includes:

[0092] The position offset detection module 401 is configured to acquire the viewpoint position coordinates of the target user in real time and determine whether the viewpoint position of the target user has shifted.

[0093] The grayscale adjustment execution module 402 is configured to adjust the pixel grayscale of the 3D composite image according to the current viewpoint position coordinates if it is determined that the viewpoint position of the target user has shifted.

[0094] The brightness adjustment execution module 403 is configured to adjust the pixel brightness of the 3D composite image after pixel grayscale adjustment based on the current viewpoint position coordinates, and display the 3D composite image after pixel brightness adjustment.

[0095] The technical solution of this application embodiment involves a head-up display that determines whether the target user's viewpoint position has shifted based on the real-time acquired viewpoint position coordinates. When it is determined that the target user's viewpoint position has shifted, the display adjusts the pixel grayscale of the 3D composite image according to the current viewpoint position coordinates. Then, based on the current viewpoint position coordinates, it adjusts the pixel brightness of the 3D composite image after the pixel grayscale adjustment to display the 3D composite image after the pixel brightness adjustment. This solves the crosstalk problem of naked-eye 3D images, avoids the overlap of left and right eye images in the viewer's eyes, and ensures that the naked-eye 3D image maintains high display parameters, thereby improving the display effect of naked-eye 3D images.

[0096] The position offset detection module 401 is configured to obtain the grid area where the target user's viewpoint is located based on the target user's viewpoint position coordinates, and determine whether the target user's viewpoint position has shifted based on whether the grid area where the target user's viewpoint is located has changed; wherein, if it is determined that the grid area where the target user's viewpoint is located has changed, it is determined that the target user's viewpoint position has shifted; if it is determined that the grid area where the target user's viewpoint is located has not changed, it is determined that the target user's viewpoint position has not shifted.

[0097] The grayscale adjustment execution module 402 is configured to obtain a matching target grayscale adjustment rule through a grayscale rule mapping table based on the current viewpoint position coordinates, and perform pixel grayscale adjustment on the 3D composite image according to the target grayscale adjustment rule; wherein, the target grayscale adjustment rule includes the pixel identifier of the grayscale to be adjusted and the grayscale source identifier of the adjusted grayscale.

[0098] The grayscale adjustment execution module 402 is further configured to, if no matching target grayscale adjustment rule is obtained from the grayscale rule mapping table based on the current viewpoint position coordinates, obtain the first viewpoint position coordinates that are closest to the current viewpoint position coordinates in the grayscale rule mapping table; and use the grayscale adjustment rule corresponding to the first viewpoint position coordinates as the target grayscale adjustment rule that matches the current viewpoint position coordinates.

[0099] The brightness adjustment execution module 403 is configured to obtain a matching target brightness adjustment rule through a brightness rule mapping table based on the current viewpoint position coordinates, and adjust the pixel brightness of the 3D composite image after pixel grayscale adjustment according to the target brightness adjustment rule; wherein, the target brightness adjustment rule includes the pixel identifier of the brightness to be adjusted and the adjusted brightness value; the brightness of each pixel in the target brightness adjustment rule is greater than or equal to the minimum brightness threshold.

[0100] The brightness adjustment execution module 403 is further configured to reduce the brightness of crosstalk pixels and increase the brightness of non-crosstalk similar pixels, so that the total pixel brightness of the 3D composite image after pixel brightness adjustment is equal to the total pixel brightness of the 3D composite image before pixel brightness adjustment.

[0101] The image processing apparatus provided in this application can execute the image processing method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the image processing method provided in any embodiment of this application.

[0102] Example 5

[0103] Figure 11 shows a schematic diagram of the structure of a head-up display 10 that can be used to implement embodiments of the present application. The head-up display is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The head-up display can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0104] As shown in Figure 11, the head-up display 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the head-up display 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0105] Multiple components in the head-up display 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, optical disk, etc.; and a communication unit 19, such as a network card, modem, wireless transceiver, etc. The communication unit 19 allows the head-up display 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0106] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as image processing methods.

[0107] In some embodiments, the image processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the image processing method by any other suitable means (e.g., by means of firmware).

[0108] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0109] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable image processing apparatus, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0110] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0111] To provide user interaction, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).

[0112] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0113] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

Claims

1. An image processing method applied to a head-up display, comprising: The viewpoint coordinates of the target user are acquired in real time, and it is determined whether the viewpoint position of the target user has shifted. If it is determined that the viewpoint position of the target user has shifted, the pixel grayscale of the 3D composite image is adjusted according to the current viewpoint position coordinates; Based on the current viewpoint coordinates, the pixel brightness of the 3D composite image after pixel grayscale adjustment is adjusted, and the 3D composite image after pixel brightness adjustment is displayed.

2. The method according to claim 1, wherein, The step of determining whether the target user's viewpoint position has shifted includes: Based on the target user's viewpoint position coordinates, obtain the grid area where the target user's viewpoint is located, and determine whether the target user's viewpoint position has shifted based on whether the grid area where the target user's viewpoint is located has changed. Specifically, if it is determined that the grid area where the target user's viewpoint has changed has been changed, it is determined that the target user's viewpoint position has shifted; if it is determined that the grid area where the target user's viewpoint has not changed has not been changed, it is determined that the target user's viewpoint position has not shifted.

3. The method according to claim 1, wherein, The step of adjusting the pixel grayscale of the 3D composite image based on the current viewpoint position coordinates includes: Based on the current viewpoint position coordinates, the matching target grayscale adjustment rule is obtained through the grayscale rule mapping table, and the pixel grayscale of the 3D composite image is adjusted according to the target grayscale adjustment rule. The target grayscale adjustment rule includes the pixel identifier of the grayscale to be adjusted and the source identifier of the adjusted grayscale.

4. The method according to claim 3, wherein, The step of obtaining the matching target grayscale adjustment rule based on the current viewpoint position coordinates through a grayscale rule mapping table includes: If no matching target grayscale adjustment rule is found in the grayscale rule mapping table based on the current viewpoint position coordinates, obtain the first viewpoint position coordinates that are closest to the current viewpoint position coordinates in the grayscale rule mapping table. The grayscale adjustment rule corresponding to the first viewpoint position coordinates is used as the target grayscale adjustment rule to match the current viewpoint position coordinates.

5. The method according to claim 1, wherein, The step of adjusting the pixel brightness of the 3D composite image after pixel grayscale adjustment based on the current viewpoint position coordinates includes: Based on the current viewpoint position coordinates, the matching target brightness adjustment rule is obtained through the brightness rule mapping table, and the pixel brightness of the 3D composite image after pixel grayscale adjustment is adjusted according to the target brightness adjustment rule. The target brightness adjustment rule includes the pixel identifier of the brightness to be adjusted and the adjusted brightness value; the brightness of each pixel in the target brightness adjustment rule is greater than or equal to the minimum brightness threshold.

6. The method according to claim 1, wherein, The step of adjusting the pixel brightness of the 3D composite image after pixel grayscale adjustment based on the current viewpoint position coordinates includes: Reduce the brightness of crosstalk pixels and increase the brightness of non-crosstalk similar pixels so that the total pixel brightness of the 3D composite image after pixel brightness adjustment is equal to the total pixel brightness of the 3D composite image before pixel brightness adjustment.

7. An image processing apparatus for use in a head-up display, comprising: The position offset detection module is configured to acquire the viewpoint position coordinates of the target user in real time and determine whether the viewpoint position of the target user has shifted. The grayscale adjustment execution module is configured to adjust the pixel grayscale of the 3D composite image according to the current viewpoint coordinates if it is determined that the viewpoint position of the target user has shifted. The brightness adjustment execution module is configured to adjust the pixel brightness of the 3D composite image after pixel grayscale adjustment based on the current viewpoint position coordinates, and then display the 3D composite image after pixel brightness adjustment.

8. The apparatus according to claim 7, wherein, The position offset detection module is configured to obtain the grid area where the target user's viewpoint is located based on the target user's viewpoint position coordinates, and determine whether the target user's viewpoint position has shifted based on whether the grid area where the target user's viewpoint is located has changed. Specifically, if it is determined that the grid area where the target user's viewpoint has changed has been changed, it is determined that the target user's viewpoint position has shifted; if it is determined that the grid area where the target user's viewpoint has not changed has not been changed, it is determined that the target user's viewpoint position has not shifted.

9. The apparatus according to claim 7, wherein, The grayscale adjustment execution module is configured to obtain the matching target grayscale adjustment rule through a grayscale rule mapping table based on the current viewpoint position coordinates, and perform pixel grayscale adjustment on the 3D composite image according to the target grayscale adjustment rule; The target grayscale adjustment rule includes the pixel identifier of the grayscale to be adjusted and the source identifier of the adjusted grayscale.

10. The apparatus according to claim 9, wherein, The grayscale adjustment execution module is further configured to, if no matching target grayscale adjustment rule is obtained from the grayscale rule mapping table based on the current viewpoint position coordinates, obtain the first viewpoint position coordinates that are closest to the current viewpoint position coordinates from the grayscale rule mapping table; and use the grayscale adjustment rule corresponding to the first viewpoint position coordinates as the target grayscale adjustment rule that matches the current viewpoint position coordinates.

11. The apparatus according to claim 7, wherein, The brightness adjustment execution module is configured to obtain a matching target brightness adjustment rule through a brightness rule mapping table based on the current viewpoint position coordinates, and adjust the pixel brightness of the 3D composite image after pixel grayscale adjustment according to the target brightness adjustment rule. The target brightness adjustment rule includes the pixel identifier of the brightness to be adjusted and the adjusted brightness value; the brightness of each pixel in the target brightness adjustment rule is greater than or equal to the minimum brightness threshold.

12. The apparatus according to claim 7, wherein, The brightness adjustment execution module is further configured to reduce the brightness of crosstalk pixels and increase the brightness of non-crosstalk similar pixels, so that the total pixel brightness of the 3D composite image after pixel brightness adjustment is equal to the total pixel brightness of the 3D composite image before pixel brightness adjustment.

13. A head-up display, the head-up display comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the image processing method according to any one of claims 1-6.

14. A computer-readable storage medium storing computer instructions for causing a processor to perform the image processing method of any one of claims 1-6.

15. A computer program product comprising a computer program that, when executed by a processor, implements the image processing method according to any one of claims 1-6.

Images