Image distortion correction method and device, equipment and medium

Abstract

The invention relates to an image distortion correction method and device, equipment and a medium. The method comprises the following steps: determining a mapping function between a physical parameter and an optical distortion parameter of a heat dissipation layer; wherein the mapping function is used for predicting the parameter value of the optical distortion parameter according to the parameter value of the physical parameter of the heat dissipation layer; taking the parameter value of the optical distortion parameter smaller than or equal to a set value as an optimization target, and reversely solving the mapping function to obtain a target parameter value of the physical parameter of the heat dissipation layer; manufacturing a heat dissipation layer with a target parameter value according to the target parameter value of the physical parameter, and integrating the manufactured heat dissipation layer into the HUD, so that image distortion introduced by the heat dissipation layer is corrected through the heat dissipation layer in the operation process of the HUD; and monitoring a display image in the running process of the HUD, and adjusting a parameter value of a driving parameter of an image generation unit in the HUD according to the display image so as to carry out image distortion compensation.

Description

Technical Field

[0001] This disclosure relates to the field of head-up display technology, and more specifically, to an image distortion correction method, apparatus, device, and medium. Background Technology

[0002] Currently, HUDs (Head-Up Displays) are widely used in aviation, automotive, and other fields. HUDs project critical vehicle information, such as speed and navigation data, onto the windshield in front of the driver, reducing the need to shift eyes and improving safety and operational efficiency.

[0003] Because the image generation unit (PGU) of a HUD typically generates a lot of heat during operation, a heat dissipation layer is needed to cool it down. However, adding a heat dissipation layer alters the original optical path, causing image distortion in the final image projected onto the windshield. Summary of the Invention

[0004] This disclosure provides a novel technical solution for image distortion correction.

[0005] Firstly, this embodiment provides an image distortion correction method, including: Determine a mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters; wherein the mapping function is used to predict the parameter values ​​of the optical distortion parameters based on the parameter values ​​of the physical parameters of the heat dissipation layer; With the optimization objective of the optical distortion parameter being less than or equal to a set value, the mapping function is solved in reverse to obtain the target parameter value of the physical parameter of the heat dissipation layer; A heat dissipation layer with the target parameter value is manufactured according to the target parameter value of the physical parameters, and the manufactured heat dissipation layer is integrated into the HUD, so that the image distortion introduced by the heat dissipation layer is corrected during the operation of the HUD; During the operation of the HUD, the displayed image is monitored, and the parameter values ​​of the driving parameters of the image generation unit in the HUD are adjusted according to the displayed image to perform image distortion compensation.

[0006] In some embodiments, the step of inversely solving the mapping function to obtain the target parameter value of the physical parameters of the heat dissipation layer, with the parameter value of the optical distortion parameter being less than or equal to a set value as the optimization objective, includes: Obtain the distortion type of the input image; Based on the distortion type, and under set constraints, with the parameter value of the optical distortion parameter being less than or equal to a set value as the optimization objective, the mapping function is solved in reverse to obtain the target parameter value of the physical parameter.

[0007] In some embodiments, the set constraints include heat dissipation constraints and processing feasibility constraints; The heat dissipation constraint condition includes that the thermal conductivity of the heat dissipation layer is greater than a set thermal conductivity. The processing feasibility constraints include that the thickness of the heat dissipation layer is greater than or equal to a set thickness, and that the surface shape of the optical surface of the heat dissipation layer is within a preset processing capability range.

[0008] In some embodiments, obtaining the image distortion type of the displayed image includes: Identify image features in the displayed image; Based on the identified image features, search for distortion type features that match the image features in the distortion type feature library; The distortion type of the displayed image is determined based on the matched distortion type features.

[0009] In some embodiments, determining the mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters includes: Without a heat dissipation layer installed in the HUD, the image generation unit is activated to project a test image, the initial optical path data of the light in the HUD is collected by an optical measurement instrument, and the initial display image is collected by an image acquisition device. When a heat dissipation layer with the set parameter values ​​of the physical parameters is installed in the HUD, the optical measurement instrument collects the current optical path data of light passing through the heat dissipation layer, and the image acquisition device collects the corresponding distorted display image. Training samples are generated based on the set values ​​of the physical parameters, the initial display image, the initial optical path data, the current optical path data, and the distorted display image. The mapping function between the physical parameters and optical distortion parameters of the heat dissipation layer is obtained by training the training samples.

[0010] In some embodiments, generating training samples based on the set parameter values ​​of the physical parameters, the initial display image, the initial optical path data, the current optical path data, and the distorted display image includes: Obtain the first deviation between the initial display image and the distorted display image; Obtain the second deviation between the initial optical path data and the current optical path data; Based on the first deviation and the second deviation, the parameter values ​​of the optical distortion parameters are generated under the set parameter values ​​of the physical parameters; The training samples are generated based on the set values ​​of the physical parameters and the corresponding values ​​of the optical distortion parameters.

[0011] In some embodiments, adjusting the driving parameters of the image generation unit in the HUD according to the displayed image to perform image distortion compensation includes: Identify the current coordinates of the preset marker points in the displayed image; Based on the current coordinates and the reference coordinates of the preset marker point, an image offset is generated; Based on the image offset, an adjustment amount matching the image offset is found from the set offset adjustment amount table; wherein, the set offset adjustment amount table stores the correspondence between the image offset and the adjustment amount of the driving parameter; The parameter values ​​of the driving parameters of the image generation unit in the HUD are adjusted according to the adjustment amount to perform image distortion compensation.

[0012] Secondly, this embodiment provides an image distortion correction device, including: A determination module is used to determine the mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters; wherein, the mapping function is used to predict the parameter values ​​of the optical distortion parameters based on the parameter values ​​of the physical parameters of the heat dissipation layer; The solution module is used to perform inverse solution on the mapping function with the parameter value of the optical distortion parameter being less than or equal to a set value as the optimization objective, so as to obtain the target parameter value of the physical parameter of the heat dissipation layer; A manufacturing module is used to manufacture a heat dissipation layer having the target parameter value according to the target parameter value of the physical parameters, and to integrate the manufactured heat dissipation layer into the HUD, so that the heat dissipation layer can correct the image distortion introduced by itself during the operation of the HUD; An adjustment module is used to monitor the displayed image during the operation of the HUD and adjust the parameter values ​​of the driving parameters of the image generation unit in the HUD according to the displayed image in order to perform image distortion compensation.

[0013] Thirdly, this embodiment provides an electronic device, including a memory and a processor. The memory is used to store computer instructions, and the processor is used to retrieve the computer instructions from the memory to perform the method as described in the first aspect.

[0014] Fourthly, this embodiment provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the method described in the first aspect.

[0015] The image distortion correction method of this disclosure can establish a mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters, and solve the mapping function in reverse with the parameter value of the optical distortion parameter being less than or equal to a set value as the optimization target to obtain the target parameter value of the physical parameters of the heat dissipation layer. In this way, a heat dissipation layer with the target parameter value can be manufactured, so that the heat dissipation layer can correct the image distortion it introduces during the operation of the HUD. By optimizing the parameter value of the physical parameters of the heat dissipation layer, it can correct the optical path to compensate for the image distortion it introduces while ensuring the heat dissipation function. Furthermore, by monitoring the displayed image and adjusting the driving parameters of the image generation unit in the HUD to perform image distortion compensation during the operation of the HUD, it can ensure that the driver always sees a clear and distortion-free display image, and ensure the stability of the correction effect in dynamic driving scenarios.

[0016] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure and, together with their description, serve to explain the principles of the present disclosure.

[0018] Figure 1 A perspective view from the driver's seat of a vehicle, provided in an embodiment of this disclosure, is shown.

[0019] Figure 2 A schematic diagram of an image distortion correction method provided in an embodiment of this disclosure is shown.

[0020] Figure 3 A schematic diagram of the structure of the image generation unit and the heat dissipation layer provided in an embodiment of this disclosure is shown.

[0021] Figure 4 One of the schematic diagrams of the relationship structure of the HUD test scenario provided in the embodiments of this disclosure is shown.

[0022] Figure 5 The second schematic diagram of the relationship structure of the HUD test scenario provided in the embodiments of this disclosure is shown.

[0023] Figure 6 A schematic diagram of an image distortion correction apparatus provided in an embodiment of this disclosure is shown.

[0024] Figure 7 A schematic diagram of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0025] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0026] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0027] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0028] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0029] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0030] Currently, HUDs (Head-Up Displays) are widely used in aviation, automotive, and other fields. HUDs project critical vehicle information, such as speed and navigation data, onto the windshield in front of the driver, reducing the need to shift eyes and improving safety and operational efficiency.

[0031] Reference Figure 1 The HUD can be installed on a vehicle, which may include a windshield 101 located at the front of the vehicle, through which the driver in the vehicle's cabin can see ahead of the vehicle. The windshield 101 may be visually positioned above the vehicle's instrument panel 102, and the driver can turn the steering wheel 103 to turn the vehicle from inside the vehicle's cabin. The HUD can project display information 105 onto the windshield 101 through one or more holes (such as hole 104) in the instrument panel 102.

[0032] However, since the image generation unit of a HUD typically generates a lot of heat during operation, a heat dissipation layer is needed to cool it down. However, adding a heat dissipation layer alters the original optical path, causing image distortion in the final image projected onto the windshield.

[0033] In related technologies, image distortion correction can be performed by adjusting the angle of the image generation unit and software image pre-distortion. On the one hand, the accuracy requirements are high and the response is slow. On the other hand, the cost is high and it cannot fundamentally solve the optical path changes and image distortion caused by the heat dissipation layer.

[0034] Based on the above analysis, such as Figure 2 As shown, this disclosure provides an image distortion correction method, including the following steps S2100 to S2400: Step S2100: Determine the mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters.

[0035] The physical parameters of the heat dissipation layer can include the surface shape parameters of its optical surface, which is typically a freeform surface. The physical parameters of the heat dissipation layer can also include its thickness, refractive index, and mounting angle.

[0036] The optical distortion parameters may include radial distortion parameters and tangential distortion parameters. The radial distortion parameters may include a first radial distortion parameter k1, a second radial distortion parameter k2, and a third radial distortion parameter k3. The tangential distortion parameters may include a first tangential distortion parameter p1 and a second tangential distortion parameter p2. For example, the optical distortion parameters may include {k1, k2, k3, p1, p2}.

[0037] The mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters can be used to predict the values ​​of the optical distortion parameters based on the values ​​of the physical parameters of the heat dissipation layer. This mapping function... The independent variable is the physical parameter X of the heat dissipation layer, and the dependent variable is... This refers to the optical distortion parameter determined by the physical parameter X of the heat dissipation layer. In other words, the input of the mapping function can be the parameter value of the physical parameter of the heat dissipation layer, and the output of the mapping function can be the parameter value of the optical distortion parameter.

[0038] For example, the physical parameter X of the heat dissipation layer = ,in, It can represent the surface shape parameters of the optical surface of the heat dissipation layer. It can indicate the thickness of the heat dissipation layer. It can represent the refractive index of the heat dissipation layer. This can represent the mounting angle of the heat dissipation layer. Mapping function. The independent variable is the physical parameter X of the heat dissipation layer. Mapping function The dependent variable is the optical distortion parameters {k1, k2, k3, p1, p2} determined by the physical parameters X of the heat dissipation layer.

[0039] After performing step S2100 above to determine the mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters, proceed to: Step S2200: With the optimization objective of the optical distortion parameter being less than or equal to a set value, the mapping function is solved in reverse to obtain the target parameter value of the physical parameters of the heat dissipation layer.

[0040] The target parameter values ​​of the physical parameters can be the optimal values ​​of the physical parameters. These target values ​​may include a first target value for the surface shape, a second target value for the thickness, a third target value for the refractive index, and a fourth target value for the mounting angle. In this embodiment, the heat dissipation layer manufactured according to the target parameter values ​​of the physical parameters has the first target value for the surface shape, the second target value for the thickness, the third target value for the refractive index, and the fourth target value for the mounting angle.

[0041] The above settings can be set according to the actual scenario and application. The optimization objective of the reverse solution of the mapping function is that the parameter value of the optical distortion parameter is less than or equal to the set value. It can also be understood as taking the minimization of the parameter value of the optical distortion parameter as the optimization objective, and calculating the target parameter value of the physical parameter of the heat dissipation layer that can offset the distortion. Then, the heat dissipation layer is processed and customized and integrated into the HUD.

[0042] In one example, step S2200, which optimizes the optical distortion parameter value to be less than or equal to a set value, performs an inverse solution on the mapping function to obtain the target parameter value of the physical parameters of the heat dissipation layer. This step may further include: under set constraints, optimizing the optical distortion parameter value to be less than or equal to a set value, performing an inverse solution on the mapping function to obtain the target parameter value of the physical parameters.

[0043] The constraints can include heat dissipation constraints and processing feasibility constraints. Heat dissipation constraints can include a thermal conductivity of the heat dissipation layer exceeding a set thermal conductivity. Processing feasibility constraints can include a thickness of the heat dissipation layer exceeding or equal to a set thickness, and the surface shape of the optical surface of the heat dissipation layer being within a preset processing capability range.

[0044] The thermal conductivity setting mentioned above can be adjusted based on practical experience and the specific scenario. For example, the thermal conductivity could be set to 10 W / (m²). K), the thermal conductivity of the heat dissipation layer produced is greater than 10 W / (m²). In the case of K), the heat dissipation layer can meet the heat dissipation requirements, indicating that heat dissipation was not sacrificed in order to correct the optical path.

[0045] The thickness setting can be adjusted based on actual experience and the scenario; for example, the thickness can be set to 2mm.

[0046] The surface shape of the aforementioned heat dissipation layer optical surface, within the preset processing capability range, may include surface curvature, asphericity, and steepness of the heat dissipation layer optical surface, all of which are within the processing limit capability range of the processing equipment.

[0047] In this example, the objective function can be to make the optical distortion parameter approach 0. Under the above-mentioned constraints, the mapping function can be solved in reverse to obtain the optimal first parameter value of the surface shape parameter, the optimal second parameter value of the thickness, the optimal third parameter value of the refractive index, and the optimal fourth parameter value of the mounting angle.

[0048] In one example, step S2200, with the optimization objective of the optical distortion parameter value being less than or equal to a set value, performs an inverse solution on the mapping function to obtain the target parameter value of the physical parameters of the heat dissipation layer. This step may further include: obtaining the distortion type of the input image; based on the distortion type, under set constraints, with the optimization objective of the optical distortion parameter value being less than or equal to a set value, performing an inverse solution on the mapping function to obtain the target parameter value of the physical parameters.

[0049] The input image can be a display image whose distortion needs to be corrected. Distortion types can include radial distortion, tangential distortion, and mixed distortion.

[0050] In this example, obtaining the distortion type of the input image can be achieved in the following way: identify the image features in the displayed image, search for the distortion type features that match the image features in the distortion type feature library based on the identified image features, and determine the distortion type of the displayed image based on the matched distortion type features.

[0051] Image features can include radial distortion features and tangential distortion features.

[0052] The aforementioned distortion type feature library stores distortion type features corresponding to different distortion types. By comparing the extracted image features with the distortion type feature library, the distortion type features that match the image features can be determined, so as to determine the distortion type of the input image based on the matched distortion type features.

[0053] In this example, the weighting of the physical parameters of the heat dissipation layer can be determined based on the distortion type of the input image. For example, for simple radial distortion, the surface shape parameters of the optical surface can be optimized first, with a weighting of 100%. Similarly, for simple tangential distortion, the mounting tilt angle can be optimized first, with a weighting of 100%. Furthermore, for mixed distortion, weighting can be allocated according to the component proportions. If the mixed distortion is 70% radial distortion and 30% tangential distortion, then the surface shape parameters could have a weighting of 70%, and the tilt angle parameters could have a weighting of 30%.

[0054] In this example, the mapping function can be solved in reverse under the aforementioned constraints, with the objective function being that the optical distortion parameter approaches 0, based on the weighted proportions of the physical parameters of the heat dissipation layer. This allows for the acquisition of the optimal parameter values ​​for the physical parameters of the heat dissipation layer. By inversely solving the mapping function based on the weighted proportions of the physical parameters, the optimal parameter values ​​can be found quickly and accurately.

[0055] After performing the above step S2200, with the optimization objective of the optical distortion parameter value being less than or equal to a set value, and inversely solving the mapping function to obtain the target parameter value of the physical parameters of the heat dissipation layer, the process proceeds to: Step S2300: A heat dissipation layer with the target parameter value is manufactured according to the target parameter value of the physical parameters, and the manufactured heat dissipation layer is integrated into the HUD so that the image distortion introduced by the heat dissipation layer is corrected during the operation of the HUD.

[0056] In this embodiment, a heat dissipation layer with target physical parameter values ​​can be manufactured using precision machining equipment, and the manufactured heat dissipation layer can be integrated into a preset position of the HUD. This preset position can be a mounting point on the optical path of the emitted light of the image generation unit and close to the heat-generating area of ​​the image generation unit, for example, referring to... Figure 3 The heat dissipation layer is directly attached to the image generation unit. On the one hand, this ensures that light can pass through the heat dissipation layer completely, so that the layer can correct for changes in the optical path and image distortion introduced by itself; on the other hand, it allows the heat dissipation layer to efficiently absorb the heat generated by the image generation unit during operation.

[0057] For example, the surface shape parameters of the fabricated heat dissipation layer have an optimal first parameter value, the thickness has an optimal second parameter value, the refractive index has an optimal third parameter value, and the mounting angle has an optimal fourth parameter value. After fabricating the heat dissipation layer, its optical performance can be tested using a laser interferometer, and its heat dissipation effect can be tested using a thermal conductivity meter. After passing the tests, the heat dissipation layer is attached to the image generation unit in the HUD.

[0058] After performing the above step S2300, a heat dissipation layer with the target parameter value is manufactured according to the target parameter value of the physical parameters, and the manufactured heat dissipation layer is integrated into the HUD, so that the heat dissipation layer corrects the image distortion introduced by itself during the operation of the HUD, and then proceeds to: Step S2400: During the operation of the HUD, the displayed image is monitored, and the driving parameters of the image generation unit in the HUD are adjusted according to the displayed image to perform image distortion compensation.

[0059] The driving parameters of the image generation unit can include basic imaging control parameters, correction and adaptation adjustment parameters, and environmental adaptation optimization parameters. Basic imaging control parameters can include brightness parameters, contrast parameters, and color parameters. Correction and adaptation adjustment parameters can include projection angle parameters, optical compensation parameters, and distortion coefficient compensation parameters. Environmental adaptation optimization parameters can include temperature compensation parameters and interference immunity and reliability parameters.

[0060] In this embodiment, the displayed image can be monitored in real time or according to a set monitoring cycle during HUD operation, and the driving parameters of the image generation unit in HUD can be adjusted by the displayed image to cope with dynamic interference during driving and maintain the continuity and stability of the optical path correction effect.

[0061] In one example, step S2400, which monitors the displayed image during HUD operation and adjusts the driving parameters of the image generation unit in the HUD according to the displayed image to perform image distortion compensation, may further include: identifying the current coordinates of a preset marker point in the displayed image; generating an image offset based on the current coordinates and the reference coordinates of the preset marker point; searching for an adjustment amount matching the image offset from a set offset adjustment table based on the image offset; and adjusting the parameter values ​​of the driving parameters of the image generation unit in the HUD according to the adjustment amount to perform image distortion compensation.

[0062] The offset adjustment table can store the correspondence between image offset and the adjustment amount of driving parameters.

[0063] Among them, the preset markers can be invisible markers, which are usually not visible to the human eye. For example, the invisible markers can be infrared points, and the number of preset markers can be 3 to 5.

[0064] In this example, the monitoring cycle can be set to 100ms according to the HUD's usage scenario. While the HUD displays driving information normally, it can project 3-5 invisible markers at the edge of the displayed image. A miniature camera can collect these invisible markers and calculate the current coordinates of each marker. It then compares the current coordinates of each marker with its corresponding reference coordinates to determine if drift exists and obtain a drift detection result. If the drift detection result indicates drift, an adjustment amount matching the icon offset of the currently displayed image can be found from the set offset adjustment table. Based on this adjustment amount, the driving parameter values ​​of the image generation unit in the HUD are adjusted, and the corrected display image is output.

[0065] This example addresses minor distortion drift caused by dynamic interference, such as real-time changes in interference factors like temperature rise during driving and vehicle vibration. It overcomes the limitations of static correction using a custom heat dissipation layer, preventing these interferences from causing minor distortion in the corrected display image.

[0066] This example enables seamless real-time compensation by automatically fine-tuning the PGU's driving parameters through monitoring key landmarks. It can quickly correct drift without manual intervention, ensuring that the driver always sees a clear and distortion-free display image and guaranteeing stable correction results in dynamic driving scenarios.

[0067] The image distortion correction method of this disclosure can establish a mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters, and solve the mapping function in reverse with the parameter value of the optical distortion parameter being less than or equal to a set value as the optimization target, to obtain the target parameter value of the physical parameters of the heat dissipation layer. In this way, a heat dissipation layer with the target parameter value can be manufactured, so that the heat dissipation layer can correct the image distortion it introduces during the operation of the HUD. By optimizing the parameter value of the physical parameters of the heat dissipation layer, it can correct the optical path to compensate for the image distortion it introduces while ensuring the heat dissipation function. Furthermore, by monitoring the displayed image and adjusting the driving parameters of the image generation unit in the HUD to perform image distortion compensation during the operation of the HUD, it can ensure that the driver always sees a clear and distortion-free display image, and ensure the stability of the correction effect in dynamic driving scenarios.

[0068] In one embodiment, the mapping function mentioned above can be a mapping function, and the step S2100 of determining the mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters can further include the following steps S2110 to S2140: In step S2110, without a heat dissipation layer installed in the HUD, the image generation unit is activated to project a test image, the initial optical path data of light in the HUD is collected by an optical measurement instrument, and the initial display image is collected by an image acquisition device.

[0069] The test image can be a standard test image, such as a raster image or a checkerboard image.

[0070] Specifically, refer to Figure 4 During HUD assembly, the heat dissipation layer is not installed initially. The image generation unit is activated to project a standard test image, and the initial optical path data of the light is recorded using optical measurement instruments. This initial optical path data may include, for example, the incident angle and refraction angle of the light. An initial display image is then captured at a pre-defined viewing surface, i.e., the eye box position, using an image acquisition device, and the reference coordinate position and reference shape parameters of each feature point in the initial display image are identified.

[0071] In step S2120, with a heat dissipation layer having set parameter values ​​for physical parameters installed in the HUD, the current optical path data of light passing through the heat dissipation layer is collected by an optical measurement instrument, and the corresponding distorted display image is collected by an image acquisition device.

[0072] Specifically, refer to Figure 5 A heat dissipation layer with set parameter values ​​is installed at a preset position, such as adhering to the image generation unit. Under the same conditions, the image generation unit projects the standard test image again and records the current optical path data of the light using an optical measuring instrument. Additionally, an image acquisition device captures a currently displayed image, i.e., a distorted display image, at the eyepiece position, and identifies the current coordinate position and shape parameters of each feature point in the distorted display image.

[0073] Step S2130: Generate training samples based on the set parameter values ​​of physical parameters, the initial display image, the initial optical path data, the current optical path data, and the corresponding distorted display image.

[0074] In step S2130, generating training samples based on the set parameter values ​​of physical parameters, the initial display image, the initial optical path data, the current optical path data, and the corresponding distorted display image can be achieved in the following way: obtaining the first deviation between the initial display image and the distorted display image; obtaining the second deviation between the initial optical path data and the current optical path data; generating the parameter values ​​of the optical distortion parameters under the set parameter values ​​of physical parameters based on the first and second deviations; and generating a training sample based on the set parameter values ​​of physical parameters and the corresponding parameter values ​​of optical distortion parameters.

[0075] Specifically, the reference coordinate position of each feature point in the initial display image can be compared with the current coordinate position of each feature point in the distorted display image. The reference shape parameter of each feature point in the initial display image can be compared with the current shape parameter of each feature point in the distorted display image. The initial optical path data and the current optical path data can also be compared to generate the corresponding optical distortion parameter values. In this way, the set parameter values ​​of the physical parameters can be associated with the corresponding optical distortion parameter values ​​to obtain a training sample.

[0076] Step S2140: Train the mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters based on the training samples.

[0077] In step S2140, the mapping function can be obtained by using various fitting methods based on the physical parameters of the heat dissipation layer of the training samples and their corresponding optical distortion parameters. .

[0078] For example, this mapping function can be obtained using any multiple linear regression method. .

[0079] For example, machine learning algorithms, such as neural network learning algorithms, can also be used to obtain this mapping function. .

[0080] In this embodiment, once the mapping function is trained, it can predict the optical path changes and image distortions that will result from any given set of physical parameters of the heat dissipation layer.

[0081] This disclosure also provides an image distortion correction device. For example... Figure 6 As shown, the image distortion correction device 600 includes: The determining module 610 is used to determine the mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters; wherein, the mapping function is used to predict the parameter values ​​of the optical distortion parameters based on the parameter values ​​of the physical parameters of the heat dissipation layer; The solving module 620 is used to perform inverse solving on the mapping function with the parameter value of the optical distortion parameter being less than or equal to a set value as the optimization objective, so as to obtain the target parameter value of the physical parameter of the heat dissipation layer; Manufacturing module 630 is used to manufacture a heat dissipation layer having the target parameter value according to the target parameter value of the physical parameters, and integrate the manufactured heat dissipation layer into the HUD, so that the heat dissipation layer corrects the image distortion introduced by itself during the operation of the HUD; The adjustment module 640 is used to monitor the displayed image during the operation of the HUD and adjust the parameter values ​​of the driving parameters of the image generation unit in the HUD according to the displayed image in order to perform image distortion compensation.

[0082] In some embodiments, the solving module 620 is specifically used to obtain the distortion type of the input image; based on the distortion type, under set constraints, with the parameter value of the optical distortion parameter being less than or equal to a set value as the optimization objective, the mapping function is solved in reverse to obtain the target parameter value of the physical parameter.

[0083] In some embodiments, the set constraints include heat dissipation constraints and processing feasibility constraints; The heat dissipation constraint condition includes that the thermal conductivity of the heat dissipation layer is greater than a set thermal conductivity. The processing feasibility constraints include that the thickness of the heat dissipation layer is greater than or equal to a set thickness, and that the surface shape of the optical surface of the heat dissipation layer is within a preset processing capability range.

[0084] In some embodiments, the solving module 620 is specifically used to identify image features in the displayed image; search for a distortion type feature that matches the identified image features in a distortion type feature library; and determine the distortion type of the displayed image based on the matched distortion type feature.

[0085] In some embodiments, the determining module 610 is specifically used to, when no heat dissipation layer is installed in the HUD, activate the image generation unit to project a test image, acquire initial optical path data of light in the HUD through an optical measurement instrument, and acquire an initial display image through an image acquisition device; when heat dissipation layers with set parameter values ​​of the physical parameters are installed in the HUD, acquire current optical path data of light passing through the heat dissipation layer through the optical measurement instrument, and acquire corresponding distorted display images through the image acquisition device; generate training samples based on the set parameter values ​​of the physical parameters, the initial display image, the initial optical path data, the current optical path data, and the distorted display image; and train a mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters based on the training samples.

[0086] In some embodiments, the determining module 610 is specifically configured to: obtain a first deviation between the initial display image and the distorted display image; obtain a second deviation between the initial optical path data and the current optical path data; generate a parameter value of the optical distortion parameter under the set parameter value of the physical parameters based on the first deviation and the second deviation; and generate the training sample based on the set parameter value of the physical parameters and the corresponding parameter value of the optical distortion parameter.

[0087] In some embodiments, the adjustment module 640 is specifically used to identify the current coordinates of a preset marker point in the displayed image; generate an image offset based on the current coordinates and the reference coordinates of the preset marker point; search for an adjustment amount matching the image offset from a set offset adjustment table based on the image offset; wherein the set offset adjustment table stores the correspondence between the image offset and the adjustment amount of the driving parameters; and adjust the parameter values ​​of the driving parameters of the image generation unit in the HUD according to the adjustment amount to perform image distortion compensation.

[0088] This disclosure also provides an electronic device, such as... Figure 7 As shown, the electronic device 700 includes a memory 710 and a processor 720. The memory 710 is used to store computer instructions, and the processor 720 is used to retrieve the computer instructions from the memory 710 to perform the method as described in any of the above method embodiments.

[0089] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the above method embodiments.

[0090] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0091] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0092] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0093] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0094] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0095] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0096] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0098] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. An image distortion correction method, comprising: Determine a mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters; wherein the mapping function is used to predict the parameter values ​​of the optical distortion parameters based on the parameter values ​​of the physical parameters of the heat dissipation layer; With the optimization objective of the optical distortion parameter being less than or equal to a set value, the mapping function is solved in reverse to obtain the target parameter value of the physical parameter of the heat dissipation layer; A heat dissipation layer with the target parameter value is manufactured according to the target parameter value of the physical parameters, and the manufactured heat dissipation layer is integrated into the HUD, so that the image distortion introduced by the heat dissipation layer is corrected during the operation of the HUD; During the operation of the HUD, the displayed image is monitored, and the parameter values ​​of the driving parameters of the image generation unit in the HUD are adjusted according to the displayed image to perform image distortion compensation.

2. The method according to claim 1, wherein, The step of optimizing the mapping function by setting the optical distortion parameter value to be less than or equal to a set value, and then inversely solving the mapping function to obtain the target parameter value of the physical parameters of the heat dissipation layer, includes: Obtain the distortion type of the input image; Based on the distortion type, and under set constraints, with the parameter value of the optical distortion parameter being less than or equal to a set value as the optimization objective, the mapping function is solved in reverse to obtain the target parameter value of the physical parameter.

3. The method according to claim 2, wherein, The set constraints include heat dissipation constraints and processing feasibility constraints; The heat dissipation constraint condition includes that the thermal conductivity of the heat dissipation layer is greater than a set thermal conductivity. The processing feasibility constraints include that the thickness of the heat dissipation layer is greater than or equal to a set thickness, and that the surface shape of the optical surface of the heat dissipation layer is within a preset processing capability range.

4. The method according to claim 2, wherein, The process of obtaining the image distortion type of the displayed image includes: Identify image features in the displayed image; Based on the identified image features, search for distortion type features that match the image features in the distortion type feature library; The distortion type of the displayed image is determined based on the matched distortion type features.

5. The method according to claim 1, wherein, The method for determining the mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters includes: Without a heat dissipation layer installed in the HUD, the image generation unit is activated to project a test image, the initial optical path data of the light in the HUD is collected by an optical measurement instrument, and the initial display image is collected by an image acquisition device. When a heat dissipation layer with the set parameter values ​​of the physical parameters is installed in the HUD, the optical measurement instrument collects the current optical path data of light passing through the heat dissipation layer, and the image acquisition device collects the corresponding distorted display image. Training samples are generated based on the set values ​​of the physical parameters, the initial display image, the initial optical path data, the current optical path data, and the distorted display image. The mapping function between the physical parameters and optical distortion parameters of the heat dissipation layer is obtained by training the training samples.

6. The method according to claim 5, wherein, The step of generating training samples based on the set parameter values ​​of the physical parameters, the initial display image, the initial optical path data, the current optical path data, and the distorted display image includes: Obtain the first deviation between the initial display image and the distorted display image; Obtain the second deviation between the initial optical path data and the current optical path data; Based on the first deviation and the second deviation, the parameter values ​​of the optical distortion parameters are generated under the set parameter values ​​of the physical parameters; The training samples are generated based on the set values ​​of the physical parameters and the corresponding values ​​of the optical distortion parameters.

7. The method according to claim 1, wherein, The step of adjusting the driving parameters of the image generation unit in the HUD according to the displayed image to perform image distortion compensation includes: Identify the current coordinates of the preset marker points in the displayed image; Based on the current coordinates and the reference coordinates of the preset marker point, an image offset is generated; Based on the image offset, an adjustment amount matching the image offset is found from the set offset adjustment amount table; wherein, the set offset adjustment amount table stores the correspondence between the image offset and the adjustment amount of the driving parameter; The parameter values ​​of the driving parameters of the image generation unit in the HUD are adjusted according to the adjustment amount to perform image distortion compensation.

8. An image distortion correction device, comprising: A determination module is used to determine the mapping function between the physical parameters of the heat dissipation layer and the optical distortion parameters; wherein, the mapping function is used to predict the parameter values ​​of the optical distortion parameters based on the parameter values ​​of the physical parameters of the heat dissipation layer; The solution module is used to perform inverse solution on the mapping function with the parameter value of the optical distortion parameter being less than or equal to a set value as the optimization objective, so as to obtain the target parameter value of the physical parameter of the heat dissipation layer; A manufacturing module is used to manufacture a heat dissipation layer having the target parameter value according to the target parameter value of the physical parameters, and to integrate the manufactured heat dissipation layer into the HUD, so that the heat dissipation layer can correct the image distortion introduced by itself during the operation of the HUD; An adjustment module is used to monitor the displayed image during the operation of the HUD and adjust the parameter values ​​of the driving parameters of the image generation unit in the HUD according to the displayed image in order to perform image distortion compensation.

9. An electronic device, comprising a memory and a processor, The memory is used to store computer instructions, and the processor is used to retrieve the computer instructions from the memory to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, perform the method of any one of claims 1-7.

Images