Image distortion correction method and system, storage medium, controller and vehicle

Abstract

The invention relates to an image distortion correction method and system, a storage medium, a controller and a vehicle, and the method is applied to the image distortion correction system, a plurality of reference positions are determined based on the eye box range of a head-up display device, set test images and standard images corresponding to the test images are respectively collected at each reference position, and the test images and the standard images are obtained. And finally, generating a distortion correction parameter according to the projection characteristic difference, and carrying out distortion correction processing on the to-be-projected image based on the distortion correction parameter. Wherein the reference position is a visual angle eye position when a user observes an image of the head-up display device, the test image is an image with distortion defects, and the standard image is an image without distortion defects. According to the invention, the precision of image distortion correction can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image distortion correction method, system, storage medium, controller and vehicle. Background Art

[0002] Currently, image distortion correction for vehicle head-up display (HUD) systems mainly uses a parameter table method based on single-point calibration and a binocular camera simulation method.

[0003] However, none of the above methods considers the impact of different viewing angles of the user on the accuracy of image distortion correction, so that the corrected image may still have distortion defects under different fields of view, resulting in low distortion correction accuracy. Summary of the Invention

[0004] The embodiment of the present application provides an image distortion correction method, which improves the accuracy of image distortion correction to solve the above-mentioned technical problems.

[0005] To achieve the above-mentioned object, according to a first aspect of the present application, a method for correcting image distortion is provided, comprising:

[0006] Applied to an image distortion correction system, the method includes:

[0007] determining a plurality of reference positions based on an eye box range of the head-up display device;

[0008] At each reference position, respectively capturing a set test image and a standard image corresponding to each test image to obtain a projection feature difference between the test image and the standard image;

[0009] generating distortion correction parameters according to the projection feature differences, and performing distortion correction processing on the image to be projected based on the distortion correction parameters;

[0010] The test image is an image with distortion defects, and the standard image is an image without distortion defects.

[0011] Optionally, the multiple reference positions are located on the same target horizontal line, and the target horizontal line divides the eye box range into two symmetrical parts.

[0012] Optionally, determining a plurality of reference positions based on an eye box range of the head-up display device includes:

[0013] Based on the target horizontal line and the eye box range, a plurality of reference points and the coordinates of each of the reference points are determined to obtain the plurality of reference positions.

[0014] Optionally, determining a plurality of reference points and the coordinates of each reference point based on the target horizontal line and the eye box range to obtain the plurality of reference positions includes:

[0015] The center of the eye box range is determined as the center point, the intersection point of the target horizontal line with the left boundary of the eye box range is determined as the left endpoint, and the intersection point with the right boundary of the eye box range is determined as the right endpoint;

[0016] Using the target horizontal line as a connecting line, the midpoint of the line segment between the left endpoint and the center point is determined as the midpoint of the left area, and the midpoint of the line segment between the right endpoint and the center point is determined as the midpoint of the right area;

[0017] The left endpoint, the midpoint of the left area, the center point, the midpoint of the right area and the right endpoint are respectively used as the reference positions and the coordinates of each reference position are obtained.

[0018] Optionally, taking the left endpoint, the midpoint of the left region, the center point, the midpoint of the right region, and the right endpoint as the reference positions and obtaining the coordinates of each reference position includes:

[0019] The left limit position that meets the preset conditions is used as the coordinate of the left endpoint, and the right limit position is used as the coordinate of the right endpoint; the preset conditions include that the acquisition range of the test image at the current position covers the eye box range;

[0020] Performing linear interpolation processing on the coordinates of the left endpoint and the coordinates of the right endpoint to obtain the coordinates of the center point;

[0021] Linear interpolation is performed on the coordinates of the center point and the coordinates of the left endpoint to obtain the coordinates of the center point of the left area, and linear interpolation is performed on the coordinates of the center point and the coordinates of the right endpoint to obtain the coordinates of the center point of the right area.

[0022] Optionally, generating a distortion correction parameter according to the projection feature difference, and performing distortion correction processing on the image to be projected based on the distortion correction parameter, includes:

[0023] Preprocessing the test image and the standard image respectively to obtain a plurality of first feature points of the test image and a plurality of second feature points of the standard image; wherein each first feature point corresponds to one second feature point;

[0024] The distortion correction parameter is generated based on the offset between each of the first feature points and the corresponding second feature point.

[0025] Optionally, preprocessing the test image and the standard image respectively to obtain a plurality of first feature points of the test image and a plurality of second feature points of the standard image includes:

[0026] Performing noise reduction processing on the test image and the standard image respectively to obtain a first noise-reduced image corresponding to the test image and a second noise-reduced image corresponding to the standard image;

[0027] performing binarization processing on the first denoised image and the second denoised image respectively to obtain a plurality of first feature point contours corresponding to the test image and a plurality of second feature point contours corresponding to the standard image;

[0028] Each of the first feature point contours and each of the second feature point contours are mapped to a set coordinate system to obtain the first feature points of the test image and the second feature points of the standard image.

[0029] Optionally, generating the distortion correction parameter based on the offset between each first feature point and the corresponding second feature point includes:

[0030] obtaining a lateral offset based on a difference between the horizontal coordinates of the first feature point and the corresponding second feature point;

[0031] A longitudinal offset is obtained based on a difference between the longitudinal coordinates of the first feature point and the corresponding second feature point, and the lateral offset and the longitudinal offset are used as the distortion correction parameters.

[0032] According to a second aspect of the present application, there is provided an image distortion correction system, comprising:

[0033] An image generating device, an image capturing device, and an image processing device communicatively connected to each other, wherein:

[0034] The image generating device is used to project a test image and a standard image corresponding to the test image;

[0035] The image acquisition device is configured to determine a plurality of reference positions based on an eye box range of the head-up display device, and to respectively acquire the test image projected by the image generation device and the standard image corresponding to each test image at each reference position;

[0036] The image processing device processes the test image and the standard image acquired by the image acquisition device to obtain a projection feature difference between the test image and the standard image, generates distortion correction parameters according to the projection feature difference, and performs distortion correction processing on the image to be projected based on the distortion correction parameters;

[0037] The reference position is the eye position of the user when observing the image of the head-up display device, the test image is an image with distortion defects, and the standard image is an image without distortion defects.

[0038] According to a third aspect of the present application, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described above are implemented.

[0039] According to a fourth aspect of the present application, a controller is further provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0040] According to a fifth aspect of the present application, a vehicle is also provided, comprising the controller as described above.

[0041] According to a sixth aspect of the present application, a computer program product is also provided, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0042] In summary, the present application first determines multiple reference positions based on the eye box range of the head-up display device. Then, at each reference position, a set test image and a standard image corresponding to each test image are collected to obtain the projection feature difference between the test image and the standard image. Finally, distortion correction parameters are generated based on the projection feature difference, and distortion correction processing is performed on the projected image based on the distortion correction parameters. The reference position is the user's eye position when observing the head-up display device image, the test image is an image with distortion defects, and the standard image is an image without distortion defects. Therefore, the present application simulates the user's eye position when observing the HUD image by determining the reference position, and achieves comprehensive coverage of the user's various viewing angles within the eye box range during the image distortion correction process. It can capture the actual deviation between the test image and the standard image at each reference position, obtain more comprehensive and accurate distortion correction parameters to perform distortion correction on the image, and thus improve the accuracy of image distortion correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0044] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same drawing numbers represent the same parts in the following description.

[0045] Figure 1 is a flowchart of a method for correcting image distortion provided in an exemplary embodiment of the present disclosure;

[0046] Figure 2 is a schematic diagram of various reference positions within the eye box provided in an exemplary embodiment of the present disclosure;

[0047] Figure 3 is a structural schematic diagram of an image distortion correction system provided in an exemplary embodiment of the present disclosure;

[0048] Figure 4 is a schematic diagram of data interaction between various devices in the image distortion correction system provided in an exemplary embodiment of the present disclosure;

[0049] Figure 5 is a schematic diagram comparing a test image and a corrected image provided in an exemplary embodiment of the present disclosure;

[0050] Figure 6 is a schematic diagram of an image distortion correction system provided in an exemplary embodiment of the present disclosure;

[0051] Figure 7 FIG. 1 is a schematic diagram of the architecture of a vehicle provided in an exemplary embodiment of the present disclosure.

[0052] Description of reference numerals:

[0053] 1-displacement platform, 2-image acquisition device, 3-image processing device, 4-image generation device, 5-image magnification device, 51-first reflector, 52-second reflector, 6-vehicle front windshield, 7-standard dot matrix template. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0055] Based on the issues mentioned in the background art, image distortion correction for vehicle head-up display systems primarily relies on the following two methods. One method is the single-point calibration method. This method selects several reference locations within the eye box, captures test images, compares them with a standard dot matrix, and calculates a correction parameter table for each location. However, this method uses only a single, fixed standard image for distortion correction at each reference location, making it incapable of correcting distorted images observed from different fields of view. Furthermore, ghosting in the initial projected image is not effectively suppressed, leading to accumulated pixel detection errors and affecting the final correction accuracy. Another method is the binocular camera fusion method, which generates correction parameters by simulating left and right eye parallax. However, the images captured by the binocular camera differ from actual human observation, and the dynamic standard map is not adapted for different fields of view. This results in a mismatch between the correction effect and the user's real-time eye position. Furthermore, the ghosting problem is further amplified in the binocular system, reducing the accuracy of distortion compensation.

[0056] This application provides a method for image distortion correction. Figure 1 The image distortion correction method provided in the embodiment of the present application includes steps S101 to S103, which are described in detail below.

[0057] Step S101: Determine multiple reference positions based on an eye box range of a head-up display device.

[0058] For head-up displays, the eyebox refers to the range within which the driver's head and eyes can move while still seeing the HUD's projected image. The eyebox is a three-dimensional region whose dimensions are determined by the HUD's optical design and the vehicle's windshield's reflective geometry. For example, a typical eyebox range might be ±100mm horizontally and ±50mm vertically.

[0059] The reference position is the user's eye position when observing the head-up display image. It's understandable that HUD image distortion is not only related to the optical system but also closely related to the user's eye position. The distortion of the HUD image projected by the head-up display device varies from user to user at different viewing angles. Therefore, it's necessary to select representative reference positions within the eye box as reference eye positions. These reference positions are then used to capture test images for distortion correction.

[0060] In summary, this application determines multiple reference positions that simulate the user's perspective eye position, and then captures images at each reference position to obtain multi-perspective samples to be corrected for distortion, thereby obtaining more comprehensive distortion defect information of the user at different perspectives, laying the foundation for subsequent improvement of the accuracy of image distortion correction.

[0061] In some embodiments, the multiple reference positions are located on the same target horizontal line, which divides the eye box range into two symmetrical parts. Figure 2 As shown, the eye box range can be represented by a rectangular area, and a horizontal line that can divide the eye box range into two symmetrical parts is obtained as the target horizontal line. The target horizontal line runs through Figure 2 The reference positions a, b, c, d, and e in .

[0062] It can be understood that the target horizontal line is a virtual line used to simulate the height of the center of the user's field of view. It corresponds to the horizontal line of sight of the user's eyes when observing the HUD image in a normal sitting position. It is the baseline that best represents the direction of the user's main visual line of sight.

[0063] The target horizontal line creates bilateral symmetry in the captured image, facilitating the subsequent determination of the coordinates of each reference position through interpolation algorithms. This also helps detect systematic deviations or mirror distortion. Furthermore, since most users move their heads more horizontally than vertically in a vehicle, prioritizing horizontal reference positions better reflects real-world usage scenarios and improves the applicability and accuracy of subsequent distortion correction.

[0064] In some embodiments, multiple reference points and the coordinates of each reference point may be determined based on the target horizontal line and the eye box range to obtain multiple reference positions.

[0065] Specifically, such as Figure 2 As shown, after determining the target horizontal line, the center point of the eye box range can be determined as the first reference point based on the principle of bilateral symmetry. The boundary points of the left and right areas of the eye box range can then be determined as the second and third reference points. After determining these three reference points, the center point of the left and right areas of the eye box range can be determined using a linear interpolation algorithm, resulting in five reference points that can meet the requirements for simulating the user's eye positions at various viewing angles. The coordinates of these five reference points can then be calculated to obtain comprehensive information about multiple reference positions.

[0066] In a specific implementation, the geometric center of the eye box range can be first determined as the center point, the intersection point of the target horizontal line with the left boundary of the eye box range is determined as the left endpoint, and the intersection point with the right boundary of the eye box range is determined as the right endpoint;

[0067] Then, using the target horizontal line as the connecting line, the midpoint of the line segment between the left endpoint and the center point is determined as the midpoint of the left area, and the midpoint of the line segment between the right endpoint and the center point is determined as the midpoint of the right area;

[0068] Finally, the left endpoint, the midpoint of the left region, the center point, the midpoint of the right region and the right endpoint are respectively used as reference positions and the coordinates of each reference position are obtained.

[0069] To facilitate the process of determining each reference point, please refer to Figure 2 , we can first determine the center point c of the eye box range, then determine the left endpoint a and the right endpoint f respectively through the target horizontal line and the boundary of the eye box range, and then determine the midpoint of line segment ac as the midpoint b of the left area, and the midpoint of line segment ce as the midpoint d of the right area.

[0070] In some embodiments, the coordinates of each reference position may be determined by:

[0071] The left limit position that meets the preset conditions is used as the coordinate of the left endpoint, and the right limit position is used as the coordinate of the right endpoint; the coordinates of the left endpoint and the right endpoint are linearly interpolated to obtain the coordinates of the center point;

[0072] The coordinates of the center point and the left endpoint are linearly interpolated to obtain the coordinates of the center point of the left area, and the coordinates of the center point and the right endpoint are linearly interpolated to obtain the coordinates of the center point of the right area.

[0073] Among them, the preset conditions may include that the acquisition range of the test image at the current position covers the eye box range, specifically referring to that the image acquisition device used for image acquisition can completely cover the entire area of the HUD projection image when shooting at the current position, that is, the image within the eye box range is not cropped or missing.

[0074] Specifically, a test image that overlaps with the eye box area can be projected into the eye box range, and then the image acquisition device can be moved horizontally from the center position of the eye box range to the left along the target horizontal line until the image acquisition device can completely capture the entire test image. When the image acquisition device continues to move to the left and the complete test image cannot be captured, the current position that can just capture the complete test image is determined to be the "left limit position", and the left limit position is used as the coordinates of the left endpoint a (x1, y1). Correspondingly, the image acquisition device can be controlled to move to the right, and the right endpoint e corresponding to the right limit position and the coordinates of the right endpoint e (x5, y5) can be determined by the same calculation method.

[0075] In some embodiments, after the coordinates of the left endpoint a and the right endpoint e are determined, since the center point c is located at the midpoint of the line segment ae, the coordinates (x3, y3) of the center point c can be calculated by linear interpolation, as follows:

[0076]

[0077] In a vehicle driving scenario, the coordinates of the center point c(x3, y3) correspond to the normal eye position when the user's head is centered.

[0078] After determining the coordinates of the center point c, the coordinates of the center point b of the left region (x2, y2) can be calculated by linear interpolation based on the coordinates of the left endpoint a, as follows:

[0079]

[0080] Correspondingly, the coordinates of the center point d of the right area (x4, y4) can be calculated by combining the coordinates of the center point c and the right endpoint b through linear interpolation, as follows:

[0081]

[0082] Through the above method, the reference positions a, b, c, d and e can be obtained.

[0083] In determining each reference position, this application determines the left and right extreme positions by determining whether the image acquisition device is capable of capturing a complete test image. This approach also avoids missing data due to image edge distortion. Furthermore, a simple linear median calculation is used to ensure that each reference point is symmetrically distributed and evenly spaced within the eye box. The coordinates of each reference point have a corresponding physical meaning, which enhances the correspondence between the subsequently calculated distortion correction parameters and the user's viewing angle.

[0084] Step S102 : At each reference position, a set test image and a standard image corresponding to each test image are respectively captured to obtain a projection feature difference between the test image and the standard image.

[0085] The test image is a distorted image, projected by the head-up display through its optical structure and ultimately formed on the vehicle's windshield. During the projection process, the test image is affected by factors such as the HUD's off-axis optical system, reflector surface errors, and assembly deviations. Consequently, a certain degree of image distortion may occur, including bright spot position shift, stretching, compression, and bending.

[0086] The standard image is an image without distortion defects, that is, the image that the head-up display device should present under ideal conditions. It can be generated by image tools such as OpenCV and specifically contains 11 rows × 33 columns = 363 bright spots. The layout of multiple bright spots is regular and numbered clearly. Each bright spot can be regarded as a "feature point". At each reference position, the standard image and the test image have a one-to-one correspondence. For example Figure 5 The test images and their corresponding standard images are shown.

[0087] Projected features can refer to extracting feature points from a test image and a standard image after projecting them. Feature points at each location in the two images are in one-to-one correspondence, and the projected feature difference is the difference between the feature points in the two images. Specifically, the projected feature difference between the test image and the standard image can be determined by locating each feature point in the test image and the standard image, and then comparing the deviation between the feature points at each location in the two images.

[0088] Step S103: generating distortion correction parameters according to the projection feature differences, and performing distortion correction processing on the image to be projected based on the distortion correction parameters.

[0089] In some embodiments, the distortion correction parameters may refer to the coordinate offsets of feature points in the test image compared to feature points in the standard image, for example, including a horizontal coordinate offset Δx and a vertical coordinate offset Δy. After determining the distortion correction parameters, the pixel positions of the projected image can be reverse mapped or compensated based on the offsets of each feature point. The distortion-corrected image is then projected by the head-up display device, pre-emptively offsetting any image distortion in the optical path. Ultimately, a geometrically correct, distortion-free image is formed on the vehicle's windshield, improving the user's visual quality of the HUD image and the user experience of the head-up display device.

[0090] In some embodiments, step S103 may include:

[0091] First, preprocessing is performed on the test image and the standard image to obtain a plurality of first feature points of the test image and a plurality of second feature points of the standard image, wherein each first feature point corresponds to a second feature point;

[0092] Next, a distortion correction parameter is generated based on the offset between each first feature point and the corresponding second feature point.

[0093] Specifically, as an example, image processing algorithms such as bilateral Gaussian filtering, image binarization, and dilation-erosion can be used to determine the outlines and center coordinates of the 363 bright spots in the test image, obtaining complete information about the multiple first feature points. Correspondingly, the same image processing algorithm can be used to extract the outlines and center coordinates of the corresponding 363 bright spots in the standard image, obtaining complete information about the multiple second feature points.

[0094] Next, the same numbering rule can be used to number each first feature point and each second feature point. For example, the numbering rule can be to use the first feature point in the upper left corner of the bright spot array as the starting point and number it (0, 0). The first number is the row number of the feature point, and the second number is its column number. Then, based on the starting point number, each first feature point is numbered one by one in an ascending row and column numbering rule. For example, the next number of the starting point number in the same row is (0, 1), and the next number of the starting point number in the same column is (1, 0), until the first feature point in the lower right corner of the bright spot array is determined to be numbered (10, 32).

[0095] After determining the number and coordinates of each first feature point and each second feature point, in some embodiments, the lateral offset Δx can be obtained based on the difference between the horizontal coordinates of the first feature point and the corresponding second feature point, and then the longitudinal offset Δy can be obtained based on the difference between the vertical coordinates of the first feature point and the corresponding second feature point, and the lateral offset and the longitudinal offset are used as distortion correction parameters.

[0096] After the distortion correction parameters are determined, a table of the distortion correction parameters may be obtained. For example, the table of the distortion correction parameters may record in detail the offset of each first feature point relative to the second feature point in the test image at each reference position.

[0097] In some embodiments, the first feature point and the second feature point are obtained by:

[0098] First, denoising is performed on the test image and the standard image respectively to obtain a first denoised image corresponding to the test image and a second denoised image corresponding to the standard image;

[0099] Next, binarization is performed on the first denoised image and the second denoised image respectively to obtain a plurality of first feature point contours corresponding to the test image and a plurality of second feature point contours corresponding to the standard image;

[0100] Finally, each first feature point contour and each second feature point contour are mapped to a set coordinate system to obtain the first feature points of the test image and the second feature points of the standard image.

[0101] In a specific implementation, the test image and the standard image can be subjected to denoising respectively to remove image noise and retain the contour edges of bright spots in the two images, so that subsequent feature point extraction is more accurate, thereby obtaining a first denoised image and a second denoised image respectively.

[0102] In some embodiments, a bilateral Gaussian filter algorithm may be used to perform noise reduction on the two images while retaining edge information of the images. The bilateral Gaussian filter function is expressed as follows:

[0103]

[0104] Among them, I(i,j) is the pixel value corresponding to the position coordinate (i,j) in the image, I(k,l) is the pixel value corresponding to the position coordinate (k,l) in the image, and σs and σr are filtering parameters used to control the effect of image filtering.

[0105] Next, the first and second denoised images can be binarized to obtain multiple first feature point contours corresponding to the test image and multiple second feature point contours corresponding to the standard image. This operation can separate the bright spot areas from the background in both images and extract the corresponding feature point contours.

[0106] The principle of image binarization processing algorithm can be expressed as:

[0107]

[0108] Among them, σ is the binarization threshold, which is used to control the effect of image binarization.

[0109] Finally, each first feature point outline and each second feature point outline can be mapped to the set coordinate system to obtain the first feature point of the test image and the second feature point of the standard image. The set coordinate system is the bright spot array. Figure 5 For example, mapping the first feature point contour to the coordinate system means locating the center point of the first feature point contour, numbering each center point, obtaining the complete information of the first feature point, and calculating the distortion correction parameters based on the information of each first feature point and the corresponding second feature point.

[0110] As an example only, the point numbered (0,0) in the previous article is the first feature point. Assuming that the coordinates of the first feature point are (x0,y0), when subsequently calculating the offset of the first feature point, you can find the second feature point with the same number (0,0) and obtain the coordinates of the point, and then calculate the horizontal coordinate offset and vertical coordinate offset between the first feature point and the second feature point respectively, and obtain the distortion correction parameters of the first feature point.

[0111] It should also be noted that the core of this application is to obtain distortion correction parameters based on the relationship between the feature points of the test image and the corresponding standard image after collecting them at each reference position. Therefore, after determining the distortion correction parameters for the test image at each reference position, it is necessary to continue to determine the distortion correction parameters for the test image at the next reference position until the distortion correction parameters for all reference positions are obtained.

[0112] like Figure 3As shown, according to the second aspect of the present disclosure, there is also provided an image distortion correction system, comprising an image generating device 4, an image acquisition device 2, and an image processing device 3 that are communicatively connected to each other, wherein:

[0113] An image generating device 4 is used to project a test image and a standard image corresponding to the test image;

[0114] an image acquisition device 2 for determining a plurality of reference positions based on an eyebox range of the head-up display device, and for acquiring, at each reference position, a test image projected by the image generation device 4 and a standard image corresponding to each test image;

[0115] an image processing device 3 for processing the test image and the standard image acquired by the image acquisition device 2 to obtain a projection characteristic difference between the test image and the standard image, generating distortion correction parameters based on the projection characteristic difference, and performing distortion correction processing on the image to be projected based on the distortion correction parameters;

[0116] Among them, the reference position is the user's eye position when observing the image of the head-up display device, the test image is an image with distortion defects, and the standard image is an image without distortion defects.

[0117] like Figure 3 As shown, image acquisition device 2, image processing device 3, and image generation device 4 are interconnected and communicate with each other to detect and correct image distortion. Image generation device 4 is part of the head-up display device and is used to generate and output images, projecting them through the optical path of the head-up display device.

[0118] For example, the image generation device 4 can project a test image and a standard image. The test image is an actual image projected onto the vehicle's windshield, exhibiting optical distortion. The standard image is an ideal reference image that has not passed through the optical path of the head-up display device. The test image and the standard image have the same numbering structure and pattern arrangement, but the former is affected by distortion, while the latter is a baseline, undistorted image. The center of the output image projection must be strictly aligned with the center of the standard dot matrix template to ensure accurate offset measurement.

[0119] The image acquisition device 2 can determine multiple reference positions based on a preset eye box range, that is, simulate the eye positions of the driver's head when observing the HUD at different positions.

[0120] The image acquisition device 2 can capture the test image projected by the image generation device at each reference position, and at the same time capture the standard image corresponding to the test image. In some embodiments, the image acquisition device 2 can be set on the displacement platform 1, and the displacement platform 1 drives the image acquisition device 2 to move to determine multiple reference positions (such as Figure 2Points a, b, c, d and e in the image acquisition device are then moved to the above reference positions by the displacement platform 1 to capture test images and standard images. Each reference position represents an actual viewing angle of the user, so that the present application supports viewing angle-related distortion correction.

[0121] In some embodiments, the image processing device 3 can perform preprocessing such as noise reduction, binarization, and feature point contour extraction on the test image and the standard image, extracting and numbering the center position of the bright spot to form a feature point set. The coordinate offsets Δx and Δy between each pair of correspondingly numbered test image feature points and standard image feature points can then be compared to obtain a projection feature difference dataset at the reference viewing angle. Finally, all offset sets can be combined into a set of distortion correction parameter tables. A set of distortion correction parameter tables is generated for each reference position and stored in a server on the vehicle or head-up display device, supporting subsequent multi-perspective correction of the projected image of the head-up display device. In subsequent practical applications, before the image generation device of the head-up display device outputs the image to be projected, its pixel coordinates can be pre-initiated with reverse offset compensation based on the distortion correction parameters, so that after passing through the optical path of the head-up display device, it can be accurately imaged at the target position, eliminating the original distortion.

[0122] In some embodiments, the system may further include an image magnification device 5 connected to the image generation device 4, wherein:

[0123] The image magnifying device 5 includes a first reflecting mirror 51 and a second reflecting mirror 52 , wherein the reflecting area of the second reflecting mirror 52 is larger than the reflecting area of the first reflecting mirror 51 ;

[0124] The first reflector 51 is used to receive the image projected by the image generating device 4 and reflect the received image to the second reflector 52. The second reflector 52 expands the light angle of the image to obtain an enlarged image and reflects the enlarged image to the target position.

[0125] like Figure 3 As shown, the image magnification device 5 is also part of the head-up display device and is used to optically magnify the image output by the image generation device 4 and then project it onto a target location, such as the vehicle's windshield. The first reflector 51 has a relatively small area and is used to receive the original image projected by the image generation device. The second reflector 52 has a larger area and is used to expand the light and reflect the magnified image.

[0126] Specifically, the image generating device 4 projects the corrected image onto the first reflector 51, and the first reflector 51 can reflect the image to the second reflector 52. The second reflector 52 expands the light angle of the incident light, that is, expands the divergence angle of the light to achieve magnification of the geometric size of the image. The second reflector 52 then reflects the magnified image to the target position for the user to view, thereby realizing image projection.

[0127] In some embodiments, the system further comprises a standard dot matrix template 7, wherein:

[0128] The standard dot matrix template 7 is placed in a preset position so that the center of the orthographic projection of the projected test image on the standard dot matrix template 7 coincides with the center of the standard dot matrix template 7. The image acquisition device 2 acquires image information of the standard dot matrix template 7 and sends the image information to the image generation device 4, which generates a standard image based on the image information.

[0129] like Figure 3 As shown, in one example, the standard dot matrix template 7 is a pre-designed and printed / displayed dot matrix pattern template, which includes an 11-row × 33-column bright spot matrix, with a total of 363 bright spots. The bright spots are numbered in order and positioned evenly. Each bright spot represents an ideal projection point and serves as a "reference anchor point" in image correction.

[0130] In practical applications, the standard dot matrix template 7 can be placed in a fixed position approximately 7 meters in front of the vehicle's windshield, typically in an imaging area equivalent to the windshield itself. The center of the standard dot matrix template 7 precisely coincides with the center of the test image projection, ensuring visual alignment of the center points of the two images, ensuring high symmetry and accuracy in subsequent offset calculations.

[0131] In some embodiments, the image acquisition device 2 may capture a standard dot matrix template 7 at a specific reference position, such as point c, the center of the eye box. The image acquisition device 2 then sends the captured image to the image processing device 3 for processing to obtain image information. This image information may include data such as the location, number, and arrangement of each bright spot in the image. The image processing device may transmit the processed image information to the image generation device 4. Based on the received image information, the image generation device 4 may generate a digitized image as a standard image, also referred to as a standard dot matrix. This standard image may subsequently be used as a reference template for comparison with the test image to calculate the corresponding distortion correction parameters.

[0132] To better understand the data interaction process between the various components in the image distortion correction system, please refer to Figure 4 The image generation device 4 can first project the test image onto the front windshield of the vehicle. The displacement platform 1 drives the image acquisition device 2 to respectively acquire the test image and the corresponding standard image at each reference position. The image acquisition device 2 then processes the two acquired images to obtain distortion correction parameters for the test image at each reference position, and stores the distortion correction parameters in a server of the vehicle or the head-up display device. This completes the pre-task of obtaining the distortion correction parameters.

[0133] In actual applications, the HUD image output by the head-up display device can be pre-corrected directly based on the distortion correction parameters. The image generating device 4 generates a corrected image through the corrected image information and sends the corrected image to the image magnifying device 5. The image magnifying device 5 then magnifies the corrected image and projects it onto the front windshield of the vehicle to form a virtual HUD image for the user to watch.

[0134] Figure 6 FIG. 1 is a block diagram of a controller 300 according to an exemplary embodiment. Figure 6 As shown, the controller 300 may include: a processor 301 , a memory 302 . The controller 300 may also include one or more of a multimedia component 303 , an input / output (I / O) component 304 , and a communication component 305 .

[0135] The processor 301 is used to control the overall operation of the controller 300 to complete all or part of the steps in the above method. The memory 302 is used to store various types of data to support the operation of the controller 300. Such data may include, for example, instructions for any application or method operating on the controller 300, as well as application-related data, such as contact information, sent and received messages, pictures, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 302 or transmitted via the communication component 305. The audio component also includes at least one speaker for outputting audio signals. The I / O component 304 provides an interface between the processor 301 and other interface modules, which may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 305 is used for wired or wireless communication between the controller 300 and other devices. Wireless communication, such as WiFi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more thereof, is not limited here. Therefore, the corresponding communication component 305 may include: a WiFi module, a Bluetooth module, an NFC module, etc.

[0136] In an exemplary embodiment, the controller 300 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned method.

[0137] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When the program instructions are executed by a controller, the steps of the above-described method are implemented. For example, the computer-readable storage medium may be the memory 302 including the program instructions. The program instructions may be executed by the processor 301 of the controller 300 to complete the steps of the above-described method.

[0138] Figure 7 is a block diagram of a vehicle provided in an embodiment of the present application, such as Figure 7 As shown, the vehicle 400 includes the controller 300 described above.

[0139] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any audio processing method described in the above method embodiments.

[0140] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0141] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the system or unit can be electrical or other forms.

[0143] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0144] In addition, the functional units in the various embodiments of the application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software program modules.

[0145] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable storage unit. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage unit, including a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage unit includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk or optical disk, etc., various media that can store program codes.

[0146] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage unit, which may include: a flash drive, a read-only storage unit, a random access memory, a magnetic disk or an optical disk, etc.

[0147] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, various simple modifications can be made to the technical solution of the present application, and these simple modifications all fall within the scope of protection of the present application.

[0148] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner unless there is any contradiction. In order to avoid unnecessary repetition, this application will not further describe various possible combinations.

[0149] In addition, the various implementation methods of the present application may be arbitrarily combined, and as long as they do not violate the concept of the present application, they should also be regarded as the contents disclosed in the present application.

[0150] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0151] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0152] The embodiments, implementation methods and related technical features of the present application can be combined and replaced with each other without conflict.

[0153] The above are merely preferred embodiments of the present application and do not constitute any form of limitation to the present application. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A method for correcting image distortion, characterized in that: Applied to an image distortion correction system, the method includes: determining a plurality of reference positions based on an eye box range of the head-up display device; At each reference position, respectively capturing a set test image and a standard image corresponding to each test image to obtain a projection feature difference between the test image and the standard image; generating distortion correction parameters according to the projection feature differences, and performing distortion correction processing on the image to be projected based on the distortion correction parameters; The reference position is the eye position of the user when observing the image of the head-up display device, the test image is an image with distortion defects, and the standard image is an image without distortion defects.

2. The method according to claim 1, characterized in that The multiple reference positions are located on the same target horizontal line, and the target horizontal line divides the eye box range into two symmetrical parts.

3. The method according to claim 2, characterized in that The determining of multiple reference positions based on the eye box range of the head-up display device includes: Based on the target horizontal line and the eye box range, a plurality of reference points and the coordinates of each of the reference points are determined to obtain the plurality of reference positions.

4. The method according to claim 3, characterized in that The step of determining a plurality of reference points and the coordinates of each reference point based on the target horizontal line and the eye box range to obtain the plurality of reference positions includes: The center of the eye box range is determined as the center point, the intersection point of the target horizontal line with the left boundary of the eye box range is determined as the left endpoint, and the intersection point with the right boundary of the eye box range is determined as the right endpoint; Using the target horizontal line as a connecting line, the midpoint of the line segment between the left endpoint and the center point is determined as the midpoint of the left area, and the midpoint of the line segment between the right endpoint and the center point is determined as the midpoint of the right area; The left endpoint, the midpoint of the left area, the center point, the midpoint of the right area and the right endpoint are respectively used as the reference positions and the coordinates of each reference position are obtained.

5. The method according to claim 4, characterized in that The step of taking the left endpoint, the midpoint of the left region, the center point, the midpoint of the right region, and the right endpoint as the reference positions and obtaining the coordinates of each reference position includes: The left limit position that meets the preset conditions is used as the coordinate of the left endpoint, and the right limit position is used as the coordinate of the right endpoint; the preset conditions include that the acquisition range of the test image at the current position covers the eye box range; Performing linear interpolation processing on the coordinates of the left endpoint and the coordinates of the right endpoint to obtain the coordinates of the center point; Linear interpolation is performed on the coordinates of the center point and the coordinates of the left endpoint to obtain the coordinates of the center point of the left area, and linear interpolation is performed on the coordinates of the center point and the coordinates of the right endpoint to obtain the coordinates of the center point of the right area.

6. The method according to claim 1, characterized in that Generating distortion correction parameters according to the projection feature differences, and performing distortion correction processing on the image to be projected based on the distortion correction parameters, includes: Preprocessing the test image and the standard image respectively to obtain a plurality of first feature points of the test image and a plurality of second feature points of the standard image; wherein each first feature point corresponds to one second feature point; The distortion correction parameter is generated based on the offset between each of the first feature points and the corresponding second feature point.

7. The method according to claim 6, characterized in that Preprocessing the test image and the standard image respectively to obtain a plurality of first feature points of the test image and a plurality of second feature points of the standard image includes: Performing noise reduction processing on the test image and the standard image respectively to obtain a first noise-reduced image corresponding to the test image and a second noise-reduced image corresponding to the standard image; performing binarization processing on the first denoised image and the second denoised image respectively to obtain a plurality of first feature point contours corresponding to the test image and a plurality of second feature point contours corresponding to the standard image; Each of the first feature point contours and each of the second feature point contours are mapped to a set coordinate system to obtain the first feature points of the test image and the second feature points of the standard image.

8. The method according to claim 6 or 7, characterized in that The generating the distortion correction parameter based on the offset between each of the first feature points and the corresponding second feature point includes: obtaining a lateral offset based on a difference between the horizontal coordinates of the first feature point and the corresponding second feature point; A longitudinal offset is obtained based on a difference between the longitudinal coordinates of the first feature point and the corresponding second feature point, and the lateral offset and the longitudinal offset are used as the distortion correction parameters.

9. An image distortion correction system, characterized in that: It includes an image generating device, an image acquiring device and an image processing device that are communicatively connected to each other, wherein: The image generating device is used to project a test image and a standard image corresponding to the test image; The image acquisition device is configured to determine a plurality of reference positions based on an eye box range of the head-up display device, and to respectively acquire the test image projected by the image generation device and the standard image corresponding to each test image at each reference position; The image processing device processes the test image and the standard image acquired by the image acquisition device to obtain a projection feature difference between the test image and the standard image, generates distortion correction parameters according to the projection feature difference, and performs distortion correction processing on the image to be projected based on the distortion correction parameters; The reference position is the eye position of the user when observing the image of the head-up display device, the test image is an image with distortion defects, and the standard image is an image without distortion defects.

10. The image distortion correction system according to claim 9, wherein: The system further comprises a displacement platform, wherein: The image acquisition device is arranged on the displacement platform, and the displacement platform drives the image acquisition device to move so as to determine the plurality of reference positions.

11. The image distortion correction system according to claim 10, wherein: The system further comprises an image magnification device connected to the image generating device, wherein: The image magnifying device includes a first reflecting mirror and a second reflecting mirror, wherein the reflecting area of the second reflecting mirror is larger than the reflecting area of the first reflecting mirror; The first reflector is used to receive the image projected by the image generating device and reflect the received image to the second reflector. The second reflector expands the light angle of the image to obtain an enlarged image and reflects the enlarged image to the target position.

12. The image distortion correction system according to claim 11, wherein: The system also includes a standard dot matrix template, wherein: The standard dot matrix template is placed at a preset position so that the center of the orthographic projection of the projected test image on the standard dot matrix template coincides with the center of the standard dot matrix template; The image acquisition device acquires image information of the standard dot matrix template and sends the image information to the image generation device. The image generation device generates the standard image according to the image information.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

14. A controller having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

15. A vehicle, characterized in that: Including the controller according to claim 14.

16. A computer program product, characterized in that The method comprises a computer program or instructions, which implements the steps of the method according to any one of claims 1 to 8 when the computer program or instructions are executed by a processor.

Images