HUD distortion correction method based on multiple linear regression
By designing the calibration source map and constructing the distortion model using the multivariate linear regression method, the problem of HUD image distortion was solved, full-area distortion correction was achieved, the cost was reduced and the correction effect was improved.
Patent Information
- Application Number
- CN202510929847.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-07
- Publication Date
- 2025-10-10
AI Technical Summary
In existing HUD technology, distortion correction methods require additional hardware chips or the storage of a large number of key point coordinates, resulting in high costs and inability to effectively correct regional distortion beyond the key point range.
A method based on multiple linear regression is adopted to design a calibration source map and collect the initial distorted image through multiple mirror reflections. The coordinates of key points are extracted, and a multiple linear regression model is constructed to fit the distortion parameters and achieve image correction.
No additional hardware chips are required, only a small amount of distortion parameter storage is needed, and distortion correction can be performed in the entire imaging area, which saves costs and has good effects.
Smart Images
Figure CN120765518A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of HUD image distortion processing, and in particular to a HUD distortion correction method based on multivariate linear regression. Background Art
[0002] Head-Up Display (HUD) technology, as a modern automotive driver-assistance technology, projects important driving information directly into the driver's field of view, significantly improving driving safety and comfort. However, in practical applications, various factors, such as design and processing errors, can cause HUD image distortion. This can cause the originally regular and clear image to become misaligned and unclear, resulting in a poor driving experience for the driver and even safety hazards such as misjudgment of information. In existing technologies, HUD vendors generally lack the ability to develop software algorithms specifically for image distortion processing. Therefore, most HUD designs incorporate a separate warping IC, provided by a third party, specifically designed to handle HUD image distortion. A few HUD vendors that do not rely on distortion chips first calibrate multiple key points within the HUD's active area, then perform bilinear interpolation processing on other locations to derive the deformation relationship between the distorted image and the original image, allowing for further distortion processing.
[0003] However, the warpingIC hardware solution can only calibrate and correct key points within the fixed number of display ranges provided by its chip, and can only address some types of distortion correction provided by the chip manufacturer during the after-sales phase. Furthermore, the use of warpingIC will incur additional hardware costs. Using the existing industry-standard key point interpolation method requires storing a large amount of key point coordinate information within the HUD. The amount of data is proportional to the distortion correction effect of the HUD. There is insufficient space within the HUD to store a large amount of key point calibration data, which affects its correction effect. Existing distortion correction methods cannot perform image correction processing for areas outside the key point range. Summary of the Invention
[0004] The technical problem to be solved by the present invention is: In order to overcome the above technical problems, the present invention provides a HUD distortion correction method based on multivariate linear regression, which solves the problem of image distortion caused by HUD image distortion. The method is applicable to HUDs of various resolutions, sizes, shapes and imaging principles, and can perform distortion correction processing on them. Only a very small space is required to store the distortion parameters, and there is no need to retain a large amount of key point coordinate data.
[0005] The technical solution adopted by the present invention to solve the technical problem is: a HUD distortion correction method based on multivariate linear regression, comprising the following steps: Step 1: Design the corresponding calibration source image based on the HUD resolution and effective area to be corrected; Step 2: Collect the initial distorted image corresponding to the calibration source image; Step 3: Obtain the corresponding coordinate values of the key points of the calibration source image and the initial distorted image respectively; Step 4: Data preprocessing and normalization: Scale the coordinates in the initial distorted image to the same range as the calibration source image, and use the anchor point coordinates as the origin to obtain unified normalized coordinates; Step 5: Calibrate the source image coordinate data polynomial and increase the dimension to obtain high-dimensional feature data; Step 6: Use the multiple linear regression model to model the distortion data in the horizontal and vertical directions to obtain the distortion parameters; Step 7: Pre-distort the HUD image according to the distortion parameters and project the image to obtain the corrected result.
[0006] In step 1, the calibration source image is designed to cover the entire HUD active area and be a regular uniform dot pattern or grid pattern slightly larger than the active area.
[0007] In step 2, the calibration source image is projected through the HUD projection unit. After multiple mirror reflections inside the HUD projection unit, the calibration source image is imaged on the front windshield. The HUD image is collected at the eye box by a distortion-free industrial camera to obtain the initial distorted image.
[0008] The calibration source image is an undistorted regular image. The coordinate system can be directly established with the upper left corner of the image as the source point, and the positive directions of the X-axis and Y-axis to the right and downward in sequence, to directly obtain the coordinate values of n key positions in the calibration source image; the coordinate information corresponding to the n key positions in the calibration source image in the initial distorted image is extracted.
[0009] The key positions are points of the dot pattern or intersections of the grid pattern.
[0010] The coordinate information of n key positions in the calibration source image within the initial distorted image is extracted using OpenCV's corner detection, checkerboard detection or feature point extraction methods.
[0011] In step 4, data preprocessing and normalization specifically include the following steps: Step 41: Calculate the ratio of the length and width of the calibration source image to the length and width of the initial distorted image, and record the smallest value as its distortion ratio; multiply the coordinates of n key positions in the calibration source image in the initial distorted image by the distortion ratio, and scale the coordinates in the initial distorted image to the same range as the calibration source image; Step 42: Record the coordinate position of the anchor point, which is generally the center of the image or obtained based on the mid-eye position calibration. The coordinate values of the n key positions of the calibration source image and the coordinate values of the n key positions of the calibration source image in the initial distorted image are respectively normalized with the anchor point coordinates as the origin to obtain unified coordinates. The coordinates of the calibration source image are labeled as source coordinates (x, y), and the coordinates of the initial distorted image are labeled as distortion coordinates (u, v).
[0012] In step 5, multiple linear regression can be used to find the relationship between the dependent variable and multiple independent variables. This is more suitable for cases where the relationship between the independent and dependent variables is linear. However, in reality, many independent and dependent variables do not have a completely linear relationship. In this case, a method can be used to convert the variables into linear ones before performing multiple linear regression. The source coordinates (x, y) are used as independent variables and polynomial transformations are performed. By increasing the degree of the independent variables, the source coordinate data is mapped to a higher-dimensional space, and the result of the source coordinate data projection in the higher-order space is obtained.
[0013] In step 6, the following regression model is constructed for the lateral distortion of the HUD: u i =w0+w1x i +w2y i +w3x i 2 +w4x i y i +w5y i 2 +w6x i 3 +w7x i y i 2 +w8x i 4 +w9x i 3 y i +w 10 x i 2 y i 2 ; The following regression model is constructed for the longitudinal distortion of the HUD: v i =w0 ’ +w1 ’ x i +w2 ’ y i +w3 ’ x i 2 +w4 ’ x i y i +w5 ’ xi 3 +w6 ’ x i 2 y i +w7 ’ x i y i 2 +w8 ’ y i 3 +w9 ’ x i 3 y i +w 10 ’ x i y i 3 +w 11 ’ y i 4 ; wherein, i represents the i-th in n groups of data; u, v, x, y are collected data, which are known items; w is a coefficient to be solved, which is an unknown item.
[0014] Because of the irregularity and asymmetry of HUD distortion, regression models are respectively constructed in horizontal and vertical directions, and the corresponding characteristic components and the number of polynomial coefficients are also adjusted according to actual situation and experimental analysis. Finally, n groups of collected data are substituted into the above equations to solve the distortion model parameters w0-w 10 and w0'-w 11 '.
[0015] In step 7, the mapping relationship of arbitrary pixel coordinates of the calibration source image and the initial distortion image is obtained according to the obtained distortion model parameters, that is, the mapping relationship of arbitrary pixel coordinates of the input original image and the observation image presented on the windshield glass is obtained, and the target image needed to be presented by the HUD is inversely distorted according to the mapping relationship, so as to obtain the preprocessed corrected HUD image, and the image is projected by the HUD to achieve the effect of correcting the image.
[0016] The beneficial effects of the present invention are as follows: a HUD distortion correction method based on multivariate linear regression of the present invention does not require additional hardware chip processing, but only requires the deployment of a corresponding algorithm program on the vehicle computer / host computer, the writing of the distortion parameters generated by the host computer calibration into the HUD, and the calling of the parameters for image preprocessing during projection, which can save costs and achieve the same or even better picture correction effect; only a very small amount of distortion coefficients need to be stored after calibration to achieve the same or even better picture correction effect; the distortion correction processing can be performed on the entire imaging display area, so that the best effective area can be selected within the entire imaging area, which is conducive to handling the situation where the effective area encountered in actual production is inconsistent with the design. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The present invention will be further described below with reference to the accompanying drawings and examples.
[0018] Figure 1 This is a flow chart of a HUD distortion correction method based on multiple linear regression of the present invention.
[0019] Figure 2 It is the calibration source map involved in the specific implementation of the present invention.
[0020] Figure 3 It is the initial distorted image involved in the specific implementation of the present invention.
[0021] Figure 4 This is a comparison diagram of the coordinates of a key point before and after HUD distortion in a specific embodiment of the present invention.
[0022] Figure 5 It is the corrected HUD image after inverse distortion involved in the specific implementation of the present invention.
[0023] Figure 6 This is the corrected image effect ultimately achieved in the specific implementation of the present invention. DETAILED DESCRIPTION
[0024] The present invention will now be described in further detail with reference to the accompanying drawings. It should be emphasized that the following description is merely exemplary and is not intended to limit the scope of the present invention and its application.
[0025] like Figure 1 As shown, the HUD distortion correction method based on multivariate linear regression of the present invention includes the following steps: Step 1: Design the corresponding calibration source image according to the HUD resolution and effective area to be corrected: the calibration source image can be designed to cover the entire HUD effective area and be a regular uniform dot pattern or grid pattern slightly larger than the effective area, such as Figure 2 shown.
[0026] Step 2: Collect the initial distorted image corresponding to the calibration source image: Specifically, the calibration source image is projected through the HUD projection unit. After multiple mirror reflections inside the HUD projection unit, the calibration source image is imaged on the front windshield. The HUD image is collected at the eye box by a distortion-free industrial camera to obtain the initial distorted image, such as Figure 3 shown.
[0027] Step 3: Obtain the coordinate values of key points in the calibration source image and the initial distorted image: The calibration source image is a regular, undistorted image. A coordinate system can be established with the upper-left corner of the image as the source point, and the positive X- and Y-axes extending downward and to the right, respectively. The coordinate values of n key locations in the calibration source image can be directly obtained. These key locations are points in a dot matrix or intersections in a grid. OpenCV methods such as corner detection, checkerboard detection, or feature point extraction can be used to extract the coordinate information corresponding to the n key locations in the calibration source image within the initial distorted image.
[0028] Step 4: Data preprocessing and normalization: scaling the coordinates in the initial distorted image to the same range as the calibration source image, and obtaining unified normalized coordinates with the anchor point coordinates as the origin; specifically, the following steps are included: Step 41: Calculate the ratio of the length and width of the calibration source image to the length and width of the initial distorted image, and record the smallest value as its distortion ratio; multiply the coordinates of n key positions in the calibration source image in the initial distorted image by the distortion ratio, and scale the coordinates in the initial distorted image to the same range as the calibration source image, such as Figure 4 As shown, the blue dots represent the distorted coordinates, the red dots represent the source coordinates, and the entire figure represents the coordinate comparison of a key point before and after HUD distortion.
[0029] Step 42: Record the coordinate position of the anchor point, which is generally the center of the image or obtained based on the mid-eye position calibration. The coordinate values of the n key positions of the calibration source image and the coordinate values of the n key positions of the calibration source image in the initial distorted image are respectively normalized with the anchor point coordinate as the origin to obtain unified coordinates. The coordinates of the calibration source image are labeled as source coordinates (x, y), and the coordinates of the initial distorted image are labeled as distortion coordinates (u, v).
[0030] Step 5: Calibrate the source image coordinate data polynomial and increase the dimension to obtain high-dimensional feature data: Multiple linear regression can be used to find the relationship between the dependent variable and multiple independent variables. It is more suitable for the linear relationship between the independent variables and the dependent variables. However, in fact, the relationship between many independent variables and dependent variables is not completely linear. In this case, the variables can be converted into linear variables through methods and then multiple linear regression can be performed.
[0031] The source coordinates (x, y) are used as independent variables for polynomial transformation. By increasing the degree of the independent variable, the source coordinate data is mapped to a high-dimensional space, and the result of projecting the source coordinate data in a high-order space is obtained. For example, if the polynomial degree p=2 is selected, then a set of point coordinates (x, y) can be expanded to (1, x, y, x 2 ,xy,y 2 ), similarly when p=3, it can be expanded to (1,x,y,x 2 ,xy,y 2 ,x 3 ,x 2 y,xy 2 ,y 3 ).
[0032] Step 6: Screen eigenvalues and build regression models: Use the multivariate linear regression model to model the distorted data in the horizontal and vertical directions to obtain distortion parameters.
[0033] In theory, p can be any positive integer greater than 1. However, if p is too small, some high-dimensional features will be lost, resulting in large errors in the distortion model results. Increasing p improves the model's fit, but also requires more computing power, and excessive data volume can lead to overfitting. Therefore, it is necessary to select appropriate feature values for both the horizontal and vertical directions and construct corresponding regression models.
[0034] Based on extensive experimental results and analysis of HUD distortion optical path imaging, a p value of 4 effectively captures high-dimensional feature data. However, further experimental analysis reveals that lateral distortion is more closely related to the source image's x-coordinate, while having less correlation with the high-dimensional feature components of its y-coordinate. Furthermore, high-order data also results in greater inefficient computing power consumption. Combining theoretical and practical results, the following regression model for HUD lateral distortion was constructed: u i =w0+w1x i +w2y i +w3x i 2 +w4x i y i +w5y i 2 +w6x i 3 +w7x i y i 2 +w8x i 4 +w9x i 3 y i +w 10 x i2 y i 2 ; Similar further analysis experiments revealed that the longitudinal distortion is affected by the combined effects of the x and y coordinates, with little correlation with the high-dimensional feature components of the x coordinate and the secondary feature components of the y coordinate. Therefore, the following regression model was constructed for the longitudinal distortion of the HUD: v i =w0 ’ +w1 ’ x i +w2 ’ y i +w3 ’ x i 2 +w4 ’ x i y i +w5 ’ x i 3 +w6 ’ x i 2 y i +w7 ’ x i y i 2 +w8 ’ y i 3 +w9 ’ x i 3 y i +w 10 ’ x i y i 3 +w 11 ’ y i 4 ; Among them, i represents the i-th one in n groups of data; u, v, x, and y are the collected data and are known items; w is the coefficient to be calculated and is an unknown item.
[0035] Due to the irregularity and asymmetry of HUD distortion, regression models are constructed for both horizontal and vertical directions. The corresponding characteristic components and the number of polynomial coefficients are adjusted according to the actual situation and experimental analysis. Finally, the n sets of collected data are substituted into the above equations and the distortion model parameters w0-w are solved respectively. 10 and w0'-w 11 '.
[0036] Step 7: Pre-distort the HUD image according to the distortion parameters and project the image to obtain the corrected result: According to the obtained distortion model parameters, the mapping relationship between the coordinates of any pixel of the calibration source image and the initial distorted image is obtained, that is, the mapping relationship between the coordinates of any pixel of the input original image and the observation image presented on the windshield is obtained. The target image to be presented by the HUD is reversely distorted based on this mapping relationship, thereby obtaining the pre-processed corrected HUD image, such as Figure 5 As shown, the image is projected by HUD to achieve the effect of correcting the image, as shown in Figure 6 shown.
[0037] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.
Claims
1. A HUD distortion correction method based on multivariate linear regression, characterized in that: The following steps are involved: Step 1: Design the corresponding calibration source image based on the HUD resolution and effective area to be corrected; Step 2: Collect the initial distorted image corresponding to the calibration source image; Step 3: Obtain the corresponding coordinate values of the key points of the calibration source image and the initial distorted image respectively; Step 4: Data preprocessing and normalization: Scale the coordinates in the initial distorted image to the same range as the calibration source image, and use the anchor point coordinates as the origin to obtain unified normalized coordinates; Step 5: Calibrate the source image coordinate data polynomial and increase the dimension to obtain high-dimensional feature data; Step 6: Use the multiple linear regression model to model the distortion data in the horizontal and vertical directions to obtain the distortion model parameters; Step 7: Pre-distort the HUD image according to the distortion model parameters, and project the image to obtain the corrected result.
2. The HUD distortion correction method based on multivariate linear regression according to claim 1, wherein: In step 1, the calibration source image is designed to cover the entire HUD active area and be a regular uniform dot pattern or grid pattern slightly larger than the active area.
3. The HUD distortion correction method based on multivariate linear regression according to claim 1, wherein: In step 2, the calibration source image is projected through the HUD projection unit. After multiple mirror reflections inside the HUD projection unit, the calibration source image is imaged on the front windshield. The HUD image is collected at the eye box by a distortion-free industrial camera to obtain the initial distorted image.
4. The HUD distortion correction method based on multivariate linear regression according to claim 1, wherein: The calibration source image takes the upper left corner of the image as the source point, and establishes a coordinate system with the positive directions of the X-axis and Y-axis to the right and downward in sequence, and directly obtains the coordinate values of n key positions in the calibration source image; extracts the coordinate information corresponding to the n key positions in the calibration source image in the initial distorted image.
5. The HUD distortion correction method based on multivariate linear regression according to claim 4, wherein: The key positions are points of the dot pattern or intersections of the grid pattern.
6. The HUD distortion correction method based on multivariate linear regression according to claim 4, wherein: The coordinate information of n key positions in the calibration source image within the initial distorted image is extracted using OpenCV's corner detection, checkerboard detection or feature point extraction methods.
7. The HUD distortion correction method based on multivariate linear regression according to claim 4, wherein: In step 4, data preprocessing and normalization specifically include the following steps: Step 41: Calculate the ratio of the length and width of the calibration source image to the length and width of the initial distorted image, and record the smallest value as its distortion ratio; multiply the coordinates of n key positions in the calibration source image in the initial distorted image by the distortion ratio, and scale the coordinates in the initial distorted image to the same range as the calibration source image; Step 42: Record the coordinate position of the anchor point, and use the coordinate values of the n key positions of the calibration source image and the coordinate values of the n key positions of the calibration source image in the initial distorted image to obtain unified normalized coordinates with the anchor point coordinates as the origin. The coordinates of the calibration source image are marked as source coordinates (x, y), and the coordinates of the initial distorted image are marked as distortion coordinates (u, v).
8. The HUD distortion correction method based on multivariate linear regression according to claim 7, wherein: In step 5, the source coordinates (x, y) are used as independent variables for polynomial transformation. By increasing the degree of the independent variable, the source coordinate data is mapped to a high-dimensional space, and the result of projecting the source coordinate data in the high-order space is obtained.
9. The HUD distortion correction method based on multivariate linear regression according to claim 8, wherein: In step 6, the following regression model is constructed for the lateral distortion of the HUD: u i =w0+w1x i +w2y i +w3x i 2 +w4x i y i +w5y i 2 +w6x i 3 +w7x i y i 2 +w8x i 4 +w9x i 3 y i +w 10 x i 2 y i 2 ; The following regression model is constructed for the longitudinal distortion of the HUD: v i =w0 ’ +w1 ’ x i +w2 ’ y i +w3 ’ x i 2 +w4 ’ x i y i +w5 ’ x i 3 +w6 ’ x i 2 y i +w7 ’ x i y i 2 +w8 ’ y i 3 +w9 ’ x i 3 y i +w 10 ’ x i y i 3 +w 11 ’ y i 4 4 Finally, the n sets of collected data, i.e., n sets of source coordinates (x, y) and distortion coordinates (u, v), are substituted into the above equations and the distortion model parameters w0-w are solved respectively. 10 and w0'-w 11 '.
10. The HUD distortion correction method based on multivariate linear regression according to claim 1, wherein: In step 7, the mapping relationship between the calibration source image and the coordinates of any pixel in the initial distorted image is obtained based on the obtained distortion model parameters. The target image to be presented by the HUD is reversely distorted based on this mapping relationship to obtain a preprocessed corrected HUD image, which is then projected using the HUD to achieve the effect of correcting the image.