HUD distortion correction method based on cubic Bezier curve fitting

Abstract

The invention provides an HUD distortion correction method based on cubic Bezier curve fitting. The HUD distortion correction method comprises the following steps: S1, collecting an initial source image and a corresponding initial source distortion image; s2, acquiring control point coordinates of the initial source image and the initial source distortion image by using a corner detection algorithm; s3, according to the control point coordinates of the initial source image and the initial source distorted image, obtaining a predistortion function of the initial source image by using cubic Bezier curve fitting; s4, performing predistortion on the initial source image by using the predistortion function to obtain a predistortion image; and S5, replacing the initial source image with the pre-distorted image, and inputting the pre-distorted image into the HUD device to obtain a corrected HUD image. According to the method, the predistortion image is projected to the automobile front windshield, so that the relative coordinate points of the HUD image imaged on the automobile front windshield and the initial source image coincide, the display effect of the improved image is consistent with the display effect of the initial source image, and the driving feeling of a driver is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of HUD distortion correction, and in particular to a HUD distortion correction method based on cubic Bezier curve fitting. Background Art

[0002] With the development of modern automotive electronic technology, HUD (Head Up Display) technology, as an emerging driving assistance technology, is gradually replacing traditional center console instrument display technology. It can display key driving information such as speed and navigation directly on the windshield in front of the driver without having to lower the head or turn the head, allowing the driver to see important driving information such as speed and navigation, greatly improving driving safety and driving experience.

[0003] However, due to factors such as the uneven curvature of the windshield, the image projected on the windshield is distorted, and the originally regular and clear image becomes misplaced and unclear, resulting in a bad driving experience for the driver and even leading to safety hazards such as misjudgment of information. Summary of the invention

[0004] The present invention proposes a HUD distortion correction method based on cubic Bezier curve fitting, which solves the problem of image distortion caused by HUD image distortion in the prior art.

[0005] The technical solution of the present invention is achieved in this way:

[0006] A first aspect of the present invention provides a HUD distortion correction method based on cubic Bezier curve fitting, comprising the following steps:

[0007] S1, collecting the initial source image and the corresponding initial source distorted image;

[0008] S2, using a corner detection algorithm to obtain the coordinates of the control points of the initial source image and the initial source distorted image;

[0009] S3, obtaining a pre-distortion function of the initial source image by fitting a cubic Bezier curve according to the control point coordinates of the initial source image and the initial source distorted image;

[0010] S4, pre-distorting the initial source image using a pre-distortion function to obtain a pre-distorted image;

[0011] S5, inputting the pre-distorted image into the HUD device instead of the initial source image to obtain a corrected HUD image.

[0012] Specifically, in step S2, a checkerboard image is used as a standard source image, and the corner points of the checkerboard are used as control points. The control point coordinates of the initial source image and the initial source distorted image are obtained by a corner point detection algorithm in MATLAB or OpenCV.

[0013] Specifically, step S3 includes the following steps:

[0014] Get the coordinates of any control point on the X-axis and Y-axis of the initial source distorted image;

[0015] Get the coordinates of any corresponding X-axis and Y-axis control points in the original source image;

[0016] According to the corresponding relationship between the control point coordinates of the initial source image and the initial source distorted image, the mapping relationship between the standard coordinate points in each row and column of the initial source image and the initial source distorted image is obtained. The mapping relationship between the standard coordinate points in each row and column is fitted by using a cubic Bezier curve to obtain the pre-distortion function of each row and column of the initial source image:

[0017] B(X)=(1-t)3·X0+3·(1-t)2·t·X1+3·(1-t)·t2·X2+t3·X3

[0018] B(Y)=(1-t)3·Y0+3·(1-t)2·t·Y1+3·(1-t)·t2·Y2+t3·Y3

[0019] Among them, X0, X1, X2, and X3 represent the X-axis coordinates of each column of the chessboard nodes of the initial source image; Y0, Y1, Y2, and Y3 represent the Y-axis coordinates of each column of the chessboard nodes of the initial source image; t represents the fitting parameter, and the value range of t is [0,1].

[0020] Furthermore, after obtaining the control point coordinates of the initial source image and the initial source distorted image, the initial source distorted image is first partitioned according to the curvatures of different areas of the car windshield, and then the mapping relationship between the standard coordinate points of each partition is fitted respectively to obtain the pre-distortion functions corresponding to different partitions of the entire initial source image.

[0021] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the HUD distortion correction method are implemented.

[0022] A third aspect of the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the HUD distortion correction method are implemented.

[0023] Compared with the prior art, the beneficial effect of the present invention lies in: the present invention obtains the control point coordinates of the initial source image and the initial source distorted image through a corner detection algorithm, and then uses a cubic Bezier curve function to fit the pre-distortion function of the initial source image, and pre-distorts each pixel point of the initial source image through the pre-distortion function to obtain a pre-distorted image, and projects the pre-distorted image onto the front windshield of the car so that the relative coordinate points of the pre-distorted image and the initial source image coincide, so that the display effect of the improved image is consistent with the display effect of the initial source image, which greatly improves the driving experience of the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 This is a flow chart of a HUD distortion correction method based on cubic Bezier curve fitting according to the present invention;

[0026] Figure 2 Schematic diagram of an initial source image in an embodiment of the present invention;

[0027] Figure 3 Schematic diagram of an initial source distortion image in an embodiment of the present invention;

[0028] Figure 4 is a dot matrix image of an initial source image in an embodiment of the present invention;

[0029] Figure 5 It is a dot matrix diagram of the initial source distorted image in the embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Reference Figures 1 to 5 In a first aspect, the present invention provides a HUD distortion correction method based on cubic Bezier curve fitting, comprising the following steps:

[0032] S1, collecting the initial source image and the corresponding initial source distorted image;

[0033] S2, using a corner detection algorithm to obtain the coordinates of the control points of the initial source image and the initial source distorted image;

[0034] S3, obtaining a pre-distortion function of the initial source image by fitting a cubic Bezier curve according to the control point coordinates of the initial source image and the initial source distorted image;

[0035] S4, pre-distorting the initial source image using a pre-distortion function to obtain a pre-distorted image;

[0036] S5, inputting the pre-distorted image into the HUD device instead of the initial source image to obtain a corrected HUD image.

[0037] Specifically, in step S1, the initial source image is obtained by using a machine vision method, using a distortion-free industrial camera and a PC end to link with each other through an image acquisition card, such as Figure 2 As shown; the initial source image is projected by the HUD projection unit, and the image is imaged on the front windshield after passing through the plane mirror and the free-form surface mirror. The initial source distortion image on the front windshield is obtained using a distortion-free industrial camera, as shown Figure 3 shown.

[0038] Specifically, in step S2, since it is not easy to extract the corresponding coordinate points after the driving information image displayed by the actual HUD is acquired, this embodiment uses a checkerboard image as the standard source image, and uses the corner points of the checkerboard as control points. The control point coordinates of the initial source image and the initial source distorted image are obtained through a corner point detection algorithm in matlab or opencv.

[0039] In this embodiment, the initial source image is an undistorted, regular image. A coordinate system is directly established with the upper left corner as the origin, and a standard control point is selected, such as Figure 4 After the image processing software simulates the distortion of the car's front windshield, the checkerboard corner points are selected as control points, and the Harris algorithm is used to obtain the coordinates of the pixel points in the initial source distorted image, with a total of 4 rows and 4 columns of 16 control point coordinates, as shown in Figure 1. Figure 5 As shown; and through verification, the Harris algorithm extracts the corner point coordinate error value on the distorted image within about 0.5%.

[0040] Specifically, step S3 includes the following steps:

[0041] Get the coordinates of any control point on the X-axis and Y-axis of the initial source distorted image;

[0042] Get the coordinates of any corresponding X-axis and Y-axis control points in the original source image;

[0043] According to the corresponding relationship between the control point coordinates of the initial source image and the initial source distorted image, the mapping relationship between the standard coordinate points in each row and column of the initial source image and the initial source distorted image is obtained, and the mapping relationship between the standard coordinate points in each row and column is fitted by using a cubic Bezier curve to obtain the pre-distortion function of each row and column of the initial source image.

[0044] In the specific implementation process, a coordinate system is established with the upper left corner of the initial source distorted image as the origin, 4×4 control points are selected in the initial source distorted image, and after the control points are selected, the coordinates of the control points are read from the established coordinate system to obtain a set of control points:

[0045]

[0046] First, the coordinates of the initial source distorted image are interpolated for any column of distortion, and the coordinates of the initial source image for any X-axis are determined to be (X2, Y0), (X2, Y1), (X2, Y2), (X2, Y3), and the coordinates of the checkerboard nodes corresponding to any X-axis in the initial source distorted image are determined to be (C2X, C2Y), (C6X, C6Y), (C10X, C10Y), (C14X, C14Y);

[0047] According to the point-to-point correspondence between the initial source image and the initial source distorted image, the mapping relationship between the standard coordinate points in this column can be obtained as follows:

[0048] (X2,Y0)→(C2X,C2Y);

[0049] (X2,Y1)→(C6X,C6Y);

[0050] (X2,Y2)→(C10X,C10Y);

[0051] (X2,Y3)→(C14X,C14Y);

[0052] By performing interpolation operations on the distortion of each column of the initial source distorted image, the mapping relationship between the standard coordinate points of each column of the initial source image and the standard coordinate points of each column of the initial source distorted image can be obtained. According to the mapping relationship, the cubic Bezier curve function (i.e., pre-distortion function) between the initial source image and the standard coordinate points of each column of the initial source distorted image can be obtained by fitting:

[0053] B(X)=(1-t)3·X0+3·(1-t)2·t·X1+3·(1-t)·t2·X2+t3·X3

[0054] Similarly, through the coordinates of the initial source distorted image, the distortion of any row is interpolated to determine that the coordinates of the initial source image for any Y axis are (X0, Y1), (X1, Y1), (X2, Y1), (X3, Y1), and the coordinate values ​​of the chessboard nodes corresponding to any Y axis in the initial source distorted image are (C4X, C4Y), (C5X, C5Y), (C6X, C6Y), (C7X, C7Y);

[0055] According to the point-to-point correspondence between the initial source image and the initial source distorted image, the mapping relationship between the standard coordinate points in the row in the initial source image and the initial source distorted image can be obtained as follows:

[0056] (X0,Y1)→(C4X,C4Y);

[0057] (X1,Y1)→(C5X,C5Y);

[0058] (X2,Y1)→(C6X,C6Y);

[0059] (X3,Y1)→(C7X,C7Y);

[0060] By performing interpolation operations on the distortion of each row of the initial source distorted image, the mapping relationship between the standard coordinate points of each row of the initial source image and the standard coordinate points of each row of the initial source distorted image can be obtained. According to the mapping relationship, the cubic Bezier curve function (i.e., pre-distortion function) between the initial source image and the standard coordinate points of each row of the initial source distorted image can be obtained by fitting:

[0061] B(Y)=(1-t)3·Y0+3·(1-t)2·t·Y1+3·(1-t)·t2·Y2+t3·Y3

[0062] Among them, X0, X1, X2, and X3 represent the X-axis coordinates of each column of the chessboard nodes of the initial source image; Y0, Y1, Y2, and Y3 represent the Y-axis coordinates of each column of the chessboard nodes of the initial source image; t represents the fitting parameter, which represents the parameter under different environments and reflection states of the windshield, and the value range of t is [0,1]. The Bezier curve can flexibly create and edit function curves by controlling the four points on the curve (the starting point, the end point, and the two separated middle points). In the face of the problem of inconsistent curvature of the car windshield, the parameter t can be flexibly changed to make corresponding changes to achieve the effect of coping with different curvatures.

[0063] Furthermore, since the curvature of the car windshield is not uniform, after obtaining the control point coordinates of the initial source image and the initial source distorted image, the initial source distorted image is first partitioned according to the curvatures of different areas of the car windshield, and then the mapping relationship between the standard coordinate points of each partition is fitted to obtain the pre-distortion functions corresponding to different partitions of the entire initial source image.

[0064] For example, the initial source image is a standard horizontal rectangle, such as Figure 2 As shown in , after various distortion processes such as refraction of the glass, the original source distorted image actually projected onto the front windshield of the car is an irregular shape, such as Figure 3 As shown. If the display effect of the projected image is to be consistent with the initial source image, the initial source image needs to be pre-distorted before the image is projected. The principle of pre-distortion is to obtain the coordinates of the standard pixel points of the initial source image and the initial source distorted image, compare them multiple times, and then fit the initial source image and the initial source distorted image through a cubic Bezier curve. According to the mapping relationship, the initial source image is pre-distorted before projection, and then the corrected image obtained after being projected onto the front windshield of the car is consistent with the display effect of the initial source image, and the details of the corrected image are smooth and complete, and the display is clear, which is conducive to the driver to obtain the required data, greatly improving the driver's driving experience and driving safety.

[0065] A second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the HUD distortion correction method are implemented.

[0066] A third aspect of the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the HUD distortion correction method are implemented.

[0067] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A HUD distortion correction method based on cubic Bezier curve fitting, characterized in that: The following steps are involved: S1, collecting the initial source image and the corresponding initial source distorted image; S2, using a corner detection algorithm to obtain the control point coordinates of the initial source image and the initial source distorted image; S3, obtaining a pre-distortion function of the initial source image by fitting a cubic Bezier curve according to the coordinates of the control points of the initial source image and the initial source distorted image; S4, pre-distorting the initial source image using a pre-distortion function to obtain a pre-distorted image; S5, inputting the pre-distorted image into the HUD device instead of the initial source image to obtain a corrected HUD image.

2. The HUD distortion correction method based on cubic Bezier curve fitting according to claim 1, characterized in that: In step S2, a checkerboard image is used as a standard source image, and the corner points of the checkerboard are used as control points. The control point coordinates of the initial source image and the initial source distorted image are obtained by a corner point detection algorithm in MATLAB or OpenCV.

3. The HUD distortion correction method based on cubic Bezier curve fitting according to claim 1, characterized in that: Step S3 includes the following steps: Get the coordinates of any control point on the X-axis and Y-axis of the initial source distorted image; Get the coordinates of any corresponding X-axis and Y-axis control points in the original source image; According to the corresponding relationship between the control point coordinates of the initial source image and the initial source distorted image, the mapping relationship between the standard coordinate points in each row and column of the initial source image and the initial source distorted image is obtained. The mapping relationship between the standard coordinate points in each row and column is fitted by using a cubic Bezier curve to obtain the pre-distortion function of each row and column of the initial source image: B(X)=(1-t) 3 ·X0+3·(1-t) 2 ·t·X1+3·(1-t)·t 2 ·X2+t 3 ·X3 B(Y)=(1-t) 3 ·Y0+3·(1-t) 2 ·t·Y1+3·(1-t)·t 2 ·Y2+t 3 ·Y3 Among them, X0, X1, X2, and X3 represent the X-axis coordinates of each column of the chessboard nodes of the initial source image; Y0, Y1, Y2, and Y3 represent the Y-axis coordinates of each column of the chessboard nodes of the initial source image; t represents the fitting parameter, and the value range of t is [0,1].

4. The HUD distortion correction method based on cubic Bezier curve fitting according to claim 3, characterized in that: After obtaining the control point coordinates of the initial source image and the initial source distorted image, the initial source distorted image is first partitioned according to the curvature of different areas of the car windshield, and then the mapping relationship between the standard coordinate points of each partition is fitted to obtain the pre-distortion functions corresponding to different partitions of the entire initial source image.

5. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the HUD distortion correction method according to any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the HUD distortion correction method according to any one of claims 1 to 4 are implemented.

Images