AR-HUD distortion calibration method with FOV arranged in separate light-emitting areas

Abstract

The present invention discloses an AR-HUD distortion calibration method with FOV arranged in separated light-emitting areas, comprising the following steps: obtaining a projection image decoding pattern of a structured light image; using a viewpoint camera to shoot and decode the structured light image projected by the AR-HUD; determining the actual light-emitting area or edge of the AR-HUD according to the decoded sample image; determining a dot matrix with a clear correspondence relationship with the projected image inside the light-emitting area, and calculating the correspondence relationship; determining a reference dot matrix on the sample image and its characteristics; determining the dot matrix on the corresponding projected image according to the reference dot matrix; calculating the difference between the reference dot matrix and the dot matrix on the corresponding projected image to obtain a pre-distortion amount; and determining a pre-distortion parameter according to the pre-distortion amount. The present invention achieves a wider and more accurate AR-HUD display effect by determining a distortion-free virtual image in the actual light-emitting area of ​​the AR-HUD, using a structured light image to intuitively reflect the light area or its edge, and determining a reference dot matrix based on this, and then calculating the pre-distortion parameters.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an AR-HUD distortion calibration method in which FOV is arranged in separate light emitting areas. Background Art

[0002] like Figure 6 As shown, the optical path of AR-HUD is complex, with multiple transmissions and reflections. In view of the uniformity of transmission, reflection and materials such as windshield, the tolerance of the installation of various devices and the free-form surface of the windshield, the final imaging usually has distortion that needs to be calibrated.

[0003] For example, in the publication (announcement) number: CN117872602A, the calibration of AR-HUD is completed with the help of structured light. The general process of its main scheme is: first, determine the dot matrix rows and columns and the projection dot matrix; then, according to the corresponding coordinates of the reference dot matrix on the sample image, find the corresponding code value on the sample image decoding pattern; finally, find the corresponding dot matrix of the reference dot matrix on the projected image according to the code value, and calculate the pre-distortion parameters with the difference between the projection dot matrix and the corresponding dot matrix.

[0004] In CN117872602A, the reference dot matrix is ​​calculated based on the simulation data of the calibration bench or workshop. When calibrating different AR-HUDs, its position and the size of the determined area remain unchanged. However, as mentioned above, due to the problems of material uniformity, device installation tolerance, and windshield free-form surface in AR-HUD devices, the position and size of the displayed area of ​​different AR-HUDs or AR-HUDs on different vehicles vary during calibration. If the edge of the display area is inside the de-distorted virtual image area determined by the reference dot matrix, the de-distorted display content will be missing. In order to prevent such missing, it is necessary to design a smaller reference dot matrix area, that is, to make certain sacrifices in the de-distorted FOV.

[0005] by Figure 7 For example, the dotted rectangle in the figure is the virtual image area after de-distortion determined by the reference dot matrix, and the solid curve is the virtual image area before de-distortion. In order to effectively calibrate each AR-HUD, it is necessary to ensure similar Figure 7 The solid curve frame is always outside the dotted rectangle frame, so it is necessary to appropriately reduce the overall size of the reference dot matrix (i.e., reduce the dotted rectangle frame). After de-distortion using this solution, the virtual image is compressed from the area determined by the solid curve frame before calibration to the dotted rectangle frame, which objectively reduces the FOV.

[0006] The existing AR-HUD distortion calibration scheme that can obtain a larger FOV calculates the dedistorted virtual image and the reference dot matrix within the area determined by the sample dot matrix, but the sample dot matrix is ​​contained in the light-emitting area. If the dedistorted virtual image can be determined in the light-emitting area, a relatively larger FOV can be obtained in theory. Summary of the invention

[0007] The object of the present invention is to provide an AR-HUD distortion calibration method with FOVs separated by light emitting areas. By performing distortion calibration on the entire light emitting area of ​​the AR-HUD, a larger field of view angle can be obtained to solve the problems in the prior art raised in the above background technology.

[0008] To achieve the above object, the present invention adopts the following technical solution: an AR-HUD distortion calibration method with FOV arranged in separate light-emitting areas, comprising the following steps:

[0009] S1: Obtain a projection image decoding pattern of a structured light image; S2: Use a viewpoint camera to shoot a structured light image projected by an AR-HUD, and decode the shot image; S3: Determine the actual light emitting area or the edge of the light emitting area of ​​the AR-HUD based on the decoded sample image; S4: Determine a dot matrix on a sample image or a virtual image that has a clear corresponding relationship with the projection image inside the light emitting area, and calculate the corresponding relationship between the projection image and the sample image; S5: Determine a reference dot matrix and its features on the sample image based on the corresponding relationship; S6: Determine a dot matrix on the corresponding projection image on the sample image based on the reference dot matrix; S7: Calculate the difference between the reference dot matrix and the corresponding dot matrix on the projection image to obtain a pre-distortion amount; S8: Determine the pre-distortion parameters based on the pre-distortion amount using an interpolation or fitting method.

[0010] Preferably, the structured light image is a sinusoidal fringe light image.

[0011] Preferably, step S1 further includes: offline calculating the decoding pattern of the projection image, and obtaining the linear expressions of the decoding pattern in the width direction and the decoding pattern in the height direction:

[0012] ;

[0013] in, is the pixel position; represents the characteristic component value; b is the intercept of the straight line expression.

[0014] Preferably, step S3 further includes: performing threshold processing on the sample image to extract the edge of the light area and its envelope.

[0015] Preferably, step S4 further includes: finding a maximum inscribed rectangle inside the boundary of the light emitting area, and using four vertices of the rectangle as a point matrix for determining the corresponding relationship.

[0016] Preferably, step S5 also includes: converting the projection lattice onto the sample image according to the homography relationship obtained in step S4 to obtain a reference lattice; and determining the characteristics of the reference lattice using a bicubic linear interpolation method.

[0017] Preferably, step S6 also includes: substituting the features of each point in the reference point matrix into the straight line expression in step S1 to obtain the corresponding point coordinates of the point on the projection image.

[0018] Preferably, step S7 also includes: determining the pre-distortion amount at the dot matrix according to the difference between the reference dot matrix and the dot matrix on the corresponding projection image.

[0019] Preferably, step S8 also includes: using a cubic spline interpolation method to determine the pre-distortion amount at each pixel on the projection image.

[0020] Preferably, step S3 also includes: improving the contrast between the foreground and the background in the decoding pattern by adjusting the initial phase of the sinusoidal stripe light.

[0021] Technical effects and advantages of the present invention: Compared with the prior art, the AR-HUD distortion calibration method with FOV arranged in separate light-emitting areas proposed by the present invention has the following advantages:

[0022] The present invention achieves a relatively larger field of view (FOV) by determining a distortion-free virtual image within the actual light-emitting area of ​​the AR-HUD. Specifically, the present invention uses a structured light image to intuitively reflect the light area or its edge, and determines the reference dot matrix based on this, and then calculates the pre-distortion parameters, which effectively solves the problem of limited FOV caused by the fixed position of the reference dot matrix in the prior art. This method not only improves the accuracy of distortion calibration, but also ensures the integrity of the displayed content, and ultimately achieves a wider and more accurate AR-HUD display effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic diagram of AR-HUD system;

[0024] Figure 2 This is an example of a dot matrix image;

[0025] Figure 3 This is an example of a dot matrix under the viewpoint camera;

[0026] Figure 4 This is an example diagram of the pre-distortion variable;

[0027] Figure 5 It is a schematic diagram of the field of view;

[0028] Figure 6 This is an example diagram of the AR-HUD optical path;

[0029] Figure 7 Schematic diagram of the virtual image area before and after de-distortion;

[0030] Figure 8 Schematic diagram of the image generated by PGU;

[0031] Fig. 9 This is the encoding and decoding flow chart of structured light;

[0032] Fig.10 A decoding value map of a row of a decoding pattern in the width direction;

[0033] Fig.11 is a schematic diagram of a decoding pattern on a sample image;

[0034] Fig.12 This is one of the edge schematic diagrams of the light-emitting area;

[0035] Fig.13 This is the second schematic diagram of the edge of the light-emitting area;

[0036] Fig.14 is the virtual image area and reference dot pattern after dedistortion;

[0037] Fig.15 The present invention is a flow chart of an AR-HUD distortion calibration method with FOV separated by a light emitting area. DETAILED DESCRIPTION

[0038] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0039] The present invention provides an AR-HUD distortion calibration method with FOV arranged in separate light-emitting areas, such as Fig.15 As shown, the following steps are included:

[0040] S1: Obtaining a projection image decoding pattern of a structured light image; wherein the structured light image is a sinusoidal stripe light image. Further comprising: calculating the decoding pattern of the projection image offline, and obtaining the linear expressions of the width direction decoding pattern and the height direction decoding pattern: ;in, is the pixel position; represents the characteristic component value; b is the intercept of the straight line expression.

[0041] S2: Use the viewpoint camera to capture the structured light image projected by the AR-HUD and decode the captured image.

[0042] S3: determining the actual light emitting area or the edge of the light emitting area of ​​the AR-HUD according to the decoded sample image; further comprising: performing thresholding processing on the sample image to extract the edge of the light emitting area and its envelope; and improving the contrast between the foreground and the background in the decoded pattern by adjusting the initial phase of the sinusoidal stripe light.

[0043] S4: determining a dot matrix on the sample image or virtual image that has a clear corresponding relationship with the projection image within the light exit area, and calculating the corresponding relationship between the projection image and the sample image; further comprising: finding a maximum inscribed rectangle within the boundary of the light exit area, and using the four vertices of the rectangle as the dot matrix for determining the corresponding relationship.

[0044] S5: determining the reference lattice and its features on the sample image according to the corresponding relationship; further comprising: converting the projected lattice to the sample image according to the homography relationship obtained in step S4 to obtain the reference lattice; and determining the features of the reference lattice using bicubic linear interpolation.

[0045] S6: Determine the corresponding dot matrix on the projection image based on the reference dot matrix on the sample image; further comprising: substituting the features of each point in the reference dot matrix into the straight line expression in step S1 to obtain the corresponding point coordinates of the point on the projection image.

[0046] S7: Calculate the difference between the reference dot matrix and the dot matrix on the corresponding projection image to obtain the pre-distortion variable; further comprising: determine the pre-distortion variable at the dot matrix according to the difference between the reference dot matrix and the dot matrix on the corresponding projection image.

[0047] S8: According to the pre-distortion amount, using an interpolation or fitting method to determine the pre-distortion parameter, further comprising: using a cubic spline interpolation method to determine the pre-distortion amount at each pixel on the projected image.

[0048] The following will be described in detail with reference to the accompanying drawings:

[0049] AR-HUD (AR-Head-up-Display): The fusion of AR technology and HUD display reflects the image through the windshield, and the image is integrated with the environment outside the car, allowing the driver to see the picture that is a combination of the displayed content and the actual environment.

[0050] Viewpoint Camera: Figure 1As shown, a camera is placed at the location of the driver's eyes (viewpoint) to obtain the image seen by the human eye.

[0051] Dot matrix image: Pattern images containing dots, such as checkerboard and circular spot images. Figure 2 This is an example image where the dot matrix image is a circular spot image.

[0052] Projected image: The image used for AR-HUD projection.

[0053] Projected dot matrix: The dot matrix in the dot matrix image projected by the AR-HUD. If the projected dot matrix image is a chessboard, the projected dot matrix is ​​composed of the corner point coordinates. If it is a circular spot image, the projected dot matrix is ​​composed of the center coordinates.

[0054] Sample image: The image of the virtual image in front of the windshield captured by the viewpoint camera. The so-called virtual image is similar to Figure 1 The AR-HUD in the image is imaged in front of the windshield. After passing through the AR-HUD's complex optical path, the virtual image in the sample image is as follows Figure 3 As shown by the hollow circle in , there is a certain deformation (i.e. distortion) in the spatial arrangement.

[0055] Sample dot matrix: The dot matrix in the extracted sample image. Figure 3 The dot matrix formed by the centers of the hollow circles in the figure is an example of a sample dot matrix.

[0056] Reference dot matrix: A dot matrix generated based on parameters such as arrangement, number of rows and columns, and position. Its function is to provide a calibration benchmark for the sample dot matrix. Figure 3 The point matrix formed by the centers of the solid circles in the figure is an example of the reference point matrix.

[0057] Target position: The position of the reference point matrix. Its value can be set, but it needs to ensure that: in the field of view of the viewpoint camera, the area determined by the reference point matrix at this position overlaps with the virtual image area. Figure 1 The reference point array shown is the target position, such as Figure 3 As shown, the reference point array (solid circle) under the camera's field of view falls within the area (virtual image area) determined by the sample point array (hollow circle).

[0058] Pre-distortion: According to the distortion characteristics of AR-HUD, the projected image is reversely distorted in advance to offset the distortion of AR-HUD itself so that the human eye sees a normal picture.

[0059] Pre-distortion variables: Figure 4As shown in the figure, assuming that the image width direction corresponds to the X direction and the height direction corresponds to the Y direction, the black solid circle is a point in the projection lattice, and the black hollow circle is a point in the sample lattice corresponding to the black solid circle and converted to the projection image. If the point in the sample lattice corresponding to the black hollow circle is to be coincident with the corresponding point in the viewpoint reference lattice (i.e., dedistortion), the black hollow circle needs to be moved to the black solid circle on the projection image, and the movement amount is the pre-distortion amount.

[0060] Predistortion parameters: Based on the relationship between the dot matrix, the predistortion variables at these dot matrixes can be determined, and then interpolation, fitting and other operations are performed based on the information of the predistortion variables corresponding to the positions of the points in the dot matrix. The predistortion variables at each pixel on the projected image can be calculated with the help of these distortion parameters.

[0061] FOV (Field of View): Also known as the field of view in optical engineering, the size of the field of view angle determines the field of view of the optical instrument. Figure 5 As shown, a larger field of view angle generally means a larger field of view.

[0062] PGU: Image generation unit. Currently, PGU technologies mainly include TFT-LCD projection (Thin Film Transistor Liquid Crystal Display), DLP projection (Digital Light Processing), LBS laser scanning projection (Laser Beam Scanning), LCOS projection (Liquid Crystal on Silicon), etc.

[0063] Structured light: light with a specific structure, common ones include: stripe light such as sinusoidal stripes and Gray code, spot light such as laser dots and pseudo-random speckle, and line laser, etc. Structured light is usually used to complete tasks such as matching and correspondence between images.

[0064] Homography transformation: Mapping a point on a projective plane to another projective plane. After mapping, the straight line is still a straight line and has the line-preserving property. This transformation can be done with a The non-singular matrix representation of is a linear transformation relationship from one plane to another.

[0065] Specifically, obtaining a dedistorted virtual image in the entire undedistorted virtual image area (i.e., the actual light emitting area of ​​the AR-HUD) can theoretically obtain a larger FOV. Figure 6As shown, the image is generated by the PGU, and then its light is controlled by multiple hardwares, and finally an image is formed in front of the car. In this process, the position of the PGU, the uniformity of the light control hardware, the installation position, etc. may have different avoidable errors. These errors will eventually affect the overall light output of the AR-HUD, that is, the size of the actual display area of ​​the virtual image. In order to ensure that each AR-HUD displays complete (undistorted) content in front of the car, the above errors need to be taken into account when the PGU generates the image.

[0066] Specific as Figure 8 As shown in the figure, the content to be displayed is compressed into the solid rectangular frame, and the part outside the solid rectangular frame is used as a tolerance area and no specific content is displayed. However, the area where AR-HUD can emit light is larger than the area shown in the figure. Figure 8 The solid line rectangular area shown in FIG. 1 and the boundary of the actual light-emitting area are on the tolerance area. If the dedistorted virtual image can be determined on the actual light-emitting area, a larger FOV can be obtained.

[0067] As mentioned above, this embodiment determines the reference dot matrix on the light-emitting area of ​​the AR-HUD and performs distortion calibration. This requires that the image used for calibration can reflect or display the edge of the light-emitting area or the light-emitting area. Some categories of structured light (such as sinusoidal stripe light) can cover the entire projected image, and they can present corresponding areas with structured light (i.e., the light-emitting area or the edge of the light-emitting area) on the corresponding sample image. Therefore, structured light is selected to complete the distortion calibration of the AR-HUD that can obtain a larger FOV.

[0068] It is assumed that the size of the structured light image is the same as that of the projected image, and the sinusoidal stripe light in the structured light is taken as an example.

[0069] As shown in Figure 9, decoding the sinusoidal stripe pattern in the leftmost column finally yields Fig. 9 The decoding pattern shown in the rightmost column. At this time, the width direction decoding pattern includes the feature code of each pixel in the projection image in the width direction, as shown in the following figure: Fig.10 As shown, each pixel on a row has its decoded value.

[0070] Similarly, the height decoding pattern contains the feature code of each pixel in the height direction. The width decoding pattern and the height decoding pattern together constitute the feature space on the projected image, in which each pixel in the projected image has its own unique features. If these decoding patterns are interpolated or fitted, the feature value of any coordinate (sub-pixel) on the projected image can be further obtained.

[0071] Similarly, each pixel in the light-emitting area or on the edge of the light-emitting area of ​​the sample image corresponding to the projected image also has its own unique characteristics. Because: a. The light-emitting area on the virtual image plane corresponds to the image that can be projected by the AR-HUD, so there is no structured light coverage outside the virtual image area; b. In order to prevent ambient light from affecting the calibration process, the calibration workshop is usually a dark field, so the grayscale value of the part outside the light-emitting area in the sample image is small and most of them are 0; c. Due to the existence of distortion, the stripes are deformed accordingly. The decoded pattern of the sample image is finally as follows Fig.11 As shown. If we further Fig.12 By fitting or interpolating the two decoded patterns in , the characteristic value of the corresponding light-emitting area or any coordinate (sub-pixel) on the edge of the light-emitting area can be obtained.

[0072] After the decoding pattern of the projection image and the decoding pattern of the sample image are obtained, they can be used to complete the matching and correspondence between the projection image and the sample image. In addition to using the decoding pattern to complete the conversion of the dot matrix between the projection image and the sample image or virtual image, the separation of the light-emitting area is completed based on the structured light or its decoding pattern in this embodiment. The specific steps are as follows:

[0073] 1) Projection image. Since the structured light image is used, the correspondence between the coordinates of the projection image decoding pattern and the sample image decoding pattern can be determined by the projection image decoding pattern and the sample image decoding pattern. Therefore, the projection of the dot matrix image and the detection of the sample dot matrix are not necessary.

[0074] In the embodiment, considering the accuracy advantage of sinusoidal stripe light, sinusoidal stripe light is selected. Fig. 9 As shown, first, the projection image decoding pattern is calculated offline; Fig.10 As shown, the linear expressions of the width direction decoding pattern and the height direction decoding pattern are further obtained. The linear expression is

[0075] ,

[0076] in, is the pixel position, Fig.10 For example, it represents the coordinates of the current pixel in a row (or width direction); represents the characteristic component value, Fig.10 For example, it means that on a certain line The width direction feature component at the coordinate; b is the intercept of the straight line expression. Since the feature consists of two components in the width direction and the height direction, the straight line expression has two corresponding ones.

[0077] These straight line expressions, projection dot matrices and their features will be used as configuration files for subsequent calculations. When using the straight line expression, the width or height coordinates of the pixel can be input to obtain the corresponding width or height feature components, and vice versa. The projection dot matrix itself is the dot matrix before the projection image is pre-distorted, and the difference between it and the distorted dot matrix will be used to calculate the pre-distortion amount. The features of the projection dot matrix are used to find the sample dot matrix corresponding to the projection dot matrix in the sample image.

[0078] 2) Shooting and decoding, that is, the viewpoint camera captures the images projected by the AR-HUD and then decodes these images.

[0079] In the embodiment, a similar Fig. 9 The multi-frequency heterodyne decoding method is used to decode the sinusoidal fringes and obtain the following Fig.11 The decoding result is shown in FIG. 1 . The decoding method of sinusoidal fringes is a mature existing solution and will not be described in detail here.

[0080] 3) Determine the light emitting area or its edge, that is, determine the actual image emitting area or the edge of the image emitting area of ​​the AR-HUD in front of the vehicle or in the sample image with the help of the structured light pattern area or its decoded pattern in the sample image.

[0081] In the embodiment, the light-emitting area is determined by decoding the sample image pattern. In order to increase the contrast between the foreground (light-emitting area) and the background (area outside the light-emitting area) in the decoding pattern, the initial phase is adjusted when the sinusoidal stripe light pattern is generated so that the decoded value increases from a value greater than the background (e.g. Fig.10 As shown in the vertical axis, it increases from 10), the final decoding pattern is as follows Fig.11 As shown in Figure 1, the minimum value in the effective decoding area is higher than the black background value. Fig.11 The left side (or right side) is binarized, and then the edge and its envelope are extracted, and finally the following is obtained: Fig.12 Results shown.

[0082] 4) Determine the dot matrix on the sample image or virtual image that has a clear correspondence with the projected image and calculate the relationship between the two. In order to ensure the integrity of the displayed content after de-distortion, this step requires determining the overall correspondence between the displayed content area on the projected image and the de-distorted virtual image area.

[0083] In the embodiment, first, find the largest inscribed rectangle (such as Fig.13 The solid rectangle in the figure), the four vertices of which (such as Fig.13 The four hollow circles in the figure are the dot matrix that determines the corresponding relationship between the sample image or virtual image and the projected image. Then, the dot matrix is ​​used to determine the corresponding relationship between the sample image or virtual image and the projected image. Figure 8 The correspondence between the vertices of the solid line rectangle (the vertices of the solid line rectangle) determines the homography transformation relationship between the projection image and the sample image.

[0084] 5) Determine the reference point array and its characteristics. Calculate the reference point array on the sample image (or virtual image) with the help of the relationship in step 4, and determine its characteristics based on the coordinates of the reference point array.

[0085] In the embodiment, first, the projection dot matrix is ​​converted to the sample image according to the homography relationship obtained in step 4, and the following is obtained: Fig.14 The solid circle in the middle shows the reference dot matrix; then, the corresponding features are found on the sample image decoding pattern according to the coordinates of the reference dot matrix. Since the reference dot matrix is ​​sub-pixel coordinates in most cases, the bicubic linear interpolation method is used here to obtain the features at these dot matrices.

[0086] 6) Determine the corresponding dot matrix. Although the dot matrix image may not be projected in this solution, the sample dot matrix that matches the projection dot matrix can still be found in the sample image based on the feature matching of the structured light. At this time, the information required for AR-HUD calibration, such as the sample dot matrix, the reference dot matrix, and the projection dot matrix, is complete. The corresponding dot matrix of the sample dot matrix or the reference dot matrix on the projection image can be determined by means of the homography relationship (or projection relationship) between the reference dot matrix (or sample dot matrix) and the projection dot matrix, or structured light feature matching, and then the difference between them can be used to calculate the pre-distortion parameters.

[0087] In the embodiment, the corresponding dot matrix of the reference dot matrix on the projected image is determined by means of structured light feature matching. The features of each point in the reference dot matrix include two quantities in the width direction and the height direction. The two quantities of a point in the reference dot matrix are respectively substituted into the corresponding straight line expression in step 1 to obtain the width direction coordinate and the height direction coordinate of the corresponding point of the point on the projected image. Solving the corresponding points of all points in the reference dot matrix in the same way will eventually obtain the corresponding dot matrix.

[0088] 7) Calculate the distortion calibration parameters. That is, calculate the pre-distortion variables at the dot matrix based on the differences between the dot matrixes, and further use interpolation or fitting methods to obtain the pre-distortion parameters. Based on the parameters, the pre-distortion variables at each coordinate point in the projected image can be obtained.

[0089] In the embodiment, the sparse pre-distortion amount at the dot matrix is ​​determined according to the difference between the projected dot matrix and the corresponding dot matrix, and the pre-distortion amount at any pixel on the projected image is further interpolated by cubic spline interpolation, that is, the pre-distortion parameter is obtained. This step is included in the existing AR-HUD distortion calibration scheme, so it is a prior art and will not be described here.

[0090] In summary, the present invention achieves a relatively larger field of view (FOV) by determining a distortion-free virtual image within the actual light-emitting area of ​​the AR-HUD. Specifically, the present invention uses a structured light image to intuitively reflect the light area or its edge, and determines the reference dot matrix based on this, and then calculates the pre-distortion parameters, which effectively solves the problem of limited FOV caused by the fixed position of the reference dot matrix in the prior art. This method not only improves the accuracy of distortion calibration, but also ensures the integrity of the displayed content, and ultimately achieves a wider and more accurate AR-HUD display effect.

[0091] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for AR-HUD distortion calibration with FOV arranged in separate light-emitting areas, characterized in that: The following steps are involved: S1: Obtaining a projection image decoding pattern of a structured light image; S2: Use the viewpoint camera to capture the structured light image projected by the AR-HUD and decode the captured image; S3: Determine the actual light emitting area or the edge of the light emitting area of ​​the AR-HUD according to the decoded sample image; S4: determining a dot matrix on the sample image or virtual image that has a clear corresponding relationship with the projection image within the light exit area, and calculating the corresponding relationship between the projection image and the sample image; S5: Determine the reference point array and its features on the sample image according to the corresponding relationship; S6: determining a corresponding dot matrix on the projection image on the sample image according to the reference dot matrix; S7: Calculate the difference between the reference dot matrix and the dot matrix on the corresponding projection image to obtain the pre-distortion variable; S8: Determine pre-distortion parameters by using an interpolation or fitting method according to the pre-distortion amount; Step S1 also includes: The decoding pattern of the projection image is calculated offline, and the linear expressions of the decoding pattern in the width direction and the decoding pattern in the height direction are obtained: y=kx+b; wherein x is the pixel position; y represents the feature component value; and b is the intercept of the linear expression; Step S3 also includes: Thresholding the sample image to extract the edge of the light area and its envelope; Step S4 also includes: Finding a maximum inscribed rectangle within the boundary of the light emitting area, with four vertices of the rectangle serving as a dot matrix for determining the corresponding relationship; Step S5 also includes: The projection dot matrix is ​​converted to the sample image according to the homography relationship obtained in step S4 to obtain a reference dot matrix; the characteristics of the reference dot matrix are determined by using a bicubic linear interpolation method; Step S6 also includes: Substitute the features of each point in the reference point matrix into the straight line expression in step S1 to obtain the corresponding point coordinates of the point on the projection image; Step S7 also includes: Determine the predistortion variable at the dot matrix according to the difference between the reference dot matrix and the dot matrix on the corresponding projection image; Step S8 also includes: The pre-distortion variable at each pixel on the projection image is determined by using a cubic spline interpolation method; Step S3 also includes: The contrast between the foreground and background in the decoding pattern is improved by adjusting the initial phase of the sinusoidal stripe light.

2. According to claim 1, the AR-HUD distortion calibration method with FOV arranged in separate light-emitting areas is characterized in that: The structured light image is a sinusoidal fringe light image.

Images