AR-HUD calibration method and related device
Through the camera, feature points extraction and coordinate mapping of the projected image of AR-HUD on the virtual image plane, the calibration parameters of AR-HUD are directly calculated, which solves the problems of cumbersome and high cost of AR-HUD calibration operations in the prior art, and realizes the Simplify operations and reduce costs.
Patent Information
- Application Number
- CN202411078654.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-06
AI Technical Summary
The existing AR-HUD calibration methods are cumbersome to operate, rely on expensive equipment, are costly, and have high calibration environment requirements, which affects efficiency.
The calibrated camera takes the projected image of AR-HUD on the virtual image plane, extracts feature points, maps the image coordinate points into world coordinate points based on the camera's calibration parameters, calculates the actual coordinate points of the corner points of the virtual image plane, and then determines the AR-HUD calibration parameters simplify operations and reduce costs.
There is no need to adjust the camera position and projected image position multiple times, and the operation is simple, reducing dependence on robotic arms and expensive equipment, reducing calibration costs, and reducing requirements for calibration environments, improving calibration efficiency.
Smart Images

Figure CN119091093A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle calibration, and in particular to an AR-HUD calibration method and related devices. Background Art
[0002] With the rapid development of smart cars, in-vehicle display technology is also constantly innovating. Among them, the Augmented Reality Head-Up Display (AR-HUD) can project virtual information directly on the driver's front windshield, such as displaying basic information such as navigation routes and vehicle speed, and can also display dynamic information such as surrounding vehicles and pedestrians in real time. It achieves seamless integration with the real driving environment and provides drivers with a more intuitive and safe driving experience.
[0003] In order to ensure that AR-HUD can accurately and stably display virtual information, it needs to be accurately calibrated. In related technologies, it is usually necessary to calibrate the camera first, and then use the calibrated camera to calibrate the AR-HUD. When calibrating the AR-HUD, it is usually necessary to adjust the angle of the binocular camera on the robotic arm and the projection position of the AR-HUD's projection image multiple times until the AR-HUD's projection image coincides with the calibration plate to achieve the calibration of the AR-HUD.
[0004] However, in the process of calibrating AR-HUD, it is necessary to calibrate the binocular camera and the projection position of the projected image multiple times, which is cumbersome and affects the calibration efficiency. At the same time, it also relies on expensive equipment such as robotic arms, which is costly and has high requirements for the calibration environment. Summary of the invention
[0005] In view of this, an embodiment of the present application provides an AR-HUD calibration method and a related device, the purpose of which is to simplify the calibration operation of the AR-HUD, improve the calibration efficiency, and reduce the calibration cost.
[0006] In a first aspect, an embodiment of the present application provides a calibration method for an AR-HUD, the method comprising:
[0007] Acquire a first calibration image; the first calibration image is obtained by photographing a projection image of an augmented reality head-up display AR-HUD located in a vehicle on a virtual image plane through a calibrated camera; the projection image is a two-dimensional image; the camera is located at a preset eye point position; the first calibration image is in a first image coordinate system; the camera is in a world coordinate system;
[0008] Extracting a first feature point of the first calibration image to obtain an image coordinate point of the first feature point;
[0009] Based on the second calibration parameter of the camera, the image coordinate point of the first feature point is mapped to the actual coordinate point of the first feature point in the world coordinate system; the second calibration parameter of the camera is obtained by calibrating the camera; the actual coordinate point of the first feature point is used to represent the actual position of the projected image;
[0010] Based on the actual coordinate point of the first feature point and the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane, the actual coordinate point of the corner point of the virtual image plane is calculated; the actual coordinate point of the corner point of the virtual image plane is used to represent the actual position of the virtual image plane;
[0011] Based on the actual coordinate points of the corner points of the virtual image plane and the actual coordinate points of the camera, first calibration parameters of the AR-HUD are calculated.
[0012] As an implementation manner, the second calibration parameter of the camera is obtained by calibrating the camera through a calibration plate; the calibration plate includes a plurality of planes, at least two of the plurality of planes are not coplanar, and each plane includes a plurality of second feature points.
[0013] As an implementation manner, the camera calibration step includes:
[0014] Acquire a second calibration image; the second calibration image is obtained by photographing the calibration plate with an uncalibrated camera; the second calibration image is in a second image coordinate system; the calibration plate is in a world coordinate system;
[0015] Extracting a second feature point of the second calibration image to obtain an image coordinate point of the second feature point;
[0016] For each second feature point of the calibration plate, the second calibration parameter is calculated based on an actual coordinate point of the second feature point on the calibration plate and an image coordinate point of the second feature point.
[0017] As an implementation manner, the calibration plate includes two planes; the second calibration image includes two second calibration images; the two second calibration images are obtained by adjusting the angle of the calibration plate once, or by adjusting the angle of the camera once.
[0018] As an implementation manner, the calibration plate includes at least three planes; and the second calibration image includes a second calibration image.
[0019] As an implementation manner, the first feature point is a corner point of a projected image in the first calibration image; the actual coordinate point of the corner point of the virtual image plane is calculated based on the actual coordinate point of the first feature point and the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane, comprising:
[0020] The distance between the image coordinate point of the corner point of the projection image and the image coordinate point of the corner point of the virtual image plane is multiplied by the size ratio to obtain the distance between the actual coordinate point of the corner point of the first calibration image and the actual coordinate point of the corner point of the virtual image plane; the size ratio is the ratio of the size of the projection image in the world coordinate system to the size of the projection image in the first image coordinate system;
[0021] The actual coordinates of the corner points of the virtual image plane are calculated based on the actual coordinates of the corner points of the first calibration image and the distances between the actual coordinates of the corner points of the first calibration image and the actual coordinates of the corner points of the virtual image plane.
[0022] As an implementation manner, adjacent planes of the calibration plate are of different colors, and / or each plane of the calibration plate is provided with a checkerboard or a dot pattern.
[0023] In a second aspect, an embodiment of the present application provides an AR-HUD calibration device, the device comprising:
[0024] An image acquisition module is used to acquire a first calibration image; the first calibration image is obtained by photographing a projection image of an augmented reality head-up display AR-HUD located in a vehicle on a virtual image plane through a calibrated camera; the projection image is a two-dimensional image; the camera is located at a preset eye point position; the first calibration image is in a first image coordinate system; the camera is in a world coordinate system;
[0025] A feature point extraction module, used to extract a first feature point of the first calibration image and obtain an image coordinate point of the first feature point;
[0026] a coordinate point mapping module, used for mapping the image coordinate point of the first feature point to the actual coordinate point of the first feature point in the world coordinate system based on the second calibration parameter of the camera; the second calibration parameter of the camera is obtained by calibrating the camera; the actual coordinate point of the first feature point is used to represent the actual position of the projected image;
[0027] a coordinate point calculation module, configured to calculate the actual coordinate point of the corner point of the virtual image plane based on the actual coordinate point of the first feature point and the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane; the actual coordinate point of the corner point of the virtual image plane is used to represent the actual position of the virtual image plane;
[0028] The AR-HUD calibration module is used to calculate the first calibration parameters of the AR-HUD based on the actual coordinate points of the corner points of the virtual image plane and the actual coordinate points of the camera.
[0029] In a third aspect, an embodiment of the present application provides an AR-HUD calibration device, the device comprising a memory and a processor:
[0030] The memory is used to store a computer program and transmit the computer program to the processor;
[0031] The processor is used to execute the computer program so that the device performs the AR-HUD calibration method described in the first aspect.
[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the device running the computer program implements the AR-HUD calibration method described in the first aspect above.
[0033] Compared with the prior art, the embodiments of the present application have the following beneficial effects:
[0034] The embodiment of the present application provides an AR-HUD calibration method and related devices. In the method, a calibrated camera is first used to shoot a projection image of an augmented reality head-up display AR-HUD located in a vehicle on a virtual image plane to obtain a first calibration image, which is a two-dimensional image. The camera is located at a preset eye point position, in a world coordinate system, and the first calibration image is in a first image coordinate system; then, a first feature point of the first calibration image is extracted to obtain an image coordinate point of the first feature point in the first image coordinate system; then, based on a second calibration parameter of the camera, the image coordinate point of the first feature point is mapped to a first feature point The actual coordinate points in the world coordinate system, wherein the second calibration parameters of the camera are obtained by calibrating the camera, and the actual coordinate points of the first feature points are used to characterize the actual position of the projected image; then based on the actual coordinate points of the first feature points, and the distance between the image coordinate points of the first feature points and the image coordinate points of the corner points on the virtual image plane, the actual coordinate points of the corner points of the virtual image plane are calculated, wherein the actual coordinate points of the corner points of the virtual image plane are used to characterize the actual position of the virtual image plane; finally, based on the actual coordinate points of the camera and the actual coordinate points of the corner points of the virtual image plane, the AR-HUD is calibrated to calculate the first calibration parameters of the AR-HUD.
[0035] In this way, the camera is located at the preset eye point position, indicating that the camera perspective is consistent with the user perspective at this time, that is, the first calibration image obtained by the camera can directly simulate the projection image of the AR-HUD seen from the user perspective, and based on the calibrated camera, the corresponding relationship between the position of the first feature point in the first calibration image obtained by shooting the two-dimensional image projected on the virtual image plane in the first image coordinate system and the position in the world coordinate system can be determined, so the actual coordinate point of the first feature point can be directly calculated using this corresponding relationship. And when the projected image is a two-dimensional image, the relative position relationship between the first feature point on it and the corner point of the virtual image plane in the world coordinate system is equivalent to the relative position relationship between the two in the first image coordinate system, so the actual coordinate point of the corner point of the virtual image plane can be determined based on this equivalent relative position relationship, and finally, based on the actual coordinate point of the camera simulating the user perspective and the actual coordinate point of the corner point of the virtual image plane simulating the projection range of the AR-HUD, the first calibration parameter of the AR-HUD can be directly calculated.
[0036] As can be seen from the above, the present application does not need to adjust the camera position (or angle) and the projection position of the projected image multiple times, and the operation is simple. At the same time, when the present application calibrates the AR-HUD, there is no need to use tools such as a mechanical arm and a calibration plate, which greatly reduces the cost.
[0037] In addition, robotic arms are usually deployed in laboratories, but the AR-HUD calibration method provided in this application does not require the use of a robotic arm. Even after a vehicle equipped with a calibrated AR-HUD is produced, it can be calibrated in a daily environment, which greatly reduces the requirements for the calibration environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0039] Figure 1 An application scenario of an AR-HUD calibration method provided in an embodiment of the present application;
[0040] Figure 2 A flowchart of an AR-HUD calibration method provided in an embodiment of the present application;
[0041] Figure 3 A schematic diagram of a first calibration image provided in an embodiment of the present application;
[0042] Figure 4 is a schematic diagram of a plane calibration plate in the related art;
[0043] Figure 5 A schematic diagram of a calibration plate provided in an embodiment of the present application;
[0044] Figure 6 A schematic diagram of a code for calculating camera intrinsic parameters provided in an embodiment of the present application;
[0045] Figure 7 A schematic diagram of a code for calculating camera extrinsic parameters provided in an embodiment of the present application;
[0046] Figure 8 A schematic diagram of a camera calibration process in the related art;
[0047] Fig. 9 A schematic diagram of a camera calibration process provided in an embodiment of the present application;
[0048] Fig.10 A schematic diagram of calculating a first calibration parameter provided in an embodiment of the present application;
[0049] Fig.11 A schematic diagram of the structure of an AR-HUD calibration device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0050] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0051] At present, the existing AR-HUD method is: first calibrate the binocular camera, place the calibration plate in front of the vehicle, observe the calibration plate through the binocular camera on the mechanical arm, adjust the position of the binocular camera or the position of the calibration plate multiple times, and then shoot the calibration plate at multiple angles through the binocular camera to determine the calibration parameters of the binocular camera. Then calibrate the AR-HUD, AR-HUD projects a three-dimensional image through preset calibration parameters, and adjusts the imaging position (also called projection position) of the AR-HUD projected on the virtual image plane multiple times by adjusting the preset calibration parameters multiple times, and also adjusts the position of the binocular camera multiple times until the projection image of the AR-HUD coincides with the calibration plate, and the calibration of the AR-HUD is completed, that is, the calibration parameters when the projection image of the AR-HUD coincides with the calibration plate are the final calibration parameters.
[0052] The calibration process of AR-HUD not only involves multiple adjustments to the binocular camera, but also requires continuous calculation of the deviation between the preset calibration parameters and the actual calibration parameters (that is, the final calibration parameters) in the process of adjusting the AR-HUD to overlap the calibration plate to determine the adjustment amount, and continuously approach the final calibration parameters. The steps are cumbersome and complicated, and difficult to operate.
[0053] Based on this, in order to solve the above problems, the embodiments of the present application provide an AR-HUD calibration method and related devices, the purpose of which is to simplify the calibration operation of AR-HUD, improve the calibration efficiency, and reduce the calibration cost. In the method, a first calibration image is obtained; the first calibration image is obtained by photographing a projection image of an augmented reality head-up display AR-HUD located in a vehicle on a virtual image plane through a calibrated camera; the projection image is a two-dimensional image; the camera is located at a preset eye point position; the first calibration image is in a first image coordinate system; the camera is in a world coordinate system; a first feature point of the first calibration image is extracted to obtain an image coordinate point of the first feature point; based on a second calibration parameter of the camera, the image coordinate point of the first feature point is mapped to an actual coordinate point of the first feature point in the world coordinate system; the second calibration parameter of the camera is obtained by calibrating the camera; the actual coordinate point of the first feature point is used to characterize the actual position of the projected image; based on the actual coordinate point of the first feature point and the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane, the actual coordinate point of the corner point of the virtual image plane is calculated; the actual coordinate point of the corner point of the virtual image plane is used to characterize the actual position of the virtual image plane; based on the actual coordinate point of the corner point of the virtual image plane and the actual coordinate point of the camera, the first calibration parameter of the AR-HUD is calculated.
[0054] It can be seen that the camera simulates the user's eye position, so the first calibration image taken can directly simulate the projection image of the AR-HUD seen from the user's perspective. The projection image is a two-dimensional image, and the relative position relationship between the first feature point on it and the corner point of the virtual image plane in the world coordinate system is equivalent to the relative position relationship between the two in the first image coordinate system. Therefore, the actual coordinate points of the corner points of the virtual image plane that simulates the projection range of the AR-HUD can be calculated, and finally the first calibration parameters of the AR-HUD can be directly calculated based on the actual coordinate points of the camera and the actual coordinate points of the corner points of the virtual image plane. There is no need to adjust the camera position and the projection position of the projected image multiple times, and the operation is simple. At the same time, there is no need to use tools such as robotic arms and calibration plates, and the cost is low. In addition, only the calibrated camera tool is needed, and the requirements for the calibration environment are greatly reduced.
[0055] For example, the embodiments of the present application can be applied to Figure 1In the scene shown. The scene includes a vehicle 101, a server 102 and a camera 103, and the server 102 and the camera 103 can be connected to each other in a wireless or wired manner. Among them, the vehicle 101 includes an AR-HUD, and the AR-HUD generates a two-dimensional projection image and projects it on a virtual image plane. The camera is located at a preset eye point position to shoot the projection image on the virtual image plane. The server 102 can adopt the implementation method provided in the embodiment of the present application to obtain the first calibration image shot by the camera 103 to calibrate the AR-HUD.
[0056] First, the server 102 may be a stand-alone server, a cluster server, or a cloud server, etc. In the above application scenario, although the action description of the implementation method provided in the embodiment of the present application is executed by the server 102; however, the embodiment of the present application is not limited in terms of the execution subject, as long as the actions disclosed in the implementation method provided in the embodiment of the present application are executed. For example, the action description of the implementation method provided in the embodiment of the present application may also be executed by a terminal device. Among them, the terminal device may be a smart phone, a computer, a tablet computer, etc. This application does not specifically limit the terminal device or server mentioned above.
[0057] Secondly, the above scenario is only an example scenario provided by the embodiment of the present application, and the embodiment of the present application is not limited to this scenario.
[0058] The specific implementation of the AR-HUD calibration method and related devices in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0059] See also Figure 2 , which is a flow chart of an AR-HUD calibration method provided in an embodiment of the present application, combined with Figure 2 As shown, it may specifically include:
[0060] S201: Acquire a first calibration image.
[0061] In the embodiment of the present application, the first calibration image refers to an image used to calibrate the AR-HUD, which can be obtained by using a calibrated camera to capture the projection image of the AR-HUD in the vehicle on the virtual image plane.
[0062] In practical applications, AR-HUD needs to use augmented reality technology to project virtual information directly into the driver's field of view.
[0063] In an embodiment of the present application, when calibrating the AR-HUD, the AR-HUD can project information with obvious identification to obtain a projection image.
[0064] Among them, different from the related art, the image projected by AR-HUD is a two-dimensional image (also called a 2D image), which is convenient for the subsequent determination of the actual coordinate points of the corner points of the virtual image plane. For details, please refer to the following embodiments.
[0065] In some embodiments, the AR-HUD may project information such as a checkerboard or a dot matrix onto a virtual image plane, but this application does not limit this.
[0066] As an example, see Figure 3 , which is a schematic diagram of a first calibration image provided by an embodiment of the present application. Figure 3 As shown, the first calibration image includes a virtual image plane, the virtual image plane includes a projection image, and the projection image includes a plurality of black and white grids, which can provide a high-precision first feature point for calibration and is easy to detect and identify.
[0067] The camera is located at a preset eye point position. In the embodiment of the present application, the preset eye point position refers to a preset eye position of a user such as a driver, which is used to simulate the user's perspective.
[0068] In the embodiment of the present application, the image coordinate system refers to a coordinate system used to describe the pixel position in the image, which can be understood as a two-dimensional coordinate system. The world coordinate system is a coordinate system that describes the position of an object in three-dimensional space. Therefore, the first calibration image is in the first image coordinate system, and the camera is in the world coordinate system.
[0069] S202: Extract first feature points of the first calibration image to obtain image coordinate points of the first feature points.
[0070] In the embodiment of the present application, the first feature point refers to a point in the first calibration image where the gray value changes dramatically or a point on the edge of the image with a large curvature, which is convenient for detection and identification. For example, if the first calibration image shows a checkerboard (that is, the projected image of the AR-HUD is a checkerboard), the boundary between the black and white grids of the checkerboard may include the first feature point. The first feature point is a feature point on the projected image.
[0071] The first feature point on the first calibration image is extracted, that is, identified and located, so that the coordinate point of the first feature point in the first image coordinate system (that is, the image coordinate point) can be obtained.
[0072] In some embodiments, the first feature point can be extracted by using an open source computer vision library (OpenCV) or deep learning to obtain the image coordinate point of the first feature point. When the first feature point is extracted by using OpenCV, it can be extracted by using methods such as extraction channel, binarization, and edge search, and this application does not limit this.
[0073] In some embodiments, a feature point detection algorithm such as the Harris corner point detection method or the Scale-Invariant Feature Transform (SIFT) algorithm may be used to search for and locate the first feature point on the first calibration image to obtain the image coordinate point of the first feature point.
[0074] S203: Mapping the image coordinate point of the first feature point to the actual coordinate point of the first feature point in the world coordinate system based on the second calibration parameter of the camera.
[0075] In the embodiment of the present application, the second calibration parameter refers to a parameter used to describe the internal geometric and optical characteristics of the camera, as well as the relative position and direction between the camera coordinate system and the world coordinate system, and can be used to indicate the correspondence between the world coordinate system and the image coordinate system. The second calibration parameter is obtained by calibrating the camera in advance.
[0076] Based on the above introduction, the first feature point is a feature point on the projection image, so the actual coordinate point of the first feature point can be used to represent the actual position of the projection image.
[0077] In some embodiments, the second calibration parameters include internal parameters (which may be referred to as internal parameters for short), external parameters (which may be referred to as external parameters for short), and distortion parameters, etc.
[0078] Among them, the focal length and principal point coordinates in the intrinsic parameters define the mapping relationship from the camera coordinate system to the image coordinate system, and the rotation matrix and translation matrix in the extrinsic parameters define the transformation relationship from the world coordinate system to the camera coordinate system. Therefore, based on the intrinsic parameters in the second calibration parameters, the first feature point in the image coordinate system can be mapped to the camera coordinate system, and then based on the extrinsic parameters in the second calibration parameters, the first feature point in the camera coordinate system can be mapped to the world coordinate system to obtain the actual coordinate point of the first feature point in the world coordinate system.
[0079] In a possible implementation of the present application, the second calibration parameters mentioned above can be obtained by calibrating the camera through a calibration plate, where the calibration plate includes multiple planes, at least two of the multiple planes are not coplanar, and each plane includes multiple second feature points.
[0080] Among the multiple planes included in the calibration plate, at least two planes are not on the same plane, which can be called a special-shaped calibration plate, that is, a calibration plate different from an ordinary plane calibration plate.
[0081] In some embodiments, adjacent planes among the multiple planes included in the calibration plate are not on the same plane.
[0082] In some embodiments, the multiple planes included in the calibration plate may not be on the same plane, which is not limited in the present application.
[0083] As an example, see Figure 4 , which is a schematic diagram of a plane calibration plate in the related art. Figure 4 As shown, the figure only includes a plane with a chessboard set on it. In practical applications, when using the plane calibration plate to calibrate the camera, it is necessary to adjust the angle of the camera or the plane calibration plate multiple times to obtain images at different viewing angles, thereby achieving camera calibration.
[0084] See also Figure 5 , which is a schematic diagram of a calibration plate provided in an embodiment of the present application. Figure 5 As shown, the calibration plate includes multiple planes, such as plane 1 to plane 5. When calibrating the camera, the calibration plate is photographed by the camera to obtain an image, and the feature points on different planes in the image can directly replace the feature points of the same plane under different viewing angles, thereby reducing the number of adjustments to the camera or calibration plate, or even eliminating the need for adjustment.
[0085] In this way, during the process of calibrating the camera using a calibration plate including multiple planes, the multiple planes can cover different viewing angles, thereby reducing or even eliminating the need to adjust the camera or the calibration plate, greatly simplifying the calibration steps and improving the calibration efficiency.
[0086] In a possible implementation of the present application, the process of calibrating the camera may include the following steps:
[0087] A1: Get the second calibration image.
[0088] In the embodiment of the present application, the second calibration image refers to an image used to calibrate a camera, which can be obtained by photographing a calibration plate including multiple planes using an uncalibrated camera.
[0089] The second calibration image is in a second image coordinate system; the calibration plate is in a world coordinate system.
[0090] A2: Extracting a second feature point of the second calibration image to obtain an image coordinate point of the second feature point.
[0091] In the embodiment of the present application, the second feature point refers to a point in the second calibration image where the grayscale value changes dramatically or a point on the edge of the image with a large curvature, which is convenient for detection and identification.
[0092] The second feature point on the second calibration image is extracted, that is, identified and located, and the coordinate point of the second feature point in the second image coordinate system (that is, the image coordinate point of the second feature point) can be obtained.
[0093] A3: For each second feature point of the calibration plate, the second calibration parameter is calculated based on the actual coordinate point of the second feature point on the calibration plate and the image coordinate point of the second feature point.
[0094] In some embodiments, the actual coordinate point of the second feature point can be measured by direct observation or triangulation, etc., which is not limited in this application.
[0095] It should be noted that the actual coordinate points of any object in the world coordinate system (such as the actual coordinate points of the camera mentioned in the embodiment of the present application, the actual coordinate points of the second feature point) can be measured using the above method and will not be repeated here.
[0096] In some embodiments, the second calibration parameter of the camera can be used as an unknown number to establish a mapping relationship from the world coordinate system to the image coordinate system, and the mapping relationship can be expressed as a set of equations, which includes the second calibration parameter as an unknown number. The least square method or other optimization algorithms are then used to solve the set of equations to obtain the second calibration parameter of the camera.
[0097] In some embodiments, based on the actual coordinate points of the second feature points and the image coordinate points, the second calibration parameter can also be calculated using the image size. For example, the image size can be expressed as 1024×1024.
[0098] As an implementation method, see Figure 6 , which is a schematic diagram of a code for calculating camera intrinsic parameters provided by an embodiment of the present application. Figure 6 As shown, the camera intrinsic parameters and distortion parameters can be calculated using the 3D coordinate points of each surface of the object (that is, the actual coordinate points of the feature points on multiple planes of the calibration plate), the image coordinate points of each surface of the object (that is, the image coordinate points of the feature points on different planes of the calibration plate in the second calibration image), and the image size.
[0099] As an implementation method, see Figure 7 , which is a schematic diagram of a code for calculating camera extrinsic parameters provided in an embodiment of the present application. The extrinsic parameters of the camera can be calculated using the 3D coordinate points of each surface of the object and the image coordinate points of each surface of the object.
[0100] In this way, during the camera calibration process, a calibration plate including multiple planes is used, which can cover different viewing angles, thereby reducing or even eliminating the need to adjust the camera or the calibration plate, thereby improving calibration efficiency.
[0101] It should be noted that the above A1-A3 are only used to distinguish the camera calibration steps and are not shown in the figure.
[0102] In a possible implementation of the present application, the calibration plate includes two planes, and the second calibration image includes two second calibration images; the two second calibration images are obtained by adjusting the angle of the calibration plate once, or by adjusting the angle of the camera once.
[0103] The calibration plate is photographed by the camera to obtain a second calibration image, and then the angle of the camera or the calibration plate can be adjusted, and the calibration plate is photographed by the camera again to obtain another second calibration image. Then, based on the image coordinate points and actual coordinate points of the second feature points on the two calibration images, the second calibration parameters of the camera are calculated to achieve the calibration of the camera.
[0104] It should be understood that three feature points constitute a plane. When the camera includes two planes, a plane can be constructed using feature points on one plane and feature points on another plane.
[0105] In this way, images under at least four viewing angles can be obtained by adjusting the angle of the camera or the calibration plate only once. A second calibration image includes images under two viewing angles. No multiple adjustments are required, which simplifies the calibration steps for the camera and can improve the calibration efficiency of the camera.
[0106] In a possible implementation manner of the present application, the calibration plate includes at least three planes; the second calibration image includes a second calibration image. The calibration plate is photographed by a camera to obtain a second calibration image. Subsequently, the second calibration parameters of the camera are calculated through the image coordinate points and actual coordinate points of the second feature points on the second calibration image to achieve calibration of the camera.
[0107] In this way, images under at least three viewing angles can be obtained without adjusting the angle of the camera or the calibration plate, which greatly simplifies the camera calibration steps, is easy to operate, and can improve the camera calibration efficiency.
[0108] As an example, see Figure 8 , which is a schematic diagram of the camera calibration process in the related art. Figure 8 As shown, when calibrating a camera by using a plane calibration plate in the related art, it is necessary to first make a calibration plate; then use an uncalibrated camera to take a photo with the calibration plate (that is, take a photo of the plane calibration plate); then, adjust the angle of the camera or the calibration plate; then repeat the process of taking photos with the calibration plate and adjusting the camera or the calibration plate to obtain at least 4 photos with the calibration plate (that is, the captured images); extract corner points (that is, feature points) from the multiple photos taken; finally, calculate the parameters of the camera (including intrinsic parameters and extrinsic parameters, etc.) based on the actual coordinate points and image coordinate points of the feature points.
[0109] As an example, see Fig. 9, which is a schematic diagram of a camera calibration process provided by an embodiment of the present application. Fig. 9 As shown, when calibrating a camera by using a calibration plate (also referred to as a special-shaped calibration plate) including multiple planes provided in an embodiment of the present application, the calibration plate can be made first; then, an uncalibrated camera is used to take a photo with the calibration plate (that is, to take a photo of the special-shaped calibration plate); then, corner points (that is, second feature points) are extracted from the taken photo (that is, the second calibration image); finally, the intrinsic and extrinsic parameters (that is, the second calibration parameters) of the camera are estimated based on the actual coordinate points and image coordinate points of the second feature points.
[0110] It can be seen from the above that optimizing the camera calibration steps can improve the calibration speed and efficiency and reduce the calibration cost.
[0111] In a possible implementation manner of the present application, adjacent planes of the calibration plate including multiple planes may be of different colors.
[0112] Exemplarily, the calibration plate includes plane 1, plane 2, plane 3, plane 4 and plane 5, wherein plane 1 is adjacent to plane 2, plane 2 is adjacent to plane 3 and plane 5, and plane 4 is adjacent to plane 3 and plane 5, and adjacent planes among these planes can be different colors.
[0113] In this way, setting adjacent planes to different colors facilitates distinguishing feature points at different viewing angles in the second calibration image, thereby calculating more accurate second calibration parameters.
[0114] Furthermore, in some embodiments, each plane of the planar panel may be of a different color, which is not limited in the present application.
[0115] In a possible implementation manner of the present application, each plane of the calibration plate is provided with a checkerboard or a dot matrix pattern.
[0116] In this way, it is easy to extract multiple feature points, which is beneficial to calculate the second calibration parameter.
[0117] It should be noted that adjacent planes among the multiple planes of the calibration plate can be of different colors. At the same time, each plane can be provided with a checkerboard or a dot matrix pattern, which is not limited in this application.
[0118] S204: Calculate the actual coordinate point of the corner point of the virtual image plane based on the actual coordinate point of the first feature point and the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane.
[0119] In the embodiment of the present application, the actual coordinate points of the corner points of the virtual image plane are used to represent the actual position of the virtual image plane.
[0120] The corner points refer to points at the corners of the virtual image plane, such as the upper left corner, upper right corner, lower left corner and lower right corner of the virtual image plane. The corner point may refer to at least one point, which is not limited in the present application.
[0121] It should be understood that the corner point on the virtual image plane and the first feature point on the projected image have the same relative position relationship in different coordinate systems. That is, in the embodiment of the present application, the relative position relationship between the corner point on the virtual image plane and the first feature point on the projected image in the world coordinate system is the same as the relative position relationship between the corner point on the virtual image plane and the first feature point on the projected image in the world coordinate system.
[0122] Therefore, by clarifying the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane (used to characterize the relative position relationship between the two), as well as the actual coordinate point of the first feature point, the actual coordinate point of the corner point of the virtual image plane can be inferred.
[0123] In a possible implementation of the present application, S204 may include: first multiplying the distance between the image coordinate points of the corner points of the projected image and the image coordinate points of the corner points of the virtual image plane by the size ratio to obtain the distance between the actual coordinate points of the corner points of the first calibration image and the actual coordinate points of the corner points of the virtual image plane; then calculating the actual coordinate points of the corner points of the virtual image plane based on the actual coordinate points of the corner points of the first calibration image and the distance between the actual coordinate points of the corner points of the first calibration image and the actual coordinate points of the corner points of the virtual image plane.
[0124] The size ratio is the ratio of the size of the projected image in the world coordinate system to the size of the projected image in the first image coordinate system;
[0125] In some embodiments, the first feature points of the first calibration image can be the corner points of the projection image. Based on the introduction of the above steps, the actual coordinate points and image coordinate points of the corner points of the projection image can be obtained, and then the size of the projection image in the first image coordinate system and the size of the projection image in the world coordinate system can be calculated.
[0126] Exemplarily, in the world coordinate system, the absolute value of the actual coordinate point of the upper left corner of the projected image minus the actual coordinate point of the upper right corner can be used as the horizontal coordinate difference of the projected image, and the horizontal coordinate difference is then multiplied by the physical size per unit coordinate of the horizontal axis to obtain the horizontal size of the projected image; the vertical coordinate difference of the projected image is then subtracted from the absolute value of the actual coordinate point of the lower left corner of the projected image, and the vertical size of the projected image is then multiplied by the physical size per unit coordinate of the vertical axis to obtain the vertical size of the projected image; the horizontal size of the projected image and the vertical size of the projected image are multiplied to obtain the size of the projected image in the world coordinate system; similarly, the size of the projected image in the first image coordinate system can be calculated, which will not be repeated here.
[0127] It should be understood that the proportional relationship between the distance between the corner points of the virtual image plane and the projected image in the first image coordinate system and the distance between the corner points of the virtual image plane and the projected image in the world coordinate system is equivalent to the proportional relationship between the size of the projected image in the first image coordinate system and the size of the projected image in the world image coordinate system. Therefore, the proportional relationship calculated above can be used to further calculate the distance between the actual coordinate points of the corner points of the first calibration image and the actual coordinate points of the corner points of the virtual image plane.
[0128] Then, based on the same world coordinate system, through the distance formula between two actual coordinate points (that is, the corner point of the projection image and the corner point of the virtual image plane), based on the actual coordinate points of the corner points of the first calibration image, and the distance between the actual coordinate points of the corner points of the first calibration image and the actual coordinate points of the corner points of the virtual image plane, the actual coordinate points of the corner points of the virtual image plane can be calculated.
[0129] Exemplarily, the corner points of the virtual image plane and the corner points of the projected image may be the upper left corner, the upper right corner, the lower left corner and the lower right corner, so that the actual coordinate points of each corner point of the virtual image plane may be calculated.
[0130] It should be noted that the actual coordinates of each corner point of the virtual image plane are calculated for example only. The actual coordinates of fewer corner points of the virtual image plane can also be calculated, and then the actual coordinates of other corner points can be calculated by size ratio. This application does not limit this.
[0131] S205: Calculate and obtain first calibration parameters of the AR-HUD based on actual coordinate points of the corner points of the virtual image plane and actual coordinate points of the camera.
[0132] In the embodiment of the present application, the camera is at a preset eye point position, and the actual coordinate point of the camera is the actual coordinate point of the eyes of the driver or other user. The actual coordinate points of the corner points of the virtual image plane are used to represent the projection range of the AR-HUD. Therefore, based on the actual coordinate points of the user's eye position and the projection range of the AR-HUD, the user's perspective can be simulated to calculate the first calibration parameter of the AR-HUD.
[0133] The first calibration parameters (also called optomechanical parameters) refer to parameters used to describe the display effect of AR-HUD, etc., and can be used to indicate the correspondence between the world coordinate system and the image coordinate system.
[0134] In some embodiments, the AR-HUD may adopt a standard focal plane, which is not limited in this application.
[0135] Exemplarily, when the AR-HUD adopts a standard focal plane, the first calibration parameter may include eye point position, field of view angle, downward viewing angle, heading angle, roll angle, virtual image distance (VID), etc.
[0136] It should be noted that the above-mentioned first calibration parameters are only examples and may include more or fewer parameters than the above-mentioned parameters, and this application does not limit this.
[0137] As an implementation method, the perpendicular bisector of the virtual image plane, the center point of the virtual image plane, and the field of view angle can be calculated by the following formula. The perpendicular bisector of the virtual image plane refers to a straight line perpendicular to the center point on the virtual image plane; the field of view angle refers to the area size of the first calibration image that can be seen by the user's eyes when viewing the virtual image plane.
[0138] In the following formula, z c represents the scale factor, which is used to adjust the scale of the coordinates (that is, the proportional relationship between the coordinates in the first image coordinate system and the coordinates in the world coordinate system), u and v represent the horizontal and vertical coordinates in the first image coordinate system, dx and dy represent the horizontal and vertical actual lengths of the pixel point in the first image coordinate system, f refers to the focal length, R refers to the rotation matrix (used to indicate the rotation direction of the angle), t refers to the displacement of the eye point position, x w ,y w and z w Used to represent the three-dimensional coordinates in the world coordinate system, f x refers to the focal length in the lateral direction of the first calibration image, f y It refers to the focal length in the longitudinal direction of the first calibration image, u0 and v0 represent the horizontal and vertical coordinates of the center point of the virtual image plane, K refers to the intrinsic parameter of the camera, and M refers to the extrinsic parameter of the camera.
[0139]
[0140] As an example, see Fig.10 , which is a schematic diagram of calculating a first calibration parameter provided in an embodiment of the present application. It should be noted that the chessboard displayed on the virtual image plane is only an example, and a dot matrix can also be displayed, which is not limited in the present application.
[0141] Combination Fig.10 As shown in the figure, A is the eye point position (that is, the position of the user's eyes). Assuming that the figure is a top view effect, ∠BAC is the horizontal field of view (HFOV). Assuming that the figure is a left view effect, ∠BAC is the vertical field of view (VFOV). The actual coordinate points of each first feature point on the projected image (also known as the 3D coordinates of the corner point) can be obtained based on the calibrated camera (see the above example for details, which will not be repeated here).
[0142] According to the fact that EF (indicating the virtual image plane) and BC (indicating the first calibration image) are proportional in the world coordinate system (also called the physical coordinate system) and the image coordinate system (also called the pixel coordinate system), that is, the proportions of the two are the same in different coordinate systems, BC 物理 / EF 物理 =BC 像素 / EF 像素 Therefore, we can get BC 物理 =EF 物理 *BC 像素 / EF 像素 .like Fig.10 As shown, ∠BAC is the field of view. The angle between vector AD and the vehicle chassis plane is the downward viewing angle. The angle between vector AD and the left-right symmetric plane of the vehicle is the heading angle. The angle between vector BC and the vehicle chassis plane is the roll angle.
[0143] Among them, ∠BAC can be obtained by measuring the eye point position, the position on the upper boundary of the virtual image plane and the position on the lower boundary of the virtual image plane (the boundary position and the center point position of the virtual image plane can be determined based on the coordinate points of the corner points of the virtual image plane calculated above); the downward viewing angle can be obtained by measuring the eye point position, the position of the center point of the virtual image plane and the position of the vehicle chassis plane; the heading angle can be obtained by measuring the eye point position, the position of the center point of the virtual image plane and the position of the left-right symmetry plane of the vehicle; the roll angle can be obtained by measuring the position of the vehicle chassis plane, the position on the upper boundary of the virtual image plane and the position on the lower boundary of the virtual image plane.
[0144] Thus, based on the contents introduced in S201-S205 above, it can be known that in the embodiment of the present application, the camera is first calibrated by a calibration plate including multiple planes, and then the AR-HUD is calibrated by the calibrated camera, and different first calibration images and second calibration images are used during calibration. There is no need to adjust the calibration plate multiple times, nor is there any need to adjust the camera position and the position of the first calibration image projected on the virtual image plane. The calibration steps are simplified, and the calibration efficiency can be improved. That is, compared with the related art that requires a calibration plate for both camera and AR-HUD calibration, the embodiment of the present application only requires a special-shaped calibration plate for camera calibration, and only a calibrated camera is needed for AR-HUD calibration, which reduces the calibration cost. At the same time, only a calibration plate is needed for camera calibration, and only a calibrated camera is needed for AR-HUD calibration. There is no need to use a mechanical arm, an expensive calibration device, to further reduce the calibration cost, and the requirements for the calibration environment are also greatly reduced.
[0145] In addition, in the prior art, the position of the binocular camera is adjusted multiple times, and the binocular camera represents the eye point position, which may cause the final eye point position to deviate from user needs. However, when calibrating the AR-HUD, the present application can directly place the camera at the preset eye point position without adjustment, thereby meeting user needs.
[0146] The above are some specific implementations of the AR-HUD calibration method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding AR-HUD calibration device. The device provided in the embodiment of the present application will be introduced from the perspective of functional modularization.
[0147] See also Fig.11 , which is a schematic diagram of the structure of an AR-HUD calibration device provided in an embodiment of the present application. The AR-HUD calibration device 1100 may include:
[0148] The image acquisition module 1110 is used to acquire a first calibration image; the first calibration image is obtained by photographing a projection image of an augmented reality head-up display AR-HUD located in a vehicle on a virtual image plane through a calibrated camera; the projection image is a two-dimensional image; the camera is located at a preset eye point position; the first calibration image is in a first image coordinate system; the camera is in a world coordinate system;
[0149] A feature point extraction module 1120 is used to extract a first feature point of the first calibration image to obtain an image coordinate point of the first feature point;
[0150] A coordinate point mapping module 1130 is used to map the image coordinate point of the first feature point to the actual coordinate point of the first feature point in the world coordinate system based on the second calibration parameter of the camera; the second calibration parameter of the camera is obtained by calibrating the camera; the actual coordinate point of the first feature point is used to represent the actual position of the projected image;
[0151] A coordinate point calculation module 1140 is used to calculate the actual coordinate point of the corner point of the virtual image plane based on the actual coordinate point of the first feature point and the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane; the actual coordinate point of the corner point of the virtual image plane is used to represent the actual position of the virtual image plane;
[0152] The AR-HUD calibration module 1150 is used to calculate the first calibration parameters of the AR-HUD based on the actual coordinate points of the corner points of the virtual image plane and the actual coordinate points of the camera.
[0153] As an implementation manner, the second calibration parameters of the camera in the above-mentioned image acquisition module 1110 are obtained by calibrating the camera through a calibration plate; the calibration plate includes multiple planes, at least two of the multiple planes are not coplanar, and each plane includes multiple second feature points.
[0154] As an implementation method, the above-mentioned camera is calibrated by the following units:
[0155] An image acquisition unit, used for acquiring a second calibration image; the second calibration image is obtained by photographing the calibration plate with an uncalibrated camera; the second calibration image is in a second image coordinate system; the calibration plate is in a world coordinate system;
[0156] A feature point unit, used for obtaining an image coordinate point of a second feature point from a second feature point of the second calibration image;
[0157] The parameter calculation unit is used to calculate the second calibration parameter for each second feature point of the calibration plate based on the actual coordinate point of the second feature point on the calibration plate and the image coordinate point of the second feature point.
[0158] As an embodiment, the calibration plate in the above-mentioned image acquisition unit includes two planes; the second calibration image of the above-mentioned image acquisition unit includes two second calibration images; the two second calibration images are obtained by adjusting the angle of the calibration plate once, or by adjusting the angle of the camera once.
[0159] As an implementation manner, the calibration plate in the above-mentioned image acquisition unit includes at least three planes; and the second calibration image of the above-mentioned image acquisition unit includes a second calibration image.
[0160] As an implementation manner, the first feature point is a corner point of the projected image in the first calibration image; the coordinate point calculation module 1140 may specifically include:
[0161] a distance obtaining unit, configured to multiply the distance between the image coordinate point of the corner point of the projection image and the image coordinate point of the corner point of the virtual image plane by a size ratio to obtain the distance between the actual coordinate point of the corner point of the first calibration image and the actual coordinate point of the corner point of the virtual image plane; the size ratio is the ratio of the size of the projection image in the world coordinate system to the size of the projection image in the first image coordinate system;
[0162] A coordinate point calculation unit is used to calculate the actual coordinate point of the corner point of the virtual image plane based on the actual coordinate point of the corner point of the first calibration image and the distance between the actual coordinate point of the corner point of the first calibration image and the actual coordinate point of the corner point of the virtual image plane.
[0163] As an implementation manner, adjacent planes of the calibration plate in the above-mentioned image acquisition unit are of different colors, and / or each plane of the calibration plate is provided with a checkerboard or a dot matrix pattern.
[0164] The embodiments of the present application also provide a corresponding AR-HUD calibration device and a computer-readable storage medium for implementing the solution provided in the embodiments of the present application.
[0165] The device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program, so that the device executes the AR-HUD calibration method described in any embodiment of the present application.
[0166] The computer-readable storage medium stores a computer program. When the computer program is executed, a device executing the computer program implements the AR-HUD calibration method described in any embodiment of the present application.
[0167] The "first" and "second" in the names such as "first" and "second" (if any) mentioned in the embodiments of the present application are only used as name identifiers and do not represent the first or second in order.
[0168] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a readable storage medium, such as a read-only memory (ROM) / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.
[0169] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely schematic, in which the unit described as a separate component may or may not be physically separated, and the component prompted as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.
[0170] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A calibration method for AR-HUD, characterized in that: include: Acquire a first calibration image; the first calibration image is obtained by photographing a projection image of an augmented reality head-up display AR-HUD located in a vehicle on a virtual image plane through a calibrated camera; the projection image is a two-dimensional image; the camera is located at a preset eye point position; The first calibration image is in a first image coordinate system; the camera is in a world coordinate system; Extracting a first feature point of the first calibration image to obtain an image coordinate point of the first feature point; Based on the second calibration parameter of the camera, the image coordinate point of the first feature point is mapped to the actual coordinate point of the first feature point in the world coordinate system; the second calibration parameter of the camera is obtained by calibrating the camera; the actual coordinate point of the first feature point is used to represent the actual position of the projected image; Based on the actual coordinate point of the first feature point and the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane, calculate the actual coordinate point of the corner point of the virtual image plane; The actual coordinate points of the corner points of the virtual image plane are used to represent the actual position of the virtual image plane; Based on the actual coordinate points of the corner points of the virtual image plane and the actual coordinate points of the camera, first calibration parameters of the AR-HUD are calculated.
2. The method according to claim 1, characterized in that The second calibration parameters of the camera are obtained by calibrating the camera through a calibration plate; the calibration plate includes a plurality of planes, at least two of the plurality of planes are not coplanar, and each plane includes a plurality of second feature points.
3. The method according to claim 2, characterized in that The camera calibration step includes: Acquire a second calibration image; the second calibration image is obtained by photographing the calibration plate with an uncalibrated camera; the second calibration image is in a second image coordinate system; the calibration plate is in the world coordinate system; Extracting a second feature point of the second calibration image to obtain an image coordinate point of the second feature point; For each second feature point of the calibration plate, the second calibration parameter is calculated based on an actual coordinate point of the second feature point on the calibration plate and an image coordinate point of the second feature point.
4. The method according to claim 3, characterized in that The calibration plate includes two planes; the second calibration image includes two second calibration images; the two second calibration images are obtained by adjusting the angle of the calibration plate once, or by adjusting the angle of the camera once.
5. The method according to claim 3, characterized in that: The calibration plate includes at least three planes; the second calibration image includes a second calibration image.
6. The method according to any one of claims 1 to 5, characterized in that: The first feature point is a corner point of a projected image in the first calibration image; the actual coordinate point of the corner point of the virtual image plane is calculated based on the actual coordinate point of the first feature point and the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane, including: The distance between the image coordinate point of the corner point of the projection image and the image coordinate point of the corner point of the virtual image plane is multiplied by the size ratio to obtain the distance between the actual coordinate point of the corner point of the first calibration image and the actual coordinate point of the corner point of the virtual image plane; the size ratio is the ratio of the size of the projection image in the world coordinate system to the size of the projection image in the first image coordinate system; The actual coordinates of the corner points of the virtual image plane are calculated based on the actual coordinates of the corner points of the first calibration image and the distances between the actual coordinates of the corner points of the first calibration image and the actual coordinates of the corner points of the virtual image plane.
7. The method according to any one of claims 2 to 5, characterized in that: Adjacent planes of the calibration plate are of different colors, and / or each plane of the calibration plate is provided with a checkerboard or a dot matrix.
8. A calibration device for AR-HUD, characterized in that: The device comprises: An image acquisition module is used to acquire a first calibration image; the first calibration image is obtained by photographing a projection image of an augmented reality head-up display AR-HUD located in a vehicle on a virtual image plane through a calibrated camera; the projection image is a two-dimensional image; the camera is located at a preset eye point position; the first calibration image is in a first image coordinate system; the camera is in a world coordinate system; A feature point extraction module, used to extract a first feature point of the first calibration image and obtain an image coordinate point of the first feature point; a coordinate point mapping module, used for mapping the image coordinate point of the first feature point to the actual coordinate point of the first feature point in the world coordinate system based on the second calibration parameter of the camera; the second calibration parameter of the camera is obtained by calibrating the camera; the actual coordinate point of the first feature point is used to represent the actual position of the projected image; a coordinate point calculation module, configured to calculate the actual coordinate point of the corner point of the virtual image plane based on the actual coordinate point of the first feature point and the distance between the image coordinate point of the first feature point and the image coordinate point of the corner point of the virtual image plane; the actual coordinate point of the corner point of the virtual image plane is used to represent the actual position of the virtual image plane; The AR-HUD calibration module is used to calculate the first calibration parameters of the AR-HUD based on the actual coordinate points of the corner points of the virtual image plane and the actual coordinate points of the camera.
9. An AR-HUD calibration device, characterized in that: The device comprises a memory and a processor: The memory is used to store a computer program and transmit the computer program to the processor; The processor is used to execute the computer program so that the device performs the steps of the AR-HUD calibration method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: A computer program is stored on the computer-readable storage medium. When the computer program is executed, a device executing the computer program implements the steps of the AR-HUD calibration method according to any one of claims 1 to 7.
