Calibration method of augmented reality head-up display system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a calibration method of an augmented reality head-up display system, electronic equipment and a storage medium, and the method comprises the steps: obtaining an AR-HUD display image and a first target image shot by an industrial camera; determining a first mapping parameter of the industrial camera relative to a first three-dimensional coordinate system according to the first target image; according to the AR-HUD display image, determining a second mapping parameter of the AR-HUD relative to a second three-dimensional coordinate system; and determining a third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameter and the second mapping parameter.

Description

Technical Field

[0001] The present disclosure relates to the technical field of head-up display, and particularly to a calibration method, an electronic device and a computer-readable storage medium for an Augmented Reality Head-Up Display (AR-HUD). Background Art

[0002] With the continuous development of the automotive industry and the increasing demand for driving safety, the Head-Up Display (HUD) is an emerging technology that significantly improves the driving experience and enhances driving safety by presenting a graphical user interface (UI) combined with the driving environment in real time. Existing head-up display systems include HUD. In order to make the HUD display accurate information, the HUD is usually calibrated when the vehicle equipped with the HUD leaves the factory. However, after the vehicle leaves the factory, it is usually necessary to recalibrate the HUD due to certain reasons (for example, replacing the windshield).

[0003] The existing calibration methods mainly adopt the way of human eye subjective judgment. Although this method is easy to operate, it has obvious deficiencies. First, the accuracy of the human eye subjective calibration is greatly affected by individual differences. Different operators have different visual and judgment criteria, which will lead to differences in the calibration results and further affect the accuracy of the displayed information. Second, this calibration method relying on manual operation is inefficient. Especially in large-scale after-sales services, manual calibration will greatly prolong the service time. Finally, for the human eye, it is difficult to accurately judge whether the specific position and size of the HUD display content meet the design requirements, thus restricting the accuracy and reliability of HUD maintenance and calibration.

[0004] Compared with HUD, the Augmented Reality Head-Up Display (AR-HUD) provides more advanced functions, including the fusion display of external actual environment information, the real-time feedback of Advanced Driving Assistance System (ADAS) data, etc. In order to meet the realization of these functions, the calibration of AR-HUD requires higher precision in the display position, angle, size, viewing distance, etc., and precise spatial alignment with the vehicle's ADAS. Since the AR-HUD needs to accurately fuse virtual information with the driver's actual visual scene in real time, this puts higher requirements on the calibration process of the system. The existing HUD calibration methods on the market cannot meet the accuracy and efficiency requirements of AR-HUD, that is, they cannot calibrate AR-HUD. Summary of the Invention

[0005] The purpose of the embodiments of the present disclosure is to provide a calibration method, an electronic device, and a computer-readable storage medium for an augmented reality head-up display system, so as to solve the problem in the related art that the AR-HUD cannot be calibrated.

[0006] To solve the above technical problems, the embodiments of the present disclosure adopt the following technical solutions:

[0007] The embodiments of the present disclosure disclose a calibration method for an augmented reality head-up display system, including: obtaining an AR-HUD display image and a first target image captured by an industrial camera, where the AR-HUD display image is an image obtained by the industrial camera capturing a reference image displayed by the AR-HUD, and the first target image is an image obtained by the industrial camera capturing the target when the target is in a target position; determining a first mapping parameter of the industrial camera relative to a first three-dimensional coordinate system according to the first target image, where the first three-dimensional coordinate system is a three-dimensional coordinate system calibrated by an advanced driver assistance system (ADAS) on the target vehicle where the AR-HUD is located; determining a second mapping parameter of the AR-HUD relative to a second three-dimensional coordinate system according to the AR-HUD display image, where the second three-dimensional coordinate system is a three-dimensional coordinate system established according to the position of the industrial camera; and determining a third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameter and the second mapping parameter.

[0008] The embodiments of the present disclosure also disclose a calibration device for an augmented reality head-up display system. The device includes: an obtaining module, configured to obtain an AR-HUD display image and a first target image captured by an industrial camera, where the AR-HUD display image is an image obtained by the industrial camera capturing a reference image displayed by the AR-HUD, and the first target image is an image obtained by the industrial camera capturing the target when the target is in a target position; a first calibration module, configured to determine a first mapping parameter of the industrial camera relative to a first three-dimensional coordinate system according to the first target image, where the first three-dimensional coordinate system is a three-dimensional coordinate system calibrated by an advanced driver assistance system ADAS on the target vehicle where the AR-HUD is located; a second calibration module, configured to determine a second mapping parameter of the AR-HUD relative to a second three-dimensional coordinate system according to the AR-HUD display image, where the second three-dimensional coordinate system is a three-dimensional coordinate system established according to the position of the industrial camera; and a third calibration module, configured to determine a third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameter and the second mapping parameter.

[0009] Embodiments of the present disclosure also disclose an electronic device, which at least includes a memory and a processor. A computer program is stored on the memory, and when the processor executes the computer program on the memory, the steps of any one of the above methods are implemented.

[0010] Embodiments of the present disclosure also disclose a computer-readable storage medium. The computer-readable medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above methods are implemented.

[0011] In various embodiments of the present disclosure, by obtaining the mapping parameters of the AR-HUD relative to the three-dimensional coordinate system of the industrial camera and the mapping parameters of the industrial camera relative to the three-dimensional coordinate system calibrated by the ADAS, the mapping parameters of the AR-HUD relative to the three-dimensional coordinate system calibrated by the ADAS are determined, thereby realizing the calibration of the AR-HUD, solving the technical problem in the prior art that the AR-HUD cannot be calibrated, and achieving the technical effect of being able to accurately calibrate the AR-HUD in scenarios where the AR-HUD needs to be calibrated. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0013] Figure 1 is a schematic diagram of an application scenario of a calibration method for an AR-HUD according to an embodiment of the present disclosure;

[0014] Figure 2 is a schematic diagram of a target, a calibration station, and an imaging light cone of an AR-HUD according to an embodiment of the present disclosure;

[0015] Figure 3 is a schematic diagram of calibration using one or more target positions according to an embodiment of the present disclosure;

[0016] Figure 4 is a schematic diagram of a view of a target vehicle from the front position of the target vehicle according to an embodiment of the present disclosure;

[0017] Figure 5 is a schematic diagram of a reference image displayed by the AR-HUD according to an embodiment of the present disclosure and the second set of feature points therein;

[0018] Figure 6 is a schematic diagram of the first set of feature points in the target according to an embodiment of the present disclosure;

[0019] Figure 7 It is a flowchart of a calibration method for an AR - HUD according to an embodiment of the present disclosure;

[0020] Figure 8 It is a flowchart of a method for determining a second mapping parameter of the AR - HUD relative to a second three - dimensional coordinate system according to the AR - HUD display image in an embodiment of the present disclosure;

[0021] Figure 9 It is a flowchart of a method for determining a second mapping parameter of the imaging light cone of the AR - HUD relative to a second three - dimensional coordinate system according to a second set of two - dimensional image coordinates and the internal parameters of an industrial camera in an embodiment of the present disclosure;

[0022] Figure 10 It is a schematic diagram of the pitch angle, yaw angle, and roll angle in an embodiment of the present disclosure;

[0023] Figure 11 It is a flowchart of a method for determining a second rotation matrix included in the second mapping parameter according to the first center point coordinate, the second center point coordinate, the second set of two - dimensional image coordinates, and the internal parameters of the industrial camera in an embodiment of the present disclosure;

[0024] Figure 12 It is a flowchart of a method for determining a third mapping parameter of the AR - HUD relative to the first three - dimensional coordinate system according to the first mapping parameter and the second mapping parameter in an embodiment of the present disclosure;

[0025] Figure 13 It is a flowchart of a method for determining a first mapping parameter of the industrial camera relative to the first three - dimensional coordinate system according to the first target image in an embodiment of the present disclosure;

[0026] Figure 14 It is a flowchart of a method for determining the first mapping parameter of the industrial camera relative to the first three - dimensional coordinate system according to the first set of two - dimensional image coordinates in an embodiment of the present disclosure;

[0027] Figure 15 It is a flowchart of a method for determining the set of three - dimensional space coordinates in an embodiment of the present disclosure;

[0028] Figure 16 It is a schematic structural diagram of a calibration device for an AR - HUD. Detailed implementation manners

[0029] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

[0030] It should be noted that the terms "first", "second", etc. in the description, claims, and the above-mentioned drawings of the embodiments of the present disclosure are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence.

[0031] Figure 1 It is a schematic diagram of an application scenario of a calibration method for an augmented reality head-up display system (AR-HUD) according to an embodiment of the present disclosure. Figure 2 It is a schematic diagram of the target position, calibration station, and imaging light cone of the AR-HUD according to an embodiment of the present disclosure. Figure 3 It is a schematic diagram of calibrating using one or more target positions according to an embodiment of the present disclosure. In an alternative solution, after the target vehicle leaves the factory, due to certain reasons (for example, replacing the front windshield, or replacing the AR-HUD, or replacing the in-vehicle ADAS, etc.), it is necessary to recalibrate the AR-HUD in the target vehicle. In order to calibrate the AR-HUD in the target vehicle, an AR-HUD calibration device is proposed in the embodiments of the present disclosure. Of course, the calibration method and calibration device proposed in the embodiments of the present disclosure can also be used to calibrate the AR-HUD on the vehicle before it leaves the factory.

[0032] As Figure 1 and Figure 2 shown, as an alternative solution, the AR-HUD calibration device in the embodiments of the present disclosure may include a control terminal and an industrial camera (it should be understood that the industrial camera can be, but is not limited to, any camera with graphic acquisition capabilities). When calibrating the AR-HUD in the target vehicle, a target displaying a set of feature points is placed in front of the target vehicle. Optionally, the target vehicle can be parked at a pre-set calibration station (as Figure 2 and Figure 3 shown), at this time, the target is in front of the calibration station. As an alternative example, when calibrating the AR-HUD in the target vehicle, the target can be placed at a fixed position, such as one of the first target position, the second target position, and the third target position as Figure 3 shown. Of course, the target can also be placed at multiple fixed positions, for example, two or three of the first target position, the second target position, and the third target position as Figure 3 shown. It can be understood that the embodiments of the present disclosure do not limit the number and position of the target positions where the target is placed.

[0033] As an alternative example, the industrial camera can be set at the position of the driver's seat (as Figure 4As shown, the industrial camera is installed and positioned through the mounting posts of the headrest of the driver's cockpit seat); the front-view camera of the ADAS (as an example, referred to as the target camera) is arranged inside or outside the vehicle cockpit. For example, it is located at the top of the front-row driving position or at the bottom of the front windshield (as Figure 1 shown). It can be understood that the embodiments of the present disclosure do not limit the positions of the industrial camera and the front-view camera of the ADAS. As an alternative example, the above-mentioned front-view camera is referred to as the target camera and is used to capture images in front of the target vehicle.

[0034] In Figure 1 the scenario, an AR-HUD and an ADAS are provided in the target vehicle, where the ADAS includes one or more sensors (for example, radars located at different parts of the target vehicle) and one or more cameras (for example, the above-mentioned front-view camera). When the target is placed at one or more target positions and a set of feature points (for example, referred to as the first set of feature points) are displayed on the target, the industrial camera and the front-view camera of the ADAS respectively capture the target to obtain corresponding target images. In addition, when the AR-HUD displays a reference image and a set of feature points (for example, referred to as the second set of feature points) are displayed in the reference image, the industrial camera also captures the reference image displayed by the AR-HUD to obtain the AR-HUD display image. As an alternative example, the embodiments of the present disclosure do not limit the position of the ADAS in the target vehicle.

[0035] As an alternative example, as Figure 1 shown, the above control terminal establishes a communication connection with the front-view camera of the ADAS and the industrial camera, and is used to obtain the target images respectively captured by the industrial camera and the front-view camera of the ADAS, and the AR-HUD display image captured by the industrial camera. Then, using the above target images and the AR-HUD display image, the mapping parameters (for example, referred to as the third mapping parameters) of the AR-HUD relative to the three-dimensional coordinate system (for example, referred to as the first three-dimensional coordinate system) calibrated by the ADAS are determined, that is, the calibration of the AR-HUD is realized.

[0036] As an alternative example, the mapping parameters in the embodiments of the present disclosure may but are not limited to include a rotation matrix and a translation vector. Optionally, the third mapping parameters of the above AR-HUD relative to the first three-dimensional coordinate system calibrated by the ADAS may but are not limited to include a third rotation matrix and a third translation vector.

[0037] As an alternative example, the above control terminal may be an industrial control computer, a notebook computer, a tablet computer, or a device deployed in the cloud, etc. It can be understood that the embodiments of the present disclosure do not limit the type or deployment location of the control terminal. For example, as Figure 1As shown, the control terminal is located outside the target vehicle and establishes a communication connection with the front-view camera and the industrial camera of the ADAS through a network connection. As an alternative example, the control terminal can also be located inside the target vehicle. For example, it can be set as an independent terminal inside the target vehicle, or integrated with the in-vehicle system of the target vehicle (i.e., the in-vehicle system can implement the steps performed by the control terminal in the embodiments of the present disclosure), or integrated with the ADAS on the target vehicle (i.e., the ADAS can implement the steps performed by the control terminal in the embodiments of the present disclosure). As another alternative example, the above-mentioned control terminal establishing a communication connection with the front-view camera and the industrial camera of the ADAS is only an example. In order to calibrate the AR-HUD, the above-mentioned control terminal can also establish a communication connection with other devices and further calibrate the AR-HUD in combination with the data collected or obtained by other devices. Among them, other devices include, for example, other devices in the ADAS except the front-view camera, such as sensors (e.g., radars) in the ADAS or cameras other than the front-view camera. That is, it can be understood that the embodiments of the present disclosure do not limit the devices connected to the control terminal.

[0038] In an alternative example, the ADAS will pre-calibrate each device in the ADAS (which can but is not limited to including: one or more sensors (e.g., radars located at different parts of the target vehicle) and one or more cameras (e.g., the above-mentioned front-view camera)), that is, the mapping parameters (e.g., can include rotation matrix and translation vector) of the two-dimensional coordinate system or three-dimensional coordinate system of the above-mentioned each device relative to the three-dimensional coordinate system calibrated by the ADAS (e.g., called the first three-dimensional coordinate system) are pre-determined. For example, the mapping parameters of the front-view camera relative to the first three-dimensional coordinate system (e.g., called the fifth mapping parameters) can be pre-determined. Optionally, the fifth mapping parameters of the above-mentioned front-view camera relative to the first three-dimensional coordinate system can but are not limited to include the fifth rotation matrix and the fifth translation vector.

[0039] In an alternative example, through the mapping parameters (e.g., can include rotation matrix and translation vector) of the two-dimensional coordinate system or three-dimensional coordinate system of each device relative to the three-dimensional coordinate system calibrated by the ADAS (e.g., called the first three-dimensional coordinate system), the data collected by each device in its own two-dimensional coordinate system or three-dimensional coordinate system can be converted into the same three-dimensional coordinate system (e.g., called the first three-dimensional coordinate system), so that the data collected by each device can be processed collaboratively in the above-mentioned same three-dimensional coordinate system (e.g., called the first three-dimensional coordinate system). For example, when the fifth mapping parameters of the front-view camera relative to the first three-dimensional coordinate system are determined, the data in the image captured by the front-view camera or the data recognized from the image can be converted into the above-mentioned same three-dimensional coordinate system (e.g., called the first three-dimensional coordinate system).

[0040] As an alternative example, as Figure 3 shown, the three-dimensional coordinate system calibrated by the above ADAS (for example, referred to as the first three-dimensional coordinate system) can be a three-dimensional coordinate system established with the front of the target vehicle as the origin. In the first three-dimensional coordinate system, the width direction of the target vehicle can be the x-axis (for example, to the right or left along the width direction of the target vehicle), the forward direction of the target vehicle can be the z-axis, and the vertical direction of the plane where the target vehicle is located can be the y-axis (for example, upward or downward perpendicular to the plane where the target vehicle is located). It can be understood that the position of the three-dimensional coordinate system calibrated by the above ADAS (for example, referred to as the first three-dimensional coordinate system) and the directions of each axis are only examples, and the embodiments of the present disclosure do not limit this.

[0041] As an alternative example, it is possible but not limited to use existing methods to determine the rotation matrix and translation vector of the two-dimensional coordinate system or three-dimensional coordinate system of each of the above devices relative to the three-dimensional coordinate system calibrated by the ADAS (for example, referred to as the first three-dimensional coordinate system).

[0042] Figure 4 is a schematic diagram of the view of the target vehicle from the front position of the target vehicle in the embodiments of the present disclosure. In Figure 4 , the vertex of the imaging light cone of the AR-HUD and the position of the industrial camera are shown. It should be noted that this imaging light cone is a schematic equivalent to the imaging coverage range of the AR-HUD, and the actual imaging range of the AR-HUD is within this imaging light cone, which can be part or all of the imaging light cone. As an alternative example, the vertex of the imaging light cone of the AR-HUD is at the position of the industrial camera, that is, the vertex of the imaging light cone of the AR-HUD and the industrial camera are at the same position. It can be understood that the vertex of the imaging light cone of the AR-HUD and the industrial camera being at the same position is only an example, and the embodiments of the present disclosure do not limit the position of the vertex of the imaging light cone of the AR-HUD and the relationship between the position of the vertex of the imaging light cone of the AR-HUD and the position of the industrial camera.

[0043] As an alternative example, as Figure 4 shown, the industrial camera is located at the center position of the eye box of the AR-HUD and is fixedly installed through the mounting post of the headrest of the driver's cockpit seat to simulate the driver's eyes. As an alternative example, in the second three-dimensional coordinate system, the position of the industrial camera is used as the origin, and the vertex of the imaging light cone of the AR-HUD is at the position of the industrial camera. As an alternative example, when the industrial camera is installed at the position of the headrest of the driver's cockpit seat, the schematic diagram of the imaging light cone of the AR-HUD is as Figure 2 and Figure 3 shown.

[0044] To solve the above problems, embodiments of the present disclosure provide a calibration method for an augmented reality head-up display system applied to Figure 1 and Figure 2 the scenarios shown in, the method comprising:

[0045] Obtain an AR-HUD display image and N target images captured by an industrial camera, where the AR-HUD display image is an image obtained by the industrial camera capturing a reference image displayed by the AR-HUD, and the N target images are images obtained by the industrial camera capturing the target when the target is located at N positions respectively. The industrial camera is located inside the target vehicle, and the target is located outside the target vehicle. N is 1 or a positive integer greater than or equal to 2;

[0046] Determine first mapping parameters of the industrial camera relative to a first three-dimensional coordinate system according to the N target images, where the first three-dimensional coordinate system is a three-dimensional coordinate system calibrated by an advanced driver assistance system ADAS on the target vehicle;

[0047] Determine second mapping parameters of the AR-HUD relative to a second three-dimensional coordinate system according to the AR-HUD display image, where the second three-dimensional coordinate system is a three-dimensional coordinate system established according to the position where the industrial camera is located;

[0048] Determine third mapping parameters of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameters and the second mapping parameters. Hereinafter, taking the target at one target position (for example, Figure 3 one of the first target position, the second target position, and the third target position shown in) as an example, the implementation process of a calibration method for an AR-HUD is described. The calibration process of multiple targets (or one target at multiple target positions respectively) located at different target positions in one calibration process is similar thereto.

[0049] Figure 7 is a flowchart of a calibration method for an AR-HUD according to an embodiment of the present disclosure. In one implementation manner, as shown in Figure 7 the method comprises:

[0050] Step S701, obtain an AR-HUD display image and a first target image captured by the industrial camera, where the AR-HUD display image is an image obtained by the industrial camera capturing a reference image displayed by the AR-HUD, and the first target image is an image obtained by the industrial camera capturing the target when the target is located at the target position;

[0051] As an optional example, the above target position may but is not limited to be Figure 3One of the first target position, the second target position, and the third target position shown in [figure]. As an optional example, the embodiments of the present disclosure do not limit the manner in which the industrial camera captures the AR-HUD display image and the first target image. For example, the industrial camera can capture an image that includes both the AR-HUD display image and the first target image at one time, or can separately capture the AR-HUD display image and the first target image, as long as the position of the industrial camera in the two captures remains unchanged. As an optional example, for the case where the AR-HUD display image and the first target image are within one image, the AR-HUD display image and the first target image can be extracted from the above-mentioned one image through image recognition.

[0052] Step S702: Determine the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first target image, where the first three-dimensional coordinate system is the three-dimensional coordinate system calibrated by the Advanced Driver Assistance System (ADAS) on the target vehicle where the AR-HUD is located;

[0053] As an optional example, the first set of feature points on the above-mentioned target can be, but are not limited to, as Figure 6 shown. The first set of two-dimensional image coordinates is used to represent the positions of the first set of feature points in the first target image. In Figure 6 , the first set of feature points is arranged in regular multiple rows and multiple columns, for example, multiple rows and multiple columns with equal spacing. This is only an example. The first set of feature points in the embodiments of the present disclosure can also be arranged in other shapes, such as a parallelogram, a circle, or a triangle, etc. As an optional example, the spacing between the first set of feature points arranged in a certain shape is known.

[0054] As an optional example, each two-dimensional image coordinate in the above-mentioned first set of two-dimensional image coordinates can be, but is not limited to, represented by (u, v), where the two-dimensional image coordinate (u i , v i ) corresponding to the i-th feature point in the first set of feature points represents that the i-th feature point is the pixel at the u-th row and v-th column in the first target image, and i is a positive integer. It can be understood that the above (u, v) is only one representation method of each two-dimensional image coordinate in the first set of two-dimensional image coordinates, and the embodiments of the present disclosure do not limit this.

[0055] For ease of understanding, the following first gives an exemplary principle description of the process of determining the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first target image. It can be understood that the following description is an example of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this:

[0056] S1-1: Obtain the two-dimensional image coordinates of the feature points on the target in the target image captured by the industrial camera. For example, identify the first set of feature points on the target in the first target image to obtain the first set of two-dimensional image coordinates;

[0057] S1-2: Obtain the three-dimensional space coordinates of the feature points on the target in the first three-dimensional coordinate system, and establish a correspondence between the above two-dimensional image coordinates and the above three-dimensional space coordinates of the same feature points. For example, establish a correspondence between the first set of two-dimensional image coordinates and a pre-determined set of three-dimensional space coordinates, where a set of three-dimensional space coordinates represents the position of the first set of feature points in the first three-dimensional coordinate system;

[0058] S1-3: Use the above correspondence and the internal parameters of the industrial camera to determine the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system. For example, determine the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the correspondence and the internal parameters of the industrial camera.

[0059] As an optional example, assume that the first mapping parameter is the parameter to be obtained. Based on the first mapping parameter and the internal parameters of the industrial camera, the two-dimensional image coordinates of the feature points obtained in S1-1 in the image captured by the industrial camera can be converted into the three-dimensional space coordinates of the feature points in the first three-dimensional coordinate system. According to the above correspondence, the three-dimensional space coordinates of the above converted feature points in the first three-dimensional coordinate system are the three-dimensional space coordinates of the feature points obtained in S1-2 in the first three-dimensional coordinate system. According to the above logic, in the case of knowing the correspondence between the above two-dimensional image coordinates and the above three-dimensional space coordinates of the same feature points and the internal parameters of the industrial camera, the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system can be determined according to the above correspondence and the internal parameters of the industrial camera.

[0060] Step S703, determine the second mapping parameter of the AR-HUD relative to the second three-dimensional coordinate system according to the AR-HUD display image, where the second three-dimensional coordinate system is a three-dimensional coordinate system established according to the position of the industrial camera;

[0061] As an optional example, as Figure 3 shown, the above second three-dimensional coordinate system can be a three-dimensional coordinate system established with the industrial camera as the origin. In the second three-dimensional coordinate system, the width direction of the target vehicle can be used as the x-axis (for example, to the right or left along the width direction of the target vehicle), the forward direction of the target vehicle can be used as the z-axis, and the vertical direction of the plane where the target vehicle is located can be used as the y-axis (for example, upward or downward perpendicular to the plane where the target vehicle is located). It can be understood that the position of the second three-dimensional coordinate system and the directions of each axis are only an example, and the embodiments of the present disclosure do not limit this.

[0062] For ease of understanding, first, an exemplary principle description of the process of determining the second mapping parameter of the AR-HUD relative to the second three-dimensional coordinate system based on the AR-HUD display image is provided. It can be understood that the following description is an example of the embodiments of the present disclosure, and the embodiments of the present disclosure are not limited thereto:

[0063] S2-1: Obtain the two-dimensional image coordinates of the feature points in the reference image in the AR-HUD display image captured by the industrial camera. For example, identify the second set of feature points in the reference image in the AR-HUD display image to obtain the second set of two-dimensional image coordinates;

[0064] S2-2: When the second three-dimensional coordinate system has the position of the industrial camera as the origin and the vertex of the imaging light cone of the AR-HUD is at the position of the industrial camera, set the second translation vector included in the second mapping parameter to a preset value; As an optional example, the above preset value indicates that the vertex of the imaging light cone of the AR-HUD coincides with the position of the industrial camera;

[0065] S2-3: According to the second set of two-dimensional image coordinates and the internal parameters of the industrial camera, the rotation angles of the AR-HUD display image relative to the axes of the second three-dimensional coordinate system (for example, pitch angle, yaw angle, and roll angle) can be determined. For example, according to the first center point coordinates, the second center point coordinates, the second set of two-dimensional image coordinates, and the internal parameters of the industrial camera, determine the second rotation matrix included in the second mapping parameter; Then, according to the pitch angle, yaw angle, and roll angle, determine the second rotation matrix included in the second mapping parameter.

[0066] As an optional example, assume that the second mapping parameter is the parameter to be obtained. When the second three-dimensional coordinate system has the position of the industrial camera as the origin and the vertex of the imaging light cone of the AR-HUD is at the position of the industrial camera, that is, the second translation vector included in the second mapping parameter can be set to a preset value, and the above preset value indicates that the vertex of the imaging light cone of the AR-HUD coincides with the position of the industrial camera. In addition, based on the second rotation matrix included in the second mapping parameter, the rotation angles of the AR-HUD display image captured by the industrial camera relative to the axes of the second three-dimensional coordinate system (for example, pitch angle, yaw angle, and roll angle) can be obtained, which are the pitch angle, yaw angle, and roll angle obtained in S2-2 above. According to the above logic, when the two-dimensional image coordinates of the feature points in the reference image in the AR-HUD display image captured by the industrial camera and the internal parameters of the industrial camera are known, the pitch angle, yaw angle, and roll angle can be determined according to the second set of two-dimensional image coordinates and the internal parameters of the industrial camera, and the second rotation matrix included in the second mapping parameter can be determined according to the pitch angle, yaw angle, and roll angle.

[0067] As an alternative example, the second set of feature points in the reference image shown in the AR-HUD display image may but is not limited to being as Figure 5 shown. The second set of two-dimensional image coordinates are used to represent the positions of the second set of feature points in the AR-HUD display image. In Figure 5 , the second set of feature points are arranged in regular multiple rows and multiple columns, for example, multiple rows and multiple columns with equal spacing. This is only an example, and the second set of feature points in the embodiments of the present disclosure may also be arranged in other shapes, such as a parallelogram, a circle, or a triangle, etc. As an alternative example, the spacing between the second set of feature points arranged in a certain shape is known.

[0068] As an alternative example, each two-dimensional image coordinate in the above-mentioned second set of two-dimensional image coordinates may but is not limited to be represented by (u, v), where the two-dimensional image coordinate (u j , v j ) corresponding to the j-th feature point in the second set of feature points indicates that the j-th feature point is the pixel at the u-th row and v-th column in the AR-HUD display image, and j is a positive integer. It can be understood that the above (u, v) is only one representation method of each two-dimensional image coordinate in the second set of two-dimensional image coordinates, and the embodiments of the present disclosure do not limit this.

[0069] Step S704: Determine the third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameter and the second mapping parameter.

[0070] For ease of understanding, the process of determining the third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameter and the second mapping parameter will be described by way of an exemplary principle first. It can be understood that the following description is an example of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this:

[0071] When the second mapping parameter of the AR-HUD relative to the second three-dimensional coordinate system where the industrial camera is located and the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system are known, the third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system can be obtained by performing arithmetic operations on the second mapping parameter and the first mapping parameter.

[0072] For example, S3-1: Concatenate the first rotation matrix and the first translation vector included in the first mapping parameter to obtain a first target matrix, where the first target matrix includes each value in the first rotation matrix and each value in the first translation vector;

[0073] S3-2: Concatenate the second rotation matrix and the second translation vector included in the second mapping parameter to obtain a second target matrix, where the second target matrix includes each value in the second rotation matrix and each value in the second translation vector;

[0074] S3-3: Determine a third rotation matrix and a third translation vector according to the first target matrix and the second target matrix, where the third mapping parameter includes the third rotation matrix and the third translation vector.

[0075] It can be understood that when the second mapping parameter of the AR-HUD relative to the second three-dimensional coordinate system where the industrial camera is located and the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system are known, a point (for example, point A) displayed by the AR-HUD can be converted into the second three-dimensional coordinate system according to the above second mapping parameter to obtain a point (for example, point B) located in the second three-dimensional coordinate system. Then, according to the above first mapping parameter, a point (for example, point B) located in the second three-dimensional coordinate system can be converted into the first three-dimensional coordinate system to obtain a point (for example, point C) located in the first three-dimensional coordinate system. Assume that the third mapping parameter is the parameter to be obtained. According to the above third mapping parameter, a point (for example, point A) displayed by the AR-HUD is converted into the first three-dimensional coordinate system to obtain a point (this point should be point C) located in the second three-dimensional coordinate system. According to the above logic, when the second mapping parameter of the AR-HUD relative to the second three-dimensional coordinate system where the industrial camera is located and the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system are known, the third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system can be determined according to the first mapping parameter and the second mapping parameter.

[0076] In the embodiments of the present disclosure, the mapping parameter may but is not limited to being represented by a rotation matrix and a translation vector. The embodiments of the present disclosure do not limit the content included in the mapping parameter or the form used to represent the mapping parameter.

[0077] By obtaining the mapping parameter of the AR-HUD relative to the three-dimensional coordinate system of the industrial camera and the mapping parameter of the industrial camera relative to the three-dimensional coordinate system calibrated by the ADAS, the mapping parameter of the AR-HUD relative to the three-dimensional coordinate system calibrated by the ADAS is determined, thereby realizing the calibration of the AR-HUD, solving the technical problem in the prior art that the AR-HUD cannot be calibrated, and achieving the technical effect of accurately calibrating the AR-HUD in scenarios where the AR-HUD needs to be calibrated.

[0078] Figure 5 It is a schematic diagram of the reference image displayed by the AR-HUD in the embodiments of the present disclosure and the second set of feature points therein. Figure 8It is a flowchart of a method for determining the second mapping parameter of an AR - HUD relative to a second three - dimensional coordinate system according to the AR - HUD display image in an embodiment of the present disclosure.

[0079] In one implementation, as Figure 5 and Figure 8 shown, determining the second mapping parameter of the AR - HUD relative to the second three - dimensional coordinate system according to the AR - HUD display image includes:

[0080] Step S801, identifying a second set of feature points in the reference image in the AR - HUD display image to obtain a second set of two - dimensional image coordinates;

[0081] Step S802, determining the second mapping parameter of the imaging light cone of the AR - HUD relative to the second three - dimensional coordinate system according to the second set of two - dimensional image coordinates and the internal parameters of the industrial camera.

[0082] As an optional example, the internal parameters of the industrial camera may include but are not limited to: the focal length of the industrial camera, the size of the captured image (or called pixel size, that is, the number of rows and columns of pixels in the image. For example, if the size of the image is 1024 * 512, it means the number of pixels included in the image is 1024 * 512, that is, one column in the image includes 1024 pixels and one row in the image includes 512 pixels), the number of pixels corresponding to the unit rotation angle of the x - axis in the second three - dimensional coordinate system relative to the industrial camera (for example, called the first unit parameter, denoted by F x ), the number of pixels corresponding to the unit rotation angle of the y - axis in the second three - dimensional coordinate system relative to the industrial camera (for example, called the second unit parameter, denoted by F y ).

[0083] Figure 9 It is a flowchart of a method for determining the second mapping parameter of the imaging light cone of the AR - HUD relative to the second three - dimensional coordinate system according to the second set of two - dimensional image coordinates and the internal parameters of the industrial camera. In one implementation, as Figure 9 shown, determining the second mapping parameter of the imaging light cone of the AR - HUD relative to the second three - dimensional coordinate system according to the second set of two - dimensional image coordinates and the internal parameters of the industrial camera includes:

[0084] Step S901, when the second three - dimensional coordinate system has the position where the industrial camera is located as the origin and the vertex of the imaging light cone of the AR - HUD is the position where the industrial camera is located, setting the second translation vector included in the second mapping parameter to a preset value. As an optional example, the above - mentioned preset value indicates that the vertex of the imaging light cone of the AR - HUD coincides with the position where the industrial camera is located.

[0085] As an alternative example, the above preset value is a vector composed of multiple 0s, which is used to represent that the vertex of the imaging light cone of the AR-HUD coincides with the position where the industrial camera is located.

[0086] Step S902: Determine the first center point coordinates according to the second set of two-dimensional image coordinates, where the first center point coordinates represent the position of the center point of the shape formed by the second set of feature points in the reference image.

[0087] As an alternative example, assume that the second set of two-dimensional image coordinates includes the two-dimensional image coordinates of 7*7 feature points, and the above 7*7 feature points are arranged equidistantly in 7 rows and 7 columns. Then, the two-dimensional image coordinates of the feature point in the 4th row and 4th column among the 7*7 feature points are determined as the first center point coordinates.

[0088] Step S903: Determine the second center point coordinates according to the size of the AR-HUD display image captured by the industrial camera, where the second center point coordinates are used to represent the position of the center point of the AR-HUD display image.

[0089] As an alternative example, assume that the size of the image captured by the industrial camera is 1024 pixels * 1024 pixels. Then, the second center point coordinates are determined as (512, 512).

[0090] Step S904: Determine the second rotation matrix included in the second mapping parameter according to the first center point coordinates, the second center point coordinates, the second set of two-dimensional image coordinates, and the internal parameters of the industrial camera.

[0091] As an alternative example, through the method provided in the embodiments of the present disclosure, the rotation matrix and translation vector of the AR-HUD relative to the three-dimensional coordinate system of the industrial camera can be determined. Furthermore, in the augmented reality scenario, the data displayed by the AR-HUD can be converted into the three-dimensional coordinate system of the industrial camera, or the data collected by the industrial camera can be converted into the three-dimensional coordinate system of the AR-HUD.

[0092] Figure 10 It is a schematic diagram of the pitch angle, yaw angle, and roll angle of the embodiments of the present disclosure. Figure 11 It is a flowchart of the method for determining the second rotation matrix included in the second mapping parameter according to the first center point coordinates, the second center point coordinates, the second set of two-dimensional image coordinates, and the internal parameters of the industrial camera in the embodiments of the present disclosure.

[0093] As an alternative example, as Figure 3As shown in the figure, in the second three-dimensional coordinate system, the width direction of the target vehicle can be used as the x-axis (also referred to as the X-axis, for example, to the right or left along the width direction of the target vehicle), the forward direction of the target vehicle can be used as the z-axis (also referred to as the Z-axis), and the vertical direction of the plane where the target vehicle is located can be used as the y-axis (also referred to as the Y-axis, for example, upward or downward perpendicular to the plane where the target vehicle is located). As an optional example, the origin of the second three-dimensional coordinate system can be the position where the industrial camera is located. It can be understood that the position of the second three-dimensional coordinate system and the directions of each axis described above are only examples, and the embodiments of the present disclosure do not limit this.

[0094] As an optional example, as Figure 2 and 3 shown, a cross-section of the imaging light cone of the AR-HUD can be, but is not limited to, parallel to the plane formed by the x-axis and the y-axis in the second three-dimensional coordinate system. As an optional example, Figure 2 and 3 the above-mentioned cross-section of the imaging light cone shown in is parallel to the plane where the target is located. It can be understood that the relationship between the imaging light cone of the AR-HUD described above and the second three-dimensional coordinate system is only an example, and the embodiments of the present disclosure do not limit this.

[0095] In one implementation manner, as Figure 5 , Figure 10 and Figure 11 shown, according to the first center point coordinates, the second center point coordinates, the second set of two-dimensional image coordinates, and the internal parameters of the industrial camera, determining the second rotation matrix included in the second mapping parameter includes:

[0096] Step S1101, when the first center point coordinates are (u1, v1) and the second center point coordinates are (u2, v2), determine the pitch angle according to the difference between u1 and u2 and the first unit parameter, where the internal parameters of the industrial camera include the first unit parameter, and the first unit parameter represents the number of pixels corresponding to the unit rotation angle relative to the x-axis in the second three-dimensional coordinate system;

[0097] Step S1102, determine the yaw angle according to the difference between v1 and v2 and the second unit parameter, where the internal parameters of the industrial camera include the second unit parameter, and the second unit parameter represents the number of pixels corresponding to the unit rotation angle relative to the y-axis in the second three-dimensional coordinate system;

[0098] Step S1103, determine the roll angle according to the second set of two-dimensional image coordinates;

[0099] Step S1104, determine the second rotation matrix according to the pitch angle, the yaw angle, and the roll angle.

[0100] As an alternative example, existing methods can be used, but are not limited to, to determine the pitch angle based on the difference between u1 and u2 and the first unit parameter.

[0101] As an alternative example, existing methods can be used, but are not limited to, to determine the yaw angle based on the difference between v1 and v2 and the second unit parameter.

[0102] As an alternative example, existing methods can be used, but are not limited to, to determine the second rotation matrix based on the pitch angle, yaw angle, and roll angle.

[0103] As an alternative example, by the method provided in the embodiments of the present disclosure, based on the given first center point coordinates (u1, v1), second center point coordinates (u2, v2), and the internal parameters of the industrial camera (F x and F y ), the pitch angle and yaw angle can be determined. As an alternative example, the pitch angle describes the rotation angle of the AR display image or the above-mentioned second set of feature points relative to the x-axis in the second three-dimensional coordinate system, and the yaw angle describes the rotation angle of the AR display image or the above-mentioned second set of feature points relative to the y-axis. By the method provided in the embodiments of the present disclosure, the pitch angle, yaw angle, and roll angle can be accurately determined, thereby accurately determining the second rotation matrix.

[0104] It can be understood that the above method is only an example, and the pitch angle and yaw angle can also be determined by other methods.

[0105] In an alternative example, to determine the roll angle based on the second set of two-dimensional image coordinates, it includes:

[0106] When the second set of feature points are arranged in K rows and L columns in the reference image, determine the roll angle according to the two-dimensional image coordinates corresponding to the feature points in the first column and the last column in at least some rows of the second set of two-dimensional image coordinates, where K and L are positive integers greater than or equal to 2.

[0107] It can be understood that the above method is only an example, and the roll angle can also be determined by other methods.

[0108] As an alternative example, to determine the roll angle according to the two-dimensional image coordinates corresponding to the feature points in the first column and the last column in at least some rows of the second set of two-dimensional image coordinates, it includes one of the following:

[0109] 1) When the second set of two-dimensional image coordinates includes the coordinates (u i1 , v i1 ) and the coordinates (u iL , v iL ), existing methods can be used, but are not limited to, to determine, based on the coordinates (u i1 , vi1 ) and coordinates (u iL , v iL ) to determine the roll angle, where the coordinates (u i1 , v i1 ) represent the position of the feature point in the first column of the i-th row in the second set of feature points in the reference image, and the coordinates (u iL , v iL ) represent the position of the feature point in the L-th column of the i-th row in the second set of feature points in the reference image, and i is a positive integer greater than or equal to 1 and less than or equal to K;

[0110] 2) Determine r initial roll angles according to the two-dimensional image coordinates corresponding to the feature points in the first column and the last column in the r-th row in the second set of two-dimensional image coordinates, where r is a positive integer greater than or equal to 1 and less than K. In the case that one of the r initial roll angles is the initial roll angle determined according to the two-dimensional image coordinates corresponding to the feature points in the first column and the last column in the j-th row in the second set of two-dimensional image coordinates, according to (u j1 , v j1 ) and (u jL , v jL ), determine the above one initial roll angle, where the coordinates (u j1 , v j1 ) represent the position of the feature point in the first column of the j-th row in the second set of feature points in the reference image, and the coordinates (u jL , v jL ) represent the position of the feature point in the L-th column of the j-th row in the second set of feature points in the reference image, and j is a positive integer greater than or equal to 1 and less than or equal to K; Determine the roll angle as the average value of the r initial roll angles.

[0111] It can be understood that the above method is only an example, and the roll angle can also be determined by other methods. As an optional example, it is possible but not limited to use existing methods to determine the roll angle according to the coordinates (u i1 , v i1 ) and the coordinates (u iL , v iL ), or, according to (u j1 , v j1 ) and (u jL , v jL ), determine the above one initial roll angle, and the embodiments of the present disclosure do not make any limitations in this regard.

[0112] Figure 12 is a flowchart of a method for determining the third mapping parameter of the AR - HUD relative to the first three - dimensional coordinate system according to the first mapping parameter and the second mapping parameter in the embodiments of the present disclosure. In one implementation, as Figure 12As shown, determining a third mapping parameter of the AR-HUD relative to a first three-dimensional coordinate system according to a first mapping parameter and a second mapping parameter includes:

[0113] Step S1201: Concatenate a first rotation matrix and a first translation vector included in the first mapping parameter to obtain a first target matrix, where the first target matrix includes each value in the first rotation matrix and each value in the first translation vector;

[0114] In an exemplary implementation, construct a first target matrix according to a given first rotation matrix and a first translation vector Where R1 represents the first rotation matrix and K1 represents the first translation vector. The first target matrix includes each value in the first rotation matrix and each value in the first translation vector. As an optional example, the first rotation matrix and the first translation vector can be concatenated into a homogeneous transformation matrix (for example, the above first target matrix) in the above manner to represent the transformation of the industrial camera relative to the first three-dimensional coordinate system.

[0115] Step S1202: Concatenate a second rotation matrix and a second translation vector included in the second mapping parameter to obtain a second target matrix, where the second target matrix includes each value in the second rotation matrix and each value in the second translation vector;

[0116] In an exemplary implementation, construct a second target matrix according to a given second rotation matrix and a second translation vector Where R2 represents the second rotation matrix and K2 represents the second translation vector. The second target matrix includes each value in the second rotation matrix and each value in the second translation vector. As an optional example, the second rotation matrix and the second translation vector can be concatenated into a homogeneous transformation matrix (for example, the above second target matrix) in the above manner to represent the transformation of the AR-HUD relative to the second three-dimensional coordinate system.

[0117] Step S1203: Determine a third rotation matrix and a third translation vector according to the first target matrix and the second target matrix, where the third mapping parameter includes the third rotation matrix and the third translation vector.

[0118] As an optional example, multiply the second target matrix by the first target matrix to obtain a third target matrix, and extract the third rotation matrix and the third translation vector from the third target matrix.

[0119] As an optional example, but not limited to, an existing method can be used to multiply the second target matrix by the first target matrix to obtain a third target matrix, and extract the third rotation matrix and the third translation vector from the third target matrix. The embodiments of the present disclosure do not limit this.

[0120] As an alternative example, by using the method provided in the embodiments of the present disclosure, a first target matrix and a second target matrix that are convenient for calculation can be obtained, thereby improving the efficiency of determining the calibration result of the AR-HUD (for example, the third rotation matrix and the third translation vector of the AR-HUD relative to the first three-dimensional coordinate system), shortening the time to obtain the calibration result of the AR-HUD. For example, multiplying the second target matrix by the first target matrix can obtain the third rotation matrix and the third translation vector of the AR-HUD relative to the first three-dimensional coordinate system.

[0121] As an alternative example, by using the method provided in the embodiments of the present disclosure, the mapping parameters (including the rotation matrix and the translation vector) of the AR-HUD relative to the first three-dimensional coordinate system can be determined. Furthermore, in the augmented reality scenario, the data displayed by the AR-HUD can be converted into the first three-dimensional coordinate system, so as to perform collaborative processing with the data collected by various devices in the ADAS and converted into the first three-dimensional coordinate system.

[0122] Figure 13 is a flowchart of a method for determining the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first target image in the disclosed embodiments. In one implementation, as Figure 13 shown, determining the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first target image includes:

[0123] Step 1301, identifying a first set of feature points on the target in the first target image to obtain a first set of two-dimensional image coordinates;

[0124] As an alternative example, the first set of feature points on the above target may include, but is not limited to, as Figure 6 shown, the first set of two-dimensional image coordinates is used to represent the positions of the first set of feature points in the first target image. In Figure 6 , the first set of feature points is arranged in regular multiple rows and multiple columns, for example, multiple rows and multiple columns with equal spacing. This is only an example, and the first set of feature points in the embodiments of the present disclosure may also be arranged in other shapes, such as a parallelogram, a circle, or a triangle, etc. As an alternative example, the spacing between the first set of feature points arranged in a certain shape is known.

[0125] As an alternative example, each two-dimensional image coordinate in the first set of two-dimensional image coordinates may include, but is not limited to, being represented by (u, v), where the two-dimensional image coordinate (u i , v i)It is indicated that the \(i\)-th feature point is the pixel at the \(u\)-th row and \(v\)-th column in the first target image, where \(i\) is a positive integer. It can be understood that the above \((u, v)\) is just one representation of each two-dimensional image coordinate in the first set of two-dimensional image coordinates, and the embodiments of the present disclosure do not limit this.

[0126] Step 1302: Determine the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first set of two-dimensional image coordinates. As an optional example, the first three-dimensional coordinate system may be the three-dimensional coordinate system calibrated by the ADAS on the target vehicle, such as Figure 3 shown.

[0127] Figure 14 is a flowchart of the method for determining the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first set of two-dimensional image coordinates in the embodiments of the present disclosure. In one implementation manner, as Figure 14 shown, determining the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first set of two-dimensional image coordinates includes:

[0128] Step S1401: Establish a correspondence between the first set of two-dimensional image coordinates and a pre-determined set of three-dimensional space coordinates, where the set of three-dimensional space coordinates represents the positions of the first set of feature points in the first three-dimensional coordinate system;

[0129] Step S1402: Determine the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the correspondence and the internal parameters of the industrial camera.

[0130] As an optional example, an existing method can be used, but is not limited to, to determine the first mapping parameter (including the first rotation matrix and the first translation vector) of the industrial camera relative to the first three-dimensional coordinate system according to the correspondence and the internal parameters of the industrial camera.

[0131] In an optional example, according to the given first set of two-dimensional image coordinates and the pre-determined set of three-dimensional space coordinates, a correspondence between them can be established. This correspondence represents the correspondence between the two-dimensional image coordinates and the three-dimensional space coordinates of the first set of feature points.

[0132] Optionally, through this correspondence and the internal parameters of the industrial camera (including parameters such as the focal length, pixel size, and principal point of the camera), the rotation matrix and translation vector of the industrial camera relative to the first three-dimensional coordinate system can be determined. The specific implementation is as follows:

[0133] Establish a correspondence between the first set of two-dimensional image coordinates and a set of three-dimensional space coordinates. Through calibration techniques, such as using a target or a calibration board, the coordinates of the feature points on the two-dimensional image can be measured, and at the same time, the positions of these feature points in the three-dimensional coordinate system can be determined.

[0134] According to the internal parameters of the industrial camera (parameters such as focal length, pixel size, principal point, etc.) and the corresponding relationships, use a camera calibration algorithm (such as the PnP algorithm) to solve the rotation matrix and translation vector of the industrial camera. These internal parameter information can be obtained through measurement during the calibration process of the industrial camera.

[0135] Through algorithm calculation, the rotation matrix and translation vector of the industrial camera relative to the first three-dimensional coordinate system can be obtained. This rotation matrix describes the rotational transformation of the feature points observed by the industrial camera relative to the first three-dimensional coordinate system, and the translation vector represents the translational transformation of the position of the industrial camera relative to the first three-dimensional coordinate system.

[0136] It can be understood that the above method is only an example, and the rotation matrix and translation vector of the industrial camera relative to the first three-dimensional coordinate system can also be determined by other methods.

[0137] For example, when the target is located at N target positions respectively (N is a positive integer greater than or equal to 2) and the industrial camera captures N target images, N corresponding relationships can be obtained according to the N target images by using a method similar to step S1401. Among them, the determination method of each corresponding relationship is the same as that of S1401. Among them, a set of three-dimensional space coordinates in each corresponding relationship can be obtained by using a method similar to Figure 15 the method, and a set of three-dimensional space coordinates in each corresponding relationship represents the three-dimensional space coordinates of the first set of feature points in one of the N target images in the first three-dimensional space. After obtaining the N corresponding relationships, the first mapping parameters of the industrial camera relative to the first three-dimensional coordinate system can be determined according to the N corresponding relationships and the internal parameters of the industrial camera.

[0138] As an optional example, existing methods can be used, but are not limited to, to determine the first mapping parameters (including the first rotation matrix and the first translation vector) of the industrial camera relative to the first three-dimensional coordinate system according to the N corresponding relationships and the internal parameters of the industrial camera.

[0139] As an optional example, through the method provided in the embodiments of the present disclosure, the rotation matrix and translation vector of the industrial camera relative to the first three-dimensional coordinate system can be determined. Furthermore, the data collected by the industrial camera can be converted into the first three-dimensional coordinate system, so that it can be cooperatively processed with the data collected by each device in ADAS and converted into the first three-dimensional coordinate system.

[0140] As an alternative example, by using the method provided in the embodiments of the present disclosure, it is possible to correspond the feature points on the two-dimensional image to the positions in the three-dimensional coordinate system, and further determine the rotation matrix and translation vector of the industrial camera relative to the three-dimensional coordinate system. In this way, the position and orientation information of the feature points in the three-dimensional space can be obtained from the two-dimensional image captured by the industrial camera, so as to perform further three-dimensional reconstruction, object pose estimation and other tasks.

[0141] Figure 15 is a flowchart of the method for determining the set of three-dimensional space coordinates in the embodiments of the present disclosure. In one implementation, as Figure 15 shown, before establishing the correspondence between the first set of two-dimensional image coordinates and the pre-determined set of three-dimensional space coordinates, the method further includes:

[0142] Step S1501: Obtain a second target image captured by a target camera, where the target camera is the front view camera of the ADAS, and the second target image is an image obtained by the target camera capturing the target when the target is at the target position;

[0143] Step S1502: Identify the first set of feature points on the target in the second target image to obtain a third set of two-dimensional image coordinates, and determine the fourth mapping parameter of the target relative to the third three-dimensional coordinate system according to the third set of two-dimensional image coordinates and the internal parameters of the target camera, where the third three-dimensional coordinate system is a three-dimensional coordinate system established according to the position where the target camera is located;

[0144] As an alternative example, as Figure 3 shown, the above-mentioned third three-dimensional coordinate system can be a three-dimensional coordinate system established with the target camera as the origin. In the third three-dimensional coordinate system, the width direction of the target vehicle can be used as the x-axis (for example, to the right or left along the width direction of the target vehicle), the forward direction of the target vehicle can be used as the z-axis, and the vertical direction of the plane where the target vehicle is located can be used as the y-axis (for example, upward or downward perpendicular to the plane where the target vehicle is located). It can be understood that the position of the third three-dimensional coordinate system and the directions of each axis are only examples, and the embodiments of the present disclosure do not limit this;

[0145] As an alternative example, it is possible but not limited to use existing methods to determine the fourth mapping parameter (including the fourth rotation matrix and the fourth translation vector) of the target relative to the third three-dimensional coordinate system according to the third set of two-dimensional image coordinates and the internal parameters of the target camera.

[0146] As an alternative example, the internal parameters of the target camera may include but are not limited to: the focal length of the target camera, the pixel size of the captured image, the number of pixels corresponding to the unit rotation angle of the x-axis in the third three-dimensional coordinate system relative to the target camera (for example, using F xthe number of pixels corresponding to the unit rotation angle of the y-axis in the third three-dimensional coordinate system relative to the target camera (for example, using F y representation).

[0147] Step S1503: Determine the sixth mapping parameter of the target relative to the first three-dimensional coordinate system according to the fourth mapping parameter and the predetermined fifth mapping parameter, where the fifth mapping parameter is the mapping parameter of the target camera calibrated by ADAS relative to the first three-dimensional coordinate system;

[0148] As an optional example, determine the sixth rotation matrix and the sixth translation vector of the target relative to the first three-dimensional coordinate system according to the fourth rotation matrix and the fourth translation vector, and the predetermined fifth rotation matrix and the fifth translation vector.

[0149] As an optional example, determining the sixth rotation matrix and the sixth translation vector of the target relative to the first three-dimensional coordinate system according to the fourth rotation matrix and the fourth translation vector, and the predetermined fifth rotation matrix and the fifth translation vector includes:

[0150] Step S1: Concatenate the fourth rotation matrix and the fourth translation vector to obtain a fourth target matrix, where the fourth target matrix includes each value in the fourth rotation matrix and each value in the fourth translation vector;

[0151] In an exemplary implementation manner, construct a fourth target matrix according to the given fourth rotation matrix and the fourth translation vector where R4 represents the fourth rotation matrix, K4 represents the fourth translation vector, and the fourth target matrix includes each value of the fourth rotation matrix and each value of the fourth translation vector. As an optional example, the fourth rotation matrix and the fourth translation vector can be concatenated into a homogeneous transformation matrix (for example, the above-mentioned fourth target matrix) in the above manner to represent the transformation of the target relative to the third three-dimensional coordinate system.

[0152] Step S2: Concatenate the fifth rotation matrix and the fifth translation vector to obtain a fifth target matrix, where the fifth target matrix includes each value in the fifth rotation matrix and each value in the fifth translation vector;

[0153] In an exemplary implementation manner, construct a fifth target matrix according to the given fifth rotation matrix and the fifth translation vector Among them, R5 represents the fifth rotation matrix, K5 represents the fifth translation vector, and the fifth target matrix includes the values of the fifth rotation matrix and the values of the fifth translation vector. As an optional example, the fifth rotation matrix and the fifth translation vector can be concatenated into a homogeneous transformation matrix (for example, the above-mentioned fifth target matrix) in the above manner to represent the transformation of the target camera relative to the first three-dimensional coordinate system.

[0154] Step S3, determine a sixth rotation matrix and a sixth translation vector according to the fourth target matrix and the fifth target matrix.

[0155] As an optional example, the fourth target matrix can be multiplied by the fifth target matrix to obtain a sixth target matrix, and the sixth rotation matrix and the sixth translation vector can be extracted from the sixth target matrix.

[0156] As an optional example, the fourth target matrix can be multiplied by the fifth target matrix in an existing manner (but not limited to this) to obtain a sixth target matrix, and the sixth rotation matrix and the sixth translation vector can be extracted from the sixth target matrix. The embodiments of the present disclosure do not limit this.

[0157] As an optional example, through the method provided in the embodiments of the present disclosure, a fourth target matrix and a fifth target matrix that are convenient for calculation can be obtained, so that the efficiency of determining the calibration result of the AR-HUD (for example, the third rotation matrix and the third translation vector of the AR-HUD relative to the first three-dimensional coordinate system) can be improved, and the time for obtaining the calibration result of the AR-HUD can be shortened. For example, multiplying the fourth target matrix and the fifth target matrix can obtain the sixth rotation matrix and the sixth translation vector of the target relative to the first three-dimensional coordinate system.

[0158] As an optional example, through the method provided in the embodiments of the present disclosure, the mapping parameters (including the rotation matrix and the translation vector) of the target relative to the first three-dimensional coordinate system can be determined, and then in the augmented reality scenario, the position of the target in the first three-dimensional coordinate system can be determined.

[0159] Step S1504, determine a set of three-dimensional space coordinates according to the first set of two-dimensional image coordinates and the sixth mapping parameters.

[0160] As an optional example, a set of three-dimensional space coordinates can be determined according to the first set of two-dimensional image coordinates and the sixth mapping parameters in an existing manner (but not limited to this).

[0161] It can be understood that the above method is only an example, and the three-dimensional space coordinates of the first set of feature points in the first three-dimensional coordinate system can also be determined by other methods.

[0162] As an alternative example, by using the method provided in the embodiments of the present disclosure, the three-dimensional spatial coordinates of the first set of feature points in the first three-dimensional coordinate system can be accurately determined, so that the rotation matrix and translation vector of the industrial camera relative to the first three-dimensional coordinate system can be further accurately determined.

[0163] As an alternative example, by using the method provided in the embodiments of the present disclosure, feature points are extracted from an image, and the internal parameters and rotation and translation matrix of the camera are used to convert two-dimensional image coordinates into three-dimensional spatial coordinates. In this way, the three-dimensional spatial coordinates of the first set of feature points in the first three-dimensional coordinate system can be used in ADAS to process the first set of feature points. For example, the positions of the first set of feature points are measured and tracked, providing more accurate environmental perception and target positioning capabilities.

[0164] As an alternative example, the embodiments of the present disclosure further provide a calibration device for an augmented reality head-up display system (AR-HUD), and the device includes:

[0165] An acquisition module, configured to acquire an AR-HUD display image and N target images captured by an industrial camera, where the AR-HUD display image is an image obtained by the industrial camera capturing a reference image displayed by the AR-HUD, and the N target images are images obtained by the industrial camera capturing the targets when the targets are respectively located at N positions. The industrial camera is located inside the target vehicle, the targets are located outside the target vehicle, and N is 1 or a positive integer greater than or equal to 2;

[0166] A first calibration module, configured to determine a first mapping parameter of the industrial camera relative to a first three-dimensional coordinate system according to the N target images, where the first three-dimensional coordinate system is a three-dimensional coordinate system calibrated by an advanced driver assistance system (ADAS) on the target vehicle;

[0167] A second calibration module, configured to determine a second mapping parameter of the AR-HUD relative to a second three-dimensional coordinate system according to the AR-HUD display image, where the second three-dimensional coordinate system is a three-dimensional coordinate system established according to the position where the industrial camera is located;

[0168] A third calibration module, configured to determine a third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameter and the second mapping parameter.

[0169] The following takes the target at one target position (for example, Figure 3 one of the first target position, the second target position, and the third target position shown) as an example to describe an AR-HUD calibration device.

[0170] Figure 16It is a schematic structural diagram of a calibration device for an AR - HUD according to an embodiment of the present disclosure. As shown in Figure 16 , the calibration device for the AR - HUD includes:

[0171] An acquisition module 161, configured to acquire an AR - HUD display image and a first target image captured by an industrial camera. Among them, the AR - HUD display image is an image obtained by the industrial camera capturing a reference image displayed by the AR - HUD, and the first target image is an image obtained by the industrial camera capturing the target when the target is at a target position;

[0172] As an optional example, the above - mentioned target position may but is not limited to Figure 3 one of the first target position, the second target position, and the third target position shown in

[0173] A first calibration module 162, configured to determine a first mapping parameter of the industrial camera relative to a first three - dimensional coordinate system according to the first target image. Among them, the first three - dimensional coordinate system is a three - dimensional coordinate system calibrated by an advanced driver assistance system (ADAS) on the target vehicle where the AR - HUD is located;

[0174] As an optional example, the first group of feature points on the above - mentioned target may but is not limited to be as shown in Figure 6 . The first group of two - dimensional image coordinates is used to represent the position of the first group of feature points in the first target image. In Figure 6 , the first group of feature points is arranged in regular multiple rows and multiple columns. For example, multiple rows and multiple columns with equal spacing. This is only an example. The first group of feature points in the embodiments of the present disclosure may also be arranged in other shapes, such as a parallelogram, a circle, or a triangle, etc. As an optional example, the spacing between the first group of feature points arranged in a certain shape is known.

[0175] As an optional example, each two - dimensional image coordinate in the above - mentioned first group of two - dimensional image coordinates may but is not limited to be represented by (u, v). Among them, the two - dimensional image coordinate (u i , v i ) corresponding to the i - th feature point in the first group of feature points represents that the i - th feature point is the pixel at the u - th row and v - th column in the first target image, and i is a positive integer. It can be understood that the above (u, v) is only one representation method of each two - dimensional image coordinate in the first group of two - dimensional image coordinates, and the embodiments of the present disclosure do not limit this.

[0176] A second calibration module 163, configured to determine a second mapping parameter of the AR - HUD relative to a second three - dimensional coordinate system according to the AR - HUD display image. Among them, the second three - dimensional coordinate system is a three - dimensional coordinate system established according to the position of the industrial camera;

[0177] As an alternative example, as Figure 3 shown, the above second three-dimensional coordinate system may be a three-dimensional coordinate system established with an industrial camera as the origin. In the second three-dimensional coordinate system, the width direction of the target vehicle may be used as the x-axis (for example, to the right or left along the width direction of the target vehicle), the forward direction of the target vehicle may be used as the z-axis, and the vertical direction of the plane where the target vehicle is located may be used as the y-axis (for example, upward or downward perpendicular to the plane where the target vehicle is located). It can be understood that the position of the second three-dimensional coordinate system and the directions of each axis are only examples, and the embodiments of the present disclosure do not limit this;

[0178] As an alternative example, the second set of feature points in the reference image displayed in the AR-HUD display image may but are not limited to being as Figure 5 shown. The second set of two-dimensional image coordinates are used to represent the positions of the second set of feature points in the AR-HUD display image. In Figure 5 , the second set of feature points are arranged in regular multiple rows and multiple columns, for example, multiple rows and multiple columns with equal spacing. This is only an example. The second set of feature points in the embodiments of the present disclosure may also be arranged in other shapes, such as a parallelogram, a circle, or a triangle, etc. As an alternative example, the spacing between the second set of feature points arranged in a shape is known.

[0179] As an alternative example, each two-dimensional image coordinate in the above second set of two-dimensional image coordinates may but is not limited to be represented by (u, v), where the two-dimensional image coordinate (u j , v j ) corresponding to the jth feature point in the second set of feature points represents that the jth feature point is the pixel at the u-th row and v-th column in the AR-HUD display image, and j is a positive integer. It can be understood that the above (u, v) is only a representation method of each two-dimensional image coordinate in the second set of two-dimensional image coordinates, and the embodiments of the present disclosure do not limit this.

[0180] The third calibration module 164 is configured to determine the third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameter and the second mapping parameter.

[0181] In the embodiments of the present disclosure, the mapping parameter may but is not limited to be represented by a rotation matrix and a translation vector. The embodiments of the present disclosure do not limit the content included in the mapping parameter or the form used to represent the mapping parameter.

[0182] By obtaining the mapping parameters of the AR-HUD with respect to the three-dimensional coordinate system of the industrial camera and the mapping parameters of the industrial camera with respect to the three-dimensional coordinate system calibrated by the ADAS, the mapping parameters of the AR-HUD with respect to the three-dimensional coordinate system calibrated by the ADAS are determined, thereby realizing the calibration of the AR-HUD, solving the technical problem in the prior art that the AR-HUD cannot be calibrated, and achieving the technical effect of accurately calibrating the AR-HUD in scenarios where the AR-HUD needs to be calibrated.

[0183] Each module in the calibration device of the AR-HUD in the embodiments of the present disclosure may, but is not limited to, correspondingly execute steps such as Figures 7 - 9 and those in 11-15. The specific process and examples may adopt the processes and examples described in the embodiments of the present disclosure for Figures 7 - 9 and 11-15, and the embodiments of the present disclosure will not repeat the description here.

[0184] The embodiments of the present disclosure further provide an electronic device, which at least includes a memory and a processor. A computer program is stored on the memory, and the processor implements the steps of the above method when executing the computer program on the memory.

[0185] In some embodiments, the processor executing the computer program may be a processing device including more than one general-purpose processing device, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be more than one dedicated processing device, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on chip (SoC), etc.

[0186] The memory may be a read-only memory (ROM), a random access memory (RAM), a phase change random access memory (PRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), an electrically erasable programmable read-only memory (EEPROM), other types of random access memory (RAM), a flash drive or other forms of flash memory, a cache, a register, a static memory, a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD) or other optical memories, a cassette tape or other magnetic storage devices, or any other possible non-transitory medium used to store information or instructions that can be accessed by a computer device.

[0187] The electronic devices according to the embodiments of the present disclosure may include, but are not limited to, fixed terminal devices such as servers, desktop computers, digital TVs, etc., and mobile terminal devices such as in-vehicle devices (e.g., head-up display devices), handheld devices (e.g., mobile phones, tablet computers, etc.), wearable devices (e.g., smart watches, smart bracelets, etc.).

[0188] The embodiments of the present disclosure also provide a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0189] The computer-readable storage medium according to the embodiments of the present disclosure may be any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device, for example, the above-mentioned memory.

[0190] The computer programs according to the embodiments of the present disclosure may be organized into one or more computer-executable components or modules. Various aspects of the present disclosure may be implemented with any number and combination of such components or modules. For example, various aspects of the present disclosure are not limited to the specific computer-executable instructions or specific components or modules shown in the drawings and described herein. Other embodiments may include different computer-executable instructions or components with more or less functions than those shown and described herein.

[0191] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features with similar functions disclosed in the present disclosure.

Claims

1. A calibration method for an augmented reality head-up display system AR-HUD, characterized in that, Including: Obtain an AR-HUD display image and a first target image captured by an industrial camera, where the AR-HUD display image is an image obtained by the industrial camera capturing a reference image displayed by the AR-HUD, and the first target image is an image obtained by the industrial camera capturing the target when the target is at a target position; Determine a first mapping parameter of the industrial camera relative to a first three-dimensional coordinate system according to the first target image, where the first three-dimensional coordinate system is a three-dimensional coordinate system calibrated by an advanced driver assistance system (ADAS) on the target vehicle where the AR-HUD is located; Determine a second mapping parameter of the AR-HUD relative to a second three-dimensional coordinate system according to the AR-HUD display image, where the second three-dimensional coordinate system is a three-dimensional coordinate system established based on the position where the industrial camera is located; Determine a third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameter and the second mapping parameter.

2. The method according to claim 1, characterized in that, The determining the second mapping parameter of the AR-HUD relative to the second three-dimensional coordinate system according to the AR-HUD display image includes: Identify a second set of feature points in the reference image in the AR-HUD display image to obtain a second set of two-dimensional image coordinates; Determine the second mapping parameter of the imaging light cone of the AR-HUD relative to the second three-dimensional coordinate system according to the second set of two-dimensional image coordinates and the internal parameters of the industrial camera.

3. The method according to claim 2, characterized in that, The determining the second mapping parameter of the imaging light cone of the AR-HUD relative to the second three-dimensional coordinate system according to the second set of two-dimensional image coordinates and the internal parameters of the industrial camera includes: When the second three-dimensional coordinate system has the position where the industrial camera is located as the origin and the vertex of the imaging light cone of the AR-HUD is the position where the industrial camera is located, set the second translation vector included in the second mapping parameter to a preset value; Determine a first center point coordinate according to the second set of two-dimensional image coordinates, where the first center point coordinate represents the position of the center point of the shape formed by the second set of feature points in the reference image; Determine a second center point coordinate according to the size of the AR-HUD display image captured by the industrial camera, where the second center point coordinate represents the position of the center point of the AR-HUD display image; Determine the second rotation matrix included in the second mapping parameter according to the first center point coordinate, the second center point coordinate, the second set of two-dimensional image coordinates, and the internal parameters of the industrial camera.

4. The method according to claim 3, characterized in that, The determining the second rotation matrix included in the second mapping parameter according to the first center point coordinate, the second center point coordinate, the second set of two-dimensional image coordinates, and the internal parameters of the industrial camera includes: When the coordinates of the first center point are (u1, v1) and the coordinates of the second center point are (u2, v2), determine the pitch angle according to the difference between u1 and u2 and the first unit parameter, where the internal parameters of the industrial camera include the first unit parameter, and the first unit parameter represents the number of pixels corresponding to the unit rotation angle relative to the x-axis in the second three-dimensional coordinate system; Determine the yaw angle according to the difference between v1 and v2 and the second unit parameter, where the internal parameters of the industrial camera include the second unit parameter, and the second unit parameter represents the number of pixels corresponding to the unit rotation angle relative to the y-axis in the second three-dimensional coordinate system; Determine the roll angle according to the second set of two-dimensional image coordinates; Determine the second rotation matrix according to the pitch angle, the yaw angle and the roll angle.

5. The method according to claim 4, characterized in that, The determining the roll angle according to the second set of two-dimensional image coordinates includes: When the second set of feature points are arranged in K rows and L columns in the reference image, determine the roll angle according to the two-dimensional image coordinates corresponding to the feature points in the first column and the last column in at least some rows of the second set of two-dimensional image coordinates, where K and L are positive integers greater than or equal to 2.

6. The method according to claim 1, characterized in that, The determining the third mapping parameter of the AR-HUD relative to the first three-dimensional coordinate system according to the first mapping parameter and the second mapping parameter includes: Concatenate the first rotation matrix and the first translation vector included in the first mapping parameter to obtain a first target matrix, where the first target matrix includes each value in the first rotation matrix and each value in the first translation vector; Concatenate the second rotation matrix and the second translation vector included in the second mapping parameter to obtain a second target matrix, where the second target matrix includes each value in the second rotation matrix and each value in the second translation vector; Determine a third rotation matrix and a third translation vector according to the first target matrix and the second target matrix, where the third mapping parameter includes the third rotation matrix and the third translation vector.

7. The method according to any one of claims 1 to 6, characterized in that, The determining the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first target image includes: Identify the first set of feature points on the target in the first target image to obtain a first set of two-dimensional image coordinates; Determine the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first set of two-dimensional image coordinates.

8. The method according to claim 7, characterized in that, The determining the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the first set of two-dimensional image coordinates includes: Establish a correspondence between the first set of two-dimensional image coordinates and a pre-determined set of three-dimensional space coordinates, where the set of three-dimensional space coordinates represents the positions of the first set of feature points in the first three-dimensional coordinate system; Determine the first mapping parameter of the industrial camera relative to the first three-dimensional coordinate system according to the correspondence and the internal parameters of the industrial camera.

9. The method according to claim 8, characterized in that, Before establishing a corresponding relationship between the first set of two-dimensional image coordinates and a predetermined set of three-dimensional space coordinates, the method further includes: Obtaining a second target image captured by a target camera, where the target camera is the front-view camera of the ADAS, and the second target image is an image obtained by the target camera capturing the target when the target is at the target position; Identifying the first set of feature points on the target in the second target image to obtain a third set of two-dimensional image coordinates, and determining a fourth mapping parameter of the target relative to a third three-dimensional coordinate system according to the third set of two-dimensional image coordinates and the internal parameters of the target camera, where the third three-dimensional coordinate system is a three-dimensional coordinate system established according to the position where the target camera is located; Determining a sixth mapping parameter of the target relative to the first three-dimensional coordinate system according to the fourth mapping parameter and a predetermined fifth mapping parameter, where the fifth mapping parameter is a mapping parameter of the target camera calibrated by the ADAS relative to the first three-dimensional coordinate system; Determining the set of three-dimensional space coordinates according to the first set of two-dimensional image coordinates and the sixth mapping parameter.

10. An electronic device, characterized in that, At least including a memory and a processor, a computer program is stored on the memory, and the processor implements the steps of the method according to any one of claims 1 to 9 when executing the computer program on the memory.

11. A computer-readable storage medium, characterized in that, The computer-readable medium stores a computer program, and the computer program implements the steps of the method according to any one of claims 1 to 9 when executed by a processor.

Images