Semi-automatic calibration method for Leiyu fusion all-in-one machine

Abstract

The invention discloses a semi-automatic calibration method for a thunder-vision fusion all-in-one machine. The method comprises the following steps: firstly, calibrating the internal reference of a camera, and calibrating the rotation and translation relationship between a 3D millimeter wave radar installation coordinate system and a camera coordinate system, thereby obtaining the internal reference of the 3D thunder-sight fusion all-in-one machine; secondly, establishing a rectangular coordinate system in an actual application scene of the 3D thunder-sight fusion all-in-one machine, and constructing a function expression of a conversion relation between the rectangular coordinate system and a radar coordinate system; by scanning unknown parameters in a conversion relation between a rectangular coordinate system and a radar coordinate system in a reasonable interval, a target coordinate detected by a radar is converted into a rectangular coordinate system coordinate and further converted into a pixel coordinate, the pixel coordinate is matched with a target detected by a video, and when the matching effect is the best, the target is identified. And obtaining the optimal conversion relation from the radar coordinate system to the pixel coordinate system.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-sensor fusion perception calibration, and in particular to a semi-automatic calibration method for a radar-vision fusion all-in-one machine. Background Art

[0002] 3D millimeter-wave radar is widely used in the fields of smart transportation and assisted driving to measure moving vehicles, pedestrians and other targets. 3D millimeter-wave radar has the characteristics of high cost performance, long detection distance, not easily affected by weather, and can work all day and all weather. However, compared with video detectors, 3D millimeter-wave radars are difficult to detect stationary targets and cannot capture visual information such as the appearance of the target. The radar-vision fusion all-in-one machine can complement each other's advantages by fusing video detection results and radar detection results, and obtain the visual information and position information of the target at the same time. In order to fuse video detection results and radar detection results, the radar-vision fusion all-in-one machine needs to calibrate the conversion relationship from radar detection results to video detection results. This conversion relationship is related to the installation environment and needs to be recalibrated after each installation. The commonly used manual calibration method manually selects radar detection results and video detection results belonging to the same target, resulting in a complex calibration process that is time-consuming and labor-intensive. Summary of the invention

[0003] The purpose of the present invention is to provide a semi-automatic calibration method for a millimeter wave radar and video fusion integrated machine in view of the problems existing in the above-mentioned prior art.

[0004] The technical solution to achieve the purpose of the present invention is: a semi-automatic calibration method for a radar-visual fusion integrated machine, the method comprising:

[0005] Step 1: calibrate the internal parameters of the camera in the laser-vision fusion integrated machine, that is, the conversion relationship from the camera coordinate system to the pixel coordinate system;

[0006] Step 2, taking the first receiving antenna of the 3D millimeter-wave radar in the radar-vision fusion integrated machine as the origin, along the radar receiving antenna array to the right as the x-axis, along the side of the radar panel downward as the y-axis, and along the normal of the radar panel as the z-axis, establish a rectangular coordinate system A; then calibrate the transformation relationship between the camera coordinate system and the rectangular coordinate system A;

[0007] Step 3, converting the target coordinates detected by the 3D millimeter wave radar into coordinates under the rectangular coordinate system A;

[0008] Step 4, based on the transformation relationship between the camera coordinate system and the rectangular coordinate system A, convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates in the camera coordinate system;

[0009] Step 5, calculating the target similarity based on the pixel coordinates of all targets detected by the camera and the pixel coordinates of all targets converted by the 3D millimeter wave radar;

[0010] Step 6, according to the target similarity, matching the target detected by the camera and the target detected by the 3D millimeter wave radar, calculating the matching score, and recording the set of all matched target combinations as S;

[0011] Step 7, based on the matching score, optimize the coordinate conversion parameters of step 3 and step 4, and calibrate the coordinate conversion relationship of the 3D millimeter wave radar and camera detection target in the radar and vision fusion integrated machine.

[0012] Furthermore, the internal parameters of the camera in the integrated laser-vision fusion machine are calibrated in step 1, and the specific formula is:

[0013]

[0014] Where, (uv) T is the pixel coordinate, (X c Y c Z c ) T is the coordinate in the camera coordinate system, Z c is the depth of field of the target in the camera coordinate system; f x and f y is the focal length of the camera, and (u0, v0) represents the intersection of the camera optical axis and the imaging plane.

[0015] Furthermore, the transformation relationship between the camera coordinate system and the rectangular coordinate system A is calibrated in step 2, and the specific formula is:

[0016]

[0017] In the formula, (xyz) T is the coordinate in the rectangular coordinate system A, R is the rotation matrix from the rectangular coordinate system A to the camera coordinate system, and T is the translation matrix from the rectangular coordinate system A to the camera coordinate system, which are:

[0018]

[0019] In the formula, R ij represents the i-th row and j-th column element of the three-dimensional coordinate rotation matrix R, i, j = 1...3; T i′j′ Represents the element of the i′th row and j′th column of the three-dimensional coordinate translation matrix T, where i′=1...3, j′=1.

[0020] Furthermore, in step 3, the target coordinates detected by the 3D millimeter wave radar are converted into coordinates in the rectangular coordinate system A, and the specific formula is:

[0021]

[0022] Where, α∈(-90°, +90°), β∈(-90°, +90°), (x′ z′) T The target coordinates detected by the 3D millimeter wave radar, (uv) T is the target coordinate detected in the video, i.e., the pixel coordinate, and C is the height difference between the radar and the target.

[0023] Furthermore, step 4 is specifically as follows:

[0024] According to the following formula:

[0025]

[0026] The coordinates (xyz) of the target in the rectangular coordinate system A T Convert to pixel coordinates (uv) T :

[0027]

[0028] Furthermore, the calculation formula of the target similarity in step 5 is:

[0029]

[0030] in,

[0031] Δu ij =u i -u′ j

[0032] Δv ij =v i -v′ j

[0033] In the formula, (u i v i ) T is the pixel coordinate of the i-th target detected by the camera, (u′ j v′ j ) T is the pixel coordinate of the jth target converted by 3D millimeter-wave radar, Δr ij is the similarity between the i-th target and the j-th target.

[0034] Furthermore, the target detected by the camera and the target detected by the 3D millimeter wave radar are matched in step 6, and a matching score is calculated, which specifically includes:

[0035] Step 6-1, for the i-th target detected by the camera, select the j-th target converted by the 3D millimeter wave radar corresponding to the minimum target similarity value and match it with the i-th target, and the set of all matched target combinations is recorded as S;

[0036] Step 6-2, calculate the matching score of the target combination set in step 6-1:

[0037]

[0038] In the formula, λ is the weight coefficient, r threshold is the target matching distance threshold, count(*) indicates the number of * that meet the conditions, and mean(*) indicates the mean of *.

[0039] Furthermore, step 7 specifically includes:

[0040] Step 7-1, perform parameter scanning using the following formula to obtain the optimal α, β and C:

[0041]

[0042] Step 7-2: Convert the target coordinates detected by the 3D millimeter wave radar into coordinates in the rectangular coordinate system A according to the following formula:

[0043]

[0044] Step 7-3, convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates according to the following formula:

[0045]

[0046] Compared with the prior art, the present invention has the following significant advantages:

[0047] (1) The coordinate transformation relationship between radar detection targets and video detection targets in the 3D millimeter-wave radar and video fusion integrated machine can be effectively calibrated.

[0048] (2) By calibrating the internal parameters of the radar and vision fusion integrated device before leaving the factory, the complexity of calibration in the working environment can be reduced.

[0049] (3) In the working environment, the target coordinates detected by the radar and the target coordinates detected by the video are automatically matched and calculated, without the need for manual operation, which can effectively reduce the complexity of the calibration process.

[0050] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flowchart of the semi-automatic calibration method for the millimeter-wave radar and video fusion integrated device.

[0052] Figure 2 Schematic diagram of the internal parameter calibration method for the 3D millimeter wave radar and video fusion integrated machine. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0055] In one embodiment, in combination Figure 1 , provides a semi-automatic calibration method for a radar-vision fusion integrated device, the method comprising:

[0056] Step 1: calibrate the internal parameters of the camera in the laser-vision fusion integrated machine, that is, the conversion relationship from the camera coordinate system to the pixel coordinate system;

[0057] Here, the calibration of the intrinsic parameters includes the calibration of the camera intrinsic parameters and the calibration of the rotation and translation relationship between the camera coordinate system and the radar installation position coordinate system. The camera intrinsic parameter calibration mainly calibrates the focal length of the camera, the intersection of the optical axis and the image plane, etc. The calibration of the camera intrinsic parameters can refer to Zhang Zhengyou's calibration method, etc.

[0058] The specific formula is:

[0059]

[0060] Where, (uv) T is the pixel coordinate, (X c Y c Z c ) T is the coordinate in the camera coordinate system, Z c is the depth of field of the target in the camera coordinate system; f x and f y is the focal length of the camera, and (u0, v0) represents the intersection of the camera optical axis and the imaging plane.

[0061] Step 2, taking the first receiving antenna of the 3D millimeter-wave radar in the radar-vision fusion integrated machine as the origin, along the radar receiving antenna array to the right as the x-axis, along the side of the radar panel downward as the y-axis, and along the normal of the radar panel as the z-axis, establish a rectangular coordinate system A; then calibrate the transformation relationship between the camera coordinate system and the rectangular coordinate system A;

[0062] Step 3, converting the target coordinates detected by the 3D millimeter wave radar into coordinates under the rectangular coordinate system A;

[0063] Step 4, based on the transformation relationship between the camera coordinate system and the rectangular coordinate system A, convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates in the camera coordinate system;

[0064] Step 5, calculating the target similarity based on the pixel coordinates of all targets detected by the camera and the pixel coordinates of all targets converted by the 3D millimeter wave radar;

[0065] Step 6, according to the target similarity, matching the target detected by the camera and the target detected by the 3D millimeter wave radar, calculating the matching score, and recording the set of all matched target combinations as S;

[0066] Here, the matching of radar and video detection targets means that in the actual working environment of the radar vision, the 3D millimeter wave radar and video detection functions are turned on at the same time;

[0067] Step 7, based on the matching score, optimize the parameters of the coordinate transformation in step 3 and step 4, and calibrate the coordinate transformation relationship of the 3D millimeter wave radar and camera detection target in the radar-vision fusion integrated machine.

[0068] Furthermore, in one embodiment, the transformation relationship between the camera coordinate system and the rectangular coordinate system A is calibrated in step 2, and the specific formula is:

[0069]

[0070] In the formula, (xyz) T is the coordinate in the rectangular coordinate system A, R is the rotation matrix from the rectangular coordinate system A to the camera coordinate system, and T is the translation matrix from the rectangular coordinate system A to the camera coordinate system, which are:

[0071]

[0072] In the formula, R ij represents the i-th row and j-th column element of the three-dimensional coordinate rotation matrix R, i, j = 1...3; T i′j′ Represents the element of the i′th row and j′th column of the three-dimensional coordinate translation matrix T, where i′=1...3, j′=1.

[0073] In step 3, the target coordinates detected by the 3D millimeter wave radar are converted into coordinates under the rectangular coordinate system A. The specific formula is:

[0074]

[0075] Where, α∈(-90°, +90°), β∈(-90°, +90°), (x′z′) T The target coordinates detected by the 3D millimeter wave radar, (uv) T is the target coordinate detected in the video, i.e., the pixel coordinate, and C is the height difference between the radar and the target.

[0076] Step 4 is as follows:

[0077] According to the following formula:

[0078]

[0079] The coordinates (xyz) of the target in the rectangular coordinate system A T Convert to pixel coordinates (uv) T :

[0080]

[0081] Specifically, the calibration method for the rotation and translation relationship between the camera coordinate system and the radar installation position coordinate system is as follows: Figure 2 As shown:

[0082] In an open space, set up a bracket, take the bracket center as the origin, horizontal right as the x-axis, vertical downward as the z-axis, and horizontal forward as the z-axis, establish a rectangular coordinate system, and place the radar-vision fusion machine on the bracket so that the first receiving antenna of the radar is at the origin, the receiving antenna array is parallel to the x-axis, the radar panel is in the xy plane, and the radar normal is toward the z-axis. i ,y i , z i )(i=1...N,N≥3) place the angle counter and use radar to detect these targets. If the target coordinates x′ detected by the radar i ≈x i , If the video detection function is enabled, fine-tune the installation position of the integrated laser and vision fusion machine. Record the pixel coordinates of the corner anti-center point detected by the video (u i , v i ) and manually match the corresponding spatial coordinates (x i ,y i , z i ). For all angular inversion targets, the following system of equations is established:

[0083]

[0084] The corresponding rotation matrix R and displacement matrix T can be solved.

[0085] Furthermore, in one embodiment, the calculation formula of the target similarity in step 5 is:

[0086]

[0087] in,

[0088] Δu ij =u i -u′ j

[0089] Δv ij =v i -v′ j

[0090] In the formula, (u i v i ) T is the pixel coordinate of the i-th target detected by the camera, (u′ j v′ j ) T is the pixel coordinate of the jth target converted by 3D millimeter-wave radar, Δr ij is the similarity between the i-th target and the j-th target.

[0091] Further, in one of the embodiments, the step 6 of matching the target detected by the camera with the target detected by the 3D millimeter wave radar and calculating the matching score specifically includes:

[0092] Step 6-1, for the i-th target detected by the camera, select the j-th target converted by the 3D millimeter wave radar corresponding to the minimum target similarity value and match it with the i-th target, and the set of all matched target combinations is recorded as S;

[0093] Step 6-2, calculate the matching score of the target combination set in step 6-1:

[0094]

[0095] In the formula, λ is the weight coefficient, r threshold is the target matching distance threshold, count(*) indicates the number of * that meet the conditions, and mean(*) indicates the mean of *.

[0096] Furthermore, step 7 specifically includes:

[0097] Step 7-1, perform parameter scanning by the following formula to obtain the optimal α, β and C, i.e. αbest , β best and C best :

[0098]

[0099] Step 7-2: Convert the target coordinates detected by the 3D millimeter wave radar into coordinates in the rectangular coordinate system A according to the following formula:

[0100]

[0101] Step 7-3, convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates according to the following formula:

[0102]

[0103] Here, the best matching result and optimal parameters [α best , β best , C best ], that is, the calibration of the coordinate transformation relationship between the 3D millimeter wave radar and video fusion integrated radar detection target and the video detection target is completed.

[0104] In one embodiment, a semi-automatic calibration system for a radar-visual fusion integrated machine is provided, the system comprising:

[0105] The first module is used to calibrate the internal parameters of the camera in the laser-vision fusion integrated machine, that is, the conversion relationship from the camera coordinate system to the pixel coordinate system;

[0106] The second module is used to establish a rectangular coordinate system A with the first receiving antenna of the 3D millimeter-wave radar in the radar-vision fusion integrated machine as the origin, the x-axis to the right along the radar receiving antenna array, the y-axis downward along the side of the radar panel, and the z-axis along the normal of the radar panel; then calibrate the transformation relationship between the camera coordinate system and the rectangular coordinate system A;

[0107] The third module is used to convert the target coordinates detected by the 3D millimeter wave radar into coordinates under the rectangular coordinate system A;

[0108] A fourth module is used to convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates in the camera coordinate system based on the transformation relationship between the camera coordinate system and the rectangular coordinate system A;

[0109] The fifth module is used to calculate the target similarity based on the pixel coordinates of all targets detected by the camera and the pixel coordinates of all targets converted by the 3D millimeter wave radar;

[0110] The sixth module is used to match the target detected by the camera and the target detected by the 3D millimeter wave radar according to the target similarity, calculate the matching score, and record the set of all matched target combinations as S;

[0111] The seventh module is used to optimize the parameters of coordinate transformation in the third module and the fourth module based on the matching score, and to calibrate the coordinate transformation relationship of the 3D millimeter wave radar and camera detection target in the radar and vision fusion all-in-one machine.

[0112] For the specific definition of the semi-automatic calibration system of the integrated thunder and vision fusion machine, please refer to the definition of the semi-automatic calibration method of the integrated thunder and vision fusion machine mentioned above, which will not be repeated here. Each module in the above-mentioned semi-automatic calibration system of the integrated thunder and vision fusion machine can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0113] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following is achieved:

[0114] Step 1: calibrate the internal parameters of the camera in the laser-vision fusion integrated machine, that is, the conversion relationship from the camera coordinate system to the pixel coordinate system;

[0115] Step 2, taking the first receiving antenna of the 3D millimeter-wave radar in the radar-vision fusion integrated machine as the origin, along the radar receiving antenna array to the right as the x-axis, along the side of the radar panel downward as the y-axis, and along the normal of the radar panel as the z-axis, establish a rectangular coordinate system A; then calibrate the transformation relationship between the camera coordinate system and the rectangular coordinate system A;

[0116] Step 3, converting the target coordinates detected by the 3D millimeter wave radar into coordinates under the rectangular coordinate system A;

[0117] Step 4, based on the transformation relationship between the camera coordinate system and the rectangular coordinate system A, convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates in the camera coordinate system;

[0118] Step 5, calculating the target similarity based on the pixel coordinates of all targets detected by the camera and the pixel coordinates of all targets converted by the 3D millimeter wave radar;

[0119] Step 6, according to the target similarity, matching the target detected by the camera and the target detected by the 3D millimeter wave radar, calculating the matching score, and recording the set of all matched target combinations as S;

[0120] Step 7, based on the matching score, optimize the parameters of the coordinate transformation in step 3 and step 4, and calibrate the coordinate transformation relationship of the 3D millimeter wave radar and camera detection target in the radar-vision fusion integrated machine.

[0121] For the specific limitations of each step, please refer to the limitations of the semi-automatic calibration method for the radar and vision fusion all-in-one device mentioned above, which will not be repeated here.

[0122] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the computer program implements:

[0123] Step 1: calibrate the internal parameters of the camera in the laser-vision fusion integrated machine, that is, the conversion relationship from the camera coordinate system to the pixel coordinate system;

[0124] Step 2, taking the first receiving antenna of the 3D millimeter-wave radar in the radar-vision fusion integrated machine as the origin, along the radar receiving antenna array to the right as the x-axis, along the side of the radar panel downward as the y-axis, and along the normal of the radar panel as the z-axis, establish a rectangular coordinate system A; then calibrate the transformation relationship between the camera coordinate system and the rectangular coordinate system A;

[0125] Step 3, converting the target coordinates detected by the 3D millimeter wave radar into coordinates under the rectangular coordinate system A;

[0126] Step 4, based on the transformation relationship between the camera coordinate system and the rectangular coordinate system A, convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates in the camera coordinate system;

[0127] Step 5, calculating the target similarity based on the pixel coordinates of all targets detected by the camera and the pixel coordinates of all targets converted by the 3D millimeter wave radar;

[0128] Step 6, according to the target similarity, matching the target detected by the camera and the target detected by the 3D millimeter wave radar, calculating the matching score, and recording the set of all matched target combinations as S;

[0129] Step 7, based on the matching score, optimize the parameters of the coordinate transformation in step 3 and step 4, and calibrate the coordinate transformation relationship of the 3D millimeter wave radar and camera detection target in the radar-vision fusion integrated machine.

[0130] For the specific limitations of each step, please refer to the limitations of the semi-automatic calibration method for the radar and vision fusion all-in-one device mentioned above, which will not be repeated here.

[0131] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A semi-automatic calibration method for a radar-vision fusion integrated machine, characterized in that: The method comprises: Step 1: calibrate the internal parameters of the camera in the laser-vision fusion integrated machine, that is, the conversion relationship from the camera coordinate system to the pixel coordinate system; Step 2, taking the first receiving antenna of the 3D millimeter-wave radar in the radar-vision fusion integrated machine as the origin, along the radar receiving antenna array to the right as the x-axis, along the side of the radar panel downward as the y-axis, and along the normal of the radar panel as the z-axis, establish a rectangular coordinate system A; then calibrate the transformation relationship between the camera coordinate system and the rectangular coordinate system A; Step 3, converting the target coordinates detected by the 3D millimeter wave radar into coordinates under the rectangular coordinate system A; Step 4, based on the transformation relationship between the camera coordinate system and the rectangular coordinate system A, convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates in the camera coordinate system; Step 5, calculating the target similarity based on the pixel coordinates of all targets detected by the camera and the pixel coordinates of all targets converted by the 3D millimeter wave radar; Step 6, according to the target similarity, matching the target detected by the camera and the target detected by the 3D millimeter wave radar, calculating the matching score, and recording the set of all matched target combinations as S; Step 7, based on the matching score, optimize the parameters of the coordinate transformation in step 3 and step 4, and calibrate the coordinate transformation relationship of the 3D millimeter wave radar and camera detection target in the radar-vision fusion integrated machine.

2. The semi-automatic calibration method of the radar and vision fusion integrated machine according to claim 1 is characterized in that: Step 1 is to calibrate the internal parameters of the camera in the integrated camera. The specific formula is: Where, (uv) T is the pixel coordinate, (X c Y c Z c ) T is the coordinate in the camera coordinate system, Z c is the depth of field of the target in the camera coordinate system; f x and f y is the focal length of the camera, and (u0, v0) represents the intersection of the camera optical axis and the imaging plane.

3. The semi-automatic calibration method of the integrated radar and visual fusion machine according to claim 2 is characterized in that: The transformation relationship between the camera coordinate system and the rectangular coordinate system A is calibrated in step 2, and the specific formula is: In the formula, (xyz) T is the coordinate in the rectangular coordinate system A, R is the rotation matrix from the rectangular coordinate system A to the camera coordinate system, and T is the translation matrix from the rectangular coordinate system A to the camera coordinate system, which are: In the formula, R ij represents the i-th row and j-th column element of the three-dimensional coordinate rotation matrix R, i, j = 1…3; T i'j' Represents the i'th row and j'th column element of the three-dimensional coordinate translation matrix T, i'=1...3, j'=1.

4. The semi-automatic calibration method of the radar and vision fusion integrated machine according to claim 3 is characterized in that: In step 3, the target coordinates detected by the 3D millimeter wave radar are converted into coordinates under the rectangular coordinate system A. The specific formula is: Where, α∈(-90°,+90°), β∈(-90°,+90°), (x'z') T The target coordinates detected by the 3D millimeter wave radar, (uv) T is the target coordinate detected in the video, i.e., the pixel coordinate, and C is the height difference between the radar and the target.

5. The semi-automatic calibration method of the radar and vision fusion integrated machine according to claim 4 is characterized in that: Step 4 is as follows: According to the following formula: The coordinates (xyz) of the target in the rectangular coordinate system A T Convert to pixel coordinates (uv) T :

6. The semi-automatic calibration method of the radar and vision fusion integrated machine according to claim 5 is characterized in that: The calculation formula for the target similarity in step 5 is: in, Δu ij =u i -u' j Δv ij =v i -v’ j In the formula, (u i v i ) T is the pixel coordinate of the i-th target detected by the camera, (u' j v' j ) T is the pixel coordinate of the jth target converted by 3D millimeter-wave radar, Δr ij is the similarity between the i-th target and the j-th target.

7. The semi-automatic calibration method of the radar and visual fusion integrated machine according to claim 6 is characterized in that: Step 6 matches the target detected by the camera with the target detected by the 3D millimeter wave radar, and calculates the matching score, which specifically includes: Step 6-1, for the i-th target detected by the camera, select the j-th target converted by the 3D millimeter wave radar corresponding to the minimum target similarity value and match it with the i-th target, and the set of all matched target combinations is recorded as S; Step 6-2, calculate the matching score of the target combination set in step 6-1: In the formula, λ is the weight coefficient, r threshold is the target matching distance threshold, count(*) indicates the number of * that meet the conditions, and mean(*) indicates the mean of *.

8. The semi-automatic calibration method of the radar and vision fusion integrated machine according to claim 7 is characterized in that: Step 7 specifically includes: Step 7-1, perform parameter scanning by the following formula to obtain the optimal α, β and C, i.e. α best , β best and C best : Step 7-2: Convert the target coordinates detected by the 3D millimeter wave radar into coordinates in the rectangular coordinate system A according to the following formula: Step 7-3, convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates according to the following formula:

9. A semi-automatic calibration system for a radar-visual fusion integrated machine based on the method described in claims 1 to 8, characterized in that: The system comprises: The first module is used to calibrate the internal parameters of the camera in the laser-vision fusion integrated machine, that is, the conversion relationship from the camera coordinate system to the pixel coordinate system; The second module is used to establish a rectangular coordinate system A with the first receiving antenna of the 3D millimeter-wave radar in the radar-vision fusion integrated machine as the origin, the x-axis to the right along the radar receiving antenna array, the y-axis downward along the side of the radar panel, and the z-axis along the normal of the radar panel; then calibrate the transformation relationship between the camera coordinate system and the rectangular coordinate system A; The third module is used to convert the target coordinates detected by the 3D millimeter wave radar into coordinates under the rectangular coordinate system A; A fourth module is used to convert the coordinates of the target in the rectangular coordinate system A into pixel coordinates in the camera coordinate system based on the transformation relationship between the camera coordinate system and the rectangular coordinate system A; The fifth module is used to calculate the target similarity based on the pixel coordinates of all targets detected by the camera and the pixel coordinates of all targets converted by the 3D millimeter wave radar; The sixth module is used to match the target detected by the camera and the target detected by the 3D millimeter wave radar according to the target similarity, calculate the matching score, and record the set of all matched target combinations as S; The seventh module is used to optimize the parameters of coordinate transformation in the third module and the fourth module based on the matching score, and to calibrate the coordinate transformation relationship of the 3D millimeter wave radar and camera detection target in the radar and vision fusion all-in-one machine.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

Images