Unmanned vehicle, millimeter wave radar and camera joint calibration method and device

By combining binocular camera and millimeter-wave radar in a special driving environment, the problem of inability to fusion of binocular camera and millimeter-wave radar detection results is solved, and the ranging accuracy and real-time observation ability are improved, and the special Stable recognition effect in the environment.

CN118711153BActive Publication Date: 2024-12-06FXB CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411196435.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-06
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

In special driving environments, binocular camera sensors are difficult to identify pedestrians and vehicles. Millimeter-wave radar has low reflectivity on non-metallic objects, resulting in missed detection. The existing imaging calibration methods cannot accurately integrate the detection results of binocular cameras and millimeter-wave radars, resulting in The accuracy of ranging is reduced.

Method used

A joint calibration method of millimeter wave radar and camera is proposed. By placing the target at the marking point, a binocular camera is used to obtain the first coordinate point and the millimeter wave radar is used to obtain the second coordinate point, and a coordinate point pair is generated, and an optimization algorithm is used to solve it. Mathematical relational model to obtain a homography matrix, achieving joint calibration of millimeter wave radar and camera.

Benefits of technology

It improves the ranging accuracy and real-time observation ability of the target object, so that the detection results of the binocular camera and millimeter-wave radar can be accurately integrated on the same target object, and enhances the recognition stability in special environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118711153B_ABST
    Figure CN118711153B_ABST
Patent Text Reader

Abstract

The present invention proposes a method and device for joint calibration of an unmanned vehicle, a millimeter-wave radar, and a camera, and relates to the field of assisted driving technology. The method comprises: placing a target object at a marking point; obtaining a first coordinate point of the target object through a camera, and obtaining a second coordinate point of the target object through a millimeter-wave radar; combining the first coordinate point and the second coordinate point to generate a coordinate point pair; according to the coordinate point pair, jointly calibrating the millimeter-wave radar and the camera, and verifying the calibration result, and determining the effect of the joint calibration of the millimeter-wave radar and the camera according to the verification result. The present invention collects the coordinate data of the corner reflector through a binocular camera and a millimeter-wave radar respectively, and combines and calibrates the coordinate data of the two to complete the joint calibration of the binocular camera and the millimeter-wave radar, so that the detection results of the binocular camera and the millimeter-wave radar for the same target object can be accurately integrated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of assisted driving technology, and in particular to a method and device for jointly calibrating an unmanned vehicle, a millimeter-wave radar, and a camera. Background Art

[0002] With the development of sensor technology and machine vision technology, obstacle testing technology based on binocular cameras is increasingly widely used in the fields of robotics, smart cars, etc. However, in special driving environments, such as foggy days, rainy days, dark nights, and dim light, it is easy to lose the target and cause misjudgment when relying solely on binocular camera sensors to assist in identifying pedestrians, cars, and other targets. When only using millimeter-wave radar sensors to identify targets such as people and cars, it is impossible to observe the visual target in real time with the naked eye, and the low reflectivity of millimeter-wave radars for non-metallic objects can easily cause missed detection. On the basis of visual recognition of targets and ranging by binocular cameras, the ranging accuracy of millimeter-wave radars is used to assist binocular cameras in jointly identifying targets, which can not only improve accuracy and maintain stability, but also observe targets in real time. However, the existing imaging calibration method is only for the calibration of the binocular camera itself, and the detection results of the binocular camera and the millimeter-wave radar for the same target cannot be accurately integrated, which still leads to reduced accuracy. Summary of the invention

[0003] The main purpose of the present invention is to propose a joint calibration method and device for an unmanned vehicle, a millimeter-wave radar and a camera, aiming to realize the joint calibration of the millimeter-wave radar and the camera.

[0004] The present invention proposes a method for joint calibration of a millimeter-wave radar and a camera, wherein the camera is a binocular camera, and the method comprises: placing a target object at a plurality of marking points according to a preset period; each time the target object is placed at a marking point, obtaining a first coordinate point of the target object through the camera, and obtaining a second coordinate point of the target object through the millimeter-wave radar; wherein the number of the first coordinate points and the second coordinate points are the same, the first coordinate point is the position of the target object vertically to the ground, and the second coordinate point is the three-dimensional spatial position of the target object; combining the plurality of first coordinate points of the target object obtained by the camera and the plurality of second coordinate points of the target object obtained by the millimeter-wave radar to generate a plurality of coordinate point pairs; generating a mathematical relationship model according to the plurality of coordinate point pairs, and using an optimization algorithm to solve the parameters in the mathematical relationship model to obtain a homography matrix, generating a joint coordinate graph of the millimeter-wave radar and the camera based on the homography matrix, and determining the effect of the joint calibration of the millimeter-wave radar and the camera according to the joint coordinate graph.

[0005] In one embodiment, determining the effect of the joint calibration of the millimeter-wave radar and the camera according to the joint coordinate diagram includes: determining the position of the calibration point according to the joint coordinate diagram; when the calibration point is at a position where the target object is perpendicular to the ground, determining that the effect of the joint calibration of the millimeter-wave radar and the camera is better; when the calibration point is not at a position where the target object is perpendicular to the ground, determining that the effect of the joint calibration of the millimeter-wave radar and the camera is poor.

[0006] The present invention also proposes a millimeter-wave radar and camera joint calibration device, which includes: a processor and a memory; a millimeter-wave radar and camera joint calibration control program is stored in the memory, and the millimeter-wave radar and camera joint calibration control program implements the steps of the millimeter-wave radar and camera joint calibration method when executed by the processor.

[0007] The present invention also proposes an unmanned vehicle, which includes the millimeter-wave radar and camera joint calibration device.

[0008] The present invention proposes a method and device for joint calibration of an unmanned vehicle, a millimeter-wave radar, and a camera, and relates to the field of assisted driving technology. The method comprises: placing a target at a marking point; obtaining a first coordinate point of the target through a camera, and obtaining a second coordinate point of the target through a millimeter-wave radar; combining the first coordinate point and the second coordinate point to generate a coordinate point pair; according to the coordinate point pair, jointly calibrating the millimeter-wave radar and the camera, and verifying the calibration result, and determining the effect of the joint calibration of the millimeter-wave radar and the camera according to the verification result. The present invention collects the coordinate data of the corner reflector through a binocular camera and a millimeter-wave radar respectively, and combines and calibrates the coordinate data of the two to complete the joint calibration of the binocular camera and the millimeter-wave radar, so that the detection results of the binocular camera and the millimeter-wave radar for the same target can be accurately integrated, which can not only improve the ranging accuracy of the target, but also observe the target in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0010] Figure 1 A method step diagram of a millimeter wave radar and camera joint calibration method of the present invention;

[0011] Figure 2 A method step diagram of another embodiment of the millimeter wave radar and camera joint calibration method of the present invention;

[0012] Figure 3 A method step diagram of another embodiment of the millimeter wave radar and camera joint calibration method of the present invention;

[0013] Figure 4 A method step diagram of another embodiment of the millimeter wave radar and camera joint calibration method of the present invention;

[0014] Figure 5 This is a method step diagram of another embodiment of the millimeter wave radar and camera joint calibration method of the present invention.

[0015] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0018] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0019] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0020] It is understandable that with the development of sensor technology and machine vision technology, obstacle testing technology based on binocular cameras is increasingly widely used in the fields of robotics, smart cars, etc. However, in special driving environments, such as foggy days, rainy days, dark nights, and dim light, it is easy to lose the target and cause misjudgment when relying solely on binocular camera sensors to assist in identifying pedestrians, cars, and other targets. When only using millimeter-wave radar sensors to identify targets such as people and cars, it is impossible to observe the visual target in real time with the naked eye, and the low reflectivity of millimeter-wave radars for non-metallic objects can easily cause missed detection. On the basis of visual recognition of targets and ranging by binocular cameras, the ranging accuracy of millimeter-wave radars is used to assist binocular cameras in jointly identifying targets, which can not only improve accuracy and maintain stability, but also observe targets in real time. However, the existing imaging calibration method is only for the calibration of the binocular camera itself, and the detection results of the binocular camera and the millimeter-wave radar for the same target cannot be accurately integrated, which still leads to reduced accuracy.

[0021] Therefore, in order to improve the accuracy of distance measurement of the target object and observe the target object in real time, the present invention proposes a millimeter wave radar and camera joint calibration method, referring to Figure 1 , the method comprising:

[0022] S100, placing the target object at the marked point;

[0023] S200, acquiring a first coordinate point of the target object through a camera, and acquiring a second coordinate point of the target object through a millimeter wave radar;

[0024] S300, combining the first coordinate point and the second coordinate point to generate a coordinate point pair;

[0025] S400: jointly calibrate the millimeter-wave radar and the camera according to the coordinate point pair, verify the calibration result, and determine the effect of the joint calibration of the millimeter-wave radar and the camera according to the verification result.

[0026] It is understandable that, on the basis of visual recognition of the target and distance measurement by the binocular camera, the ranging accuracy of the millimeter-wave radar is used to assist the binocular camera in jointly identifying the target, which can not only improve the accuracy and maintain stability, but also observe the target in real time. However, the existing imaging calibration methods are all for the calibration of the binocular camera itself, lacking the joint calibration of the binocular camera and the millimeter-wave radar, resulting in the inability to accurately merge the detection results of the binocular camera and the millimeter-wave radar for the same target. Therefore, the present invention obtains the first coordinate point of the target object through the binocular camera, and obtains the second coordinate point of the target object through the millimeter-wave radar, and combines the first coordinate point and the second coordinate point to generate a coordinate point pair, and completes the joint calibration of the binocular camera and the millimeter-wave radar through the coordinate point pair, so that the detection results of the binocular camera and the millimeter-wave radar for the same target can be accurately merged, which solves the technical problem that the imaging calibration method in the prior art cannot realize the joint calibration and the detection results cannot be accurately merged, which can not only improve the ranging accuracy of the target object, but also observe the target object in real time.

[0027] In this embodiment, the target object is specifically a corner reflector. The camera is specifically implemented using a binocular camera, which simultaneously obtains the coordinate parameters of the corner reflector at different angles through two parallel lenses, and calculates the depth information of the corner reflector by measuring the distance difference between the two lenses. In addition, the millimeter wave radar obtains the position information, speed information, angle information, etc. of the corner reflector by transmitting a millimeter wave signal to the corner reflector and measuring the reflection time and intensity of the millimeter wave signal. This enables the millimeter wave radar to achieve accurate measurement of the corner reflector.

[0028] In practical applications, the corner reflector is placed at the marking point, and the first coordinate point of the corner reflector is obtained by the binocular camera. The first coordinate point is the depth information. The depth information in this embodiment refers to the point where the corner reflector is perpendicular to the ground; and the second coordinate point of the corner reflector is obtained by the millimeter wave radar. The second coordinate point is the position information. The position information in this embodiment refers to the three-dimensional spatial position of the corner reflector. Among them, the three-dimensional spatial position of the corner reflector consists of length, width and height. The depth information obtained by the binocular camera and the position information obtained by the millimeter wave radar are combined to generate a coordinate point pair. The millimeter wave radar and the camera are jointly calibrated according to the coordinate point pair to generate the corresponding calibration result, and the calibration result is verified and analyzed, and then the effect of the joint calibration of the millimeter wave radar and the camera is determined according to the verification result. For example, when the calibration point is verified to be at the position where the corner reflector is perpendicular to the ground, it is determined that the effect of the joint calibration of the millimeter wave radar and the camera is better, otherwise it is determined that the effect of the joint calibration of the millimeter wave radar and the camera is poor. In this embodiment, the coordinate data of the corner reflector is collected by the binocular camera and the millimeter-wave radar respectively, and the coordinate data of the two are combined and calibrated to complete the joint calibration of the binocular camera and the millimeter-wave radar, so that the detection results of the binocular camera and the millimeter-wave radar for the same target can be accurately integrated, which can not only improve the ranging accuracy of the target, but also observe the target in real time.

[0029] The present invention proposes a method for joint calibration of millimeter-wave radar and camera, the method comprising: placing a target at a marking point; obtaining a first coordinate point of the target through a camera, and obtaining a second coordinate point of the target through a millimeter-wave radar; combining the first coordinate point and the second coordinate point to generate a coordinate point pair; according to the coordinate point pair, jointly calibrating the millimeter-wave radar and the camera, and verifying the calibration result, and determining the effect of the joint calibration of the millimeter-wave radar and the camera according to the verification result. The present invention collects the coordinate data of the corner reflector through a binocular camera and a millimeter-wave radar respectively, and combines and calibrates the coordinate data of the two to complete the joint calibration of the binocular camera and the millimeter-wave radar, so that the detection results of the binocular camera and the millimeter-wave radar for the same target can be accurately integrated, which can not only improve the ranging accuracy of the target, but also observe the target in real time.

[0030] In one embodiment, referring to Figure 2 , the placing of the target object at the marking point comprises:

[0031] S110, placing the target object at multiple marking points according to a preset period; and each time the target object is placed at a marking point, executing the method steps of obtaining the first coordinate point of the target object through a camera and obtaining the second coordinate point of the target object through a millimeter wave radar.

[0032] It is understandable that the present embodiment is provided with nine marking points, and before the control camera and the millimeter-wave radar obtain the coordinate points of the corner reflector, the corner reflector is placed at the nine marking points according to a preset period. The preset period can be set according to user needs. For example, when the preset period is set to 30s, the corner reflectors are placed at the nine marking points according to a period of placement once every 30s. Specifically, the corner reflector is placed at the first marking point, and after an interval of 30s, the corner reflector is placed at the second marking point, and after an interval of 30s, the corner reflector is placed at the third marking point, and so on. It should be understood that each time the corner reflector is placed at a marking point, the first coordinate point of the corner reflector is obtained once by the binocular camera, and the second coordinate point of the corner reflector is obtained once by the millimeter-wave radar, until the binocular camera obtains the first coordinate points of the nine corner reflectors at the nine marking points, and the millimeter-wave radar obtains the second coordinate points of the nine corner reflectors at the nine marking points.

[0033] In one embodiment, referring to Figure 3 , the acquiring the first coordinate point of the target object by using a camera, and the acquiring the second coordinate point of the target object by using a millimeter wave radar include:

[0034] S210, acquiring a plurality of first coordinate points of the target object through the camera, and acquiring a plurality of second coordinate points of the target object through the millimeter wave radar;

[0035] The number of the first coordinate points and the number of the second coordinate points are the same, the first coordinate points are the positions of the target object vertically to the ground, and the second coordinate points are the three-dimensional spatial positions of the target object.

[0036] It can be understood that in this embodiment, each time the corner reflector is placed at a marking point, the first coordinate point of the corner reflector is acquired once by the binocular camera, and the second coordinate point of the corner reflector is acquired once by the millimeter-wave radar. Specifically, the corner reflector is placed at the first marking point, and the first coordinate point of the corner reflector is acquired once by the binocular camera, and the second coordinate point of the corner reflector is acquired once by the millimeter-wave radar. After an interval of 30 seconds, the corner reflector is placed at the second marking point, and the first coordinate point of the corner reflector is acquired once by the binocular camera, and the second coordinate point of the corner reflector is acquired once by the millimeter-wave radar. And so on, until the corner reflector is placed at nine different marking points, the binocular camera acquires the first coordinate points of the nine corner reflectors at the nine marking points, and the millimeter-wave radar acquires the second coordinate points of the nine corner reflectors at the nine marking points, respectively.

[0037] In one embodiment, referring to Figure 4 , combining the first coordinate point and the second coordinate point to generate a coordinate point pair includes:

[0038] S310, combining a plurality of first coordinate points of the target object acquired by the camera and a plurality of second coordinate points of the target object acquired by the millimeter wave radar to generate a plurality of coordinate point pairs.

[0039] It can be understood that in this embodiment, the corner reflectors are placed at nine marking points according to a preset period. Each time the corner reflector is placed at a marking point, the first coordinate point of the corner reflector is obtained once by the binocular camera, and the second coordinate point of the corner reflector is obtained once by the millimeter wave radar. Thus, the binocular camera obtains the first coordinate points of the nine corner reflectors at the nine marking points, and the millimeter wave radar obtains the second coordinate points of the nine corner reflectors at the nine marking points. Therefore, the number of the first coordinate points and the second coordinate points is the same, the number of the first coordinate points is nine, and the number of the second coordinate points is also nine, and the nine first coordinate points and the nine second coordinate points constitute nine coordinate point pairs. Since the first coordinate point is the position of the corner reflector vertical to the ground, that is, the depth information of the corner reflector; the second coordinate point is the three-dimensional spatial position of the corner reflector, that is, the position information of the corner reflector, the nine first coordinate points and the nine second coordinate points constitute nine coordinate point pairs, the purpose of which is to establish an accurate spatial relationship between the millimeter wave radar and the binocular camera, so as to achieve accurate fusion of the two sensor data and improve the perception accuracy and reliability in the application device. Therefore, the coordinate point pair can be regarded as the spatial relationship parameter between the millimeter-wave radar and the stereo camera.

[0040] In one embodiment, referring to Figure 5 , the joint calibration of the millimeter wave radar and the camera according to the coordinate point pair includes:

[0041] S410: Generate a mathematical relationship model according to the plurality of coordinate point pairs, and use an optimization algorithm to solve parameters in the mathematical relationship model to obtain a homography matrix.

[0042] It can be understood that in the present embodiment, nine first coordinate points and nine second coordinate points constitute nine coordinate point pairs, and a mathematical relationship model is generated based on the nine coordinate point pairs, thereby establishing a mathematical relationship model between the binocular camera and the millimeter-wave radar. Based on an optimization algorithm such as the least squares method, the parameters in the mathematical relationship model are solved to obtain the homography matrix. It should be understood that the homography matrix is ​​used to fuse the data acquired by the millimeter-wave radar and the data acquired by the binocular camera, and to establish a coordinate connection between the common points of the millimeter-wave radar and the binocular camera, so that the coordinate parameters of the millimeter-wave radar and the binocular camera are closely connected to form a whole, thereby realizing the concept of a stereoscopic vision system. Therefore, the homography matrix can be regarded as a mapping relationship between the millimeter-wave radar and the binocular camera. ‌‌‌

[0043] In one embodiment, referring to Figure 5, the verification of the calibration result and determining the effect of the joint calibration of the millimeter wave radar and the camera according to the verification result include:

[0044] S420: Generate a joint coordinate map of the millimeter-wave radar and the camera based on the homography matrix, and determine an effect of the joint calibration of the millimeter-wave radar and the camera according to the joint coordinate map.

[0045] It can be understood that in this embodiment, the homography matrix is ​​mainly used to describe the mapping relationship between the millimeter-wave radar and the binocular camera. Using the extracted and matched feature points, the homography matrix is ​​calculated by the least squares method or other optimization algorithms to generate the position and shape of the corner reflector in the three-dimensional space, and then generate a joint coordinate map of the millimeter-wave radar and the binocular camera, so as to achieve the splicing and fusion between the coordinate parameters of the millimeter-wave radar and the coordinate parameters of the binocular camera, and complete the joint calibration of the binocular camera and the millimeter-wave radar. According to the position of the calibration point in the joint coordinate map of the millimeter-wave radar and the camera, the effect of the joint calibration of the millimeter-wave radar and the camera can be further judged.

[0046] In one embodiment, referring to Figure 5 , the effect of determining the joint calibration of the millimeter wave radar and the camera according to the joint coordinate diagram includes:

[0047] S430, determining the position of the calibration point according to the joint coordinate graph;

[0048] S440: When the calibration point is at a position where the target object is perpendicular to the ground, it is determined that the joint calibration of the millimeter wave radar and the camera has a better effect;

[0049] S450: When the calibration point is not at a position where the target object is perpendicular to the ground, it is determined that the effect of the joint calibration of the millimeter wave radar and the camera is poor.

[0050] It can be understood that in this embodiment, the effect of the joint calibration of the millimeter-wave radar and the camera can be judged according to the position of the calibration point in the joint coordinate diagram of the millimeter-wave radar and the camera. For example, when the calibration point is verified to be at the position where the corner reflector is perpendicular to the ground according to the joint coordinate diagram, it is determined that the effect of the joint calibration of the millimeter-wave radar and the camera is good; when the calibration point is not verified to be at the position where the corner reflector is perpendicular to the ground according to the joint coordinate diagram, it is determined that the effect of the joint calibration of the millimeter-wave radar and the camera is poor.

[0051] The present invention proposes a millimeter-wave radar and camera joint calibration device, which includes: a processor and a memory; a millimeter-wave radar and camera joint calibration control program is stored in the memory, and when the millimeter-wave radar and camera joint calibration control program is executed by the processor, the steps of the millimeter-wave radar and camera joint calibration method described in the above embodiment are implemented.

[0052] It is understandable that the existing imaging calibration methods are all for the calibration of the binocular camera itself, lacking the joint calibration of the binocular camera and the millimeter-wave radar, resulting in the inability to accurately fuse the detection results of the binocular camera and the millimeter-wave radar for the same target. Therefore, the present invention obtains the first coordinate point of the target through the binocular camera, and obtains the second coordinate point of the target through the millimeter-wave radar, and combines the first coordinate point and the second coordinate point to generate a coordinate point pair, and completes the joint calibration of the binocular camera and the millimeter-wave radar through the coordinate point pair, so that the detection results of the binocular camera and the millimeter-wave radar for the same target can be accurately fused, solving the technical problem that the imaging calibration method in the prior art cannot realize the joint calibration and the detection results cannot be accurately fused, which can not only improve the ranging accuracy of the target, but also observe the target in real time.

[0053] The present invention proposes an unmanned vehicle, which includes the millimeter wave radar and camera joint calibration device described in the above embodiment. The specific control logic of the unmanned vehicle refers to the above embodiment. Since the unmanned vehicle adopts all the technical solutions of all the above embodiments, it at least has all the beneficial effects brought by the technical solutions of the above embodiments, which will not be repeated here one by one.

[0054] The above descriptions are only optional embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A millimeter wave radar and camera joint calibration method, characterized in that: The camera is a binocular camera, and the millimeter wave radar and camera joint calibration method includes: Place the target at multiple marking points according to a preset cycle; Each time the target object is placed at a marking point, a first coordinate point of the target object is acquired through the camera, and a second coordinate point of the target object is acquired through the millimeter wave radar; The number of the first coordinate points and the second coordinate points is the same, the first coordinate points are positions of the target object vertically to the ground, and the second coordinate points are positions of the target object in three-dimensional space; combining a plurality of first coordinate points of the target object acquired by the camera and a plurality of second coordinate points of the target object acquired by the millimeter wave radar to generate a plurality of coordinate point pairs; A mathematical relationship model is generated according to the plurality of coordinate point pairs, and an optimization algorithm is used to solve the parameters in the mathematical relationship model to obtain a homography matrix, a joint coordinate graph of the millimeter-wave radar and the camera is generated based on the homography matrix, and an effect of the joint calibration of the millimeter-wave radar and the camera is determined according to the joint coordinate graph.

2. The millimeter wave radar and camera joint calibration method according to claim 1, characterized in that: The effect of determining the joint calibration of the millimeter wave radar and the camera according to the joint coordinate diagram includes: Determine the position of the calibration point according to the joint coordinate diagram; When the calibration point is at a position where the target object is perpendicular to the ground, it is determined that the joint calibration of the millimeter wave radar and the camera has a better effect; When the calibration point is not at a position where the target object is perpendicular to the ground, it is determined that the effect of the joint calibration of the millimeter wave radar and the camera is poor.

3. A millimeter wave radar and camera joint calibration device, characterized in that: The millimeter wave radar and camera joint calibration device comprises: Processor and memory; The memory stores a millimeter-wave radar and camera joint calibration control program, and when the millimeter-wave radar and camera joint calibration control program is executed by the processor, the steps of the millimeter-wave radar and camera joint calibration method as described in any one of claims 1-2 are implemented.

4. An unmanned vehicle, characterized in that: The unmanned vehicle includes the millimeter-wave radar and camera joint calibration device as described in claim 3.