Method and system for calibrating a conversion relationship between two roadside sensors

Abstract

The present invention discloses a method and system for calibrating the conversion relationship between two roadside sensors, relating to the technical field of roadside sensor calibration. The method comprises: transforming the position coordinates obtained by a camera device using a first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in a coordinate system used for a bird's-eye view; transforming the position coordinates obtained by a radar device using a rigid transformation matrix to obtain the position coordinates of the position coordinates obtained by the radar device in a coordinate system used for a bird's-eye view; aligning the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view with the position coordinates obtained by the radar device in the coordinate system used for the bird's-eye view, and then determining the conversion relationship between the camera device and the radar device. The present invention can obtain a more accurate conversion relationship and achieve precise conversion of position coordinates.

Description

Technical Field

[0001] The present invention relates to the technical field of roadside sensor calibration, and in particular to a method and system for calibrating a conversion relationship between two roadside sensors. Background Art

[0002] Roadside sensors are used to detect information about traffic participants on the road. They need to provide accurate location information for traffic participants and are a crucial component of vehicle-road collaboration and digital twin systems. The current technical solutions for calibrating different roadside sensors are as follows:

[0003] 1) The invention patent, publication number "CN111383285A," titled "A Method and System for Calibration Based on Millimeter-Wave Radar and Camera Sensor Fusion," requires manual calibration, such as requiring an engineer to walk a straight line through different areas of the scene with a handheld corner reflector. Collecting data at urban intersections or highways poses a risk. If road closures require approval from traffic control authorities, not only does this disrupt normal traffic flow, but replacing roadside sensors also requires additional road closures, which is inefficient.

[0004] 2) The invention patent with publication number "CN112558023A" and subject name "Sensor calibration method and device" proposes a method that does not require road closures. Specifically, the mapping relationship between sensors is established through high-precision maps. However, there are often no high-precision maps for reference on actual roads.

[0005] 3) The invention patent with publication number "CN112836737A" and subject name "A method for online calibration of roadside combined perception equipment based on vehicle-road data fusion", as well as the invention patent with publication number "WO2022206978A1" and subject name "A method for calibration of roadside millimeter-wave radar based on vehicle-mounted positioning device", both propose technical solutions for calculating sensor mapping relationships by providing calibration target data through vehicle-mounted high-precision positioning equipment, but require relatively expensive professional equipment to record and feedback data in real time, and also require the vehicle equipped with the positioning equipment to drive multiple times within the detection range to complete the calibration. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and specifically provide a method and system for calibrating the conversion relationship between two roadside sensors, as follows:

[0007] 1) In a first aspect, the present invention provides a method for calibrating a conversion relationship between two roadside sensors. The specific technical solution is as follows:

[0008] After the camera device and the radar are time-aligned, the position coordinates of each preset target in the corresponding coordinate system are monitored at the same time using the camera device and the radar;

[0009] Based on the first transformation matrix, performing an inverse perspective projection transformation on the image monitored by the camera device at the same time to obtain a bird's-eye view, and transforming the position coordinates obtained by the camera device using the first transformation matrix to obtain the position coordinates obtained by the camera device in a coordinate system used for the bird's-eye view;

[0010] Calculating a rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view, and using the rigid transformation matrix to transform the position coordinates obtained by the radar to obtain the position coordinates of the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view;

[0011] registering the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view with the position coordinates of the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view;

[0012] The conversion relationship between the camera device and the radar is determined based on the conversion relationship of the position coordinates for achieving the registration and the position coordinates after the registration.

[0013] The beneficial effects of the method for calibrating the conversion relationship between two roadside sensors provided by the present invention are as follows:

[0014] The position coordinates of the target obtained by any roadside sensor are the coordinates of the coordinate system used by the roadside sensor. In actual application, the position coordinates of the targets obtained by different roadside sensors need to be converted. Since the conversion between the position coordinates of the targets obtained by any two roadside sensors needs to be completed through the conversion relationship between the two roadside sensors, the accuracy of the conversion relationship determines whether the conversion of the position coordinates is accurate. The present invention converts the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view and the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view. Registration can obtain a more accurate conversion relationship, and the process of obtaining the conversion relationship is simpler and lower in cost than the existing technology (for example: the invention patent with publication number "CN112836737A" and subject name "A method for online calibration of roadside combined perception equipment based on vehicle-road data fusion", and the invention patent with publication number "WO2022206978A1" and subject name "A method for calibration of roadside millimeter-wave radar based on vehicle-mounted positioning device"). Through the obtained conversion relationship, the precise conversion of position coordinates in different coordinate systems (coordinate systems used by different roadside sensors) can be achieved.

[0015] Based on the above solution, the method for calibrating the conversion relationship between two roadside sensors of the present invention can be further improved as follows.

[0016] Furthermore, the position coordinates obtained by the camera device are transformed using a first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in a coordinate system used for the bird's-eye view, including:

[0017] The first transformation formula is used to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view. The first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, and i is a positive integer.

[0018] Furthermore, the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view is calculated, including:

[0019] The ICP algorithm or CPD algorithm is used to calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

[0020] Furthermore, the conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

[0021] 2) In a second aspect, the present invention further provides a system for calibrating the conversion relationship between two roadside sensors. The specific technical solution is as follows:

[0022] It includes a position coordinate acquisition module, a bird's-eye view acquisition module, a position coordinate conversion module, a rigid transformation matrix determination module, a registration module and a conversion relationship determination module;

[0023] The position coordinate acquisition module is used to: after time alignment of the camera device and the radar, use the camera device and the radar to monitor and obtain the position coordinates of each preset target in the corresponding coordinate system at the same time;

[0024] The bird's-eye view acquisition module is used to: perform inverse perspective projection transformation on the image monitored by the camera device at the same time based on the first transformation matrix to obtain a bird's-eye view;

[0025] The position coordinate conversion module is used to: transform the position coordinates obtained by the camera device using a first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in a coordinate system used for the bird's-eye view;

[0026] The rigid transformation matrix determination module is used to: calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view;

[0027] The position coordinate conversion module is further used for: using a rigid transformation matrix to transform the position coordinates obtained by the radar to obtain the position coordinates of the position coordinates obtained by the radar in a coordinate system used in the bird's-eye view;

[0028] The registration module is used to: register the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view with the position coordinates of the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view;

[0029] The conversion relationship determination module is used to determine the conversion relationship between the camera device and the radar according to the conversion relationship of the position coordinates for achieving registration and the position coordinates after registration.

[0030] Based on the above solution, the system for calibrating the conversion relationship between two roadside sensors of the present invention can be further improved as follows.

[0031] Furthermore, the position coordinate conversion module is specifically used for:

[0032] The first transformation formula is used to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view. The first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, and i is a positive integer.

[0033] Furthermore, the rigid transformation matrix determination module is specifically used for:

[0034] The ICP algorithm or CPD algorithm is used to calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

[0035] Furthermore, the conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

[0036] 3) In a third aspect, the present invention further provides an electronic device, comprising a processor coupled to a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the electronic device implements any one of the above-mentioned methods for calibrating the conversion relationship between two roadside sensors.

[0037] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is loaded and executed by a processor so that the computer implements any of the above-mentioned methods for calibrating the conversion relationship between two roadside sensors.

[0038] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0040] Figure 1 A schematic flow chart of a method for calibrating a conversion relationship between two roadside sensors according to an embodiment of the present invention;

[0041] Figure 2 Schematic diagram of the structure of a system for calibrating a conversion relationship between two roadside sensors according to an embodiment of the present invention;

[0042] Figure 3 The figure is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0044] like Figure 1 As shown, a system for calibrating a conversion relationship between two roadside sensors according to an embodiment of the present invention includes the following steps:

[0045] S1. After time alignment of the camera device and the radar, the camera device and the radar are used to monitor and obtain the position coordinates of each preset target in the corresponding coordinate system at the same time;

[0046] After time alignment between the camera and radar, target detection can be performed on the camera's image using algorithms such as the YOLO or ViD algorithm. N targets are detected. Since these algorithms output a target detection frame, the center point of the bottom edge of the target detection frame is used as the position coordinate. The radar detects the target point, and its position coordinates can be directly obtained. A total of M targets are detected.

[0047] The same multiple targets can be selected from N targets and M targets as preset targets. For example, n preset targets are selected. When N=M, all targets can be used as preset targets, and in this case, N=M=n.

[0048] The area for target tracking using cameras and radars can be set according to actual conditions, such as a designated road area.

[0049] Among them, the position coordinates of the target obtained by the camera device are the coordinates in the coordinate system used by the camera device, and the position coordinates of the target obtained by the radar are the coordinates in the coordinate system used by the radar. The coordinate system used by the camera device can be established according to actual conditions, and the coordinate system used by the radar can also be established according to actual conditions.

[0050] The imaging device may be a camera, the radar may be a millimeter wave radar, or other radars may be selected according to actual conditions.

[0051] For ease of expression, the set of position coordinates of each preset target in the corresponding coordinate system (the coordinate system used by the camera device) obtained by monitoring the camera device is recorded as P1, and the set of position coordinates of each preset target in the corresponding coordinate system (the coordinate system used by the radar) obtained by monitoring the radar is recorded as R1.

[0052] S2. Based on the first transformation matrix, perform an inverse perspective projection transformation on the image monitored by the camera device at the same time to obtain a bird's-eye view, and use the first transformation matrix to transform the position coordinates obtained by the camera device to obtain the position coordinates of the position coordinates obtained by the camera device in a coordinate system used for the bird's-eye view;

[0053] The first transformation matrix can be obtained in advance by selecting four pairs of points.

[0054] The method of transforming the position coordinates obtained by the camera device using the first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view includes:

[0055] The first transformation formula is used to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view. The first transformation formula is: 2i =H1P 1i, where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, and i is a positive integer.

[0056] S3. Calculate a rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view, and use the rigid transformation matrix to transform the position coordinates obtained by the radar to obtain the position coordinates of the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view;

[0057] The ICP algorithm (Iterative closest point) or the CPD algorithm (Coherent point drift) is used to calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

[0058] Among them, the position coordinates obtained by the radar are transformed using the rigid transformation matrix to obtain the position coordinates of the position coordinates obtained by the radar in the coordinate system used in the bird's-eye view. This is specifically achieved through the second transformation formula. The second transformation formula is: R 2i =H2R 1i , where H2 represents the rigid transformation matrix, R 1i Indicates the position coordinates of the i-th preset target in R1 in the coordinate system used by the radar, R 2i Indicates: R 1i The coordinates of the position in the coordinate system used by the Bird's Eye view.

[0059] S4. aligning the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view with the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view;

[0060] For ease of description, the set of position coordinates obtained by radar in the coordinate system used in the bird's-eye view is recorded as R2, which includes R 11 、R 12 …R 1i …R 1n The position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view are marked as P2, P2 includes P 11 、P 12 …P 1i …P 1n .

[0061] Among them, S4 specifically includes:

[0062] S40, record the bird's-eye view (also called BEV view) obtained in S2 as I BEV , I BEV The coordinate system used is in proportional scaling with the geodetic coordinate system. BEV The transformation matrix between the used coordinate system and the geodetic coordinate system can be obtained in advance through calibration. This transformation matrix is ​​recorded as the second transformation matrix. The second transformation matrix is ​​used to transform each position coordinate in P2 (the position coordinate obtained by the camera device in the coordinate system used for the bird's-eye view) to obtain the position coordinate of each position coordinate in P2 in the geodetic coordinate system. This is specifically achieved by the following formula:

[0063] C 2i =H3P 2i

[0064] Among them, H3 represents the second transformation matrix, P 2i represents the position coordinates of the i-th preset target in P2 in the coordinate system used in the bird's-eye view, C 2i Indicates: P 2i The coordinates of the position in the geodetic coordinate system.

[0065] The set of position coordinates of each position coordinate in P2 in the geodetic coordinate system is recorded as C2, which includes C 11 、C 12 …C 1i …C 1n .

[0066] S41 , superimposing the detection frame of each preset target on the bird's-eye view according to the position coordinates of each preset target in C2 in the earth coordinate system.

[0067] S42. Multiply each position coordinate in R1 by the coefficient scale to transform each position coordinate in R1 into the coordinate system used in the bird's-eye view. This is specifically achieved by the following formula:

[0068] R 3i =scale×R 1i

[0069] Among them, R 3i Indicates that R is multiplied by the coefficient scale. 1i Convert the position coordinates to the coordinate system used by the bird's-eye view, transform each position coordinate in R1 into the set of position coordinates in the coordinate system used by the bird's-eye view by means of a scale coefficient, and record it as R3. The position coordinates in R3 can be displayed on the bird's-eye view.

[0070] S43. Rotate and / or translate (including horizontal movement and / or vertical movement) each position coordinate in R3 to coincide with the detection frame where each position coordinate in C2 is superimposed on the bird's-eye view, thereby achieving alignment between the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view and the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view.

[0071] The position coordinates of each preset target in C2 in the earth coordinate system and each position coordinate in R3 are registered position coordinates.

[0072] In another embodiment, S4 can be implemented manually.

[0073] S5. Determine a conversion relationship between the camera device and the radar according to the conversion relationship of the position coordinates for achieving registration and the position coordinates after registration.

[0074] The conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

[0075] S5 specifically includes:

[0076] S50 , recording the rotation angle and translation displacement when the registration is achieved, and obtaining the adjusted position coordinates corresponding to each position coordinate in R3 , and recording the set of all adjusted position coordinates as R4 .

[0077] S51. Derivation of the conversion relationship between any position coordinate in R3 and the adjusted position coordinate corresponding to the position coordinate (i.e., the conversion relationship of the position coordinates for achieving registration) is as follows:

[0078]

[0079] Among them, any position coordinate in R3 is (x r3 ,y r3 ), (x r3 ,y r3 ) corresponds to the adjusted position coordinates in R4 (x r4 ,y r4 ), θ represents the rotation angle, Δx represents the horizontal displacement, and Δy represents the vertical displacement.

[0080] S52, using formula C 2i =H3P 2i 、R 3i =scale×R 1i and The homography transformation matrix H used to characterize the transformation relationship can be obtained:

[0081]

[0082] Among them, h 00 、h 01 、h 02 、h 10 、h 11 、h 12 、h 20 、h 21 and h 22 is an element in the homography transformation matrix H.

[0083] At this time, the target is tracked using the time-aligned camera and radar pair. The position coordinates of the same target monitored at any same time in the corresponding coordinate system satisfy:

[0084]

[0085] Among them, the position coordinates in the coordinate system used by the camera device are (u, v), and the position coordinates in the coordinate system used by the radar are (x, y), and (u, v) and (x, y) are: the position coordinates of the same target monitored by the time-aligned camera device and radar at any same time, and s represents the coefficient of the homogeneous coordinate.

[0086] Through the homography transformation matrix H, the position coordinates in the coordinate system used by the camera device can be accurately converted to the coordinate system used by the radar, and the position coordinates in the coordinate system used by the second preset roadside sensor can also be accurately converted to the coordinate system used by the camera device.

[0087] The beneficial effects of the method for calibrating the conversion relationship between two roadside sensors provided by the present invention are as follows:

[0088] The position coordinates of the target obtained by any roadside sensor are the coordinates of the coordinate system used by the roadside sensor. In practical applications, the position coordinates of the target obtained by different roadside sensors need to be converted. Since the conversion between the position coordinates of the target obtained by any two roadside sensors needs to be completed through the conversion relationship (homography transformation matrix) between the two roadside sensors, the accuracy of the conversion of the position coordinates determines whether the conversion of the position coordinates is accurate. The present invention aligns the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view with the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view. A more accurate transformation relationship (homography transformation matrix) can be obtained, and the process of obtaining the transformation relationship (homography transformation matrix) is simpler and lower in cost than that of the existing technology (for example: the invention patent with publication number "CN112836737A" and subject name "A method for online calibration of a roadside combined perception device based on vehicle-road data fusion", and the invention patent with publication number "WO2022206978A1" and subject name "A method for calibrating a roadside millimeter-wave radar based on a vehicle-mounted positioning device"). Through the obtained transformation relationship (homography transformation matrix), the precise conversion of position coordinates in different coordinate systems (coordinate systems used by different roadside sensors) can be achieved.

[0089] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the scope of protection of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0090] like Figure 2 As shown, a system 200 for calibrating a transformation relationship between two roadside sensors according to an embodiment of the present invention includes a position coordinate acquisition module 201, a bird's-eye view acquisition module 202, a position coordinate conversion module 203, a rigid transformation matrix determination module 204, a registration module 205, and a transformation relationship determination module 206;

[0091] The position coordinate acquisition module 201 is used to: after time alignment of the camera device and the radar, use the camera device and the radar to monitor and obtain the position coordinates of each preset target in the corresponding coordinate system at the same time;

[0092] The bird's-eye view acquisition module 202 is configured to: perform an inverse perspective projection transformation on the image monitored by the camera device at the same time based on the first transformation matrix to obtain a bird's-eye view;

[0093] The position coordinate conversion module 203 is used to: transform the position coordinates obtained by the camera device using a first transformation matrix to obtain the position coordinates obtained by the camera device in a coordinate system used in the bird's-eye view;

[0094] The rigid transformation matrix determination module 204 is used to: calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view;

[0095] The position coordinate conversion module 203 is further configured to: transform the position coordinates obtained by the radar using a rigid transformation matrix to obtain the position coordinates of the position coordinates obtained by the radar in a coordinate system used in the bird's-eye view;

[0096] The registration module 205 is used to register the position coordinates in the coordinate system used for the bird's-eye view obtained by the camera device with the position coordinates in the coordinate system used for the bird's-eye view obtained by the radar;

[0097] The conversion relationship determination module 206 is used to determine the conversion relationship between the camera device and the radar using the aligned position coordinates.

[0098] Optionally, in the above technical solution, the position coordinate conversion module 203 is specifically used to:

[0099] The first transformation formula is used to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view. The first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, and i is a positive integer.

[0100] Optionally, in the above technical solution, the rigid transformation matrix determination module 204 is specifically configured to:

[0101] The ICP algorithm or CPD algorithm is used to calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

[0102] Optionally, in the above technical solution, the conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

[0103] It should be noted that the beneficial effects of the system 200 for calibrating the conversion relationship between two roadside sensors provided in the above embodiment are the same as the beneficial effects of the method for calibrating the conversion relationship between two roadside sensors, and will not be repeated here. In addition, when implementing its functions, the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0104] like Figure 3 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330. The at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any of the above-mentioned methods for calibrating a conversion relationship between two roadside sensors, specifically:

[0105] The electronic device 300 may vary significantly due to different configurations or performance, and may include one or more processors 320 (Central Processing Units, CPUs) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, which is loaded and executed by the one or more processors 320 to enable the electronic device 300 to implement any of the methods for calibrating the conversion relationship between two roadside sensors provided in the above embodiments. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The electronic device 300 may also include other components for implementing device functions, which will not be detailed here. The electronic device may specifically be a computer, etc.

[0106] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above-mentioned methods for calibrating the conversion relationship between two roadside sensors.

[0107] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0108] In an exemplary embodiment, a computer program product or computer program is also provided. The computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform any of the aforementioned methods for calibrating a conversion relationship between two roadside sensors.

[0109] It should be noted that the terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects and to define a specific order or precedence. Where appropriate, the order used for similar objects may be interchanged, such that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0110] Those skilled in the art will appreciate that the present invention may be implemented as a system, method, or computer program product. Therefore, the present invention may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present invention may be implemented in the form of a computer program product embodied in one or more computer-readable media containing computer-readable program code.

[0111] Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0112] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for calibrating the conversion relationship between two roadside sensors, characterized in that: include: After time alignment of the camera device and the radar, the camera device and the radar are used to monitor and obtain the position coordinates of each preset target in the corresponding coordinate system at the same time; performing an inverse perspective projection transformation on the image monitored by the camera device at the same time based on a first transformation matrix to obtain a bird's-eye view, and transforming the position coordinates obtained by the camera device using the first transformation matrix to obtain position coordinates of the position coordinates obtained by the camera device in a coordinate system used for the bird's-eye view; calculating a rigid transformation matrix between a coordinate system used by the radar and a coordinate system used by the bird's-eye view, and transforming position coordinates obtained by the radar using the rigid transformation matrix to obtain position coordinates of the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view; Registering the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view with the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view: S40, record the bird's-eye view obtained in S2 as I BEV , I BEV The coordinate system used is in proportional scaling with the geodetic coordinate system. BEV The transformation matrix between the used coordinate system and the geodetic coordinate system is recorded as the second transformation matrix. The second transformation matrix is ​​used to transform each position coordinate in the position coordinate P2 in the coordinate system used for the bird's-eye view obtained by the camera device to obtain the position coordinate of each position coordinate in P2 in the geodetic coordinate system. This is specifically achieved by the following formula: C 2i =H3P 2i Among them, H3 represents the second transformation matrix, P 2i represents the position coordinates of the i-th preset target in P2 in the coordinate system used in the bird's-eye view, C 2i Indicates: P 2i Position coordinates in the geodetic coordinate system; The set of position coordinates of each position coordinate in P2 in the geodetic coordinate system is recorded as C2, which includes C 11 、C 12 …C 1i …c 1n ; S41, superimposing the detection frame of each preset target on the bird's-eye view according to the position coordinates of each preset target in the geodetic coordinate system in C2; S42. Multiply each position coordinate in R1 by the coefficient scale to transform each position coordinate in R1 into the coordinate system used in the bird's-eye view. This is specifically achieved by the following formula: R 3i =scale×R 1i Among them, R 3i Indicates that R is multiplied by the coefficient scale. 1i Convert the position coordinates to the coordinate system used by the bird's-eye view, transform each position coordinate in R1 into the position coordinates in the coordinate system used by the bird's-eye view by using the scale coefficient, and record the set as R3, and display the position coordinates in R3 on the bird's-eye view; S43. Rotate and / or translate each position coordinate in R3 so that each position coordinate in R3 coincides with a detection frame in which each position coordinate in C2 is superimposed on the bird's-eye view, thereby achieving registration between the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view and the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view. The conversion relationship between the camera device and the radar is determined according to the conversion relationship of the position coordinates for achieving the registration and the position coordinates after the registration.

2. The method for calibrating the conversion relationship between two roadside sensors according to claim 1, characterized in that: Transforming the position coordinates obtained by the camera device using the first transformation matrix to obtain position coordinates of the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view includes: The position coordinates of the position coordinates obtained by the camera device in the coordinate system used in the bird's-eye view are obtained using a first transformation formula. The first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, and i is a positive integer.

3. A method for calibrating the conversion relationship between two roadside sensors according to claim 1 or 2, characterized in that: Calculating a rigid transformation matrix between a coordinate system used by the radar and a coordinate system used by the bird's-eye view includes: The ICP algorithm or the CPD algorithm is used to calculate a rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

4. A method for calibrating the conversion relationship between two roadside sensors according to claim 1 or 2, characterized in that: The conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

5. A system for calibrating the conversion relationship between two roadside sensors, characterized in that: It includes a position coordinate acquisition module, a bird's-eye view acquisition module, a position coordinate conversion module, a rigid transformation matrix determination module, a registration module and a conversion relationship determination module; The position coordinate acquisition module is used to: after time alignment of the camera device and the radar, use the camera device and the radar to monitor and obtain the position coordinates of each preset target in the corresponding coordinate system at the same time; The bird's-eye view acquisition module is configured to: perform an inverse perspective projection transformation on the image monitored by the camera device at the same time based on a first transformation matrix to obtain a bird's-eye view; The position coordinate conversion module is configured to: transform the position coordinates obtained by the camera device using the first transformation matrix to obtain position coordinates of the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view; The rigid transformation matrix determination module is used to: calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view; The position coordinate conversion module is further configured to: transform the position coordinates obtained by the radar using the rigid transformation matrix to obtain the position coordinates of the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view; The registration module is used to register the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view with the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view: S40, record the bird's-eye view obtained in S2 as I BEV , I BEV The coordinate system used is in proportional scaling with the geodetic coordinate system. BEV The transformation matrix between the used coordinate system and the geodetic coordinate system is recorded as the second transformation matrix. The second transformation matrix is ​​used to transform each position coordinate in the position coordinate P2 in the coordinate system used for the bird's-eye view obtained by the camera device to obtain the position coordinate of each position coordinate in P2 in the geodetic coordinate system. This is specifically achieved by the following formula: C 2i =H3P 2i Among them, H3 represents the second transformation matrix, P 2i represents the position coordinates of the i-th preset target in P2 in the coordinate system used in the bird's-eye view, C 2i Indicates: P 2i Position coordinates in the geodetic coordinate system; The set of position coordinates of each position coordinate in P2 in the geodetic coordinate system is recorded as C2, which includes C 11 、C 12 …C 1i …C 1n ; S41, superimposing the detection frame of each preset target on the bird's-eye view according to the position coordinates of each preset target in the geodetic coordinate system in C2; S42. Multiply each position coordinate in R1 by the coefficient scale to transform each position coordinate in R1 into the coordinate system used in the bird's-eye view. This is specifically achieved by the following formula: R 3i =scale×R 1i Among them, R 3i Indicates that R is multiplied by the coefficient scale. 1i Convert the position coordinates to the coordinate system used by the bird's-eye view, transform each position coordinate in R1 into the position coordinates in the coordinate system used by the bird's-eye view by using the scale coefficient, and record the set as R3, and display the position coordinates in R3 on the bird's-eye view; S43. Rotate and / or translate each position coordinate in R3 so that each position coordinate in R3 coincides with a detection frame in which each position coordinate in C2 is superimposed on the bird's-eye view, thereby achieving registration between the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view and the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view. The conversion relationship determination module is used to determine the conversion relationship between the camera device and the radar according to the conversion relationship of the position coordinates for achieving registration and the position coordinates after registration.

6. The system for calibrating the conversion relationship between two roadside sensors according to claim 5, characterized in that: The position coordinate conversion module is specifically used for: The position coordinates of the position coordinates obtained by the camera device in the coordinate system used in the bird's-eye view are obtained using a first transformation formula. The first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, and i is a positive integer.

7. A system for calibrating the conversion relationship between two roadside sensors according to claim 5 or 6, characterized in that: The rigid transformation matrix determination module is specifically used for: The ICP algorithm or the CPD algorithm is used to calculate a rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

8. A system for calibrating the conversion relationship between two roadside sensors according to claim 5 or 6, characterized in that: The conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, and the memory stores at least one computer program. The at least one computer program is loaded and executed by the processor so that the electronic device implements a method for calibrating the conversion relationship between two roadside sensors as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer to implement a method for calibrating a conversion relationship between two roadside sensors as described in any one of claims 1 to 4.

Images