Method and system for calibrating transformational relationship between two roadside sensors

Abstract

The invention discloses a method and system for calibrating the conversion relation between two roadside sensors, and relates to the technical field of roadside sensor calibration, and the method comprises the steps: converting a position coordinate obtained through a camera device through a first transformation matrix, obtaining a position coordinate of the position coordinate obtained by the camera device in a coordinate system used by a bird's-eye view; transforming the position coordinates obtained by the radar by using a rigid transformation matrix to obtain position coordinates of the position coordinates obtained by the radar in a coordinate system used by the aerial view; the position coordinates of the position coordinates obtained by the camera device in the coordinate system used in the aerial view are registered with the position coordinates of the position coordinates obtained by the radar in the coordinate system used in the aerial view, and then the conversion relation between the camera device and the radar is determined. According to the method, a more accurate conversion relation can be obtained, and accurate conversion of the position coordinates is realized.

Description

Background Art

[0002] Roadside sensors are used to detect information about traffic participants on the road. The sensors need to provide accurate location information of traffic participants and are an important part of vehicle-road collaboration and digital twin systems. The current technical solutions for calibrating different roadside sensors are as follows:

[0003] 1) In the invention patent with the publication number of "CN111383285A" and the subject name of "A calibration method and system based on millimeter-wave radar and camera sensor fusion", it is necessary to use artificial calibration objects for calibration, such as engineers holding corner reflectors and walking in straight lines in different areas of the scene. There is a certain risk for engineers to collect data at urban intersections or highways. If the road closure needs to be applied to the traffic control department, it will not only affect normal traffic, but also require the application for road closure again to replace the roadside sensor, which is very inefficient.

[0004] 2) The invention patent with publication number "CN112558023A" and subject name "Sensor calibration method and device" proposes a method that does not require road closures. Specifically, a mapping relationship between sensors is established through high-precision maps, but there are often no high-precision maps for reference on actual roads.

[0005] 3) The invention patent with publication number "CN112836737A" and subject name "A method for online calibration of roadside combined perception equipment based on vehicle-road data fusion", as well as the invention patent with publication number "WO2022206978A1" and subject name "A method for roadside millimeter-wave radar calibration based on vehicle-mounted positioning device" both proposed a technical solution for calculating the sensor mapping relationship by providing calibration target data through a vehicle-mounted high-precision positioning device, but it requires relatively expensive professional equipment to record and feedback data in real time, and also requires the vehicle equipped with the positioning device to drive multiple times within the detection range to complete the calibration. Summary of the invention

[0006] The technical problem to be solved by the present invention is to address the deficiencies of the prior art and specifically provide a method and system for calibrating the conversion relationship between two roadside sensors, as follows:

[0007] 1) In a first aspect, the present invention provides a method for calibrating a conversion relationship between two roadside sensors, and the specific technical solution is as follows:

[0008] After the camera device and the radar are time-aligned, the position coordinates of each preset target in the corresponding coordinate system are monitored at the same time using the camera device and the radar;

[0009] Based on the first transformation matrix, an inverse perspective projection transformation is performed on the image monitored by the camera device at the same time to obtain a bird's-eye view, and the position coordinates obtained by the camera device are transformed by using the first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in a coordinate system used by the bird's-eye view;

[0010] Calculating a rigid transformation matrix between a coordinate system used by the radar and a coordinate system used by the bird's-eye view, and using the rigid transformation matrix to transform the position coordinates obtained by the radar to obtain the position coordinates of the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view;

[0011] registering the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view with the position coordinates of the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view;

[0012] The conversion relationship between the camera device and the radar is determined based on the conversion relationship of the position coordinates for achieving the registration and the position coordinates after the registration.

[0013] The beneficial effects of a method for calibrating a conversion relationship between two roadside sensors provided by the present invention are as follows:

[0014] The position coordinates of the target obtained by any roadside sensor are the coordinates of the coordinate system used by the roadside sensor. In practical applications, the position coordinates of the targets obtained by different roadside sensors need to be converted. Since the conversion between the position coordinates of the targets obtained by any two roadside sensors needs to be completed through the conversion relationship between the two roadside sensors, the accuracy of the conversion relationship determines whether the conversion of the position coordinates is accurate. The present invention converts the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view and the position coordinates of the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view. Registration can obtain a more accurate conversion relationship, and the process of obtaining the conversion relationship is simpler and less costly than the prior art (for example, the invention patent with publication number "CN112836737A" and subject name "A roadside combined perception device online calibration method based on vehicle-road data fusion", and the invention patent with publication number "WO2022206978A1" and subject name "A roadside millimeter-wave radar calibration method based on a vehicle-mounted positioning device"). Through the obtained conversion relationship, the precise conversion of position coordinates in different coordinate systems (coordinate systems used by different roadside sensors) can be achieved.

[0015] On the basis of the above solution, a method for calibrating the conversion relationship between two roadside sensors of the present invention can be further improved as follows.

[0016] Further, the position coordinates obtained by the camera device are transformed by using the first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view, including:

[0017] The first transformation formula is used to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used in the bird's-eye view. The first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, i is a positive integer.

[0018] Furthermore, the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view is calculated, including:

[0019] The ICP algorithm or the CPD algorithm is used to calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

[0020] Furthermore, the conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

[0021] 2) In a second aspect, the present invention further provides a system for calibrating a conversion relationship between two roadside sensors, and the specific technical solution is as follows:

[0022] It includes a position coordinate acquisition module, a bird's-eye view acquisition module, a position coordinate conversion module, a rigid transformation matrix determination module, a registration module and a conversion relationship determination module;

[0023] The position coordinate acquisition module is used to: after time alignment of the camera device and the radar, use the camera device and the radar to monitor and obtain the position coordinates of each preset target in the corresponding coordinate system at the same time;

[0024] The bird's-eye view acquisition module is used to: perform inverse perspective projection transformation on the image monitored by the camera device at the same time based on the first transformation matrix to obtain a bird's-eye view;

[0025] The position coordinate conversion module is used to: transform the position coordinates obtained by the camera device using a first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in a coordinate system used in a bird's-eye view;

[0026] The rigid transformation matrix determination module is used to: calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view;

[0027] The position coordinate conversion module is also used for: transforming the position coordinates obtained by the radar using a rigid transformation matrix to obtain the position coordinates of the position coordinates obtained by the radar in a coordinate system used in the bird's-eye view;

[0028] The registration module is used to: register the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view with the position coordinates of the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view;

[0029] The conversion relationship determination module is used to determine the conversion relationship between the camera device and the radar according to the conversion relationship of the position coordinates for achieving registration and the position coordinates after registration.

[0030] On the basis of the above solution, the system for calibrating the conversion relationship between two roadside sensors of the present invention can also be improved as follows.

[0031] Furthermore, the position coordinate conversion module is specifically used for:

[0032] The first transformation formula is used to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used in the bird's-eye view. The first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, i is a positive integer.

[0033] Furthermore, the rigid transformation matrix determination module is specifically used for:

[0034] The ICP algorithm or the CPD algorithm is used to calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

[0035] Furthermore, the conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

[0036] 3) In a third aspect, the present invention further provides an electronic device, comprising a processor, the processor being coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so that the electronic device implements any one of the above-mentioned methods for calibrating the conversion relationship between two roadside sensors.

[0037] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is loaded and executed by a processor so that a computer implements any one of the above-mentioned methods for calibrating the conversion relationship between two roadside sensors.

[0038] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0040] Figure 1 A schematic flow chart of a method for calibrating a conversion relationship between two roadside sensors according to an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the structure of a system for calibrating a conversion relationship between two roadside sensors according to an embodiment of the present invention;

[0042] Figure 3 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0044] like Figure 1 As shown, a system for calibrating a conversion relationship between two roadside sensors according to an embodiment of the present invention includes the following steps:

[0045] S1. After time alignment of the camera device and the radar, the position coordinates of each preset target in the corresponding coordinate system are monitored at the same time by using the camera device and the radar;

[0046] After the camera and the radar are time-aligned, the image monitored by the camera can be detected by the YOLO algorithm or the ViD algorithm, and N targets are detected. Since the YOLO algorithm or the ViD algorithm will output the detection frame of the target, the bottom center point of the detection frame of the target is used as the position coordinate. The target point is detected by the radar, and the position coordinate of the target can be directly obtained. A total of M targets are detected.

[0047] The same multiple targets can be selected from N targets and M targets as preset targets. For example, n preset targets are selected. When N=M, all targets can be used as preset targets, and at this time N=M=n.

[0048] Among them, the area for target tracking using the camera device and the radar can be set according to actual conditions, such as a designated road area.

[0049] Among them, the position coordinates of the target obtained by the camera device are the coordinates in the coordinate system used by the camera device, and the position coordinates of the target obtained by the radar are the coordinates in the coordinate system used by the radar. The coordinate system used by the camera device can be established according to actual conditions, and the coordinate system used by the radar can also be established according to actual conditions.

[0050] The imaging device may be a camera, the radar may be a millimeter wave radar, or other radars may be selected according to actual conditions.

[0051] For ease of expression, the set of position coordinates of each preset target in the corresponding coordinate system (the coordinate system used by the camera device) monitored by the camera device is recorded as P1, and the set of position coordinates of each preset target in the corresponding coordinate system (the coordinate system used by the radar) monitored by the radar is recorded as R1.

[0052] S2. Based on the first transformation matrix, perform inverse perspective projection transformation on the image monitored by the camera device at the same time to obtain a bird's-eye view, and use the first transformation matrix to transform the position coordinates obtained by the camera device to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view;

[0053] The first transformation matrix can be obtained in advance by selecting four pairs of points.

[0054] The method of transforming the position coordinates obtained by the camera device using the first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view includes:

[0055] The first transformation formula is used to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used in the bird's-eye view. The first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, i is a positive integer.

[0056] S3, calculating a rigid transformation matrix between a coordinate system used by the radar and a coordinate system used by the bird's-eye view, and using the rigid transformation matrix to transform the position coordinates obtained by the radar to obtain the position coordinates of the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view;

[0057] The ICP algorithm (Iterative closest point) or CPD (Coherent point drift) algorithm is used to calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

[0058] The rigid transformation matrix is ​​used to transform the position coordinates obtained by the radar to obtain the position coordinates of the position coordinates obtained by the radar in the coordinate system used in the bird's-eye view. This is specifically achieved through the second transformation formula. The second transformation formula is: 2i =H2R 1i , where H2 represents the rigid transformation matrix, R 1i represents the position coordinates of the i-th preset target in R1 in the coordinate system used by the radar, R 2i Indicates: R 1i The coordinates of the position in the coordinate system used by the Bird's Eye view.

[0059] S4, aligning the position coordinates in the coordinate system used for the bird's-eye view obtained by the camera device with the position coordinates in the coordinate system used for the bird's-eye view obtained by the radar;

[0060] For ease of description, the set of position coordinates obtained by radar in the coordinate system used in the bird's-eye view is denoted as R2, which includes R 11 , R 12 …R 1i …R 1n , the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view are marked as P2, P2 includes P 11 , P 12 …P 1i …P 1n .

[0061] Among them, S4 specifically includes:

[0062] S40, record the bird's-eye view (also called BEV view) obtained in S2 as I BEV , I BEV The coordinate system used is proportional to the geodetic coordinate system. BEVThe transformation matrix between the used coordinate system and the geodetic coordinate system can be obtained in advance by calibration. The transformation matrix is ​​recorded as the second transformation matrix. The second transformation matrix is ​​used to transform each position coordinate in P2 (the position coordinate obtained by the camera device in the coordinate system used for the bird's-eye view) to obtain the position coordinate of each position coordinate in P2 in the geodetic coordinate system, which is specifically implemented by the following formula:

[0063] C 2i =H3P 2i

[0064] Among them, H3 represents the second transformation matrix, P 2i represents the position coordinates of the i-th preset target in P2 in the coordinate system used in the bird's-eye view, C 2i Indicates: P 2i The coordinates of the location in the geodetic coordinate system.

[0065] The set of position coordinates of each position coordinate in P2 in the geodetic coordinate system is recorded as C2, and C2 includes C 11 , C 12 …C 1i …C 1n .

[0066] S41 , superimposing the detection frame of each preset target on the bird's-eye view according to the position coordinates of each preset target in C2 in the earth coordinate system.

[0067] S42, multiplying each position coordinate in R1 by the coefficient scale, so as to transform each position coordinate in R1 into the coordinate system used in the bird's-eye view, which is specifically achieved by the following formula:

[0068] R 3i =scale×R 1i

[0069] Among them, R 3i It means that R is multiplied by the coefficient scale. 1i Convert to the position coordinates in the coordinate system used by the bird's-eye view, transform each position coordinate in R1 to the set of position coordinates in the coordinate system used by the bird's-eye view by means of a scale coefficient, and record it as R3. The position coordinates in R3 can be displayed on the bird's-eye view.

[0070] S43. Rotate and / or translate (including horizontal movement and / or vertical movement) each position coordinate in R3 to make each position coordinate in R3 coincide with the detection frame where each position coordinate in C2 superimposed on the bird's-eye view is located, so as to achieve registration between the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view and the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view.

[0071] Among them, the position coordinates of each preset target in C2 in the earth coordinate system and each position coordinate in R3 are the registered position coordinates.

[0072] In another embodiment, S4 can be implemented manually.

[0073] S5. Determine the conversion relationship between the camera device and the radar according to the conversion relationship of the position coordinates for achieving the registration and the position coordinates after the registration.

[0074] The conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

[0075] S5 specifically includes:

[0076] S50, recording the rotation angle and translation displacement when the registration is achieved, and obtaining the adjusted position coordinates corresponding to each position coordinate in R3, and recording the set of all adjusted position coordinates as R4.

[0077] S51, deriving the conversion relationship between any position coordinate in R3 and the adjusted position coordinate corresponding to the position coordinate (i.e., the conversion relationship of the position coordinate for achieving registration) as follows:

[0078]

[0079] Among them, the coordinates of any position in R3 are (x r3 ,y r3 ), (x r3 ,y r3 ) corresponds to the adjusted position coordinates in R4 as (x r4 ,y r4 ), θ represents the rotation angle, Δx represents the horizontal displacement, and Δy represents the vertical displacement.

[0080] S52, using formula C 2i =H3P 2i , R 3i =scale×R 1i and The homography transformation matrix H used to characterize the transformation relationship can be obtained:

[0081]

[0082] Among them, h 00 、h 01 、h 02 、h 10 、h 11 、h 12 、h 20 、h 21 and h 22 is the element in the homography transformation matrix H.

[0083] At this time, the camera and radar are used to track the target. The position coordinates of the same target monitored at any same time in the corresponding coordinate system satisfy:

[0084]

[0085] Among them, the position coordinates in the coordinate system used by the camera device are (u, v), the position coordinates in the coordinate system used by the radar are (x, y), and (u, v) and (x, y) are: the position coordinates of the same target monitored by the camera device and radar at any same time after time alignment, and s represents the coefficient of the homogeneous coordinates.

[0086] Through the homography transformation matrix H, the position coordinates in the coordinate system used by the camera device can be accurately converted to the coordinate system used by the radar, and the position coordinates in the coordinate system used by the second preset roadside sensor can also be accurately converted to the coordinate system used by the camera device.

[0087] The beneficial effects of a method for calibrating a conversion relationship between two roadside sensors provided by the present invention are as follows:

[0088] The position coordinates of the target obtained by any roadside sensor are the coordinates of the coordinate system used by the roadside sensor. In practical applications, the position coordinates of the targets obtained by different roadside sensors need to be converted. Since the conversion between the position coordinates of the targets obtained by any two roadside sensors needs to be completed through the conversion relationship (homography transformation matrix) between the two roadside sensors, the accuracy of the conversion relationship (homography transformation matrix) determines whether the conversion of the position coordinates is accurate. The present invention aligns the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view with the position coordinates of the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view. A more accurate transformation relationship (homography transformation matrix) can be obtained, and the process of obtaining the transformation relationship (homography transformation matrix) is simpler and low-cost than the prior art (for example: the invention patent with publication number "CN112836737A" and subject name "A roadside combined perception device online calibration method based on vehicle-road data fusion", and the invention patent with publication number "WO2022206978A1" and subject name "A roadside millimeter-wave radar calibration method based on a vehicle-mounted positioning device"). Through the obtained transformation relationship (homography transformation matrix), the precise conversion of position coordinates in different coordinate systems (coordinate systems used by different roadside sensors) can be achieved.

[0089] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0090] like Figure 2 As shown, a system 200 for calibrating a conversion relationship between two roadside sensors according to an embodiment of the present invention includes a position coordinate acquisition module 201, a bird's-eye view acquisition module 202, a position coordinate conversion module 203, a rigid transformation matrix determination module 204, a registration module 205 and a conversion relationship determination module 206;

[0091] The position coordinate acquisition module 201 is used to: after time alignment of the camera device and the radar, use the camera device and the radar to monitor and obtain the position coordinates of each preset target in the corresponding coordinate system at the same time;

[0092] The bird's-eye view acquisition module 202 is used to: perform inverse perspective projection transformation on the image monitored by the camera device at the same time based on the first transformation matrix to obtain a bird's-eye view;

[0093] The position coordinate conversion module 203 is used to: transform the position coordinates obtained by the camera device using a first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in a coordinate system used in a bird's-eye view;

[0094] The rigid transformation matrix determination module 204 is used to: calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view;

[0095] The position coordinate conversion module 203 is further used to: transform the position coordinates obtained by the radar using a rigid transformation matrix to obtain the position coordinates of the position coordinates obtained by the radar in a coordinate system used in the bird's-eye view;

[0096] The registration module 205 is used to: register the position coordinates of the position coordinates obtained by the camera device in the coordinate system used for the bird's-eye view with the position coordinates of the position coordinates obtained by the radar in the coordinate system used for the bird's-eye view;

[0097] The conversion relationship determination module 206 is used to determine the conversion relationship between the camera device and the radar using the aligned position coordinates.

[0098] Optionally, in the above technical solution, the position coordinate conversion module 203 is specifically used for:

[0099] The first transformation formula is used to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used in the bird's-eye view. The first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, i is a positive integer.

[0100] Optionally, in the above technical solution, the rigid transformation matrix determination module 204 is specifically used for:

[0101] The ICP algorithm or the CPD algorithm is used to calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

[0102] Optionally, in the above technical solution, the conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

[0103] It should be noted that the beneficial effects of the system 200 for calibrating the conversion relationship between two roadside sensors provided by the above embodiment are the same as the beneficial effects of the method for calibrating the conversion relationship between two roadside sensors, which will not be repeated here. In addition, when the system provided by the above embodiment realizes its functions, it only takes the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided by the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0104] like Figure 3 As shown, an electronic device 300 of an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any one of the above methods for calibrating the conversion relationship between two roadside sensors, specifically:

[0105] The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements any one of the methods for calibrating the conversion relationship between two roadside sensors provided in the above embodiments. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The electronic device 300 may also include other components for realizing the functions of the device, which will not be described in detail here. The electronic device may specifically be a computer, etc.

[0106] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any one of the above-mentioned methods for calibrating the conversion relationship between two roadside sensors.

[0107] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0108] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes any of the above methods for calibrating a conversion relationship between two roadside sensors.

[0109] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0110] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0111] Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0112] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for calibrating a conversion relationship between two roadside sensors, characterized in that: include: After the camera device and the radar are time-aligned, the position coordinates of each preset target in the corresponding coordinate system are monitored and obtained at the same time by using the camera device and the radar; Based on the first transformation matrix, performing inverse perspective projection transformation on the image monitored by the camera device at the same time to obtain a bird's-eye view, and using the first transformation matrix to transform the position coordinates obtained by the camera device to obtain position coordinates of the position coordinates obtained by the camera device in a coordinate system used by the bird's-eye view; Calculating a rigid transformation matrix between a coordinate system used by the radar and a coordinate system used by the bird's-eye view, and transforming a position coordinate obtained by the radar using the rigid transformation matrix to obtain position coordinates of the position coordinate obtained by the radar in the coordinate system used by the bird's-eye view; aligning the position coordinates of the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view with the position coordinates of the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view; The conversion relationship between the camera device and the radar is determined according to the conversion relationship of the position coordinates for achieving the registration and the position coordinates after the registration.

2. A method for calibrating a conversion relationship between two roadside sensors according to claim 1, characterized in that: Using the first transformation matrix to transform the position coordinates obtained by the camera device to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view includes: The position coordinates of the position coordinates obtained by the camera device in the coordinate system used in the bird's-eye view are obtained by using a first transformation formula, wherein the first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, i is a positive integer.

3. A method for calibrating the conversion relationship between two roadside sensors according to claim 1 or 2, characterized in that: Calculating a rigid transformation matrix between a coordinate system used by the radar and a coordinate system used by the bird's-eye view includes: The ICP algorithm or the CPD algorithm is used to calculate and obtain a rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

4. A method for calibrating a conversion relationship between two roadside sensors according to claim 1 or 2, characterized in that: The conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

5. A system for calibrating the conversion relationship between two roadside sensors, characterized in that: It includes a position coordinate acquisition module, a bird's-eye view acquisition module, a position coordinate conversion module, a rigid transformation matrix determination module, a registration module and a conversion relationship determination module; The position coordinate acquisition module is used to: after time alignment of the camera device and the radar, use the camera device and the radar to monitor and obtain the position coordinates of each preset target in the corresponding coordinate system at the same time; The bird's-eye view acquisition module is used to: perform inverse perspective projection transformation on the image monitored by the camera device at the same time based on the first transformation matrix to obtain a bird's-eye view; The position coordinate conversion module is used to: transform the position coordinates obtained by the camera device using the first transformation matrix to obtain the position coordinates of the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view; The rigid transformation matrix determination module is used to: calculate the rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view; The position coordinate conversion module is further used to: transform the position coordinates obtained by the radar using the rigid transformation matrix to obtain the position coordinates of the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view; The registration module is used to: register the position coordinates of the position coordinates obtained by the camera device in the coordinate system used by the bird's-eye view with the position coordinates of the position coordinates obtained by the radar in the coordinate system used by the bird's-eye view; The conversion relationship determination module is used to determine the conversion relationship between the camera device and the radar according to the conversion relationship of the position coordinates for achieving registration and the position coordinates after registration.

6. A system for calibrating the conversion relationship between two roadside sensors according to claim 5, characterized in that: The position coordinate conversion module is specifically used for: The position coordinates of the position coordinates obtained by the camera device in the coordinate system used in the bird's-eye view are obtained by using a first transformation formula, wherein the first transformation formula is: 2i =H1P 1i , where H1 represents the first transformation matrix, P 1i Indicates: the position coordinates of the i-th preset target in P1 in the coordinate system used by the camera device, P 2i Indicates: P 1i The position coordinates in the coordinate system used in the bird's-eye view, P1, represent: a set of position coordinates of each preset target in the coordinate system used by the camera device monitored by the camera device, 1≤i≤n, i is a positive integer.

7. A system for calibrating the conversion relationship between two roadside sensors according to claim 5 or 6, characterized in that: The rigid transformation matrix determination module is specifically used for: The ICP algorithm or the CPD algorithm is used to calculate and obtain a rigid transformation matrix between the coordinate system used by the radar and the coordinate system used by the bird's-eye view.

8. A system for calibrating the conversion relationship between two roadside sensors according to claim 5 or 6, characterized in that: The conversion relationship between the camera device and the radar is represented by a homography transformation matrix.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, and at least one computer program is stored in the memory. The at least one computer program is loaded and executed by the processor so that the electronic device implements a method for calibrating the conversion relationship between two roadside sensors as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the computer implements a method for calibrating a conversion relationship between two roadside sensors as described in any one of claims 1 to 4.

Images