Calibration matrix determination method, device and computer readable storage medium

CN118643255BActive Publication Date: 2024-11-19ZHEJIANG DAHUA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411099370.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-19
Estimated Expiration
2044-08-12

AI Technical Summary

Technical Problem

In the prior art, the accuracy of target calibration is low and the error is large, which affects the safety of subsequent target tracking and monitoring, especially in complex traffic scenarios, it is difficult to achieve high-precision target calibration.

Method used

By combining the data of radar and video sensors, multiple calibration matrices are constructed, and calibration errors are calculated using the set of test points, and calibration matrix with the minimum error is selected as the target calibration matrix to achieve automated high-precision target calibration.

Benefits of technology

It improves the accuracy and efficiency of target calibration, is suitable for complex traffic scenarios, reduces the error rate of manual intervention, realizes fully automatic and high-precision coordinate conversion and calibration, and improves the accuracy of the overall target calibration matrix.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118643255B_ABST
    Figure CN118643255B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a calibration matrix determination method, device and computer-readable storage medium, including: obtaining first data in a first coordinate system acquired by a radar for a target scene, and second data in a second coordinate system acquired by a video sensor; calibrating the target scene based on the first data and the second data to obtain N calibration points and a test point set; selecting M calibration matrices from a plurality of calibration matrices constructed based on at least four calibration points of the N calibration points; testing the M calibration matrices through the test point set to obtain calibration errors of each calibration matrix determined based on the test point set; determining the calibration matrix corresponding to the minimum value of the calibration error as the target calibration matrix in the target scene. By determining the preferred calibration matrix in this way, calibration can be achieved more accurately, and the accuracy of calibration matrix determination can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a calibration matrix determination method, device and computer-readable storage medium. Background Art

[0002] In actual monitoring, identification or tracking scenarios, target calibration is often used to perform target monitoring, tracking and safety judgments. For example, radar-based target tracking monitoring technology and video sensor-based target tracking monitoring technology are gaining more and more attention.

[0003] With the rapid development of monitoring and recognition technology, the requirements for target calibration are becoming increasingly higher. However, the target calibration methods currently provided by related technologies often have problems such as low calibration accuracy and large errors, which makes it impossible to effectively and accurately track or monitor subsequent targets based on the calibration results, seriously affecting the safety of the scene. Summary of the invention

[0004] The embodiments of the present invention provide a calibration matrix determination method, device and computer-readable storage medium to achieve high-precision target calibration without manual intervention, thereby effectively improving the efficiency and accuracy of calibration matrix determination.

[0005] In a first aspect, an embodiment of the present invention provides a method for determining a calibration matrix, comprising:

[0006] Acquire first data in a first coordinate system obtained by collecting data of a target scene by a radar, and second data in a second coordinate system obtained by collecting data of the target scene by a video sensor; calibrate the target scene based on the first data in the first coordinate system and the second data in the second coordinate system to obtain N calibration points and a test point set; select M calibration matrices from a plurality of calibration matrices constructed based on at least four calibration points among the N calibration points, where N is an integer greater than or equal to 4, and M is an integer greater than or equal to 1; test the M calibration matrices through the test point set to respectively obtain calibration errors of each calibration matrix determined based on the test point set; determine the calibration matrix corresponding to the minimum value of the calibration error as the target calibration matrix under the target scene.

[0007] As an optional manner, the testing the M calibration matrices through the test point set to obtain calibration errors of the calibration matrices determined based on the test point set respectively further includes:

[0008] Based on the fact that each of the M calibration matrices includes test points, a calibration matrix including a number of test points not less than a threshold among the M calibration matrices is determined as a calibration matrix that needs to be tested; the calibration matrix that needs to be tested is tested, and calibration errors of each calibration matrix in the calibration matrix that needs to be tested, which is determined based on the test point set, are obtained respectively.

[0009] As an optional manner, the testing the M calibration matrices through the test point set to obtain calibration errors of the calibration matrices determined based on the test point set respectively includes:

[0010] Determine the sum of calibration errors of each calibration matrix in the calibration matrix to be tested based on each test point in the test point set; and determine the sum of calibration errors as the calibration error obtained by the corresponding calibration matrix based on the test point set.

[0011] As an optional manner, the testing the M calibration matrices through the test point set to obtain calibration errors of the calibration matrices determined based on the test point set respectively includes:

[0012] Determine a mean calibration error of each calibration matrix in the calibration matrix to be tested based on each test point in the test point set; determine the mean calibration error as the calibration error obtained by the corresponding calibration matrix based on the test point set.

[0013] As an optional manner, the selecting M calibration matrices from a plurality of calibration matrices constructed based on at least four calibration points among the N calibration points includes:

[0014] A calibration matrix is ​​constructed based on at least four calibration points among the N calibration points to generate H calibration matrices, where H is an integer greater than or equal to M; based on a matrix screening condition, the M calibration matrices are screened from the H calibration matrices; the matrix screening condition includes that the polygon area corresponding to the calibration matrix is ​​not less than a threshold area, and there are no calibration points with three collinear points among the calibration points corresponding to the calibration matrix.

[0015] As an optional manner, constructing a calibration matrix based on at least four calibration points of the N calibration points to generate H calibration matrices includes:

[0016] In an exhaustive manner, four calibration points are selected in turn from the N calibration points to construct a basic calibration matrix; for each basic calibration matrix, a new calibration matrix is ​​constructed based on at least one calibration point corresponding to the remaining calibration points; the basic calibration matrix and the redundant calibration matrix in the new calibration matrix corresponding to the basic calibration matrix are deleted to obtain the H calibration matrices.

[0017] As an optional manner, the target scene is calibrated based on the first data in the first coordinate system and the second data in the second coordinate system to obtain N calibration points and a test point set, including:

[0018] The first data in the first coordinate system is fused with the second data in the second coordinate system to obtain third data based on the fusion of the first coordinate system corresponding to the radar and the second coordinate system corresponding to the video sensor; the target scene is calibrated by the third data to obtain the N calibration points represented by the third data and the test point set represented by the third data.

[0019] As an optional manner, the fusing the first data in the first coordinate system with the second data in the second coordinate system to obtain third data based on the fusion of the first coordinate system corresponding to the radar and the second coordinate system corresponding to the video sensor includes:

[0020] Map the first data based on the first coordinate system to the second coordinate system to obtain the fourth data based on the second coordinate system; map the second data based on the second coordinate system to the first coordinate system to obtain the fifth data based on the first coordinate system; merge the fourth data and the fifth data to obtain the third data.

[0021] As an optional manner, the method further includes:

[0022] According to the distance between the test point and the reference line in the second coordinate system, a target weighted value is determined based on the correspondence between the distance and the weighted value; and according to the target weighted value, the coordinates of the test point in the second coordinate system are weighted and mapped to the coordinates in the first coordinate system.

[0023] In a second aspect, an embodiment of the present invention provides a method for determining a traffic calibration matrix, including:

[0024] Acquire first scene data in a radar coordinate system and second scene data in a video coordinate system respectively collected by a radar and a video sensor in a traffic scene; calibrate the traffic scene based on the first scene data and the second scene data to obtain N calibration points and a test point set based on the traffic scene; select at least four calibration points from the N calibration points multiple times to construct M calibration matrices in the traffic scene, where N is an integer greater than or equal to 4, and M is an integer greater than or equal to 1; test the M calibration matrices in the traffic scene through the test point set to obtain calibration errors of each calibration matrix determined based on the test point set; determine the calibration matrix corresponding to the minimum value of the calibration error as the target calibration matrix in the traffic scene.

[0025] In a third aspect, an embodiment of the present application provides a calibration matrix determination device, which includes modules / units / technical means for executing the method in the first aspect above.

[0026] Exemplarily, the device may include:

[0027] An acquisition module, used to acquire first data in a first coordinate system obtained by a radar collecting data on a target scene, and second data in a second coordinate system obtained by a video sensor collecting data on the target scene;

[0028] A determination module, configured to calibrate the target scene based on the first data in the first coordinate system and the second data in the second coordinate system to obtain N calibration points and a test point set; select M calibration matrices from a plurality of calibration matrices constructed based on at least four calibration points among the N calibration points, where N is an integer greater than or equal to 4, and M is an integer greater than or equal to 1;

[0029] A processing module is used to test the M calibration matrices through the test point set to obtain calibration errors of each calibration matrix determined based on the test point set; and determine the calibration matrix corresponding to the minimum calibration error as the target calibration matrix under the target scene.

[0030] As an optional manner, the processing module is further used for:

[0031] Based on the fact that each of the M calibration matrices includes test points, a calibration matrix including a number of test points not less than a threshold among the M calibration matrices is determined as a calibration matrix that needs to be tested; the calibration matrix that needs to be tested is tested, and calibration errors of each calibration matrix in the calibration matrix that needs to be tested, which is determined based on the test point set, are obtained respectively.

[0032] As an optional manner, the processing module is specifically used for:

[0033] Determine the sum of calibration errors of each calibration matrix in the calibration matrix to be tested based on each test point in the test point set; and determine the sum of calibration errors as the calibration error obtained by the corresponding calibration matrix based on the test point set.

[0034] As an optional manner, the processing module is specifically used for:

[0035] Determine a mean calibration error of each calibration matrix in the calibration matrix to be tested based on each test point in the test point set; determine the mean calibration error as the calibration error obtained by the corresponding calibration matrix based on the test point set.

[0036] As an optional manner, the determining module is specifically used for:

[0037] A calibration matrix is ​​constructed based on at least four calibration points among the N calibration points to generate H calibration matrices, where H is an integer greater than or equal to M; based on a matrix screening condition, the M calibration matrices are screened from the H calibration matrices; the matrix screening condition includes that the polygon area corresponding to the calibration matrix is ​​not less than a threshold area, and there are no calibration points with three collinear points among the calibration points corresponding to the calibration matrix.

[0038] As an optional manner, the determining module is specifically used for:

[0039] In an exhaustive manner, four calibration points are selected in turn from the N calibration points to construct a basic calibration matrix; for each basic calibration matrix, a new calibration matrix is ​​constructed based on at least one calibration point corresponding to the remaining calibration points; the basic calibration matrix and the redundant calibration matrix in the new calibration matrix corresponding to the basic calibration matrix are deleted to obtain the H calibration matrices.

[0040] As an optional manner, the determining module is specifically used for:

[0041] The first data in the first coordinate system is fused with the second data in the second coordinate system to obtain third data based on the fusion of the first coordinate system corresponding to the radar and the second coordinate system corresponding to the video sensor; the target scene is calibrated by the third data to obtain the N calibration points represented by the third data and the test point set represented by the third data.

[0042] As an optional manner, the processing module is specifically used for:

[0043] Map the first data based on the first coordinate system to the second coordinate system to obtain the fourth data based on the second coordinate system; map the second data based on the second coordinate system to the first coordinate system to obtain the fifth data based on the first coordinate system; merge the fourth data and the fifth data to obtain the third data.

[0044] As an optional manner, the processing module is further used for:

[0045] According to the distance between the test point and the reference line in the second coordinate system, a target weighted value is determined based on the correspondence between the distance and the weighted value; and according to the target weighted value, the coordinates of the test point in the second coordinate system are weighted and mapped to the coordinates in the first coordinate system.

[0046] In a fourth aspect, an embodiment of the present application provides a calibration matrix determination system, the system comprising a radar device, a video sensor device, a processor, and a memory;

[0047] The processor is used to: obtain first data in a first coordinate system obtained by a radar performing data collection on a target scene, and second data in a second coordinate system obtained by a video sensor performing data collection on the target scene; calibrate the target scene based on the first data in the first coordinate system and the second data in the second coordinate system to obtain N calibration points and a test point set; select M calibration matrices from a plurality of calibration matrices constructed based on at least four calibration points among the N calibration points, where N is an integer greater than or equal to 4, and M is an integer greater than or equal to 1; test the M calibration matrices through the test point set to respectively obtain calibration errors of each calibration matrix determined based on the test point set; and determine the calibration matrix corresponding to the minimum value of the calibration error as the target calibration matrix under the target scene.

[0048] As an optional manner, the radar device is used to collect data on the target scene to obtain first data in a first coordinate system; the video sensor is used to collect data on the target scene to obtain second data in a second coordinate system.

[0049] In a fifth aspect, the present application provides an electronic device, comprising: at least one processor; and a memory and a communication interface communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored in the memory, so that the device executes the first aspect or any optional implementation of the first aspect.

[0050] In a sixth aspect, a computer-readable storage medium is provided for storing instructions, which, when executed, enable the above-mentioned first aspect or any optional implementation of the first aspect.

[0051] In the seventh aspect, the application provides a computer program product, which includes: a computer program code, when the computer program code is run on a computer, it enables the computer to execute the above-mentioned first aspect or any optional implementation of the first aspect.

[0052] The beneficial effects of the present invention are as follows:

[0053] The embodiment of the present invention can perform target calibration more accurately and quickly by calculating the calibration error of multiple calibration matrices based on the test set, thereby determining the preferred calibration area, and can effectively improve the calibration accuracy. It can be applied to a variety of complex target calibration scenarios, for example, it is applicable to the calibration of complex traffic scenarios such as channelized lanes, curves, tunnels, highways, and intersections. In addition, based on the embodiment of the present application, it can combine the calibration matrix with the error calculation results of the test point set to achieve automatic calibration, effectively reducing the error rate of manually determining the target calibration matrix, so that the obtained target calibration matrix has higher accuracy and stronger adaptability, and better reduces the intervention of the manual calibration matrix. In addition, the embodiment of the present application also provides a calibration method based on the fusion of different sensors. For example, based on the importance of the calibration accuracy of the combination of radar and video to data fusion, a more flexible and accurate radar-visual integrated coordinate fusion method is provided, which can realize the fully automatic high-precision calibration of the coordinate transformation relationship of different sensors without manual intervention, improve the work efficiency of the coordinate transformation relationship calibration, and help improve the accuracy of the overall target calibration matrix. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Schematic diagram of data calibration / fusion in radar video intelligent monitoring provided for related technologies;

[0055] Figure 2 A schematic diagram of a calibration matrix determination method provided in an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of video and radar calibration in a radar-vision integrated device provided by an embodiment of the present invention;

[0057] Figure 4 A schematic diagram of a multi-point calibration optimization process provided by an embodiment of the present invention;

[0058] Figure 5 A schematic diagram of a scenario requiring determination of a target calibration matrix provided by an embodiment of the present invention;

[0059] Figure 6A schematic diagram of a block division of a calibration area provided by an embodiment of the present invention;

[0060] Figure 7 A schematic diagram of the structure of a scene calibration device provided in an embodiment of the present invention;

[0061] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described below with reference to the accompanying drawings and examples. However, the exemplary embodiments can be implemented in various forms and should not be understood as being limited to the embodiments described herein; on the contrary, these embodiments are provided to make the present invention more comprehensive and complete, and to fully convey the concepts of the exemplary embodiments to those skilled in the art. The same figure marks in the figures represent the same or similar structures, and thus their repeated descriptions will be omitted. The words expressing position and direction described in the present invention are all explained using the accompanying drawings as examples, but changes can be made as needed, and the changes made are all included in the scope of protection of the present invention. The drawings of the present invention are only used to illustrate the relative position relationship and do not represent the true proportions.

[0063] It should be noted that specific details are explained in the following description in order to fully understand the present invention. However, the present invention can be implemented in a variety of other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation methods disclosed below. The subsequent description of the specification is a preferred implementation method for implementing the present application, but the description is for the purpose of illustrating the general principles of the present application and is not intended to limit the scope of the present application. The scope of protection of the present application shall be determined as defined by the attached claims. The following is a specific description of the traffic calibration matrix determination method, device and computer-readable storage medium provided in an embodiment of the present invention in conjunction with the accompanying drawings.

[0064] In actual monitoring, identification or tracking scenarios, target calibration is often used to perform target monitoring, tracking and safety judgments. For example, radar-based target tracking monitoring technology and video sensor-based target tracking monitoring technology are gaining more and more attention.

[0065] With the rapid development of monitoring and recognition technology, the requirements for target calibration are becoming increasingly higher. However, the methods for determining the optimal calibration matrix provided by the current related technologies often have the problems of low calibration accuracy and large errors, which results in the inability to effectively and accurately perform subsequent target tracking or monitoring based on the selected calibration matrix, seriously affecting the safety of the scene.

[0066] In addition, in order to further improve the probability of target detection and recognition, target detection is often performed by combining radar and video sensors. For example, the advantage of radar is that it can obtain accurate spatial information and motion information of the target, and the advantage of video sensors is the high-precision target recognition accuracy. By effectively fusing radar data and video sensor data, higher target recognition accuracy, motion information and spatial position information can be obtained.

[0067] Among them, the relevant technology is based on the process of combining radar and video sensors to detect targets, such as Figure 1 The figure below is a schematic diagram of radar-video collaborative monitoring. The left picture is a video monitoring scene, and the right picture is a radar monitoring scene. There are three targets in the left picture (video monitoring scene) and four targets in the right picture (radar monitoring area). Among them, the analysis of the radar target trajectory shows that the behavior of target D is somewhat dangerous (driving on the line and curves). However, the radar device cannot identify the target, that is, the radar can detect dangerous behavior of the target, but does not know who the target is. At this time, it is necessary to associate / fuse the radar target and the video target for better target monitoring.

[0068] However, when radar and video are combined for target detection, the calibration error is often large and target recognition is ambiguous during the calibration / fusion process due to the inconsistency between radar data and video data. Figure 1 In the scenario shown, during calibration, video target 3 may be associated with radar target D, and a non-motor vehicle may be considered to be in dangerous behavior, while the correct association is that video target 1 is associated with radar target D, which is a wrong association result. Subsequent event warnings and processing, such as sound warnings or license plate capture, all depend on the correct radar-visual calibration / data fusion results. This shows the importance of calibration accuracy based on the combination of radar and video to data fusion.

[0069] Based on this, an embodiment of the present invention provides a calibration matrix determination method for realizing more accurate and efficient multi-point calibration of autonomous scenarios based on the combination of different acquisition devices. For example, in a traffic scenario, the calibration matrix determination method provided in the present application can be used to perform an overall calibration error calculation on multiple calibration matrices to be selected through a set of test points, and the calibration matrix with the smallest calibration error is determined as the target calibration matrix, that is, the preferred calibration matrix. Based on the preferred calibration matrix, the target calibration can be performed more accurately and quickly without human intervention and additional sensor assistance, thereby effectively improving the determination accuracy of the calibration matrix and the accuracy of subsequent target tracking based on the calibration matrix.

[0070] The types of acquisition devices used in the target calibration combined with different acquisition devices described in the embodiments of the present application may be two or more. The types of acquisition devices may be radar, video, or other sensors, etc. The embodiments of the present application do not make too many restrictions here. For example, the embodiments of the present application may include two different types of acquisition devices, namely radar devices or video sensor devices, wherein the radar and the video sensor may be installed integrally or separately, which is not limited here.

[0071] like Figure 2 As shown, the calibration matrix determination method provided by the embodiment of the present invention specifically includes the following steps:

[0072] S201: Acquire first data in a first coordinate system obtained by collecting data of a target scene by a radar.

[0073] Exemplarily, the embodiments of the present application can detect and track the target scene based on the radar, collect the number of targets and point cloud data, for example, obtain a set of target point cloud positions (x, y) in a first coordinate system corresponding to the radar.

[0074] S202: Acquire second data in a second coordinate system obtained by collecting data of the target scene by a video sensor.

[0075] Exemplarily, the embodiments of the present application can detect and track the target scene based on the video sensor, and acquire the position of the target frame, for example, obtain a target sequence and a corresponding target frame sequence in a second coordinate system corresponding to the video sensor.

[0076] S203: Calibrate the target scene based on the first data in the first coordinate system and the second data in the second coordinate system to obtain N calibration points and a test point set.

[0077] The N calibration points described in the embodiment of the present application can be obtained by manual calibration or by calibration in other ways, which is not limited here.

[0078] As an example, the N calibration points in the embodiment of the present application may be a calibration point set used to determine a calibration matrix, or a calibration point list, etc.

[0079] It is understandable that for different scenarios, for example, different road sections, the corresponding calibration point set will also change accordingly. Among them, the calibration points between different road sections may overlap, which does not affect the subsequent determination of the calibration matrix of the target road section based on the calibration point set in the embodiment of the present application.

[0080] As an example, in the embodiments of the present application, corresponding calibration point sets can be set based on different road sections or scenes.

[0081] As an example, the test point set in the embodiment of the present application can be determined for a vehicle in the target scene. For example, in a simplified scenario, the test point set in the embodiment of the present application can be formed by the coordinate data of the only vehicle in the target scene based on the integrated fusion of radar and vision. constitute.

[0082] As an example, when the embodiment of the present application calibrates the target scene based on the first data in the first coordinate system and the second data in the second coordinate system, the first data in the first coordinate system and the second data in the second coordinate system can be fused to obtain third data based on the fusion of the first coordinate system corresponding to the radar and the second coordinate system corresponding to the video sensor, and then the target scene is calibrated using the third data to obtain the N calibration points represented by the third data and a set of test points represented by the third data.

[0083] Exemplarily, an embodiment of the present application can map the first data based on the first coordinate system to the second coordinate system to obtain fourth data based on the second coordinate system, and map the second data based on the second coordinate system to the first coordinate system to obtain fifth data based on the first coordinate system, and then fuse the fourth data with the fifth data to obtain the third data.

[0084] S204: Selecting M calibration matrices from a plurality of calibration matrices constructed based on at least four calibration points among the N calibration points, where N is an integer greater than or equal to 4, and M is an integer greater than or equal to 1.

[0085] As an example, in an embodiment of the present application, a calibration matrix is ​​constructed based on at least four calibration points of the N calibration points, and H calibration matrices can be obtained, where H is an integer greater than or equal to M. Then, based on a matrix screening condition, the M calibration matrices can be screened from the H calibration matrices. Among them, a matrix screening condition provided in an embodiment of the present application may include that the area of ​​a polygon corresponding to the calibration matrix is ​​not less than a threshold area, and there are no calibration points with three points collinear among the calibration points corresponding to the calibration matrix.

[0086] Further, in the embodiment of the present application, H calibration matrices can be obtained based on the N calibration points in the following manner:

[0087] For example, an embodiment of the present application can select four calibration points from the N calibration points in sequence in an exhaustive manner to construct a basic calibration matrix, and then, for each basic calibration matrix, construct a new calibration matrix based on at least one calibration point corresponding to the remaining calibration points, and finally delete the redundant calibration matrices in the basic calibration matrix and the new calibration matrix obtained according to the basic calibration matrix to obtain the H calibration matrices.

[0088] S205: Testing the M calibration matrices through the test point set to obtain calibration errors of the respective calibration matrices determined based on the test point set.

[0089] As an example, the test points in the embodiments of the present application may be manually set; or, the test points may be obtained based on preset test point generation rules through a test point generation device, such as a test point generation model; or, the test points may be obtained by selecting part of the collected historical data as test points, for example, half of the collected historical data may be used for training, and half for testing.

[0090] As an example, the number of test points in the embodiment of the present application can be a fixed value; or, the number of test points in the embodiment of the present application can be set according to actual conditions. For example, the number of test points for a more complex road section can be greater than the number of test points corresponding to a simple road section.

[0091] S206: Determine a calibration matrix corresponding to the minimum calibration error as a target calibration matrix in the target scene.

[0092] As an example, in order to effectively reduce the complexity of determining the preferred calibration matrix and improve the efficiency of determining the preferred calibration matrix, when performing a calibration matrix test based on a test point set, the embodiment of the present application can screen out the calibration matrix that needs to be tested from the M calibration matrices.

[0093] Exemplarily, an embodiment of the present application can determine, based on the fact that each of the M calibration matrices includes test points, a calibration matrix among the M calibration matrices including a number of test points not less than a threshold as a calibration matrix that needs to be tested, and then test the calibration matrix that needs to be tested to obtain the calibration errors of each calibration matrix in the calibration matrix that needs to be tested, determined based on the test point set.

[0094] Exemplarily, the embodiment of the present application can determine the calibration matrix that includes one or more test points in the M calibration matrices as the calibration matrix that needs to be tested based on the fact that each calibration matrix in the M calibration matrices includes a test point, and then test the calibration matrix that needs to be tested to obtain the calibration errors of each calibration matrix in the calibration matrix that needs to be tested based on the test point set. For example, when the test point is exactly located on the common calibration line of adjacent calibration matrices.

[0095] As an example, there are multiple ways to obtain the calibration errors determined by each calibration matrix based on the test point set in the embodiment of the present application, which are not limited to the following ways:

[0096] Error determination 1: Determined based on the sum of calibration errors.

[0097] As an example, an embodiment of the present application can determine the sum of the calibration errors obtained by testing each test point in the test point set for each calibration matrix in the calibration matrix that needs to be tested, and then determine the sum of the calibration errors as the calibration error obtained by the corresponding calibration matrix based on the test point set.

[0098] Error determination 2: Determined based on the calibration error mean.

[0099] As an example, an embodiment of the present application can determine the mean calibration error of each calibration matrix in the calibration matrix that needs to be tested based on each test point in the test point set, and then determine the mean calibration error as the calibration error obtained by the corresponding calibration matrix based on the test point set.

[0100] It should be noted that the above process steps in the embodiment of the present application do not constitute a limitation on the embodiment of the present application. The present application may modify the above process steps in the embodiment of the present application according to actual conditions. Figure 2 The process steps are adaptively adjusted to be more suitable for actual scene applications. For example, the above steps S201 and S202 can be executed simultaneously.

[0101] Through the above method, the embodiment of the present invention calculates the calibration error of multiple calibration matrices based on the test set respectively, so as to determine the preferred calibration area, which can perform target calibration more accurately and quickly, effectively improve the calibration accuracy, and can be applicable to a variety of complex target calibration scenarios, for example, it is suitable for the calibration of complex traffic scenes such as channelized lanes, curves, tunnels, highways, intersections, etc., and based on the embodiment of the present application, it is possible to combine the calibration matrix and the error calculation results of the test point set to realize automatic calibration, effectively reducing the error rate of manually determining the target calibration matrix, so that the obtained target calibration matrix has higher accuracy and stronger adaptability, and better reduces the intervention of the manual calibration matrix.

[0102] Furthermore, in order to further improve the accuracy of calibration matrix determination, the embodiment of the present application also provides a method for determining the calibration matrix based on radar and video fusion when performing scene calibration, for example, when determining the calibration matrix for a traffic scene, which is described in detail below.

[0103] As an example, the test point positions in the test point set described in the embodiment of the present application can be determined in the following manner:

[0104] The embodiment of the present application can collect data on the test point based on the video sensor and the radar respectively, obtain the first coordinate based on the video coordinate system and the second coordinate based on the radar coordinate system corresponding to the test point, and then fuse the first coordinate with the second coordinate to obtain the third coordinate of the test point based on the fusion of radar and video. Finally, the third coordinate is used to represent the position of the test point.

[0105] Specifically, the embodiment of the present application fuses the first coordinate with the second coordinate to obtain the third coordinate of the test point based on radar and video fusion, including:

[0106] The first coordinate data of the test point based on the video coordinate system is mapped to data in the radar coordinate system to obtain fourth coordinate data, and the second coordinate data of the test point based on the radar coordinate system is mapped to data in the video coordinate system to obtain fifth coordinate data; finally, the fourth coordinate data and the fifth coordinate data are fused to obtain the third coordinate data of the test point based on the fusion of radar and video.

[0107] As an example, in order to better ensure the smoothness of the calibration straight line, the embodiment of the present application can map the first coordinate data in the video coordinate system to the data in the radar coordinate system at the test point. When the fourth coordinate data is obtained, the target weighted value is determined based on the distance between the test point and the reference line in the video coordinate system and the correspondence between the distance and the weighted value. Then, the first coordinate data in the video coordinate system is weighted and mapped to the data in the radar coordinate system according to the target weighted value.

[0108] For example, Figure 3 As shown, it is a schematic diagram of video and radar calibration / target tracking in a radar-vision integrated machine provided in an embodiment of the present application.

[0109] Among them, in the calibration phase, it is assumed that Figure 3The points ABCDEF shown in are multiple calibration points set in the current traffic scenario in the embodiment of the present application, and it is assumed that in the subsequent calibration application / target fusion link, the video intelligence and radar tracking algorithms can accurately detect the number and position of vehicles in the traffic scenario (excluding targets outside the sensor monitoring range and obstruction).

[0110] In the embodiment of the present application, the position of the corresponding target frame in the traffic scene can be given in real time based on the video tracking algorithm. The following target sequence and corresponding target frame sequence can be established:

[0111]

[0112] Among them, the above sequence Indicates the video target ID. Indicates the position of the video target frame. For details, see the following formula 1.

[0113] Formula 1

[0114] in, Represents the pixel coordinates of the upper left, upper right, lower left, and lower right points respectively.

[0115] Similarly, the radar can detect and track the number and point cloud data of targets in the traffic scene, and can establish the following target sequence and corresponding point cloud set sequence:

[0116]

[0117] Among them, the above sequence Indicates the radar target ID, Represents the radar target point cloud position (x, y) set.

[0118] Based on the data sets obtained from video and radar measurements, the following mapping relationship can be constructed:

[0119]

[0120] Among them, based on the contents of the above formulas 2 to 5, the embodiments of the present application Map the (x, y) coordinates in the radar coordinate system to the video coordinates (u, v) one by one. The video coordinates (u, v) are mapped one by one to (x, y) in the radar coordinate system.

[0121] Furthermore, the embodiment of the present application performs calibration matrix calculation based on radar coordinates and video coordinates.

[0122] Exemplarily, the present application embodiment is based on the data in the calibration data group. Point series and Point series, where Represents the point index, and constructs the homogeneous equation shown in the following formula 6:

[0123]

[0124] Then, by further transforming the above formula 6, we can get the following formulas 7 to 9:

[0125]

[0126] Among them, the above formulas 7 and 8 can be regarded as a constraint, and then mathematically the above formulas 7 and 8 can be equivalently processed into the following formulas 10 and 11:

[0127]

[0128] From the analysis of the right side of formula 10 and formula 11 above, we can understand that the above formula 10 and formula 11 focus on the proportional value, so we can let It is expressed by the following formula 12: It is expressed by the following formula 13:

[0129]

[0130] Through the above processing, the obtained formula 12 and formula 13 are the calculation formulas for realizing the image pixel points and radar coordinate points. Based on this, the coefficient matrix shown in the following formula 14 can be constructed:

[0131]

[0132] Among them, in the above formula 14 .

[0133] Based on the above formula 14, we know that there are 8 unknown quantities to be solved. In theory, only four pairs of marking points are needed to solve them. It is understandable that in practical applications, the more points the better. Here, for the convenience of calculation, it is only introduced as a simplified example. Based on this, using four pairs of marking points, the measurement equation shown in the following formula 15 can be constructed:

[0134]

[0135] Furthermore, the embodiment of the present application can simplify the above formula 15 by the following formula 16:

[0136]

[0137] Based on the above formula 16, the following formula 17 can be obtained:

[0138]

[0139] Based on this, the embodiment of the present application obtains the coefficient solution result and substitutes it into the above formula 12 and formula 13, that is, the conversion relationship between the image pixel points and the radar coordinate points.

[0140] Through the above method, the embodiment of the present application also provides a calibration method based on the fusion of different sensors. For example, based on the importance of calibration accuracy of the combination of radar and video to data fusion, a more flexible and accurate radar-vision integrated coordinate fusion method is provided, which can realize fully automatic high-precision calibration of coordinate transformation relationships of different sensors without manual intervention, thereby improving the work efficiency of coordinate transformation relationship calibration, and helping to improve the accuracy of the overall target calibration matrix.

[0141] In order to better understand the contents described in the embodiments of the present application, the following Figure 4 , further introduces the processes involved in the embodiments of the present application in detail.

[0142] S401: Initialize the relevant parameters of the scene calibration.

[0143] As an example, the embodiment of the present application can set the size of the threshold area in the matrix screening condition according to actual conditions, and the embodiment of the present application can set the straight line judgment threshold according to actual conditions, etc., which is not limited here.

[0144] S402: Configure multiple calibration points.

[0145] As an example, the embodiments of the present application may configure the calibration points by manual configuration or by other algorithm generation methods, which is not limited here.

[0146] As an example, the multiple calibration points configured in the embodiment of the present application may be a calibration point set used to determine a calibration matrix, or a calibration point list, etc.

[0147] It is understandable that for different scenarios, for example, different road sections, the corresponding calibration point set will also change accordingly. Among them, the calibration points between different road sections may overlap, which does not affect the subsequent determination of the calibration matrix of the target road section based on the calibration point set in the embodiment of the present application.

[0148] As an example, in the embodiments of the present application, corresponding calibration point sets can be set based on different road sections or scenes.

[0149] S403: Distributed exhaustive calculation of the calibration data set.

[0150] As an example, the calibration data group described in the embodiment of the present application can be understood as coordinate data information of the calibration matrix.

[0151] Exemplarily, assuming that a set of calibration data includes at least four pairs of (u, v) and (x, y) points, the set of calibration data can determine a unique calibration matrix.

[0152] See above Figure 3 As shown, the specific number of groups corresponding to the six calibration points ABCDEF included in the scene can be determined by the number and distribution of specific calibration points in the scene. Usually, the "distributed exhaustive" method can be used to determine the calibration data group.

[0153] For example, 4 calibration points are randomly selected from the 6 calibration points ABCDEF to determine the basic data set, and then it is determined whether there are 3 points in the basic data set that are collinear. If so, the basic data set is eliminated. Then, it is further determined whether the polygon area enclosed by each data set in the remaining basic data sets is less than the threshold area, and the basic data sets with an area less than the threshold area are eliminated. Then, a new set of calibration data sets is formed for any one of the remaining basic data sets based on at least one calibration point in the corresponding remaining calibration points.

[0154] S404: Merge the calibration data groups.

[0155] As an example, the embodiment of the present application may delete redundant calibration data groups in the calibration data groups obtained in the above step S403.

[0156] Among them, the possible calibration data groups of the six calibration points ABCDEF in this scenario are as follows:

[0157] Calibration data set 1: AFEB;

[0158] Calibration data set 2: ACDE;

[0159] Calibration data set 3: AFDC;

[0160] Calibration data set 4: AEDC;

[0161] Calibration data set 5: BEDC;

[0162] Calibration data group 6: ACDEF;

[0163] Calibration data set 7: ABCDEF.

[0164] It can be understood that the above 7 groups of calibration data groups may correspond to 7 calibration results or calibration matrices.

[0165] S405: Calculate the radar video calibration matrix.

[0166] As an example, step S405 may calculate the calibration matrix based on the radar and video fusion method described above.

[0167] S406: Extract test data.

[0168] As an example, an embodiment of the present application may assume that the current scene is a scene in an ideal environment. For example, in an equipment monitoring environment, there is only one car. Data is collected from the car to obtain a point column (u, v, x, y) corresponding to the car after radar and video fusion. The point column (u, v, x, y) can then be used as a test data set for subsequent calibration matrix testing.

[0169] S407: Optimizing the calibration matrix.

[0170] As an example, an embodiment of the present application can determine the sum of calibration errors of each calibration matrix in the calibration matrix where the test point sequence is located based on testing each test point in the test point sequence, and then determine the calibration matrix corresponding to the minimum value of the sum of the calibration errors as the preferred matrix.

[0171] For example, when calculating (x, y) based on (u, v), the embodiment of the present application may first calculate the nearest test point i of (u, v), and then calculate the nearest test point i based on the corresponding test point i. , Select the calibration matrix j with the smallest error.

[0172] See also Figure 5 As shown in the figure, for the test point, it can be seen that there are at least two calibration matrix options, namely the calibration matrix determined by AFEB and the calibration matrix determined by BEDC, where the test point column can be formed by the only vehicle in the ideal scenario. constitute.

[0173] It is assumed that the point sequence of the test point is as shown in the following formula 18:

[0174]

[0175] Here, m means there are m points in the test point column.

[0176] Furthermore, in the embodiment of the present application, based on the j-th calibration matrix, the calibration error obtained by testing the i-th test point can be expressed as the following formula 19:

[0177]

[0178] Then the sum of the calibration errors of the j-th calibration data group based on the test point sequence can be shown in the following formula 20:

[0179]

[0180] It can be understood that the embodiment of the present application can obtain the error of each calibration matrix at the position of the test point i through the above formula 19, and the sum of the calibration errors corresponding to each calibration matrix can be obtained through the above formula 20, so that the calibration matrix corresponding to the minimum sum of the calibration errors can be determined as the preferred matrix.

[0181] Furthermore, when a test point is completely located in a certain calibration matrix, in order to determine the target calibration matrix more quickly and conveniently, the calibration matrix where the test point is located may be determined as the corresponding target calibration matrix.

[0182] For example, Figure 6 As shown, it is assumed that the current traffic scene can be divided into three calibration areas, which are determined by ABEF, BEDC, and CDHG. If the test point, such as test point T1, is located in the image area determined by ABEF, the calibration matrix determined by ABEF can be used. When the test point, such as test point T2, is located in the image area determined by BCDE, the calibration matrix determined by BCDE can be used. The corresponding radar coordinate system also has a similar division to the image.

[0183] Furthermore, when the embodiment of the present application includes calculations across multiple calibration areas, the embodiment of the present application may also execute step S408, ie, smoothing the calibration straight line.

[0184] Exemplarily, an embodiment of the present application can determine a target weighted value based on the distance between the test point and the reference line in the second coordinate system and based on the correspondence between the distance and the weighted value, and then map the coordinates of the test point in the second coordinate system to the coordinates in the first coordinate system after weighting according to the target weighted value.

[0185] Based on the same inventive concept, Figure 7 As shown, an embodiment of the present invention further provides a calibration matrix determination device, including:

[0186] An acquisition module 701 is used to acquire first data in a first coordinate system obtained by a radar collecting data on a target scene, and second data in a second coordinate system obtained by a video sensor collecting data on the target scene;

[0187] A determination module 702 is used to calibrate the target scene based on the first data in the first coordinate system and the second data in the second coordinate system to obtain N calibration points and a test point set; select M calibration matrices from a plurality of calibration matrices constructed based on at least four calibration points of the N calibration points, where N is an integer greater than or equal to 4, and M is an integer greater than or equal to 1;

[0188] The processing module 703 is used to test the M calibration matrices through the test point set to obtain calibration errors of each calibration matrix determined based on the test point set; and determine the calibration matrix corresponding to the minimum calibration error as the target calibration matrix under the target scene.

[0189] As an optional manner, the processing module 703 is further configured to:

[0190] Based on the fact that each of the M calibration matrices includes test points, a calibration matrix including a number of test points not less than a threshold among the M calibration matrices is determined as a calibration matrix that needs to be tested; the calibration matrix that needs to be tested is tested, and calibration errors of each calibration matrix in the calibration matrix that needs to be tested, which is determined based on the test point set, are obtained respectively.

[0191] As an optional manner, the processing module 703 is specifically used for:

[0192] Determine the sum of calibration errors of each calibration matrix in the calibration matrix to be tested based on each test point in the test point set; and determine the sum of calibration errors as the calibration error obtained by the corresponding calibration matrix based on the test point set.

[0193] As an optional manner, the processing module 703 is specifically used for:

[0194] Determine a mean calibration error of each calibration matrix in the calibration matrix to be tested based on each test point in the test point set; determine the mean calibration error as the calibration error obtained by the corresponding calibration matrix based on the test point set.

[0195] As an optional manner, the determining module 702 is specifically configured to:

[0196] A calibration matrix is ​​constructed based on at least four calibration points among the N calibration points to generate H calibration matrices, where H is an integer greater than or equal to M; based on a matrix screening condition, the M calibration matrices are screened from the H calibration matrices; the matrix screening condition includes that the polygon area corresponding to the calibration matrix is ​​not less than a threshold area, and there are no calibration points with three collinear points among the calibration points corresponding to the calibration matrix.

[0197] As an optional manner, the determining module 702 is specifically configured to:

[0198] In an exhaustive manner, four calibration points are selected in turn from the N calibration points to construct a basic calibration matrix; for each basic calibration matrix, a new calibration matrix is ​​constructed based on at least one calibration point corresponding to the remaining calibration points; the basic calibration matrix and the redundant calibration matrix in the new calibration matrix corresponding to the basic calibration matrix are deleted to obtain the H calibration matrices.

[0199] As an optional manner, the determining module 702 is specifically configured to:

[0200] The first data in the first coordinate system is fused with the second data in the second coordinate system to obtain third data based on the fusion of the first coordinate system corresponding to the radar and the second coordinate system corresponding to the video sensor; the target scene is calibrated by the third data to obtain the N calibration points represented by the third data and the test point set represented by the third data.

[0201] As an optional manner, the processing module 703 is specifically used for:

[0202] Map the first data based on the first coordinate system to the second coordinate system to obtain the fourth data based on the second coordinate system; map the second data based on the second coordinate system to the first coordinate system to obtain the fifth data based on the first coordinate system; merge the fourth data and the fifth data to obtain the third data.

[0203] As an optional manner, the processing module 703 is further configured to:

[0204] According to the distance between the test point and the reference line in the second coordinate system, a target weighted value is determined based on the correspondence between the distance and the weighted value; and according to the target weighted value, the coordinates of the test point in the second coordinate system are weighted and mapped to the coordinates in the first coordinate system.

[0205] It should be understood that the embodiments of the calibration matrix determination device described above are merely illustrative. For example, the division of the modules is merely a logical function division. There may be other division methods in actual implementation, such as multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. The functional modules in the embodiments may be integrated into a processing module, or the modules may be physically present, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional units. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may be stored in a computer-readable storage medium.

[0206] Since the principle of solving the problem by the calibration matrix determination device is basically consistent with the calibration matrix determination method, the implementation of the calibration matrix determination device can refer to the implementation of the calibration matrix determination method, which will not be repeated here.

[0207] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, such as Figure 8 As shown, it includes: a processor 810 and a memory 820 for storing executable instructions of the processor 810;

[0208] The processor 810 is configured to execute the instructions to implement the calibration method of the different sensors.

[0209] In the specific implementation process, the device may have relatively large differences due to different configurations or performances, and may include one or more processors 810, memory 820, and computer-readable storage medium 830. The memory 820 and / or computer-readable storage medium 830 include one or more applications 831 or data 832. The memory 820 and / or computer-readable storage medium 830 may also include one or more operating systems 833, such as Windows, Mac OS, Linux, IOS, Android, Unix, FreeBSD, etc. Among them, the memory 820 and the computer-readable storage medium 830 may be short-term storage or persistent storage. The application 831 may include one or more modules ( Figure 8 ), each module may include a series of instruction operations. Further, the processor 810 may be configured to communicate with the computer-readable storage medium 830, and execute a series of instruction operations in the computer-readable storage medium 830 on the device. The device may also include one or more power supplies ( Figure 8 ); one or more network interfaces 840, wherein the network interface 840 includes a wired network interface 841 and / or a wireless network interface 842; and one or more input / output interfaces 843.

[0210] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program code runs on a computer, the computer implements the calibration matrix determination method.

[0211] Based on the same inventive concept, an embodiment of the present invention further provides a computer program product, which includes: computer program code, and when the computer program code is executed on a computer, the computer implements the calibration matrix determination method.

[0212] The calibration matrix determination method, device and computer-readable storage medium provided by the embodiment of the present invention firstly calculate the calibration error of multiple calibration matrices based on the test set to determine the preferred calibration area, which can more accurately and quickly perform target calibration, effectively improve the calibration accuracy, and can be applied to a variety of complex target calibration scenarios, for example, it is applicable to the calibration of complex traffic scenarios such as channelized lanes, curves, tunnels, highways, and intersections. Moreover, based on the embodiment of the present application, it can combine the calibration matrix with the error calculation results of the test point set to achieve automatic calibration, effectively reduce the error rate of manually determining the target calibration matrix, so that the obtained target calibration matrix has higher accuracy and stronger adaptability, and better reduces the intervention of the manual calibration matrix. In addition, the embodiment of the present application also provides a calibration method based on the fusion of different sensors. For example, based on the importance of the calibration accuracy of the combination of radar and video to data fusion, a more flexible and accurate radar-visual integrated coordinate fusion method is provided, which can realize the fully automatic high-precision calibration of the coordinate transformation relationship of different sensors without manual intervention, improve the work efficiency of the coordinate transformation relationship calibration, and help improve the accuracy of the overall target calibration matrix.

[0213] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0214] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0215] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0216] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0217] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for determining a calibration matrix, characterized in that: include: Acquire first data in a first coordinate system obtained by a radar collecting data on a target scene, and second data in a second coordinate system obtained by a video sensor collecting data on the target scene; Fusing the first data in the first coordinate system with the second data in the second coordinate system to obtain third data based on the fusion of the first coordinate system corresponding to the radar and the second coordinate system corresponding to the video sensor; Calibrate the target scene using the third data to obtain N calibration points represented by the third data and a test point set represented by the third data; Select four calibration points from the N calibration points in turn to construct a basic calibration matrix in an exhaustive manner; for each basic calibration matrix, construct a new calibration matrix based on at least one calibration point corresponding to the remaining calibration points; delete redundant calibration matrices in the basic calibration matrix and a new calibration matrix corresponding to the basic calibration matrix to obtain H calibration matrices; Based on the matrix screening condition, M calibration matrices are obtained by screening from the H calibration matrices; The matrix screening condition includes that the polygon area corresponding to the calibration matrix is ​​not less than the threshold area, and there are no calibration points with three points collinear among the calibration points corresponding to the calibration matrix, wherein N is an integer greater than or equal to 4, M is an integer greater than or equal to 1, and H is an integer greater than or equal to M; Testing the M calibration matrices through the test point set to obtain calibration errors of the calibration matrices determined based on the test point set; The calibration matrix corresponding to the minimum calibration error is determined as the target calibration matrix in the target scene.

2. The method according to claim 1, characterized in that The testing of the M calibration matrices by the test point set to obtain calibration errors of the calibration matrices determined based on the test point set also includes: Based on the fact that each of the M calibration matrices includes a test point, determining a calibration matrix including a number of test points not less than a threshold value among the M calibration matrices as a calibration matrix to be tested; The calibration matrices to be tested are tested, and calibration errors of the calibration matrices to be tested that are determined based on the test point set are obtained.

3. The method according to claim 2, characterized in that The testing the M calibration matrices by using the test point set to obtain calibration errors of the calibration matrices determined based on the test point set includes: Determine the sum of calibration errors of each calibration matrix in the calibration matrix to be tested based on each test point in the test point set; The sum of the calibration errors is determined as the calibration error obtained by the corresponding calibration matrix based on the test point set.

4. The method according to claim 2, characterized in that The testing the M calibration matrices by using the test point set to obtain calibration errors of the calibration matrices determined based on the test point set includes: Determine a calibration error mean value of each calibration matrix in the calibration matrix to be tested based on each test point in the test point set; The calibration error mean is determined as the calibration error obtained by the corresponding calibration matrix based on the test point set.

5. The method according to claim 1, characterized in that The fusing the first data in the first coordinate system with the second data in the second coordinate system to obtain third data based on the fusion of the first coordinate system corresponding to the radar and the second coordinate system corresponding to the video sensor includes: Mapping the first data based on the first coordinate system to the second coordinate system to obtain fourth data based on the second coordinate system; Mapping the second data based on the second coordinate system to the first coordinate system to obtain fifth data based on the first coordinate system; The fourth data and the fifth data are fused to obtain the third data.

6. The method according to claim 5, characterized in that The method further comprises: Determine a target weighted value according to the distance between the test point and the reference line in the second coordinate system and based on the corresponding relationship between the distance and the weighted value; The coordinates of the test point in the second coordinate system are weighted and mapped to the coordinates in the first coordinate system according to the target weighted value.

7. A method for determining a traffic calibration matrix, characterized in that: include: Acquire first scene data in a radar coordinate system and second scene data in a video coordinate system respectively collected by a radar and a video sensor in a traffic scene; Fusing the first data in the first coordinate system with the second data in the second coordinate system to obtain third data based on the fusion of the first coordinate system corresponding to the radar and the second coordinate system corresponding to the video sensor; Calibrate the target scene using the third data to obtain N calibration points represented by the third data and a test point set represented by the third data; Select four calibration points from the N calibration points in turn to construct a basic calibration matrix in an exhaustive manner; for each basic calibration matrix, construct a new calibration matrix based on at least one calibration point corresponding to the remaining calibration points; delete redundant calibration matrices in the basic calibration matrix and a new calibration matrix corresponding to the basic calibration matrix to obtain H calibration matrices; Based on the matrix screening condition, M calibration matrices are obtained by screening from the H calibration matrices; The matrix screening condition includes that the polygon area corresponding to the calibration matrix is ​​not less than the threshold area, and there are no calibration points with three points collinear among the calibration points corresponding to the calibration matrix, wherein N is an integer greater than or equal to 4, M is an integer greater than or equal to 1, and H is an integer greater than or equal to M; Testing the M calibration matrices in the traffic scenario through the test point set, and obtaining calibration errors of each calibration matrix determined based on the test point set; The calibration matrix corresponding to the minimum calibration error is determined as the target calibration matrix in the traffic scene.

8. A calibration matrix determination device, characterized in that: include: An acquisition module, used to acquire first data in a first coordinate system obtained by a radar collecting data on a target scene, and second data in a second coordinate system obtained by a video sensor collecting data on the target scene; a determination module, configured to fuse the first data in the first coordinate system with the second data in the second coordinate system to obtain third data based on the fusion of the first coordinate system corresponding to the radar and the second coordinate system corresponding to the video sensor; Calibrate the target scene using the third data to obtain N calibration points represented by the third data and a test point set represented by the third data; select four calibration points from the N calibration points in sequence to construct a basic calibration matrix in an exhaustive manner; for each basic calibration matrix, construct a new calibration matrix based on at least one calibration point corresponding to the remaining calibration points; delete redundant calibration matrices in the basic calibration matrix and the new calibration matrix obtained according to the basic calibration matrix to obtain H calibration matrices; Based on the matrix screening condition, M calibration matrices are obtained by screening from the H calibration matrices; The matrix screening condition includes that the polygon area corresponding to the calibration matrix is ​​not less than the threshold area, and there are no calibration points with three points collinear among the calibration points corresponding to the calibration matrix, wherein N is an integer greater than or equal to 4, M is an integer greater than or equal to 1, and H is an integer greater than or equal to M; A processing module is used to test the M calibration matrices through the test point set to obtain calibration errors of each calibration matrix determined based on the test point set; and determine the calibration matrix corresponding to the minimum calibration error as the target calibration matrix under the target scene.

9. An electronic device, characterized in that: include: a processor and a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the calibration matrix determination method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program code, and when the computer program code is executed on a computer, the computer implements the calibration matrix determination method according to any one of claims 1 to 6.