Radar and video calibration method and device and storage medium

Abstract

The invention discloses a radar and video calibration method and device and a storage medium, and belongs to the technical field of radar detection.The radar and video calibration method comprises the steps that a radar track and a video track of at least one target in an intersection area are obtained, the intersection area comprises a plurality of divided areas, and each divided area comprises a plurality of calibration blocks; determining a video-to-radar track corresponding to the video track according to the initial calibration parameter of each divided region; an association point group corresponding to each calibration block is determined from all radar tracks and video-to-radar tracks, and each association point group comprises a radar track point and a video-to-radar track point; and according to the association point groups corresponding to all the calibration blocks and the initial calibration parameters, determining the target calibration parameters corresponding to each divided region, thereby realizing accurate calibration between the radar and the video, and effectively improving the calibration precision and the calibration efficiency.

Description

Technical Field

[0001] The present application belongs to the field of radar detection technology, and in particular, relates to a radar and video calibration method, device and storage medium. Background Art

[0002] At present, in order to improve the operational efficiency, service level and safety of road traffic, more and more detection equipment are being deployed on the road.

[0003] The most common detection equipment for road traffic is visual sensors (such as cameras) and radars. With the gradual improvement of road traffic detection accuracy, traffic monitoring based on radar and video data fusion is inevitable. The key to achieving data fusion is the accurate calibration between radar and video. The current calibration methods on the market generally have the problems of low calibration accuracy and poor calibration effect, which in turn affects the data fusion effect of radar and video. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a radar and video calibration method, device and storage medium, which can achieve accurate calibration between radar and video and improve the calibration effect.

[0005] In a first aspect, the present application provides a radar and video calibration method, comprising:

[0006] Acquire a radar track and a video track of at least one target in an intersection area, wherein the intersection area includes a plurality of divided areas, and each of the divided areas includes a plurality of calibrated blocks;

[0007] Determine the video-to-radar trajectory corresponding to the video trajectory according to the initial calibration parameters of each of the divided areas;

[0008] Determine, from all the radar tracks and the video-to-radar tracks, a correlation point group corresponding to each calibration block, each of the correlation point groups including a radar track point and a video-to-radar track point;

[0009] According to the associated point groups corresponding to all the calibration blocks and the initial calibration parameters, target calibration parameters corresponding to each of the divided areas are determined.

[0010] In some embodiments, the radar and video calibration method further includes:

[0011] Determine the shape and area of ​​the lanes in the intersection area;

[0012] Dividing the area where the lane is located according to the shape to obtain the multiple divided areas;

[0013] Each of the divided areas is meshed, and the plurality of calibration blocks are selected from the meshed divided areas according to the positions of the vertex meshes.

[0014] In some embodiments, the radar and video calibration method further includes:

[0015] Determine a first coordinate of each vertex in each of the divided areas in the radar coordinate system and a second coordinate in the video coordinate system;

[0016] The initial calibration parameters of the corresponding divided area are determined according to all the first coordinates and the second coordinates corresponding to each divided area.

[0017] In some embodiments, determining the associated point group corresponding to each calibration block from all the radar tracks and the video-to-radar tracks includes:

[0018] Determine, from all the video-to-radar tracks, the associated tracks that are successfully associated with each of the radar tracks, and the corresponding number of successful associations;

[0019] According to the radar track, the corresponding association track and the number of association successes, a group of association points corresponding to each calibration block is determined.

[0020] In some embodiments, determining, from all the video-to-radar tracks, an associated track that is successfully associated with each of the radar tracks, and the corresponding number of successful associations, includes:

[0021] Determine the gate interval of the radar trajectory point at the current moment in the radar trajectory;

[0022] Capture all video-to-radar trajectory points that fall into the gate interval at the current moment;

[0023] When one of the video-to-radar trajectory points is captured, the video-to-radar trajectory where the video-to-radar trajectory point is located is used as an associated trajectory successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated;

[0024] When multiple video-to-radar trajectory points are captured, a target trajectory point is determined from the multiple video-to-radar trajectory points according to preset parameter weights and the radar trajectory point at the current moment; the video-to-radar trajectory where the target trajectory point is located is used as an associated trajectory that is successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated.

[0025] In some embodiments, determining the associated point group corresponding to each calibration block according to the radar track, the corresponding associated track and the number of successful associations includes:

[0026] Determine the radar track where the radar track point entering the current calibration block is located as the target radar track;

[0027] Obtaining the associated track that is successfully associated with the target radar track at the current moment, and the corresponding number of successful associations;

[0028] When the number of successful associations is not less than a preset number, the track points of the associated track and the track points of the target radar track at the current moment are used as the associated point group of the calibration block, and the next calibration block is selected as the current calibration block, and the step of returning to the step of determining the radar track where the radar track point entering the current calibration block is located as the target radar track is executed;

[0029] When the number of successful associations is less than the preset number, the radar track that enters the calibration block next time is determined as the target radar track, and the step of obtaining the associated track that is successfully associated with the target radar track at the current moment and the corresponding number of successful associations is returned to be executed.

[0030] In some embodiments, determining the target calibration parameter corresponding to each of the divided areas according to the associated point groups corresponding to all the calibration blocks and the initial calibration parameters includes:

[0031] Merging the radar track points in the associated point group corresponding to each of the divided areas to obtain a first point set of the corresponding divided area, and merging the video-to-radar track points in the associated point group corresponding to each of the divided areas to obtain a second point set of the corresponding divided area;

[0032] Determine a relationship matrix for each of the divided areas according to the first point set and the second point set;

[0033] According to the relationship matrix and the initial calibration parameters, target calibration parameters corresponding to each of the divided areas are determined.

[0034] In a second aspect, the present application provides a radar and video calibration device, comprising:

[0035] An acquisition unit, used to acquire a radar track and a video track of at least one target in an intersection area, wherein the intersection area includes a plurality of divided areas, and each of the divided areas includes a plurality of calibration blocks;

[0036] A first determining unit, configured to determine a video-to-radar trajectory corresponding to the video trajectory according to an initial calibration parameter of each of the divided areas;

[0037] A second determining unit is used to determine, from all the radar tracks and the video-to-radar tracks, a correlation point group corresponding to each of the calibration blocks, each of the correlation point groups including a radar track point and a video-to-radar track point;

[0038] The third determination unit is used to determine the target calibration parameter corresponding to each of the divided areas according to the associated point groups corresponding to all the calibration blocks and the initial calibration parameters.

[0039] In some embodiments, the radar and video calibration device further includes a division unit, which is used to:

[0040] Determine the shape and area of ​​the lanes in the intersection area;

[0041] Dividing the area where the lane is located according to the shape to obtain the multiple divided areas;

[0042] Each of the divided areas is meshed, and the plurality of calibration blocks are selected from the meshed divided areas according to the positions of the vertex meshes.

[0043] In some embodiments, the dividing unit is further configured to:

[0044] Determine a first coordinate of each vertex in each of the divided areas in the radar coordinate system and a second coordinate in the video coordinate system;

[0045] The initial calibration parameters of the corresponding divided area are determined according to all the first coordinates and the second coordinates corresponding to each divided area.

[0046] In some embodiments, the second determining unit is configured to:

[0047] Determine, from all the video-to-radar tracks, the associated tracks that are successfully associated with each of the radar tracks, and the corresponding number of successful associations;

[0048] According to the radar track, the corresponding association track and the number of association successes, a group of association points corresponding to each calibration block is determined.

[0049] In some embodiments, the second determining unit is configured to:

[0050] Determine the gate interval of the radar trajectory point at the current moment in the radar trajectory;

[0051] Capture all video-to-radar trajectory points that fall into the gate interval at the current moment;

[0052] When one of the video-to-radar trajectory points is captured, the video-to-radar trajectory where the video-to-radar trajectory point is located is used as an associated trajectory successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated;

[0053] When multiple video-to-radar trajectory points are captured, a target trajectory point is determined from the multiple video-to-radar trajectory points according to preset parameter weights and the radar trajectory point at the current moment; the video-to-radar trajectory where the target trajectory point is located is used as an associated trajectory that is successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated.

[0054] In some embodiments, the second determining unit is configured to:

[0055] Determine the radar track where the radar track point entering the current calibration block is located as the target radar track;

[0056] Obtaining the associated track that is successfully associated with the target radar track at the current moment, and the corresponding number of successful associations;

[0057] When the number of successful associations is not less than a preset number, the track points of the associated track and the track points of the target radar track at the current moment are used as the associated point group of the calibration block, and the next calibration block is selected as the current calibration block, and the step of returning to the step of determining the radar track where the radar track point entering the current calibration block is located as the target radar track is executed;

[0058] When the number of successful associations is less than the preset number, the radar track that enters the calibration block next time is determined as the target radar track, and the step of obtaining the associated track that is successfully associated with the target radar track at the current moment and the corresponding number of successful associations is returned to be executed.

[0059] In some embodiments, the third determining unit is used to:

[0060] Merging the radar track points in the associated point group corresponding to each of the divided areas to obtain a first point set of the corresponding divided area, and merging the video-to-radar track points in the associated point group corresponding to each of the divided areas to obtain a second point set of the corresponding divided area;

[0061] Determine a relationship matrix for each of the divided areas according to the first point set and the second point set;

[0062] According to the relationship matrix and the initial calibration parameters, target calibration parameters corresponding to each of the divided areas are determined.

[0063] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any of the above-mentioned radar and video calibration methods.

[0064] In a fourth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the radar and video calibration method described in any one of the above items is implemented.

[0065] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the radar and video calibration methods described above.

[0066] The radar and video calibration method, device, storage medium, electronic device and computer program product provided in the embodiments of the present application obtain the radar trajectory and video trajectory of at least one target in the intersection area, the intersection area includes multiple divided areas, each divided area includes multiple calibration blocks; determine the video-to-radar trajectory corresponding to the video trajectory according to the initial calibration parameters of each divided area; determine the associated point group corresponding to each calibration block from all radar trajectories and video-to-radar trajectories, each associated point group includes a radar trajectory point and a video-to-radar trajectory point; determine the target calibration parameters corresponding to each divided area according to the associated point groups and initial calibration parameters corresponding to all calibration blocks, that is, by dividing the intersection area and using the initial calibration parameters of each divided area and the associated point groups captured by each calibration block to calculate the target calibration parameters, so as to minimize the calibration error caused by the detection environment or installation of the radar-vision integrated machine, realize accurate calibration between radar and video, and effectively improve the calibration accuracy and calibration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0068] Figure 1 It is a flowchart of a radar and video calibration method provided in an embodiment of the present application;

[0069] Figure 2 It is a schematic diagram showing the demarcated blocks in the divided area provided in the embodiment of the present application;

[0070] Figure 3is another flow chart of the radar and video calibration method provided in an embodiment of the present application;

[0071] Figure 4 A schematic diagram of the structure of a radar and video calibration device provided in an embodiment of the present application;

[0072] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0073] Figure 6 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0074] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0075] The most common detection equipment for road traffic is visual sensors (such as cameras) and radars. With the gradual improvement of road traffic detection accuracy, traffic monitoring based on radar and video data fusion is inevitable. The key to achieving data fusion is the calibration between radar and video. Simply put, calibration is to establish the correspondence between radar data and video data, to achieve data alignment between radar and visual sensors, so that the data of radar and visual sensors can be compared and fused in the same coordinate system, thereby producing more accurate and consistent perception results.

[0076] The current calibration method for radar-visual integrated devices used for road detection on the market is to directly calculate the conversion matrix between radar data and video data based on the radar coordinates and video coordinates of the four vertices of the intersection area. The conversion matrix serves as the data correspondence between the radar and visual sensors (calibration parameters). Since only the coordinates of the four vertices of the entire intersection area are referenced during the calculation of the calibration parameters, it is difficult to overcome the errors caused by the detection environment (such as the shape of the intersection) or the equipment installation, and the calibration effect is poor.

[0077] To solve at least one of the above technical problems, the embodiments of the present application provide a radar and video calibration method, device, storage medium, electronic device and computer program product.

[0078] See also Figure 1 , Figure 11 is a flow chart of a radar and video calibration method provided in an embodiment of the present application. The radar and video calibration method is applied to an electronic device, and can be performed by an electronic device or a functional module or functional entity in the electronic device. The radar and video calibration method includes the following steps 101-104, wherein:

[0079] 101. Obtain a radar track and a video track of at least one target in an intersection area, where the intersection area includes a plurality of divided areas, and each divided area includes a plurality of calibrated blocks.

[0080] Among them, radar tracks and video tracks are composed of a series of continuous track points. A radar-visual integrated machine can be installed at the intersection to obtain the track points of the radar tracks and video tracks of each target passing through the intersection in real time. The targets mainly include various types of motor vehicles, non-motor vehicles, and moving objects such as people. The radar-visual integrated machine can collect real-time radar raw data and video raw data for the intersection area, and obtain stable and reliable radar tracks for each target by filtering the radar raw data, removing static points, and target identification and tracking. By performing target identification and tracking, filtering, and trajectory optimization on the video raw data, stable and reliable video tracks for each target are obtained.

[0081] In some embodiments, the radar and video calibration method further includes the following steps:

[0082] Determine the shape and area of ​​the lanes in the intersection area;

[0083] Divide the area where the lane is located according to the shape to obtain the multiple divided areas;

[0084] Each divided area is meshed, and the plurality of calibration blocks are selected from the meshed divided area according to the positions of the vertex meshes.

[0085] Among them, the detection range of the radar vision equipment (i.e. the intersection area) is spatially split according to the shape characteristics of the intersection where the radar vision integrated machine is installed (such as long straight intersections, curved intersections, sloping intersections, etc.), and the calibration parameters of the local area are subsequently determined based on the trajectory data in each split local area, rather than directly determining the overall calibration parameters based on the trajectory data in the entire intersection area. This can minimize the local errors caused by objective factors such as intersection curvature, distortion, ups and downs, etc.

[0086] Specifically, when performing spatial splitting, for a single-lane or multi-lane intersection area, first find the two side edges and stop lines of the outermost lanes, and take the two side edges and the area enclosed by the straight lines at a certain distance (such as 200 meters) from the stop line to the lane as the area where the lane is located, that is, the area that needs to be split. After that, split the area based on the shape of the intersection and the position of the stop line. For example, for a long straight intersection, starting from the stop line, along the extension direction of the lane, a rectangular area (i.e., a divided area) is divided every 60 meters, and for a curved intersection, a rectangular area is divided every 30 meters. In addition, to further improve the accuracy, the boundary lines of the divided areas can be manually fine-tuned.

[0087] Next, see Figure 2 , Figure 2 is a schematic diagram showing the calibration blocks in the divided area provided in the embodiment of the present application. When selecting the calibration blocks, the divided area is first converted into a grid, and then a suitable grid block is selected from the grid as the calibration block (for example Figure 2 The black grid block in the figure is the calibration block). In the subsequent calibration process, the radar trajectory points and video trajectory points passing through it are captured through the calibration block. In order to ensure a high quality of trajectory point capture, scattered and relatively typical grid blocks should be selected as calibration blocks as much as possible, so that a faster and more accurate calibration effect can be achieved with the least blocks in the future. Following the above-mentioned grid block selection principle, the two vertex grids at the top of the lane area can be found as the first selected calibration blocks. After that, the next calibration block is selected on the diagonal line of the calibration block until all the grid blocks on the diagonal line are selected, and then the left and right two are selected from the next row of grids as calibration blocks. Calibration blocks are continued to be selected on the diagonal line of these two calibration blocks. The above steps are repeated until calibration blocks are selected in each row of grids in the lane area.

[0088] 102. Determine a video-to-radar trajectory corresponding to the video trajectory according to initial calibration parameters of each divided area.

[0089] Among them, according to the initial calibration parameters of each divided area, the points of the video trajectory within it can be mapped to the radar coordinate system to obtain the video-to-radar trajectory.

[0090] In some embodiments, the radar and video calibration method further includes the following steps:

[0091] Determine a first coordinate of each vertex in each divided area in the radar coordinate system and a second coordinate in the video coordinate system;

[0092] According to all the first coordinates and the second coordinates corresponding to each divided area, the initial calibration parameters of the corresponding divided area are determined.

[0093] The video coordinate system is the image coordinate system, and the initial calibration parameters are the initial conversion matrix from the video coordinate system to the radar coordinate system. Different divided areas have different initial calibration parameters. The initial calibration parameters of each divided area can be calculated based on the radar coordinates (first coordinates) and video coordinates (second coordinates) of the vertices of the area. Assuming that the first coordinates of the four vertices corresponding to each divided area constitute matrix A, and the second coordinates of the four vertices constitute matrix B, the calculation formula of the initial conversion matrix C can be: A = BC.

[0094] It should be pointed out that by giving different divided areas different initial calibration parameters instead of using the same initial calibration parameters for the entire area, the problem of large calibration parameter errors caused by the side mounting angle of the radar vision integrated device and environmental influences can be avoided, which is conducive to improving the accuracy of subsequent calibration and adaptability to the environment.

[0095] In addition, due to the volatility and possibility of false detection of the detected target, a linear filtering algorithm can be used to compensate the video-to-radar trajectory (i.e., the video trajectory after the coordinate system conversion) to ensure the stability and reliability of the trajectory data. The linear filtering algorithm includes but is not limited to the Kalman filtering algorithm. The following is an introduction to the Kalman filtering algorithm, which mainly includes the following five formulas:

[0096] 1. Prior estimation

[0097]

[0098] 2. Calculate the error covariance

[0099]

[0100] 3. Calculate Kalman gain

[0101]

[0102] 4. Revise your estimates

[0103]

[0104] 5. Update error covariance

[0105]

[0106] Among them, x^ is the state matrix, A is the state transfer matrix, z k is the measurement equation, Q is the process error, and R is the measurement error. The process error Q and the measurement error are set manually, for example, Q is a 2*2 matrix, and the measurement error R is a 2*1 matrix. Their values ​​are fixed and need to be modified through algorithm debugging until the best effect is achieved.

[0107] State Matrix Among them, in the state matrix x^: x is the x-direction distance in the radar coordinate system (i.e., the x value of the video-to-radar trajectory point), y is the y-direction distance in the radar coordinate system (i.e., the y value of the video-to-radar trajectory point). Δx is the change in x under the cycle time. Δy is the change in y under the cycle time.

[0108] The state transfer matrix A is designed as a 4*4 matrix as follows, for example:

[0109]

[0110] Measurement equation z k The expression for z k =Hx k ;

[0111] Among them, z k is the observed value, H is the measurement matrix, x k is the measured value, which has the same structure as the state matrix and is the value actually observed by sensors (i.e., the coordinate value of the radar-to-video trajectory point).

[0112] 103. Determine a correlation point group corresponding to each calibration block from all radar tracks and video-to-radar tracks, each correlation point group including a radar track point and a video-to-radar track point.

[0113] In some embodiments, see Figure 3 , Figure 3 1 is another flow chart of the radar and video calibration method provided in an embodiment of the present application. The above step 103 may specifically include the following steps 1031 and 1032, wherein:

[0114] 1031. Determine, from all video-to-radar tracks, an associated track that is successfully associated with each radar track and a corresponding number of successful associations.

[0115] Among them, when performing radar and video calibration, it is necessary to calibrate based on the radar data and video data of the same target, so it is necessary to find the video track that belongs to the same target as the radar track, and each video-to-radar track is obtained by converting the video track into a coordinate system, and two tracks belonging to the same target can be considered as mutually related tracks, that is, it is necessary to find the associated track of each radar track in the end. Considering that the track is composed of track points at multiple consecutive moments collected in real time, whether the two tracks are successfully associated mainly refers to whether the two corresponding track points at a single moment (i.e., a radar track point and a video-to-radar track point) are successfully associated, and the number of successful associations is equal to the total number of track points in the two tracks that are successfully associated with each other.

[0116] In some embodiments, the correlation between trajectories may be determined by combining the gate technique and the nearest neighbor method to effectively improve the accuracy and efficiency of data association and target tracking, that is, the above step 1031 further includes:

[0117] Determine the gate interval of the radar track point at the current moment in the radar track;

[0118] Capture all video-to-radar trajectory points that fall into the gate interval at the current moment;

[0119] When a video-to-radar trajectory point is captured, the video-to-radar trajectory where the video-to-radar trajectory point is located is used as an associated trajectory that is successfully associated with the radar trajectory;

[0120] When multiple video-to-radar trajectory points are captured, the target trajectory point is determined from the multiple video-to-radar trajectory points according to the preset parameter weights and the radar trajectory point at the current moment; the video-to-radar trajectory where the target trajectory point is located is used as the associated trajectory that is successfully associated with the radar trajectory.

[0121] Among them, considering that there are multiple detection targets (such as multiple vehicles and multiple people) in the intersection area, the gates (gate intervals) of each target and their relationship with the observation points can be managed and represented in the form of a matrix. The establishment of the gate matrix helps to determine which observation points fall into the gates of which targets, thereby realizing effective data association. When establishing the gate matrix, it is usually necessary to use the measurement data of the previous moment to predict the position of each target at the current moment, and set the corresponding gates accordingly. Then, all observation points at the current moment are compared with the gates of each target, and the gate matrix is ​​updated according to whether the observation point falls into the gate. The gate is an area or space set according to certain criteria (such as statistical distance, probability distribution, etc.) with the predicted position of the tracked target (i.e., the coordinate position of the radar trajectory point) as the center, and is also called a gate interval. The gate matrix can be defined for each area in the intersection area. The gate matrix parameters include: trajectory position (i.e., x, y coordinate values), trajectory survival time tim (i.e., the number of successful associations), and type information type (i.e., the probability value of target types such as vehicles and pedestrians). The number of matrix parameters can be modified through the relevant server, and can be fine-tuned according to the initial calibration to avoid too many wrong point pairs. In addition, weights can be set for the above parameters, for example, the weight of x is 35, the weight of y is 25, the weight of tim is 30, and the weight of type is 10. These weights are mainly used to calculate the weighted average of the deviation between the video-to-radar track points and the radar track points that fall into the wave gate interval.

[0122] Each radar track point has a gate interval, and the video-to-radar track points falling into the gate interval need to be captured. If there is only one video-to-radar track point in the gate interval at the current moment, the radar track point and the video-to-radar track point are most likely the track points of the same target. The video-to-radar track corresponding to the video-to-radar track point can be directly used as the associated track of the radar track, and the id information of the video-to-radar track is recorded, and the number of successful associations with the radar track corresponding to the id is updated, so that the number of successful associations + 1. If there are multiple video-to-radar track points in the gate interval at the current moment, the deviation value between each video-to-radar track point and the radar track point is calculated according to the x value, y value, number of successful associations and type value of the video-to-radar track point and the radar track point, and then the corresponding weighted average value is calculated based on the set weight (preset parameter weight) and the deviation value, and the video-to-radar track point with the smallest weighted average value is selected as the target track point, and the video-to-radar track where the target track point is located is used as the associated track.

[0123] 1032. Determine the associated point group corresponding to each calibration block according to the radar track, the corresponding associated track and the number of successful associations.

[0124] In some embodiments, the above step 1032 specifically includes:

[0125] Determine the radar track where the radar track point entering the current calibration block is located as the target radar track;

[0126] Obtain the associated trajectory that is successfully associated with the target radar trajectory at the current moment, and the corresponding number of successful associations;

[0127] When the number of successful associations is not less than the preset number, the track points of the current associated track and the track points of the target radar track are used as the associated point group of the calibration block, and the next calibration block is selected as the current calibration block, and the step of returning to the above step of determining the radar track where the radar track point entering the current calibration block is located as the target radar track is executed;

[0128] When the number of successful associations is less than the preset number, the radar track that enters the calibration block next time is determined as the target radar track, and the process returns to the above step of obtaining the associated track that is successfully associated with the target radar track at the current moment, and the corresponding number of successful associations.

[0129] Among them, when the radar track enters any calibration block for the first time, the associated track (i.e., video to radar track) currently associated with it is queried from the record, and the number of successful associations between the radar track and the associated track is obtained. If the threshold is reached, the track point coordinates vx and vy of the current associated track and the track point coordinates lx and ly of the current radar track are obtained, and they are filled into the corresponding values ​​of the calibration block (i.e., these two track points are used as the associated point group of the calibration block), thereby completing the filling of the calibration block, and continuing to select the next calibration block for filling. If the threshold is not reached, the current radar track is discarded, and the next radar track entering the calibration block is waited for, and the calibration block is continued to be judged and filled. When the filling is completed, the next calibration block is selected for filling. In this way, the filling of all calibration blocks is completed.

[0130] 104. Determine the target calibration parameters corresponding to each divided area according to the associated point groups corresponding to all the calibration blocks and the initial calibration parameters.

[0131] In some embodiments, please see Figure 3 , the above step 104 specifically includes:

[0132] 1041. Merge the radar track points in the associated point group corresponding to each divided area to obtain a first point set of the corresponding divided area, and merge the video-to-radar track points in the associated point group corresponding to each divided area to obtain a second point set of the corresponding divided area.

[0133] 1042. Determine a relationship matrix for each divided area according to the first point set and the second point set;

[0134] 1043. Determine the target calibration parameters corresponding to each divided area according to the relationship matrix and the initial calibration parameters.

[0135] Among them, when all the calibration blocks are filled, the RANSAC (Random Sample Consensus) algorithm can be used to eliminate the erroneous filling data in each divided area, and finally form two related point sets, namely the first point set obtained by merging the radar trajectory points, and the second point set obtained by merging the video to radar trajectory points. After that, the overdetermined equations are solved to calculate the relationship matrix between the two point sets as the relationship matrix of the divided area.

[0136] Since the trajectory points in the first point set and the second point set of each divided area are captured by the dispersed and typical calibration areas, although the number of these captured trajectory points is small, the quality is high, so that a more accurate relationship matrix calculation can be achieved through a smaller number of points. When the initial calibration parameters are subsequently calibrated using the relationship matrix of each divided area, more accurate calibration parameters can be obtained, avoiding the impact of the installation of the integrated radar and vision equipment or other factors on the calibration of local parameters, thereby effectively improving the overall calibration speed while ensuring the overall calibration reliability.

[0137] For example, the relationship matrix of each partition area can be solved by the following formula:

[0138]

[0139] In the matrix, x' is the x-coordinate value of each point in the first point set, y' is the y-coordinate value of each point in the first point set, and w' defaults to 0. x is the x-coordinate value of each point in the second point set, y is the y-coordinate value of each point in the second point set, and z defaults to 0. h1-h9 are the element values ​​of the relationship matrix.

[0140] Afterwards, the initial calibration parameters of each divided area are adjusted through the relationship matrix to obtain the final target calibration parameters of the divided area. For example, for any divided area, the initial calibration parameters are multiplied by the relationship matrix, and the resulting matrix is ​​used as the target calibration parameter. At this point, the automatic calibration of the initial calibration parameters of each divided area is completed, and relatively accurate calibration parameters of each divided area are obtained. Subsequently, for the radar tracks and video tracks collected in real time by the radar-vision integrated device, the corresponding target calibration parameters can be selected based on the divided area to which each track belongs to realize spatial calibration and data fusion, which can effectively improve the fusion and anti-interference capabilities of radar and video data.

[0141] As can be seen from the above, the radar and video calibration method provided in the embodiment of the present application obtains the radar trajectory and video trajectory of at least one target in the intersection area, the intersection area includes multiple divided areas, and each divided area includes multiple calibration blocks; the video-to-radar trajectory corresponding to the video trajectory is determined according to the initial calibration parameters of each divided area; from all radar trajectories and video-to-radar trajectories, the associated point group corresponding to each calibration block is determined, and each associated point group includes a radar trajectory point and a video-to-radar trajectory point; according to the associated point groups and initial calibration parameters corresponding to all calibration blocks, the target calibration parameters corresponding to each divided area are determined, that is, by dividing the intersection area, and using the initial calibration parameters of each divided area and the associated point groups captured by each calibration block to calculate the target calibration parameters, so as to minimize the calibration error caused by the detection environment or installation of the radar-vision integrated machine, and realize accurate calibration between radar and video, which effectively improves the calibration accuracy and calibration efficiency.

[0142] According to the method described in the above embodiment, the present application embodiment also provides a radar and video calibration device, which is used to perform the steps in the above radar and video calibration method. Figure 4 , Figure 4 2 is a schematic diagram of the structure of a radar and video calibration device 200 provided in an embodiment of the present application. The radar and video calibration device 200 is applied to an electronic device, and the radar and video calibration method can be performed by the electronic device or a functional module or functional entity in the electronic device. Specifically, the radar and video calibration device 200 includes an acquisition unit 201, a first determination unit 202, a second determination unit 203, and a third determination unit 204, wherein:

[0143] An acquisition unit 201 is used to acquire a radar track and a video track of at least one target in an intersection area, where the intersection area includes a plurality of divided areas, each of which includes a plurality of calibrated blocks;

[0144] A first determining unit 202 is used to determine the video-to-radar trajectory corresponding to the video trajectory according to the initial calibration parameters of each divided area;

[0145] A second determining unit 203 is used to determine a correlation point group corresponding to each calibration block from all radar tracks and video-to-radar tracks, each correlation point group including a radar track point and a video-to-radar track point;

[0146] The third determining unit 204 is used to determine the target calibration parameters corresponding to each divided area according to the associated point groups corresponding to all the calibration blocks and the initial calibration parameters.

[0147] In some embodiments, the radar and video calibration device 200 further includes a division unit, which is used to:

[0148] Determine the shape and area of ​​the lanes in the intersection area;

[0149] Divide the area where the lane is located according to the shape to obtain the multiple divided areas;

[0150] Each divided area is meshed, and the plurality of calibration blocks are selected from the meshed divided area according to the positions of the vertex meshes.

[0151] In some embodiments, the partitioning unit is further configured to:

[0152] Determine a first coordinate of each vertex in each divided area in the radar coordinate system and a second coordinate in the video coordinate system;

[0153] According to all the first coordinates and the second coordinates corresponding to each divided area, the initial calibration parameters of the corresponding divided area are determined.

[0154] In some embodiments, the second determining unit 203 is used to:

[0155] Determine the associated tracks successfully associated with each radar track and the corresponding number of successful associations from all video-to-radar tracks;

[0156] According to the radar track, the corresponding associated track and the number of successful associations, the associated point group corresponding to each calibration block is determined.

[0157] In some embodiments, the second determining unit 203 is used to:

[0158] Determine the gate interval of the radar track point at the current moment in the radar track;

[0159] Capture all video-to-radar trajectory points that fall into the gate interval at the current moment;

[0160] When a video-to-radar trajectory point is captured, the video-to-radar trajectory where the video-to-radar trajectory point is located is used as an associated trajectory successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated;

[0161] When multiple video-to-radar trajectory points are captured, the target trajectory point is determined from the multiple video-to-radar trajectory points according to the preset parameter weights and the radar trajectory point at the current moment; the video-to-radar trajectory where the target trajectory point is located is used as the associated trajectory that is successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated.

[0162] In some embodiments, the second determining unit 203 is used to:

[0163] Determine the radar track where the radar track point entering the current calibration block is located as the target radar track;

[0164] Obtain the associated track that is successfully associated with the target radar track at the current moment, and the corresponding number of successful associations;

[0165] When the number of successful associations is not less than a preset number, the track points of the associated track and the track points of the target radar track at the current moment are used as the associated point group of the calibration block, and the next calibration block is selected as the current calibration block, and the step of returning to the step of determining the radar track where the radar track point entering the current calibration block is located as the target radar track is executed;

[0166] When the number of successful associations is less than the preset number, the radar track that enters the calibration block next time is determined as the target radar track, and the step of obtaining the associated track that is successfully associated with the target radar track at the current moment and the corresponding number of successful associations is returned to be executed.

[0167] In some embodiments, the third determining unit 204 is used to:

[0168] Merging the radar track points in the associated point group corresponding to each divided area to obtain a first point set of the corresponding divided area, and merging the video-to-radar track points in the associated point group corresponding to each divided area to obtain a second point set of the corresponding divided area;

[0169] Determine a relationship matrix for each divided area according to the first point set and the second point set;

[0170] According to the relationship matrix and the initial calibration parameters, the target calibration parameters corresponding to each divided area are determined.

[0171] It should be noted that the specific details of each module unit in the above-mentioned radar and video calibration device 200 have been described in detail in the embodiment of the above-mentioned radar and video calibration method, and will not be repeated here.

[0172] As can be seen from the above, the radar and video calibration device 200 provided in the embodiment of the present application obtains the radar trajectory and video trajectory of at least one target in the intersection area through the acquisition unit 201, and the intersection area includes multiple divided areas, each divided area includes multiple calibration blocks; the first determination unit 202 determines the video-to-radar trajectory corresponding to the video trajectory according to the initial calibration parameters of each divided area; the second determination unit 203 determines the associated point group corresponding to each calibration block from all radar trajectories and video-to-radar trajectories, and each associated point group includes a radar trajectory point and a video-to-radar trajectory point; the third determination unit 204 determines the target calibration parameters corresponding to each divided area according to the associated point groups and initial calibration parameters corresponding to all calibration blocks, that is, by dividing the intersection area, and using the initial calibration parameters of each divided area and the associated point group captured by each calibration block to calculate the target calibration parameters, so as to minimize the calibration error caused by the detection environment or installation of the radar-vision integrated machine, and achieve accurate calibration between radar and video, and effectively improve the calibration accuracy and calibration efficiency.

[0173] In some embodiments, the radar and video calibration device in the embodiments of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a server, which can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The embodiments of the present application are not specifically limited.

[0174] In some embodiments, Figure 5 As shown, an embodiment of the present application further provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, each process of the above-mentioned radar and video calibration method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0175] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0176] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0177] The electronic device 400 includes but is not limited to: a radio frequency unit 401, a network module 402, an audio output unit 403, an input unit 404, a sensor 405, a display unit 406, a user input unit 407, an interface unit 408, a memory 409, a processor 410 and other components.

[0178] In some embodiments, the processor 410 implements the following steps when executing the program:

[0179] Acquire a radar track and a video track of at least one target in an intersection area, the intersection area includes a plurality of divided areas, each divided area includes a plurality of calibrated blocks;

[0180] Determine the video-to-radar trajectory corresponding to the video trajectory according to the initial calibration parameters of each divided area;

[0181] Determine the associated point group corresponding to each calibration block from all radar tracks and video-to-radar tracks, each associated point group includes a radar track point and a video-to-radar track point;

[0182] According to the associated point groups and initial calibration parameters corresponding to all calibration blocks, the target calibration parameters corresponding to each divided area are determined.

[0183] In some embodiments, the radar and video calibration method further includes:

[0184] Determine the shape and area of ​​the lanes in the intersection area;

[0185] Divide the area where the lane is located according to the shape to obtain the multiple divided areas;

[0186] Each divided area is meshed, and the plurality of calibration blocks are selected from the meshed divided area according to the positions of the vertex meshes.

[0187] In some embodiments, the radar and video calibration method further includes:

[0188] Determine a first coordinate of each vertex in each divided area in the radar coordinate system and a second coordinate in the video coordinate system;

[0189] According to all the first coordinates and the second coordinates corresponding to each divided area, the initial calibration parameters of the corresponding divided area are determined.

[0190] In some embodiments, determining the associated point group corresponding to each calibration block from all radar tracks and video-to-radar tracks includes:

[0191] Determine the associated tracks successfully associated with each radar track and the corresponding number of successful associations from all video-to-radar tracks;

[0192] According to the radar track, the corresponding associated track and the number of successful associations, the associated point group corresponding to each calibration block is determined.

[0193] In some embodiments, determining the associated track successfully associated with each radar track and the corresponding number of successful associations from all video-to-radar tracks includes:

[0194] Determine the gate interval of the radar track point at the current moment in the radar track;

[0195] Capture all video-to-radar trajectory points that fall into the gate interval at the current moment;

[0196] When a video-to-radar trajectory point is captured, the video-to-radar trajectory where the video-to-radar trajectory point is located is used as an associated trajectory successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated;

[0197] When multiple video-to-radar trajectory points are captured, the target trajectory point is determined from the multiple video-to-radar trajectory points according to the preset parameter weights and the radar trajectory point at the current moment; the video-to-radar trajectory where the target trajectory point is located is used as the associated trajectory that is successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated.

[0198] In some embodiments, determining the associated point group corresponding to each calibration block according to the radar track, the corresponding associated track and the number of successful associations includes:

[0199] Determine the radar track where the radar track point entering the current calibration block is located as the target radar track;

[0200] Obtain the associated track that is successfully associated with the target radar track at the current moment, and the corresponding number of successful associations;

[0201] When the number of successful associations is not less than a preset number, the track points of the associated track and the track points of the target radar track at the current moment are used as the associated point group of the calibration block, and the next calibration block is selected as the current calibration block, and the step of returning to the step of determining the radar track where the radar track point entering the current calibration block is located as the target radar track is executed;

[0202] When the number of successful associations is less than the preset number, the radar track that enters the calibration block next time is determined as the target radar track, and the step of obtaining the associated track that is successfully associated with the target radar track at the current moment and the corresponding number of successful associations is returned to be executed.

[0203] In some embodiments, the method of determining the target calibration parameters corresponding to each divided area according to the associated point groups and initial calibration parameters corresponding to all calibration blocks includes:

[0204] Merging the radar track points in the associated point group corresponding to each divided area to obtain a first point set of the corresponding divided area, and merging the video-to-radar track points in the associated point group corresponding to each divided area to obtain a second point set of the corresponding divided area;

[0205] Determine a relationship matrix for each divided area according to the first point set and the second point set;

[0206] According to the relationship matrix and the initial calibration parameters, the target calibration parameters corresponding to each divided area are determined.

[0207] Those skilled in the art will appreciate that the electronic device 400 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 410 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0208] It should be understood that in the embodiment of the present application, the input unit 404 may include a graphics processor (Graphics Processing Unit, GPU) 4041 and a microphone 4042, and the graphics processor 4041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 406 may include a display panel 4061, and the display panel 4061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 407 includes a touch panel 4071 and at least one of other input devices 4072. The touch panel 4071 is also called a touch screen. The touch panel 4071 may include two parts: a touch detection device and a touch controller. Other input devices 4072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0209] The memory 409 can be used to store software programs and various data. The memory 409 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 409 may include a volatile memory or a non-volatile memory, or the memory 409 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 409 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0210] The processor 410 may include one or more processing units; the processor 410 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 410.

[0211] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned radar and video calibration method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0212] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0213] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned radar and video calibration method when executed by a processor.

[0214] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0215] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0216] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0217] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0218] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0219] In the description of the present application, “plurality” means two or more.

[0220] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0221] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A radar and video calibration method, characterized in that: include: Acquire a radar track and a video track of at least one target in an intersection area, wherein the intersection area includes a plurality of divided areas, and each of the divided areas includes a plurality of calibrated blocks; Determine the video-to-radar trajectory corresponding to the video trajectory according to the initial calibration parameters of each of the divided areas; Determine, from all the radar tracks and the video-to-radar tracks, a correlation point group corresponding to each calibration block, each of the correlation point groups including a radar track point and a video-to-radar track point; According to the associated point groups corresponding to all the calibration blocks and the initial calibration parameters, target calibration parameters corresponding to each of the divided areas are determined.

2. The radar and video calibration method according to claim 1, characterized in that: The radar and video calibration method also includes: Determine the shape and area of ​​the lanes in the intersection area; Dividing the area where the lane is located according to the shape to obtain the multiple divided areas; Each of the divided areas is meshed, and the plurality of calibration blocks are selected from the meshed divided areas according to the positions of the vertex meshes.

3. The radar and video calibration method according to claim 1, characterized in that: The radar and video calibration method also includes: Determine a first coordinate of each vertex in each of the divided areas in the radar coordinate system and a second coordinate in the video coordinate system; The initial calibration parameters of the corresponding divided area are determined according to all the first coordinates and the second coordinates corresponding to each divided area.

4. The radar and video calibration method according to claim 1, characterized in that: The step of determining a group of associated points corresponding to each calibration block from all the radar tracks and the video-to-radar tracks includes: Determine, from all the video-to-radar tracks, the associated tracks that are successfully associated with each of the radar tracks, and the corresponding number of successful associations; According to the radar track, the corresponding association track and the number of association successes, a group of association points corresponding to each calibration block is determined.

5. The radar and video calibration method according to claim 4, characterized in that: The determining, from all the video-to-radar tracks, an associated track that is successfully associated with each radar track and a corresponding number of successful associations includes: Determine the gate interval of the radar trajectory point at the current moment in the radar trajectory; Capture all video-to-radar trajectory points that fall into the gate interval at the current moment; When one of the video-to-radar trajectory points is captured, the video-to-radar trajectory where the video-to-radar trajectory point is located is used as an associated trajectory successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated; When multiple video-to-radar trajectory points are captured, a target trajectory point is determined from the multiple video-to-radar trajectory points according to preset parameter weights and the radar trajectory point at the current moment; the video-to-radar trajectory where the target trajectory point is located is used as an associated trajectory that is successfully associated with the radar trajectory, and the corresponding number of successful associations is accumulated.

6. The radar and video calibration method according to claim 4, characterized in that: Determining the associated point group corresponding to each calibration block according to the radar track, the corresponding associated track and the number of successful associations includes: Determine the radar track where the radar track point entering the current calibration block is located as the target radar track; Obtaining the associated track that is successfully associated with the target radar track at the current moment, and the corresponding number of successful associations; When the number of successful associations is not less than a preset number, the track points of the associated track and the track points of the target radar track at the current moment are used as the associated point group of the calibration block, and the next calibration block is selected as the current calibration block, and the step of returning to the step of determining the radar track where the radar track point entering the current calibration block is located as the target radar track is executed; When the number of successful associations is less than the preset number, the radar track that enters the calibration block next time is determined as the target radar track, and the step of obtaining the associated track that is successfully associated with the target radar track at the current moment and the corresponding number of successful associations is returned to be executed.

7. The radar and video calibration method according to claim 1, characterized in that: The step of determining the target calibration parameter corresponding to each of the divided areas according to the associated point groups corresponding to all the calibration blocks and the initial calibration parameters includes: Merging the radar track points in the associated point group corresponding to each of the divided areas to obtain a first point set of the corresponding divided area, and merging the video-to-radar track points in the associated point group corresponding to each of the divided areas to obtain a second point set of the corresponding divided area; Determine a relationship matrix for each of the divided areas according to the first point set and the second point set; According to the relationship matrix and the initial calibration parameters, target calibration parameters corresponding to each of the divided areas are determined.

8. A radar and video calibration device, characterized in that: include: An acquisition unit, used to acquire a radar track and a video track of at least one target in an intersection area, wherein the intersection area includes a plurality of divided areas, and each of the divided areas includes a plurality of calibration blocks; A first determining unit, configured to determine a video-to-radar trajectory corresponding to the video trajectory according to an initial calibration parameter of each of the divided areas; A second determining unit is used to determine, from all the radar tracks and the video-to-radar tracks, a correlation point group corresponding to each of the calibration blocks, each of the correlation point groups including a radar track point and a video-to-radar track point; The third determination unit is used to determine the target calibration parameter corresponding to each of the divided areas according to the associated point groups corresponding to all the calibration blocks and the initial calibration parameters.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the radar and video calibration method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the radar and video calibration method according to any one of claims 1 to 7 is implemented.

Images