An intersection multi-radar track matching method based on target detection tracking

Abstract

The application discloses a kind of intersection multi-radar track matching methods based on target detection tracking, comprising the following steps: step S1, the time synchronization of multiple devices at intersection is carried out;Step S2, the space conversion of each radar track completing time synchronization is carried out;Step S3, the detection tracking of vehicle is carried out to electric police image, and multi-radar track matching is carried out based on detection tracking result;Step S4, multi-radar target track is fused by target detection tracking result matching, step S5, when visual detection frame respectively and multi-radar track match, the track between multiple radars is matched.The application mainly relies on camera image to carry out multi-radar track matching, and camera device is not susceptible to environmental influence, and clear and complete scene image can be shot, so the adaptability of the application to different scenes is stronger.

Description

Technical Field

[0001] This invention belongs to the field of radar signal processing technology, specifically a method for matching multiple radar tracks at intersections based on target detection and tracking. Background Technology

[0002] Traffic radars are typically installed on poles at one of the entrances to an intersection, such as traffic light poles or electronic warning poles, to detect real-time vehicle position, speed, heading angle, type, and other target characteristic information. The detection range of a radar is usually ellipsoidal; a radar device installed at one entrance can only detect the central area of ​​the intersection and the opposite section of the road. This means a single radar cannot track the complete trajectory of a vehicle throughout the intersection. For a crossroads, one traffic radar needs to be installed at each of the four entrances to cover the entire intersection area. Furthermore, the overlapping trajectories of multiple radars in the shared detection area need to be matched and fused to form the complete trajectory of a vehicle passing through the intersection.

[0003] In practical applications, radar equipment has some limitations in its detection capabilities. For example, multiple radars may detect the same target at inconsistent positions, making it difficult to match and connect the trajectories of multiple radars in a shared detection area. This patent, by incorporating video information and using visual detection and tracking technology to match and connect the trajectories of multiple radars, effectively avoids the problems caused by the limitations of radar equipment mentioned above, improves the accuracy of multi-radar trajectory matching, and thus achieves continuous and stable tracking of vehicles throughout the entire intersection area.

[0004] Multi-radar data fusion methods can be categorized into centralized and distributed processing. Centralized processing involves concentrating the point information from each individual radar at a multi-radar processing center, then performing point correlation, track establishment, and filtering tracking. The advantage of this method is minimal information loss, but data correlation is relatively difficult and susceptible to anomalies and noise. Distributed processing involves first establishing tracks on each individual radar processor, then sending them to the multi-radar processing center for matching and fusion to form a fused multi-radar track. The advantage of this method is strong local independent tracking capability, and with the help of matching algorithms, it can achieve global surveillance capability. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-radar trajectory matching method for intersections based on target detection and tracking, in order to solve the problems raised in the background art by using complex logic to solve the problems raised in the background art, resulting in high algorithm complexity, weak adaptability in changing scenarios, and poor algorithm matching and fusion effect.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] A method for matching multiple radar tracks at intersections based on target detection and tracking includes the following steps:

[0008] Step S1: Synchronize the time of multiple devices at the intersection; specifically, synchronize the radar devices and electronic police cameras by referencing the same clock source.

[0009] Step S2: Spatial transformation is performed on the radar trajectory that has completed time synchronization for subsequent matching and fusion.

[0010] Spatial transformation includes the following steps:

[0011] Step S201, for the collected latitude and longitude coordinates [lon, lat] and pixel coordinates [o x o y The transformation is performed using a 2x2 matrix, as shown below:

[0012]

[0013] Rewriting the matrix will convert the matrix representation into an expression, as follows:

[0014] o x = a*lon + b*lat

[0015] o y =c*lon+d*lat

[0016] In the formula, a, b, c, and d are the four matrix values ​​that need to be obtained from the transformation matrix. All the values ​​can be obtained by matching four sets of pixel-latitude and longitude pairs.

[0017] Step S3 involves detecting and tracking vehicles in the electronic police images, and performing multi-radar trajectory matching based on the detection and tracking results. Vehicle detection and tracking employ the YOLO target detection algorithm and the BYTETRACK target tracking algorithm, specifically including the following steps:

[0018] Step S301, YOLO object detection algorithm, first divides the input image into S×S cells;

[0019] Step S302: Use each cell to detect targets whose center point falls within that cell, that is, output the detection box of the corresponding target and the confidence level of the detection box;

[0020] Step S303: The BYTETRACK target tracking algorithm uses the Kalman filter algorithm to predict the coordinates of the detection box in the current frame, and uses the Hungarian algorithm to match the Kalman filter result with the high-confidence detection box and low-confidence detection box output by the target detection algorithm respectively. The successfully matched detection box is assigned a unified ID for continuous tracking.

[0021] Step S4: Match and fuse the target trajectories of multiple radars through the target detection and tracking results. Specifically, in the overlapping area of ​​the detection range of multiple radars and the field of view of the camera, match the multiple radar trajectories with the detection boxes respectively.

[0022] Step S5: After the visual detection box is matched with the trajectory of each of the multiple radars, the trajectories between the multiple radars are matched.

[0023] According to the above technical solution, step S4, matching the multiple radar trajectories with the detection boxes, includes the following steps:

[0024] Step S401: Calculate the center pixel coordinates of the visual detection box, as follows:

[0025]

[0026]

[0027] In the formula, m x Let m be the x-coordinate of the detection box center. y Let l be the y-coordinate of the detection box center. x r is the x-coordinate of the top-left corner of the detection box. x l is the x-coordinate of the bottom right corner of the detection box. y Let l be the y-coordinate of the top-left corner of the detection box. y The y-coordinate of the bottom right corner of the detection box;

[0028] Step S402: Calculate the distance L between the center pixel coordinates of the visual detection box and the radar projection pixel coordinates, as shown in the following formula:

[0029]

[0030] In the formula, m x Let m be the x-coordinate of the detection box center. y Let y be the center coordinate of the detection box, o x Let x be the radar projection pixel x-coordinate, o y y-coordinate of radar projection pixel;

[0031] Step S403: Calculate the distances for all visual and radar detection results at the current time to obtain the distance matrix M, as shown in the following formula:

[0032]

[0033] Among them, L i，j Let L represent the distance L between the i-th visual detection result and the j-th radar detection result; then match the visual detection box with its nearest radar detection result.

[0034] According to the above technical solution, in step S403, the i-th visual detection result matches the j-th radar detection result when the following condition is met:

[0035] L i，j =min(L t，1n )

[0036] In the formula, L i，j Let L represent the distance L between the i-th visual detection result and the j-th radar detection result.

[0037] According to the above technical solution, step S5, matching the trajectories between multiple radars, includes the following steps:

[0038] Step S501: Associate the multiple radar trajectories that successfully match the same ID detection box;

[0039] Step S502: By performing a weighted average of the four-directional radar data, the final fused target result is output.

[0040] According to the above technical solution, in step S1, time synchronization is achieved using the BeiDou satellite navigation system.

[0041] According to the above technical solution, in step S303, the Kalman filter algorithm predicts the detection box coordinates of the current frame as follows:

[0042] The Kalman filter algorithm makes predictions based on the following three formulas:

[0043] x k =Ax k-1 +Bu k-1 (1)

[0044] In the formula, x k It is the coordinate of the detection box predicted by the filter at the current moment, x k-1 These are the coordinates of the detection box after filter correction at the previous time step, A is the state transition matrix, and u k-1 is the parameter of the external influence on the system at the previous moment, B is the input control matrix, and Equation 1 is used for state prediction.

[0045] P k =AP k-1 A T +Q (2)

[0046] In the formula, p k It is the error matrix at the current time, p k-1 Equation 2 is the error matrix of the previous time step, A is the state transition matrix, Q is the prediction noise covariance matrix, and Equation 2 is the error matrix prediction.

[0047] K k =P k HT (HOKHT+R) -1 (3)

[0048] In the formula, k k It is the Kalman gain at the current moment, p k H is the error matrix at the current moment, H is the observation matrix, R is the measurement noise covariance matrix, and Equation 3 is the Kalman gain calculation.

[0049] The Kalman filter algorithm performs corrections based on the following two formulas:

[0050]

[0051] In the formula, x k k is the coordinate of the detection box predicted by the filter at the current moment. k It is the Kalman gain at the current time step, H is the observation matrix, and z is the Kalman gain at the current time step. k Equation 4 is the state correction, calculated by the target detection algorithm at the current moment, and its output is the observation value. It is the final Kalman filter result, that is, the coordinates of the detection box after filter correction at the current moment;

[0052]

[0053] In the formula, p k It is the error matrix before the current time step, k k H is the Kalman gain at the current time step, H is the observation matrix, and I is the identity matrix. It is the error matrix updated at the current time, and Equation 5 is the error matrix update.

[0054] According to the above technical solution, in step S303, matching the Kalman filter result with the high-confidence detection boxes and low-confidence detection boxes output by the target detection algorithm using the Hungarian algorithm includes the following steps:

[0055] Step B1: Generate an association matrix based on the distance between the Kalman filter results and the detection boxes, and subtract the minimum value of each row of the association matrix;

[0056] Specifically, the correlation matrix is ​​a two-dimensional matrix, with the number of rows representing the number of detection boxes output by the Kalman filter, the number of columns representing the number of detection boxes output by the detection algorithm, and the matrix value representing the distance between the center coordinates of the corresponding detection box output by the filter and the center coordinates of the detection box output by the detection algorithm.

[0057] Step B2: Subtract the minimum value of each column from the new matrix.

[0058] Step B3: Connect all the zeros in the new matrix using row and column lines, and check if the current allocation is optimal. If the row and column lines do not connect all the elements of the matrix, find the minimum element among the elements that are not connected, subtract the minimum element from the remaining elements, and add the minimum element to the elements at the intersection of the row and column lines. If the row and column lines connect all the elements of the matrix, find the zero element corresponding to each row and the zero element corresponding to each column, and find the optimal match based on the zero element. The row and column corresponding to the position of 0 in the matrix are successfully matched. That is, if the position [1,1] in the matrix is ​​0, the first detection box of the filter result and the first detection box of the target detection algorithm are successfully matched.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] This invention primarily relies on camera images for multi-radar trajectory matching. Camera devices are less affected by environmental conditions and can capture clear and complete scene images, thus demonstrating strong adaptability to different scenarios. Secondly, radar detection performance is susceptible to sudden fluctuations due to environmental interference, placing high demands on existing algorithms and requiring complex logic to achieve good matching results. This invention, however, utilizes mainstream deep learning algorithms to quickly and accurately obtain visual detection and tracking results, and completes multi-radar trajectory matching with simple algorithmic logic. This invention offers advantages such as simple implementation, high accuracy, fast matching, strong real-time performance, and low computational cost, achieving excellent real-time matching and fusion results. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the intersection equipment installation of the present invention;

[0062] Figure 2 This is a rendering of the four-directional radar projection electronic police camera of the present invention;

[0063] Figure 3 This is a diagram illustrating the target detection and tracking effect of the present invention.

[0064] Figure 4 This is a schematic diagram of multi-radar trajectory matching in this invention;

[0065] Figure 5 This is a schematic diagram of the Hungarian algorithm matching method of the present invention. Figure 1 ;

[0066] Figure 6 This is a schematic diagram of the Hungarian algorithm matching method of the present invention. Figure 2 ;

[0067] Figure 7 This is a schematic diagram of the Hungarian algorithm matching method of the present invention. Figure 3 ;

[0068] Figure 8 This is a schematic diagram of the Hungarian algorithm matching method of the present invention. Figure 4 ;

[0069] Figure 9 This is a schematic diagram of the Hungarian algorithm matching method of the present invention. Figure 5 . Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Example 1

[0072] like Figures 1 to 4 As shown, a multi-radar track matching method for intersections based on target detection and tracking includes the following steps:

[0073] Step S1: Synchronize the time of multiple devices at the intersection; specifically, synchronize the radar devices and electronic police cameras by referencing the same clock source.

[0074] Step S2: Spatial transformation is performed on the radar trajectory that has completed time synchronization for subsequent matching and fusion.

[0075] Spatial transformation includes the following steps:

[0076] Step S201, for the collected latitude and longitude coordinates [lon, lat] and pixel coordinates [o x o y The transformation is performed using a 2x2 matrix, as shown below:

[0077]

[0078] Rewriting the matrix will convert the matrix representation into an expression, as follows:

[0079] o x = a*lon + b*lat

[0080] o y =c*lon+d*lat

[0081] In the formula, a, b, c, and d are the four matrix values ​​that need to be obtained from the transformation matrix. All the values ​​can be obtained by matching four sets of pixel-latitude and longitude pairs.

[0082] Step S3 involves detecting and tracking vehicles in the electronic police images, and performing multi-radar trajectory matching based on the detection and tracking results. Vehicle detection and tracking employ the YOLO target detection algorithm and the BYTETRACK target tracking algorithm, specifically including the following steps:

[0083] Step S301, YOLO object detection algorithm, first divides the input image into S×S cells;

[0084] Step S302: Use each cell to detect targets whose center point falls within that cell, that is, output the detection box of the corresponding target and the confidence level of the detection box;

[0085] Step S303: The BYTETRACK target tracking algorithm uses the Kalman filter algorithm to predict the coordinates of the detection box in the current frame, and uses the Hungarian algorithm to match the Kalman filter result with the high-confidence detection box and low-confidence detection box output by the target detection algorithm respectively. The successfully matched detection box is assigned a unified ID for continuous tracking.

[0086] Step S4: Match and fuse the target trajectories of multiple radars through the target detection and tracking results. Specifically, in the overlapping area of ​​the detection range of multiple radars and the field of view of the camera, match the multiple radar trajectories with the detection boxes respectively.

[0087] Step S5: After the visual detection box is matched with the trajectory of each of the multiple radars, the trajectories between the multiple radars are matched.

[0088] This invention primarily relies on camera images for multi-radar trajectory matching. Camera devices are less affected by environmental conditions and can capture clear and complete scene images, thus demonstrating strong adaptability to different scenarios. Secondly, radar detection performance is susceptible to sudden fluctuations due to environmental interference, placing high demands on existing algorithms and requiring complex logic to achieve good matching results. This invention, however, utilizes mainstream deep learning algorithms to quickly and accurately obtain visual detection and tracking results, and completes multi-radar trajectory matching with simple algorithmic logic. This invention offers advantages such as simple implementation, high accuracy, fast matching, strong real-time performance, and low computational cost, achieving excellent real-time matching and fusion results.

[0089] Example 2

[0090] This embodiment is a further refinement of Embodiment 1.

[0091] In step S4, matching the multiple radar trajectories with the detection boxes includes the following steps:

[0092] Step S401: Calculate the center pixel coordinates of the visual detection box, as follows:

[0093]

[0094]

[0095] In the formula, m x Let m be the x-coordinate of the detection box center. y Let l be the y-coordinate of the detection box center. x r is the x-coordinate of the top-left corner of the detection box. x l is the x-coordinate of the bottom right corner of the detection box. y r is the y-coordinate of the top-left corner of the detection box. y The y-coordinate of the bottom right corner of the detection box;

[0096] Step S402: Calculate the distance L between the center pixel coordinates of the visual detection box and the radar projection pixel coordinates, as shown in the following formula:

[0097]

[0098] In the formula, m x Let m be the x-coordinate of the detection box center. y Let y be the center coordinate of the detection box, o x Let x be the radar projection pixel x-coordinate, o y y-coordinate of radar projection pixel;

[0099] Step S403: Calculate the distances for all visual and radar detection results at the current time to obtain the distance matrix M, as shown in the following formula:

[0100]

[0101] Among them, L i，j Let L represent the distance L between the i-th visual detection result and the j-th radar detection result; then match the visual detection box with its nearest radar detection result.

[0102] In step S403, the i-th visual detection result matches the j-th radar detection result when the following condition is met:

[0103] L i，j =min(L t，1n )

[0104] In the formula, L i，j Let L represent the distance L between the i-th visual detection result and the j-th radar detection result.

[0105] Step S5, matching the trajectories between multiple radars, includes the following steps:

[0106] Step S501: Associate the multiple radar trajectories that successfully match the same ID detection box;

[0107] Step S502: By performing a weighted average of the four-directional radar data, the final fused target result is output.

[0108] In step S1, time synchronization is achieved using the BeiDou Navigation Satellite System. Time synchronization can also be achieved using NTP, PTP, or GPS.

[0109] In step S303, the Kalman filter algorithm predicts the detection box coordinates of the current frame as follows:

[0110] The Kalman filter algorithm makes predictions based on the following three formulas:

[0111] x k =Ax k-1 +Bu k-1 (1)

[0112] In the formula, x k It is the coordinate of the detection box predicted by the filter at the current moment, x k-1 These are the coordinates of the detection box after filter correction at the previous time step, A is the state transition matrix, and u k-1 is the parameter of the external influence on the system at the previous moment, B is the input control matrix, and Equation 1 is used for state prediction.

[0113] P k =AP k-1 A T +Q (2)

[0114] In the formula, p k It is the error matrix at the current time, p k-1 Equation 2 is the error matrix of the previous time step, A is the state transition matrix, Q is the prediction noise covariance matrix, and Equation 2 is the error matrix prediction.

[0115] K k =P k H T HP k H T +R) -1 (3)

[0116] In the formula, k k It is the Kalman gain at the current moment, p k H is the error matrix at the current moment, H is the observation matrix, R is the measurement noise covariance matrix, and Equation 3 is the Kalman gain calculation.

[0117] The Kalman filter algorithm performs corrections based on the following two formulas:

[0118]

[0119] In the formula, x kk is the coordinate of the detection box predicted by the filter at the current moment. k It is the Kalman gain at the current time step, H is the observation matrix, and z is the Kalman gain at the current time step. k Equation 4 is the state correction, calculated by the target detection algorithm at the current moment, and its output is the observation value. It is the final Kalman filter result, that is, the coordinates of the detection box after filter correction at the current moment;

[0120]

[0121] In the formula, p k It is the error matrix before the current time step, k k H is the Kalman gain at the current time step, H is the observation matrix, and I is the identity matrix. It is the error matrix updated at the current time, and Equation 5 is the error matrix update.

[0122] In step S303, matching the Kalman filter result with the high-confidence and low-confidence detection boxes output by the target detection algorithm using the Hungarian algorithm includes the following steps:

[0123] Step B1: Generate an association matrix based on the distance between the Kalman filter results and the detection boxes, and subtract the minimum value of each row of the association matrix;

[0124] Step B2: Subtract the minimum value of each column from the new matrix.

[0125] Step B3: Connect all the zeros in the new matrix using row and column lines, and check if the current allocation is optimal. If the row and column lines do not connect all the elements of the matrix, find the minimum element among the elements that are not connected, subtract the minimum element from the remaining elements, and add the minimum element to the elements at the intersection of the row and column lines. If the row and column lines connect all the elements of the matrix, find the zero element corresponding to each row and the zero element corresponding to each column, and find the optimal match based on the zero element. The row and column corresponding to the position of 0 in the matrix are successfully matched. That is, if the position [1,1] in the matrix is ​​0, the first detection box of the filter result and the first detection box of the target detection algorithm are successfully matched.

[0126] The following example illustrates the matching steps of the Hungarian algorithm:

[0127] Step one: First, generate a correlation matrix based on the Kalman filter results and the distance between the detection boxes, such as... Figure 5 As shown:

[0128] Then subtract the minimum value of each row in the correlation matrix, such as... Figure 6 As shown:

[0129] Step two, subtract the minimum value of each column from the new matrix, such as... Figure 7 As shown:

[0130] Step 3: Connect all the zeros in the new matrix using row and column lines, and check if this is the optimal allocation. Figure 8 As shown, the row and column lines do not connect all the elements of the matrix:

[0131] Then, subtract the minimum element (i.e., the distance of 1 between filter result one and algorithm result four) from the remaining elements, and add the minimum element to the elements at the intersections of the row and column lines. The resulting new matrix is ​​shown in the table below. After connecting all the 0s in the new matrix with the row and column lines, all the elements of the matrix can be connected, as shown below. Figure 9 As shown;

[0132] The optimal match can be found based on the 0 elements in the matrix. Taking a one-to-one match as an example, filter result 1 matches algorithm result 4 successfully, filter result 2 matches algorithm result 3 successfully, filter result 3 matches algorithm result 2 successfully, and filter result 4 matches algorithm result 1 successfully. At this time, the matching cost (the sum of the distances of the matching pairs) is the minimum, which is 111.

[0133] This invention utilizes image information and performs multi-radar matching based on visual detection and tracking results to effectively solve the above problems. Specifically, addressing the issue of inconsistent positions when multiple radars detect the same target, this patent projects the radar's latitude and longitude positions onto the image coordinate system and performs matching within a certain range near the detection box, effectively reducing the probability of matching failure.

[0134] Example 3

[0135] The inventive concept of this invention is as follows:

[0136] Equipment installation locations at intersections, such as Figure 1 As shown, four radars are installed at the four entrances of the intersection, forming a complete detection area covering the center of the intersection and all four entrances. The following explanation uses a vehicle moving from right to left as an example. In this invention, the radar installed at the opposite entrance is called the main radar, and the radars at other entrances are called secondary radars. When a vehicle enters the intersection, the main radar will detect it first, while the secondary radars continue to track the vehicle outside the detection area of ​​the main radar. For vehicles at that entrance, a traffic enforcement camera located at that entrance provides video information. The camera's field of view can include the central area of ​​the intersection and part of the opposite entrance area.

[0137] The specific process of this invention is described below: First, the time of multiple devices at the intersection is synchronized. Specifically, the radar devices and traffic camera cameras are referenced to the same clock source, and time synchronization is performed using NTP, PTP, GPS, or Beidou, etc.

[0138] This involves spatially converting the radar trajectories from the same time period to facilitate subsequent matching and fusion.

[0139] The spatial transformation specifically involves matching and fusion performed in the image coordinate system, requiring the projection of each radar trajectory onto the image coordinate system. Radar-camera joint calibration yields the transformation matrix from the radar coordinate system to the camera coordinate system. Specifically, this is achieved by acquiring ten sets of latitude / longitude-pixel matching pairs, and then fitting the coordinate system transformation matrix through geometric transformation. For any pair of acquired latitude / longitude coordinates [lon, lat] and pixel coordinates [o...]... x o y The transformation can be performed using a 2x2 matrix, as shown in the following equation:

[0140]

[0141] The matrix form can be expressed as an expression, as shown in the following formula:

[0142] o x = a*lon + b*lai

[0143] o y =c*lon+d*lat

[0144] Here, a, b, c, and d are the four matrix values ​​needed to obtain the transformation matrix. All values ​​can be obtained from four sets of pixel-latitude / longitude matching pairs. Using more pixel-latitude / longitude matching pairs can improve the fitting accuracy. This allows us to obtain the coordinate system transformation matrix.

[0145] This example uses an intersection to demonstrate the effect of projecting four-way radar data from the intersection onto an image from an imported electronic traffic enforcement camera. Figure 2 As shown, the dots with different numbers represent vehicle targets detected by radars from different directions. Specifically, dot 1 represents a target detected by the main radar (installed at the camera's opposite entrance), dot 2 represents a target detected by secondary radar 1 (installed at the camera's own entrance), dot 3 represents a target detected by secondary radar 2 (installed at the camera's left entrance), and dot 4 represents a target detected by secondary radar 3 (installed at the camera's right entrance). It can be seen that vehicles in the middle of the intersection can be simultaneously monitored by radars from four directions. Subsequently, multi-radar matching and fusion will be performed in the intersection area.

[0146] After time synchronization and spatial transformation, the targets detected by multiple single radars at the same time are projected onto the electronic police image coordinate system, generally close to the location of the corresponding vehicle in the image, such as... Figure 2 As shown.

[0147] Next, vehicle detection and tracking are performed on the electronic police images, and multi-radar trajectory matching is conducted based on the detection and tracking results. Specifically, the mainstream target detection algorithm YOLO and target tracking algorithm BYTETRACK are used as the vehicle detection and tracking algorithms of this invention. The target detection algorithm can provide the location of the vehicle in the image, and the target tracking algorithm can assign a unique ID to the vehicle in the image. The YOLO target detection algorithm uses a single convolutional neural network model to achieve end-to-end target detection: first, the input image is divided into an S×S grid, and then each cell is responsible for detecting targets whose center point falls within that cell, i.e., outputting B bounding boxes and their confidence scores. The BYTETRACK target tracking algorithm uses the Kalman filter algorithm to predict the coordinates of the detection boxes in the current frame, and then uses the Hungarian algorithm to match them with the high-confidence detection boxes and low-confidence detection boxes output by the target detection algorithm, respectively. The successfully matched detection boxes are assigned a unified ID for continuous tracking. Figure 3 The rectangle and the number in the top left corner of the image shown are... Figure 2 The detection and tracking results corresponding to the image shown.

[0148] Next, we will explain how to match and fuse target trajectories from multiple radars using target detection and tracking results.

[0149] In the overlapping area between the multi-radar detection range and the camera's field of view, the multi-radar trajectories are first matched with the detection boxes. Specifically, the center pixel coordinates of the visual detection boxes are first calculated:

[0150]

[0151]

[0152] In the formula, m x Let m be the x-coordinate of the detection box center. y Let l be the y-coordinate of the detection box center. x r is the x-coordinate of the top-left corner of the detection box. x l is the x-coordinate of the bottom right corner of the detection box. y The y-coordinate of the top left corner of the detection box ， r y The y-coordinate of the bottom right corner of the detection box 。

[0153] Then, the distance L between the center pixel coordinates of the visual inspection box and the radar projection pixel coordinates is calculated, as shown in the following formula:

[0154]

[0155] In the formula, m x Let m be the x-coordinate of the detection box center. y Let y be the center coordinate of the detection box, o xLet x be the radar projection pixel x-coordinate, o y The y-coordinate of the radar projection pixel.

[0156] Calculate the distances for all visual and radar detection results at the current time to obtain the distance matrix M, as shown in the following formula:

[0157]

[0158] Among them, L i，j Let L represent the distance L between the i-th visual detection result and the j-th radar detection result. The visual detection box is matched with its nearest neighbor radar detection result; that is, the i-th visual detection result matches the j-th radar detection result when the following condition is met:

[0159] L i，j =min(L t,1n )

[0160] After the visual detection box is matched with each of the multiple radar trajectories, the next step is to match the multiple radar trajectories with each other.

[0161] Specifically, multiple radar trajectories that successfully match the same ID detection box are associated to complete the multi-radar trajectory matching.

[0162] For example Figure 4 The ID234 detection box shown first successfully matched with the four nearest radar targets in each of the four directions: primary radar target ID509 (point 1), secondary radar target ID940 (point 2), secondary radar target ID777 (point 3), and secondary radar target ID437 (point 4). Successful matching means that the radar target and the visual target correspond to the same vehicle; that is, all four radar targets correspond to the same vehicle. Therefore, the radar trajectories of these four directions can be correlated, thus successfully fusing the multi-radar tracks corresponding to the same target (the vehicle in the ID234 detection box). Finally, by weighted averaging the four-directional radar data, the final fused target result is output.

[0163] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0164] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for matching multiple radar tracks at intersections based on target detection and tracking, characterized in that: Includes the following steps: Step S1: Synchronize the time of multiple devices at the intersection; Specifically, the radar equipment and the electronic police camera are synchronized by referencing the same clock source. Step S2: Spatial transformation is performed on the radar trajectory that has completed time synchronization for subsequent matching and fusion. Spatial transformation includes the following steps: Step S201, for the collected latitude and longitude coordinates and pixel coordinates Use a 2 The matrix is ​​transformed as follows: Rewrite the matrix, converting the matrix form into an expression, as follows: In the formula, These are the four matrix values ​​that need to be obtained from the transformation matrix. All the values ​​can be obtained by matching four sets of pixel-latitude and longitude pairs. Step S3 involves detecting and tracking vehicles in the electronic police images, and performing multi-radar trajectory matching based on the detection and tracking results. Vehicle detection and tracking employ the YOLO target detection algorithm and the BYTETRACK target tracking algorithm, specifically including the following steps: Step S301, YOLO object detection algorithm, first segment the input image into The cell; Step S302: Use each cell to detect targets whose center point falls within that cell, that is, output the detection box of the corresponding target and the confidence score of the detection box; Step S303: The BYTETRACK target tracking algorithm uses the Kalman filter algorithm to predict the coordinates of the detection box in the current frame, and then uses the Hungarian algorithm to match the Kalman filter result with the high-confidence and low-confidence detection boxes output by the target detection algorithm. Successfully matched detection boxes are assigned a unified ID for continuous tracking. Specifically, the Kalman filter algorithm predicts the coordinates of the detection box in the current frame as follows: The Kalman filter algorithm makes predictions based on the following three formulas: （1） In the formula, These are the coordinates of the detection box predicted by the filter at the current moment. These are the coordinates of the detection box after filter correction at the previous time step, and A is the state transition matrix. is the parameter of the external influence on the system at the previous moment, B is the input control matrix, and Equation 1 is used for state prediction. （2） In the formula, It is the error matrix at the current time. Equation 2 is the error matrix of the previous time step, A is the state transition matrix, Q is the prediction noise covariance matrix, and Equation 2 is the error matrix prediction. （3） In the formula, It is the Kalman gain at the current moment. H is the error matrix at the current moment, H is the observation matrix, R is the measurement noise covariance matrix, and Equation 3 is the Kalman gain calculation. The Kalman filter algorithm performs corrections based on the following two formulas: （4） In the formula, These are the coordinates of the detection box predicted by the filter at the current moment. H is the Kalman gain at the current time step, and H is the observation matrix. Equation 4 is the state correction, calculated by the target detection algorithm at the current moment, and its output is the observation value. It is the final Kalman filter result, that is, the coordinates of the detection box after filter correction at the current moment; （5） In the formula, It is the error matrix before the current time step. H is the Kalman gain at the current time step, H is the observation matrix, and I is the identity matrix. This is the updated error matrix at the current time, and Equation 5 is the error matrix update; Step S4: Match and fuse the target trajectories of multiple radars through the target detection and tracking results. Specifically, in the overlapping area of ​​the detection range of multiple radars and the field of view of the camera, match the multiple radar trajectories with the detection boxes respectively. Step S5: After the visual detection box is matched with the trajectory of each of the multiple radars, the trajectories between the multiple radars are matched.

2. The intersection multi-radar trajectory matching method based on target detection and tracking according to claim 1, characterized in that: In step S4, matching the multiple radar trajectories with the detection boxes includes the following steps: Step S401: Calculate the center pixel coordinates of the visual detection box, as follows: In the formula, The x-coordinate of the detection box center is Let y be the center coordinate of the detection box. The x-coordinate of the top-left corner of the detection box. The x-coordinate of the bottom right corner of the detection box. The y-coordinate of the top left corner of the detection box. The y-coordinate of the bottom right corner of the detection box; Step S402: Calculate the distance between the center pixel coordinates of the visual detection box and the radar projection pixel coordinates. As shown in the following formula: In the formula, The x-coordinate of the detection box center is Let y be the center coordinate of the detection box. The x-coordinate of the radar projection pixel. y-coordinate of radar projection pixel; Step S403: Calculate the distances for all visual and radar detection results at the current time to obtain the distance matrix. As shown in the following formula: in, Indicates the first The visual detection result and the first Distance of each radar detection result Match the visual detection box with its nearest radar detection result.

3. The intersection multi-radar track matching method based on target detection and tracking according to claim 2, characterized in that: In step S403, the i-th visual detection result matches the j-th radar detection result when the following condition is met: In the formula, Indicates the first The visual detection result and the first Distance of each radar detection result .

4. The intersection multi-radar track matching method based on target detection and tracking according to claim 1, characterized in that: Step S5, matching the trajectories between multiple radars, includes the following steps: Step S501: Associate the multiple radar trajectories that successfully match the same ID detection box; Step S502: By performing a weighted average of the four-directional radar data, the final fused target result is output.

5. The intersection multi-radar track matching method based on target detection and tracking according to claim 1, characterized in that: In step S1, time synchronization is achieved using the BeiDou satellite navigation system.

6. The intersection multi-radar track matching method based on target detection and tracking according to claim 1, characterized in that: In step S303, matching the Kalman filter result with the high-confidence and low-confidence detection boxes output by the target detection algorithm using the Hungarian algorithm includes the following steps: Step B1: Generate an association matrix based on the distance between the Kalman filter results and the detection boxes, and subtract the minimum value of each row of the association matrix; Step B2: Subtract the minimum value of each column from the new matrix. Step B3: Connect all the zeros in the new matrix using row and column lines, and check if this is the optimal allocation. If the row and column lines do not connect all the elements of the matrix, find the minimum element among the elements that are not connected, subtract the minimum element from the remaining elements, and add the minimum element to the elements at the intersection of the row and column lines. If the row and column lines connect all the elements of the matrix, find the zero element corresponding to each row and the zero element corresponding to each column, and find the optimal match based on the zero element.

Images