A congestion detection method for intersections based on multi-target tracking

Through multi-target tracking technology combining signal light status and vehicle trajectory matching, the problem that existing traffic congestion detection methods are difficult to accurately detect congestion at intersections is solved, real-time and accurate congestion detection at traffic intersections is achieved, and scientific guidance is provided for improving traffic management.

CN118571022BActive Publication Date: 2024-11-12HEFEI XUNTU INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411046359.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-12
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

The existing traffic congestion detection methods are difficult to accurately detect congestion at intersections, especially the single video tracking technology cannot effectively analyze the status of vehicles queues and signal lights, which makes it difficult to promptly and accurately inform traffic congestion.

Method used

The intersection congestion detection method based on multi-target tracking is adopted, real-time video stream is obtained through the traffic intersection camera, the signal light area is intercepted for status recognition, and combined with vehicle trajectory matching, scientific and efficient congestion analysis is used to detect congestion at traffic intersections.

Benefits of technology

Real-time and accurate detection of traffic intersections is achieved, traffic congestion can be effectively judged, traffic congestion can be provided, traffic congestion can be improved, and traffic management efficiency and accuracy are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118571022B_ABST
    Figure CN118571022B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a method for detecting congestion at an intersection based on multi-target tracking, which belongs to the field of traffic condition supervision. The method for detecting congestion at an intersection includes: pulling a real-time video stream based on an existing camera at an intersection, decoding the picture of the real-time video stream, and recording the time; intercepting the signal light area in the picture, and performing reasoning to obtain the display status of the signal light; establishing a target tracker to obtain the target position of the vehicle in the picture, and performing trajectory matching on the target position to obtain a set of vehicles for vehicle tracking; and performing congestion detection based on the display status of the signal light and the set of vehicles in the picture. The method for detecting congestion at an intersection can identify the vehicle motion state in combination with the signal light state, and use a scientific and efficient congestion analysis method to analyze traffic intersections, providing guidance for improving traffic congestion.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of traffic condition supervision, and in particular to a road intersection congestion detection method based on multi-target tracking. Background Art

[0002] In cities, with the rapid growth of the number of motor vehicles, road congestion is becoming increasingly serious, especially during rush hours such as commuting. Vehicles waiting in line at intersections continue to accumulate. When the length of the vehicle queue gradually increases to exceed the length of the road section, queue overflow will occur. The overflowing vehicles will hinder the operation of vehicles in other directions at the intersection. In severe cases, it will cause the intersection to "deadlock", causing regional traffic congestion. It is particularly important to report traffic congestion in real time, accurately and effectively.

[0003] To solve the road traffic problem, intelligent transportation system has become a new research hotspot, among which traffic congestion detection is one of the key technologies. Many practical methods have been proposed for traffic congestion detection. For example, the ground sensor coil, microwave detector and radar used by traditional ITS can realize vehicle counting but it is difficult to detect congestion. For example, traffic congestion detection based on GPS data often has the phenomenon that the original data is difficult to obtain and data is easily missing. With the development of artificial intelligence, especially computer vision technology, intelligent transportation system based on video analysis has been gradually promoted. Compared with traditional methods, traffic video has a large monitoring range, more traffic information and low monitoring cost, which has certain advantages. However, the existing video monitoring only has a single video vehicle tracking, which cannot accurately detect traffic congestion. Summary of the invention

[0004] The purpose of the embodiment of the present invention is to provide a method for detecting congestion at an intersection based on multi-target tracking. The method can identify the vehicle movement state in combination with the signal light state, and use a scientific and efficient congestion analysis method to analyze traffic intersections, thereby providing guidance for improving traffic congestion.

[0005] In order to achieve the above object, an embodiment of the present invention provides a method for detecting road congestion at an intersection based on multi-target tracking, the method comprising:

[0006] According to the existing traffic intersection camera, pull the real-time video stream, decode the picture of the real-time video stream, and record the time;

[0007] intercepting a signal light region in the picture and performing inference to obtain a display state of the signal light;

[0008] Establishing a target tracker to obtain a target position of the vehicle in the picture, and performing trajectory matching on the target position to obtain a vehicle set for vehicle tracking;

[0009] Congestion detection is performed based on the display status of the signal light and the set of vehicles in the picture.

[0010] Optionally, the capturing of the traffic light area in the picture and performing reasoning to obtain the display state of the traffic light includes:

[0011] Use the traffic light area images in different traffic data scenarios as training sets;

[0012] The training set is sent to resnet and calibrated according to the category set by the classification to train the resnet;

[0013] The traffic light area in the picture is sent to the trained resnet to obtain the red, yellow and green display status of the traffic light.

[0014] Optionally, the establishing of a target tracker to obtain a target position of the vehicle in the picture, and performing track matching on the target position to obtain a vehicle set for vehicle tracking includes:

[0015] Get the detection frame and corresponding detection score of the vehicle in each frame;

[0016] Determine whether the detection score corresponding to the detection frame is greater than a first preset threshold;

[0017] When the detection score corresponding to the detection frame is greater than the first preset threshold, classifying the detection frame as a high confidence frame;

[0018] When the detection score corresponding to the detection frame is less than the first preset threshold, determining whether the detection score corresponding to the detection frame is greater than a second preset threshold;

[0019] When the detection score corresponding to the detection frame is less than the first preset threshold and greater than the second preset threshold, classifying the detection frame as a low confidence frame;

[0020] When the detection score corresponding to the detection frame is less than the second preset threshold, discarding the detection frame;

[0021] Obtaining a tracking trajectory set in the previous frame, wherein the tracking trajectory set includes the latest matched detection frame, trajectory frame, target vehicle ID, target vehicle appearance time, and target vehicle latest time;

[0022] Input the tracking trajectory set in the previous frame into the Kalman filter algorithm to obtain the trajectory frame of the tracking trajectory set in the current frame;

[0023] Calculating the similarity between the detection frame and the trajectory frame;

[0024] Based on the similarity between the detection frame and the trajectory frame, the detection frame and the tracking trajectory set are matched by the Hungarian algorithm to obtain the latest tracking trajectory set.

[0025] Optionally, the performing congestion detection according to the display state of the signal light and the vehicle state in the picture includes:

[0026] Obtaining the display status of the signal light;

[0027] When the display state of the signal light is green, obtaining the latest tracking trajectory set;

[0028] Acquire the vehicle set in the picture according to the tracking trajectory set ,in, represents the target vehicle, represents the detection box, Indicates the target vehicle appearance time, Indicates the latest time when the target vehicle appears. Indicates the target vehicle's ID parameter, Indicates the number of target vehicles;

[0029] Gather the vehicles The parameters are brought into formula (1). When formula (1) is satisfied, the traffic intersection is judged to be congested:

[0030] , formula (1)

[0031] in, Indicates an indicator function. If the function's parameter expression is true, the function return value is 1, otherwise it is 0. Indicates the time threshold for a vehicle to stay in an area used to determine traffic congestion. Indicates the threshold value of the number of vehicles staying in the area used to determine traffic congestion.

[0032] Optionally, calculating the similarity between the detection frame and the trajectory frame includes:

[0033] Acquire a detection frame and a trajectory frame in the picture as a target image;

[0034] Dividing the target image into a plurality of patches, each of which has a uniform size;

[0035] Each of the patches is expanded along the rows into a column vector, so that a target image can be represented as multiple feature vectors;

[0036] Normalize the patch to the range [0,1];

[0037] Add the corresponding position code to each patch so that each patch is converted into a position feature ,in Indicates the serial number parameter of the patch. Represents location characteristics, represents the normalized patch, Indicates positional encoding;

[0038] Through CSP-Net Convert to feature matrix ,in Indicates the number of rows, which is the size of the last layer of the CSP+Net network output. Indicates length, represents the set of real numbers;

[0039] The feature matrix Multiply them with the three embedding layers of the transformer respectively, that is, and Multiply to get the embedding matrix ;

[0040] Concatenate the embedding layer matrix to obtain the attention feature ,in represents the target image;

[0041] The attention feature is fed into the re-ID network, and the similarity score between the detection box and the trajectory box is calculated according to formula (2):

[0042] Similarity score=a·IOU+(1-a)·SLM, formula (2)

[0043] Among them, Similarity score represents the similarity score, a represents the weight factor, IOU represents the overlap between the detection box and the trajectory box, SLM represents the similarity learning module replaced by the re-id network, and the attention feature is sent to the similarity learning module.

[0044] Optionally, matching the detection frame and the tracking trajectory set by using the Hungarian algorithm based on the similarity between the detection frame and the trajectory frame to obtain the latest tracking trajectory set includes:

[0045] Obtaining the similarity between the detection frame and the trajectory frame;

[0046] According to the similarity, matching is performed using the Hungarian algorithm, and high confidence boxes that are not matched to the tracking track set and tracking tracks that are not matched to the detection box are retained;

[0047] Associating the tracking tracks in the tracking track set remaining from the first association with the low confidence box;

[0048] Keep the tracking tracks that are still not matched to the detection box after association, and delete the low confidence boxes that are not matched to the tracking tracks;

[0049] The high confidence box that is not matched to the tracking trajectory is taken as a new tracking trajectory and saved;

[0050] All tracking trajectories are combined to obtain a new tracking trajectory set.

[0051] Optionally, matching the detection frame and the tracking trajectory set by using the Hungarian algorithm based on the similarity between the detection frame and the trajectory frame to obtain the latest tracking trajectory set includes:

[0052] Get the similarity between the trajectory box in the tracking trajectory predicted in the previous frame and the detection box in the current frame;

[0053] Establishing a matching cost matrix between the tracking trajectory and the detection frame according to the similarity;

[0054] The matching of the detection frame and the tracking trajectory is constrained according to formula (3):

[0055] , formula (3)

[0056] in, represents the matching cost matrix, is the coefficient, which takes 1 or 0. is the index of the detection box, is the index of the track box. and When the corresponding frame is is 1, and other matching pairs are 0.

[0057] Optionally, the performing congestion detection according to the display state of the signal light and the vehicle state in the picture includes:

[0058] Marking an area for determining traffic congestion on the screen;

[0059] Traverse the vehicle collection , in the vehicle collection When the center point coordinates of the detection box in are not within the area for judging traffic congestion, the vehicle set is deleted. The target vehicle in

[0060] The vehicle set that has deleted the target vehicle Substitute into formula (1), and when formula (1) is satisfied, the traffic intersection is judged to be congested.

[0061] Optionally, the target position of the vehicle in the picture is within the area for judging traffic congestion in the lanes corresponding to the left-turn, straight-ahead and right-turn light panels of the traffic lights in the picture.

[0062] Through the above technical scheme, the present invention provides a method for detecting congestion at an intersection based on multi-target tracking. According to the existing camera at the traffic intersection, a real-time video stream can be pulled, and the picture of the real-time video stream can be decoded and the time of the video can be recorded. After the picture is obtained, the signal light area in the picture can be intercepted, and reasoning can be performed to obtain the display status of the signal light. After obtaining the display status of the signal light, a target tracker can be established, so that the target position of the vehicle in the picture can be obtained, and the trajectory of the target position can be matched, so that the vehicle set for vehicle tracking can be obtained. Vehicle congestion detection can be performed based on the display status of the signal light and the vehicle set in the picture. The method for detecting congestion at the intersection can identify the vehicle motion state in combination with the signal light state, and use a scientific and efficient congestion analysis method to analyze the traffic intersection, providing guidance for improving traffic congestion.

[0063] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0065] Figure 1 is a flow chart of a method for detecting congestion at an intersection according to an embodiment of the present invention;

[0066] Figure 2 is a flow chart of obtaining a signal light display state of a method for detecting congestion at an intersection according to an embodiment of the present invention;

[0067] Figure 3 is a first flow chart of obtaining a tracking trajectory set of a method for detecting congestion at an intersection according to an embodiment of the present invention;

[0068] Figure 4 is a flow chart of congestion detection of a method for detecting congestion at an intersection according to an embodiment of the present invention;

[0069] Figure 5 is a flow chart of obtaining similarity of a method for detecting congestion at an intersection according to an embodiment of the present invention;

[0070] Figure 6 is a second flow chart of obtaining a tracking trajectory set of a method for detecting congestion at an intersection according to an embodiment of the present invention;

[0071] Figure 7 is a third flow chart of obtaining a tracking trajectory set of a method for detecting congestion at an intersection according to an embodiment of the present invention;

[0072] Figure 8 The present invention is a flowchart of a method for detecting congestion at an intersection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0074] Figure 1 1 is a flow chart of a method for detecting congestion at an intersection according to an embodiment of the present invention. In the present invention, the detection process may include:

[0075] In step S1, a real-time video stream is pulled according to an existing traffic intersection camera, and the picture of the real-time video stream is decoded and the time is recorded.

[0076] In step S2, the traffic light area in the picture is captured and inference is performed to obtain the display status of the traffic light.

[0077] In step S3, a target tracker is established to obtain the target position of the vehicle in the picture, and track matching is performed on the target position to obtain a vehicle set for vehicle tracking.

[0078] In step S4, congestion detection is performed based on the display status of the traffic lights and the vehicle collection in the picture.

[0079] In the present invention, when performing congestion detection, a real-time video stream can be pulled according to an existing traffic intersection camera, and the picture of the real-time video stream can be decoded, and the current time can be recorded while decoding the picture. After decoding the video stream, the signal light area in the picture can be intercepted, and then reasoning can be performed to obtain the display state of the signal light. After obtaining the state of the signal light, a target tracker can be established, and the target position of the vehicle in the picture can be obtained by the target tracker, and the target position can be the position of the vehicle. After obtaining the target position, the target position can be tracked, so that a vehicle set for vehicle tracking can be obtained. According to the obtained display state of the signal light and the vehicle set for vehicle tracking, congestion detection can be performed to detect whether the intersection is congested. The intersection congestion detection method can identify the vehicle motion state in combination with the signal light state, and adopts a scientific and efficient congestion analysis method to analyze the traffic intersection, providing guidance for improving traffic congestion.

[0080] In one embodiment of the present invention, Figure 2 As shown, the process of obtaining the display status of the signal light may include:

[0081] In step S5, the traffic light area images under different traffic data scenarios are used as training sets.

[0082] In step S6, the training set is sent to the resnet and calibrated according to the categories set by the classification to train the resnet.

[0083] In step S7, the traffic light area in the picture is sent to the trained resnet to obtain the red, yellow and green display status of the traffic light.

[0084] In the present invention, when identifying a traffic light, the traffic light area in the picture can be captured first. Resnet is used as a split network, and the algorithm model uses the model weights after training. The training set can be traffic light images from different traffic data scenarios, and then can be calibrated according to the categories set by the classification, and the TensorRT accelerated reasoning framework can be used for reasoning during the reasoning process, i.e., model verification. After training resnet, the traffic light area in the captured picture can be sent to resnet, so that the red, yellow, and green display status of the traffic light can be obtained.

[0085] In one embodiment of the present invention, Figure 3 As shown, the first process of obtaining the tracking trajectory set may include:

[0086] In step S8, a detection frame and a corresponding detection score of the vehicle in each frame are obtained.

[0087] In step S9, it is determined whether the detection score corresponding to the detection frame is greater than a first preset threshold.

[0088] In step S10, when the detection score corresponding to the detection frame is greater than a first preset threshold, the detection frame is classified as a high confidence frame.

[0089] In step S11, when the detection score corresponding to the detection frame is less than the first preset threshold, it is determined whether the detection score corresponding to the detection frame is greater than the second preset threshold.

[0090] In step S12, when the detection score corresponding to the detection frame is less than a first preset threshold and greater than a second preset threshold, the detection frame is classified as a low confidence frame.

[0091] In step S13, when the detection score corresponding to the detection frame is less than a second preset threshold, the detection frame is discarded.

[0092] In step S14, a tracking trajectory set in the previous frame is obtained, and the tracking trajectory set includes the latest matched detection frame, trajectory frame, target vehicle ID, target vehicle appearance time, and the latest appearance time of the target vehicle.

[0093] In step S15, the trajectory set in the previous frame is input into the Kalman filter algorithm to obtain the trajectory frame of the tracking trajectory set in the current frame.

[0094] In step S16, the similarity between the detection frame and the trajectory frame is calculated.

[0095] In step S17, the detection frame and the tracking trajectory set are matched by the Hungarian algorithm based on the similarity between the detection frame and the trajectory frame to obtain the latest tracking trajectory set.

[0096] In the present invention, when obtaining a tracking trajectory set, a detection frame and a corresponding detection score about the vehicle can be first obtained through a detector. The detection frame and the detection score can be obtained by a target detection algorithm through detection of the picture. After obtaining the detection frame and the detection score, it can be determined whether the detection score corresponding to the detection frame is greater than a first preset threshold. When the detection score corresponding to the detection frame is greater than the first preset threshold, the detection frame can be classified as a high confidence frame. When the detection score corresponding to the detection frame is between the first preset threshold and the second preset threshold, wherein the first preset threshold can be greater than the second preset threshold, the detection frame can be classified as a low confidence frame. If the detection score corresponding to the detection frame is less than the second preset threshold, it can be indicated that the detection frame has not detected the vehicle, and the detection frame can be discarded.

[0097] After obtaining the detection frame, the tracking trajectory set in the previous frame can be obtained. The tracking trajectory set may include multiple tracking trajectories, and each tracking trajectory may include the latest matched detection frame, trajectory frame, target vehicle ID, target vehicle appearance time and target vehicle latest time. After obtaining the tracking trajectory set in the previous frame, the tracking trajectory set can be input into the Kalman filter algorithm, so that the trajectory frame of the tracking trajectory set in the current frame can be obtained. After obtaining the trajectory frame, the similarity between the detection frame and the trajectory frame can be calculated. According to the similarity, the trajectories corresponding to the detection frame and the trajectory frame can be matched by the Hungarian algorithm, so that the latest tracking trajectory set can be obtained. After matching the detection frame and the tracking trajectory to obtain the tracking trajectory set, according to the parameters in the tracking trajectory set, the vehicle information in the picture, such as vehicle ID, vehicle appearance duration and other information, can be obtained.

[0098] In one embodiment of the present invention, Figure 4 As shown, the congestion detection process may include:

[0099] In step S18, the display status of the signal light is acquired.

[0100] In step S19 , when the display state of the traffic light is green, the latest tracking trajectory set is acquired.

[0101] In step S20, the vehicle set in the picture is obtained according to the tracking trajectory set ,in, represents the target vehicle, represents the detection box, Indicates the target vehicle appearance time, Indicates the latest time when the target vehicle appears. Indicates the target vehicle's ID parameter, Indicates the number of target vehicles.

[0102] In step S21, the vehicle group Substitute into formula (1), and when formula (1) is satisfied, the traffic intersection is judged to be congested:

[0103] , formula (1)

[0104] in, Indicates an indicator function. If the function's parameter expression is true, the function return value is 1, otherwise it is 0. Indicates the time threshold for a vehicle to stay in an area used to determine traffic congestion. Indicates the threshold value of the number of vehicles staying in the area used to determine traffic congestion.

[0105] In the present invention, when detecting traffic congestion at an intersection, the display state of the traffic light can be obtained. When the display state of the traffic light is green, the latest tracking trajectory set can be obtained. Because the tracking trajectory set can include the vehicle ID, vehicle location and the time when the vehicle appears, the vehicle set in the picture can be obtained based on the tracking trajectory set. ,in, represents the target vehicle, represents the detection box, Indicates the target vehicle appearance time, Indicates the latest time when the target vehicle appears. Indicates the target vehicle's ID parameter, represents the number of target vehicles. At the beginning, the vehicle set Can be an empty set. After that, the vehicle can be assembled Substitute the parameters in formula (1) into the formula (1). If the vehicle set If the parameters satisfy formula (1), it can be judged that the traffic intersection is congested.

[0106] In one embodiment of the present invention, Figure 5 As shown, the process of obtaining similarity may include:

[0107] In step S22, the detection frame and trajectory frame in the screen are acquired as the target image.

[0108] In step S23, the target image is divided into a plurality of patches, each patch having a consistent size.

[0109] In step S24, each patch is expanded along the row into a column vector, so that a target image can be represented as multiple feature vectors.

[0110] In step S25, the patches are normalized to the range of [0, 1].

[0111] In step S26, the corresponding position code is added to each patch so that each patch is converted into a position feature ,in Indicates the serial number parameter of the patch. Represents location characteristics, represents the normalized patch, Represents positional encoding.

[0112] In step S27, CSP-Net Convert to feature matrix ,in Indicates the number of rows, which is the size of the last layer of the CSP+Net network output. Indicates length, Represents the set of real numbers.

[0113] In step S28, the feature matrix Multiply them with the three embedding layers of the transformer respectively, that is, and Multiply to get the embedding matrix .

[0114] In step S29, the embedding layer matrix is ​​concatenated to obtain the attention feature ,in Represents the target image.

[0115] In step S30, the attention feature is fed into the re-ID network, and the similarity score between the detection box and the trajectory box is calculated according to formula (2):

[0116] Similarity score=a·IOU+(1-a)·SLM, formula (2)

[0117] Among them, Similarity score represents the similarity score, a represents the weight factor, IOU represents the overlap between the detection box and the trajectory box, SLM represents the similarity learning module replaced by the re-id network, and the attention feature is sent to the similarity learning module.

[0118] In the present invention, calculating the similarity between the detection box and the trajectory box can be used to prepare for the subsequent matching of the trajectories corresponding to the detection box and the trajectory box. When calculating the similarity, the target image can be made more robust through the operations of steps S23, S24 and S25. Because there is a relative position relationship between each patch, this information can be retained through position encoding. Therefore, a corresponding position encoding can be added to each patch so that each patch can be converted into a position feature. Convert to feature matrix After that, the feature can be made more robust. Then the feature matrix can be multiplied with the three embedding layers of the transformer to obtain the embedding layer matrix. The embedding layer matrix is ​​concatenated to obtain the attention feature. ,The feature vectors of the trajectory frame and the detection frame can be obtained by sending the attention matrix into the re-id network, and then the similarity can be calculated by matching, that is, the attention matrix can be sent to the similarity learning module replaced by the re-id network in formula (2), and then the similarity score of the detection frame and the trajectory frame can be obtained according to the formula (2). The similarity score can represent the similarity between the detection frame and the trajectory frame.

[0119] In one embodiment of the present invention, Figure 6 As shown, the second process of obtaining the tracking trajectory set may include:

[0120] In step S31, the similarity between the detection frame and the trajectory frame is obtained.

[0121] In step S32, matching is performed using the Hungarian algorithm according to the similarity, and high confidence boxes that are not matched to the tracking trajectory set and tracking trajectories that are not matched to the detection box are retained.

[0122] In step S33, the tracking tracks in the tracking track set remaining from the first association are associated with the low confidence box.

[0123] In step S34, the tracking tracks that are still not matched to the detection frame after association are retained, and the low-confidence frames that are not matched to the tracking tracks are deleted.

[0124] In step S35 , the high confidence box that is not matched to the tracking trajectory is taken as a new tracking trajectory and saved.

[0125] In step S36, all tracking trajectories are collected to obtain a new tracking trajectory set.

[0126] In the present invention, when the detection frame and the tracking track are matched based on the similarity using the Hungarian algorithm, the similarity of the detection frame and the track frame can be obtained. After obtaining the similarity, the Hungarian algorithm can be used for matching, and the high confidence frame that is not matched to the tracking track set and the tracking track that is not matched to the detection frame can be retained. The tracking track can be one of the tracking track set. The above matching can be the first association between the detection frame and the tracking track. After the first association, the tracking track in the tracking track set remaining from the first association can be associated with the low confidence frame. The tracking track that is still not matched to the detection frame after the re-association can be retained, and the low confidence frame that is not matched to the tracking track can be deleted, because the low confidence frame can be identified as a background that does not contain any object. The high confidence frame that is not matched to the tracking track can be considered as a newly appeared vehicle, and can be used as a new tracking track and can be saved. Collecting all the tracking tracks can obtain a new tracking track set, which can include all the information of the vehicle in the picture.

[0127] In one embodiment of the present invention, Figure 7 As shown, the third process of obtaining the tracking trajectory set may include:

[0128] In step S37, the similarity between the trajectory box in the tracking trajectory predicted in the previous frame and the detection box in the current frame is obtained.

[0129] In step S38, a matching cost matrix of the tracking trajectory and the detection box is established according to the similarity.

[0130] In step S39, the matching of the detection frame and the tracking trajectory is constrained according to formula (3):

[0131] , formula (3)

[0132] in, represents the matching cost matrix, is the coefficient, which takes 1 or 0. is the index of the detection box, is the index of the track box. and When the corresponding frame is is 1, and other matching pairs are 0.

[0133] The Hungarian algorithm can be used to solve the matching problem of bipartite graphs and can be widely applied to the assignment problem. The assignment problem in the tracking algorithm can be described as the problem of finding the optimal match of the detection box in the current frame in response to the predicted trajectory of the previous frame. The purpose is to achieve the maximum number of matches while ensuring a certain accuracy. Therefore, the similarity between the trajectory box in the tracking trajectory predicted in the previous frame and the detection box in the current frame can be obtained. Based on the similarity, the matching cost matrix of the tracking trajectory and the detection box can be established. Then, the matching problem of the tracking trajectory predicted in the previous frame and the detection box can be converted into the problem of solving the global optimal solution of the matching cost matrix. That is, according to formula (3), the matching of the detection box and the tracking trajectory can be constrained. In formula (3), when the matching and When the corresponding frame is When the detection box and the trajectory box match one-to-one, the coefficient is 1.

[0134] In one embodiment of the present invention, Figure 8 As shown, the process of detecting congestion may include:

[0135] In step S40, an area for determining traffic congestion is marked on the screen.

[0136] In step S41, traverse the vehicle set , in the vehicle collection When the center point coordinates of the detection box in are not within the area for judging traffic congestion, delete the vehicle set The target vehicle.

[0137] In step S42, the vehicle set that has deleted the target vehicle Substitute into formula (1), and when formula (1) is satisfied, the traffic intersection is judged to be congested.

[0138] In the present invention, when the vehicle is assembled When the parameters are substituted into formula (1), some vehicles in areas that do not need to be detected may be included. If all the parameters are substituted into formula (1), inaccurate detection may occur. Therefore, the area used to determine traffic congestion can be marked on the screen first, and then the vehicle set can be traversed. In this vehicle collection When the center point coordinates of the detection frame in the image are not within the area for judging traffic congestion, it can be determined that the vehicle corresponding to the detection frame will not affect the traffic congestion, and the target vehicle in the vehicle set can be deleted, and then the deleted vehicle set can be The parameters are introduced into formula (1) to more accurately judge the congestion situation at the traffic intersection.

[0139] In one embodiment of the present invention, the target position of the vehicle in the picture may be within the area where traffic congestion is judged in the lanes corresponding to the left-turn, straight-ahead and right-turn traffic lights in the picture. When detecting the target position of the vehicle, it is sufficient to detect the vehicle within the area where traffic congestion is judged.

[0140] Through the above technical scheme, the present invention provides a method for detecting congestion at an intersection based on multi-target tracking. According to the existing camera at the traffic intersection, a real-time video stream can be pulled, and the picture of the real-time video stream can be decoded and the time of the video can be recorded. After the picture is obtained, the signal light area in the picture can be intercepted, and reasoning can be performed to obtain the display status of the signal light. After obtaining the display status of the signal light, a target tracker can be established, so that the target position of the vehicle in the picture can be obtained, and the trajectory of the target position can be matched, so that the vehicle set for vehicle tracking can be obtained. Vehicle congestion detection can be performed based on the display status of the signal light and the vehicle set in the picture. The method for detecting congestion at the intersection can identify the vehicle motion state in combination with the signal light state, and use a scientific and efficient congestion analysis method to analyze the traffic intersection, providing guidance for improving traffic congestion.

[0141] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0145] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0146] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0147] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0148] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0149] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting traffic congestion at an intersection based on multi-target tracking, characterized in that: The intersection congestion detection method comprises: According to the existing traffic intersection camera, pull the real-time video stream, decode the picture of the real-time video stream, and record the time; intercepting a signal light region in the picture and performing inference to obtain a display state of the signal light; Establishing a target tracker to obtain a target position of the vehicle in the picture, and performing trajectory matching on the target position to obtain a vehicle set for vehicle tracking; Performing congestion detection according to the display state of the signal light and the set of vehicles in the picture; Establishing a target tracker to obtain the target position of the vehicle in the picture, and performing track matching on the target position to obtain a vehicle set for vehicle tracking includes: Get the detection frame and corresponding detection score of the vehicle in each frame; Determine whether the detection score corresponding to the detection frame is greater than a first preset threshold; When the detection score corresponding to the detection frame is greater than the first preset threshold, classifying the detection frame as a high confidence frame; When the detection score corresponding to the detection frame is less than the first preset threshold, determining whether the detection score corresponding to the detection frame is greater than a second preset threshold; When the detection score corresponding to the detection frame is less than the first preset threshold and greater than the second preset threshold, classifying the detection frame as a low confidence frame; When the detection score corresponding to the detection frame is less than the second preset threshold, discarding the detection frame; Obtaining a tracking trajectory set in the previous frame, wherein the tracking trajectory set includes the latest matched detection frame, trajectory frame, target vehicle ID, target vehicle appearance time, and target vehicle latest time; Input the tracking trajectory set in the previous frame into the Kalman filter algorithm to obtain the trajectory frame of the tracking trajectory set in the current frame; Calculating the similarity between the detection frame and the trajectory frame; Based on the similarity between the detection frame and the trajectory frame, the detection frame and the tracking trajectory set are matched by the Hungarian algorithm to obtain the latest tracking trajectory set; Calculating the similarity between the detection frame and the trajectory frame includes: Acquire a detection frame and a trajectory frame in the picture as a target image; Dividing the target image into a plurality of patches, each of which has a uniform size; Each of the patches is expanded along the rows into a column vector, so that a target image is represented as multiple feature vectors; Normalize the patch to the range [0,1]; Add the corresponding position code to each patch so that each patch is converted into a position feature ,in Indicates the serial number parameter of the patch. Represents location characteristics, represents the normalized patch, Indicates positional encoding; Through CSP-Net Convert to feature matrix ,in Indicates the number of rows, which is the size of the last layer of the CSP+Net network output. Indicates length, represents the set of real numbers; The feature matrix Multiply them with the three embedding layers of the transformer respectively, that is, and Multiply to get the embedding matrix ; Concatenate the embedding layer matrix to obtain the attention feature ,in represents the target image; The attention feature is fed into the re-ID network, and the similarity score between the detection box and the trajectory box is calculated according to formula (2): Similarity score=a·IOU+(1-a)·SLM, formula (2) Among them, Similarity score represents the similarity score, a represents the weight factor, IOU represents the overlap between the detection box and the trajectory box, SLM represents the similarity learning module replaced by the re-id network, and the attention feature is sent to the similarity learning module.

2. The method for detecting intersection congestion according to claim 1, characterized in that: The intercepting of the traffic light area in the picture and performing reasoning to obtain the display state of the traffic light includes: Use the traffic light area images in different traffic data scenarios as training sets; The training set is sent to resnet and calibrated according to the category set by the classification to train the resnet; The traffic light area in the picture is sent to the trained resnet to obtain the red, yellow and green display status of the traffic light.

3. The method for detecting intersection congestion according to claim 2, characterized in that: The performing congestion detection according to the display state of the signal light and the vehicle state in the picture includes: Obtaining the display status of the signal light; When the display state of the signal light is green, obtaining the latest tracking trajectory set; Acquire the vehicle set in the picture according to the tracking trajectory set ,in, represents the target vehicle, represents the detection box, Indicates the target vehicle appearance time, Indicates the latest time when the target vehicle appears. Indicates the target vehicle's ID parameter, Indicates the number of target vehicles; Gather the vehicles The parameters are brought into formula (1). When formula (1) is satisfied, the traffic intersection is judged to be congested: , Formula (1) in, Indicates an indicator function. If the function's parameter expression is true, the function return value is 1, otherwise it is 0. Indicates the time threshold for a vehicle to stay in an area used to determine traffic congestion. Indicates the threshold value of the number of vehicles staying in the area used to determine traffic congestion.

4. The method for detecting intersection congestion according to claim 1, characterized in that: The matching of the detection frame and the tracking trajectory set by using the Hungarian algorithm based on the similarity between the detection frame and the trajectory frame to obtain the latest tracking trajectory set includes: Obtaining the similarity between the detection frame and the trajectory frame; According to the similarity, matching is performed using the Hungarian algorithm, and high confidence boxes that are not matched to the tracking track set and tracking tracks that are not matched to the detection box are retained; Associating the tracking tracks in the tracking track set remaining from the first association with the low confidence box; Keep the tracking tracks that are still not matched to the detection box after association, and delete the low confidence boxes that are not matched to the tracking tracks; The high confidence box that is not matched to the tracking trajectory is taken as a new tracking trajectory and saved; All tracking trajectories are combined to obtain a new tracking trajectory set.

5. The method for detecting intersection congestion according to claim 4, characterized in that: The method of matching the detection frame and the tracking trajectory set by using the Hungarian algorithm based on the similarity between the detection frame and the trajectory frame to obtain the latest tracking trajectory set includes: Get the similarity between the trajectory box in the tracking trajectory predicted in the previous frame and the detection box in the current frame; Establishing a matching cost matrix between the tracking trajectory and the detection frame according to the similarity; The matching of the detection frame and the tracking trajectory is constrained according to formula (3): , Formula (3) in, represents the matching cost matrix, is the coefficient, which takes 1 or 0. is the index of the detection box, is the index of the track box. and When the corresponding frame is is 1, and other matching pairs are 0.

6. The method for detecting intersection congestion according to claim 3, characterized in that: The performing congestion detection according to the display state of the signal light and the vehicle state in the picture includes: Marking an area for determining traffic congestion on the screen; Traverse the vehicle collection , in the vehicle collection When the center point coordinates of the detection box in are not within the area for judging traffic congestion, the vehicle set is deleted. The target vehicle in The vehicle set that has deleted the target vehicle Substitute into formula (1), and when formula (1) is satisfied, the traffic intersection is judged to be congested.

7. The method for detecting intersection congestion according to claim 6, characterized in that: The target position of the vehicle in the picture is within the area for judging traffic congestion of the lanes corresponding to the left-turn, straight-ahead and right-turn light panels of the signal lights in the picture.