A multi-target detection and tracking statistical method and device based on edge computing

Abstract

The present invention provides a multi-target detection and tracking statistical method and device based on edge computing, which relates to the field of edge computing and target detection and tracking technology. The present invention extracts the target detection frame of each video frame by preprocessing the acquired target video data; inputs the target detection frame of each video frame into the improved multi-target tracking algorithm for trajectory matching and tracking, and obtains the target trajectory queue of all targets in each video frame; divides the target video monitoring area into a number of rectangular detection areas; then, all targets in the target trajectory queue are matched with the rectangular detection areas in turn, and it is determined whether the target trajectory passes through each rectangular detection area, and statistics are performed according to the rules, so as to obtain the statistical results of the multi-target trajectory in each rectangular detection area, which is the multi-target statistical tracking result. The present invention is suitable for application scenarios that require real-time performance and computational efficiency, especially when processing a large number of fast-moving targets, with high matching efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of edge computing and target detection and tracking technology, and in particular to a multi-target detection and tracking statistical method and device based on edge computing. Background Art

[0002] With the continuous integration and deepening application of technologies such as the Internet of Things (IoT), big data, and artificial intelligence (AI), edge computing, as an emerging computing model, is gradually becoming a key technology to solve the needs of large-scale data processing and real-time response. In many fields such as intelligent transportation, intelligent manufacturing, and smart cities, the demand for real-time statistics and tracking of multi-channel targets is becoming increasingly urgent. For example, in intelligent transportation systems, it is necessary to conduct real-time statistics and tracking of multiple targets such as vehicles and pedestrians on the road to optimize traffic management and improve road traffic efficiency and safety. Traditional monitoring systems usually rely on the cloud for data processing. Therefore, they are affected by network latency and bandwidth and are difficult to meet real-time requirements. They also face problems such as strong network dependence, low data security, unstable performance, high cost, and standard compatibility. Edge computing technology can significantly reduce data transmission latency and improve the real-time response capability of the system by directly processing data on nodes at the edge of the network.

[0003] With the continuous advancement of equipment computing technology and video detection technology, the accuracy and efficiency of multi-channel target statistics have been significantly improved. However, these technologies have also brought about the problem of explosive growth in data volume. The traditional centralized processing model has been unable to cope with such a huge amount of data.

[0004] In view of this, the applicant filed this application after studying the existing technology. Summary of the invention

[0005] The present invention aims to provide a multi-target detection and tracking statistical method and device based on edge computing to solve the problems of high latency, bandwidth bottleneck, security and accuracy encountered by existing methods.

[0006] In order to solve the above technical problems, the present invention is implemented through the following technical solutions:

[0007] A multi-target detection and tracking statistical method based on edge computing, comprising:

[0008] S1, obtaining target video data of a target video monitoring area to be detected; wherein the target video monitoring area is divided into a plurality of rectangular detection areas;

[0009] S2, preprocessing the target video data to extract the target detection frame of each video frame;

[0010] S3, input the target detection frame of each video frame into the improved multi-target tracking algorithm for trajectory matching and tracking, and obtain the target trajectory queue of all targets in each video frame;

[0011] S4, matching the rectangular detection areas of all the targets in the target trajectory queue in turn, determining whether the target trajectory passes through each rectangular detection area, and performing statistics according to the rules, thereby obtaining the statistical results of the multi-target trajectories in each rectangular detection area, that is, the multi-target statistical tracking result; wherein the rules are:

[0012] Establish a global queue with the same number as the number of rectangular detection areas, and store the target trajectories passing through the corresponding rectangular detection areas in a one-to-one correspondence;

[0013] According to the current position of each target track in the rectangular detection area, the target track is added to the global queue corresponding to the current rectangular detection area, so as to obtain the statistical results of the target track in each rectangular detection area;

[0014] Two direction counter queues are defined to count the number of target trajectories that start from the two ends of the target video monitoring area and pass through each rectangular detection area in sequence, so as to realize the statistics of bidirectional moving targets in the target video monitoring area.

[0015] Preferably, preprocessing the target video data includes:

[0016] The target video data is input into the pre-trained YOLO series target detection model for frame-by-frame detection. After different scale feature processing, non-maximum suppression processing, and bounding box deduplication, the target detection frame of each video frame is obtained; wherein, the target detection frame includes the center point coordinates, confidence level and class number of the target frame.

[0017] Preferably, an existing trajectory library is defined and initialized, and the target detection frame of each video frame is matched with the trajectory of the existing trajectory library to obtain the confirmed trajectory and unconfirmed trajectory of the current video frame, and the confirmed trajectory is marked and added to the existing trajectory library; wherein the confirmed trajectory is the trajectory of the target detection frame successfully matched with the trajectory in the existing trajectory library; the unconfirmed trajectory is the trajectory of the target detection frame that is not successfully matched with the trajectory in the existing trajectory library; the existing trajectory library contains all the trajectories marked by the previous video frame, and the initialization state of the existing trajectory library is empty.

[0018] Preferably, the improved multi-target tracking algorithm is an improved ByteTrack multi-target tracking algorithm, including a Kalman filter and a Hungarian algorithm, and S3 specifically includes:

[0019] According to the confidence of the target detection frame and a preset trajectory threshold, the target detection frame is divided into a high-score detection frame and a low-score detection frame;

[0020] The high-score detection frame is matched with the confirmed track for the first time, by calculating the intersection over union (IoU) of the track box area of ​​the high-score detection frame and the confirmed track, and then comparing the IoU with the set IoU threshold, and obtaining the first matching result after optimal matching by the Hungarian algorithm; wherein the first matching result includes: the track Fmtrack that was successfully matched for the first time, the track Futrack that was not successfully matched for the first time, and the target detection frame Fudetection that was not successfully matched for the first time;

[0021] The low-score detection frame is matched with Futrack for the second time, and the IoU of the two is calculated, the IoU threshold is adjusted, and the Hungarian algorithm is used for matching to obtain the second matching result; wherein the second matching result includes: the track Smtrack that was successfully matched for the second time, the track Sutrack that was not successfully matched for the second time, and the target detection frame Sudetection that was not successfully matched for the second time;

[0022] Match Fudetection with the unconfirmed track for the third time, calculate the IoU of the two, adjust the IoU threshold, and match them using the Hungarian algorithm to obtain the third matching result; wherein the third matching result includes: the track Tmtrack that was successfully matched for the third time, the track Tutrack that was not successfully matched for the third time, and the target detection box Tudetection that was not successfully matched for the third time;

[0023] Update the successfully matched tracks Fmtrack, Smtrack and Tmtrack, and add them to the target track queue;

[0024] Initialize a temporary loss queue and add Sutrack to the temporary loss queue; discard Sudetection and Tutrack;

[0025] Add the Tudetection to the target track queue and mark it as an unconfirmed track, update the existing track library, and continue to match the next video frame until all video frames in the target video data are completed;

[0026] During the trajectory matching process, a Kalman filter is used to predict the position of the target of the confirmed trajectory in the current video frame.

[0027] Preferably, the improved multi-target tracking algorithm is an improved DeepSORT algorithm, which includes appearance metric and motion metric, and the specific steps of S3 include:

[0028] Use the pre-trained initial feature model to extract the appearance features of the target detection box of each video frame;

[0029] A Kalman filter is used to predict the target position of each trajectory of the existing trajectory library in the current video frame, and the distance between the predicted target position and all target detection frames detected in the current video frame is calculated to obtain a distance matrix, i.e., a motion metric;

[0030] Calculate the similarity of appearance features between each target detection frame detected in the current video frame and the trajectory of the existing trajectory library to obtain a similarity feature matrix, i.e., appearance metric;

[0031] The distance matrix and the similarity feature matrix are integrated to obtain a comprehensive cost matrix; wherein the comprehensive cost matrix is ​​used to evaluate the matching quality between each target detection frame and the trajectory of the existing trajectory library;

[0032] According to the comprehensive cost matrix, the Hungarian algorithm is used to match and compare each target detection frame with the trajectory of the existing trajectory library; if the comprehensive cost matrix is ​​lower than the set threshold, the match is successful, the trajectory corresponding to the target detection frame is added to the target trajectory queue, and the successfully matched trajectory is updated; if the target detection frame fails to match the trajectory of the existing trajectory library, the trajectory corresponding to the target detection frame is added to the target trajectory queue as a new trajectory; if the trajectory of the existing trajectory library fails to match the target detection frame, the trajectory corresponding to the existing trajectory library is added to the temporarily lost queue;

[0033] Continue matching the next video frame until all video frames in the target video data are completed.

[0034] Preferably, when updating the successfully matched trajectory, a Kalman filter is used to predict and update the trajectory information; wherein the trajectory information includes the target ID, target position, target state, the state estimation of the Kalman filter and the covariance matrix of the Kalman filter.

[0035] Preferably, it also includes: performing time judgment on each trajectory in the temporarily lost queue, if the trajectory reappears within a preset time frame and is successfully matched, adding it to the target trajectory queue; if the trajectory still does not appear after exceeding the preset time, the trajectory is considered to be a permanently lost trajectory and is deleted from the temporarily lost queue.

[0036] Preferably, the S4 is specifically:

[0037] The target video monitoring area is divided into m rectangular detection areas, which are divided into , ,..., ;

[0038] Initialize the global queue set QUEUE={ , ,..., }; wherein the dimension of the global queue set QUEUE is the same as the number of the rectangular detection areas, and is used to store the target trajectory passing through the corresponding rectangular detection area, that is, Represents the first global queue, used to store target trajectory; Represents the mth global queue, used to store target trajectory;

[0039] Define two direction counter queues and ; To count the target from The direction passes through each rectangular detection area in turn and is currently in The number of target trajectories at the location; To count the target from The direction passes through each rectangular detection area in turn and is currently in The number of target trajectories at the location;

[0040] All target tracks in the target track queue are matched with the rectangular detection area in turn to obtain the current position of each target track in the rectangular detection area; if the current target track is in , then add the current target trajectory to ;in, , They represent the i-th rectangular detection area and the global queue respectively;

[0041] If the current target trajectory is in , then add the current target trajectory to , and determine whether the target of the current target trajectory exists in other queues of QUEUE. If so, then The value increases by 1; otherwise The value remains unchanged; if the current target trajectory is in , then add the current target trajectory to , and determine whether the target of the current target trajectory exists in other queues of QUEUE. If so, then The value increases by 1; otherwise The value remains unchanged; until all target trajectories in the target trajectory queue are matched, and the global queue QUEUE set and the direction counter queue are obtained by statistics. and , in order to achieve statistics of two-way movement goals.

[0042] Preferably, it also includes: regularly detecting the position change of the rectangular detection area where the target track in the global queue set QUEUE is located. If the current detected target track exists in the current rectangular detection area for more than a preset time, it is considered to be an invalid target track, and the target track is cleared from the global queue QUEUE to reduce the situation of track misdetection.

[0043] The present invention also provides a multi-target detection, tracking and statistics device based on edge computing, comprising:

[0044] An acquisition module, used to acquire target video data of a target video monitoring area to be detected; wherein the target video monitoring area is divided into a plurality of rectangular detection areas;

[0045] A target detection module, used to pre-process the target video data and extract the target detection frame of each video frame;

[0046] The multi-target tracking module is used to input the target detection frame of each video frame into the improved multi-target tracking algorithm for trajectory matching and tracking, and obtain the target trajectory queue of all targets in each video frame;

[0047] The result statistics module is used to match the rectangular detection areas of all targets in the target trajectory queue in turn, determine whether the target trajectory passes through each rectangular detection area, and perform statistics according to the rules, so as to obtain the statistical results of the multi-target trajectories in each rectangular detection area, that is, the multi-target statistical tracking results; wherein the rules are:

[0048] Establish a global queue with the same number as the number of rectangular detection areas, and store the target trajectories passing through the corresponding rectangular detection areas in a one-to-one correspondence;

[0049] According to the current position of each target track in the rectangular detection area, the target track is added to the global queue corresponding to the current rectangular detection area, so as to obtain the statistical results of the target track in each rectangular detection area;

[0050] Two direction counter queues are defined to count the number of target trajectories that start from the two ends of the target video monitoring area and pass through each rectangular detection area in sequence, so as to realize the statistics of bidirectional moving targets in the target video monitoring area.

[0051] The present invention also provides a multi-target detection, tracking and statistics device based on edge computing, including a processor and a memory, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement a multi-target detection, tracking and statistics method based on edge computing as described above.

[0052] The present invention also provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a device where the computer-readable storage medium is located, the multi-target detection and tracking statistical method based on edge computing as described above is implemented.

[0053] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention detects the target video data, extracts the target detection frame, inputs the improved multi-target tracking algorithm to perform trajectory matching and tracking calculation, assigns each target a trajectory ID, and obtains the target trajectory queue of all targets in each video frame. Then, the targets in the target trajectory queue are matched and counted in several rectangular detection areas in turn, and it is determined whether the target trajectory passes through each rectangular detection area, so as to obtain the result of multi-target statistical tracking.

[0055] The improved ByteTrack multi-target tracking algorithm adopted in the present invention integrates the Kalman filter and the Hungarian algorithm. In terms of trajectory processing strategy, the target detection frame of the video frame is matched with the existing trajectory library, and the "confirmed trajectory" and "unconfirmed trajectory" are clearly distinguished. Then, multiple fine matchings are performed, which is more suitable for application scenarios requiring real-time performance and computational efficiency. Especially when processing a large number of fast-moving targets, it not only enhances the algorithm's ability to finely manage trajectories of different stability levels, but also significantly improves the matching efficiency and accuracy.

[0056] The improved DeepSORT algorithm adopted in the present invention not only takes into account the motion trajectory of the target, but also utilizes the target appearance features extracted by the deep learning model, and then performs fusion detection and matching, which enables it to have stronger target re-identification capabilities in complex scenes.

[0057] The present invention introduces a more sophisticated multiple rectangular area detection method and designs a corresponding number of global queues to strictly distinguish and track the specific process of the target entering each rectangular area, thus avoiding statistical errors caused by fluctuations in the center position of the detection frame. At the same time, the establishment of the data queue enables the system to clearly record the movement trajectory of the target. No matter how the target detection frame changes, as long as it passes through these areas in a specific order, the system can accurately perform traffic statistics and direction judgment.

[0058] In addition, the present invention also introduces a loss queue and time judgment mechanism. When a track disappears temporarily due to occlusion, false detection, etc., the algorithm adds it to the loss queue and makes a judgment based on the time factor, which significantly reduces the false detection and deletion of tracks caused by external interference, and further improves the robustness and stability of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 A schematic diagram of a multi-target detection and tracking statistical method based on edge computing provided in Example 1 of the present invention.

[0061] Figure 2 Flow chart of the improved ByteTrack multi-target tracking algorithm provided in Example 1 of the present invention.

[0062] Figure 3 This is a flow chart of the improved DeepSort multi-target tracking algorithm provided in Example 1 of the present invention.

[0063] Figure 4 This is a statistical flow chart of three rectangular detection areas provided in Example 1 of the present invention.

[0064] Figure 5 A schematic diagram of a multi-target detection, tracking and statistics device based on edge computing provided in Example 2 of the present invention.

[0065] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0067] Embodiment 1

[0068] Existing smart devices lack big data computing capabilities and cannot realize visual processing and artificial intelligence applications. The existing solution is to transmit data to the cloud platform for computing. When processing large amounts of real-time data, traditional cloud computing models often face problems such as high latency, bandwidth bottlenecks, and security because the data needs to be transmitted over the network to a remote data center for processing.

[0069] The present invention aims to overcome the limitations of the prior art and tailor an efficient multi-channel target statistics method based on edge computing for the fields of intelligent transportation and smart cities.

[0070] Embodiment 1 of the present invention provides a multi-target detection, tracking and statistics method based on edge computing, which can be implemented by a multi-target detection, tracking and statistics device based on edge computing (hereinafter referred to as the multi-target detection, tracking and statistics device), and in particular, is executed by one or more processors in the multi-target detection, tracking and statistics device.

[0071] In this embodiment, the multi-target detection, tracking and statistical device may be an electronic device equipped with a processor, which has a computer program of the edge computing-based multi-target detection, tracking and statistical method and can be executed, such as a computer, a smart phone, a smart tablet, a workstation, etc., which is not limited here.

[0072] In this embodiment, the system hardware configuration used is as follows:

[0073] The core part of the system of the present invention adopts a high-performance quad-core RK3588 processor (CPU) as the main control chip. Its powerful processing capability supports smooth video decoding up to 30fps, ensuring that every frame is clear and unobstructed. Video access is flexible and diverse, compatible with multiple connection methods such as network ports, Wi-Fi and 5G, meeting access needs in different scenarios.

[0074] To further improve computing performance, the system is ingeniously equipped with dual computing power cards and equipped with the cutting-edge Hailo edge AI processor (GPU). With ultra-low power consumption (only 2.5W), compact design and economical cost, it achieves an amazing 26TOPS real-time deep learning reasoning capability at the edge, providing solid support for target recognition and tracking in complex scenarios.

[0075] In terms of data storage, the system has a built-in large-capacity hard disk (optional up to 1TB) to ensure the safe storage and long-term retention of video data. At the same time, it supports seamless connection to the cloud platform through the national standard GB / T28181 protocol to achieve remote data upload and sharing, providing strong data support for smart city management.

[0076] Of course, other suitable hardware configurations may also be used, which are not limited here.

[0077] like Figure 1 As shown, a multi-target detection and tracking statistical method based on edge computing includes steps S1 to S4.

[0078] S1, obtaining target video data of a target video monitoring area to be detected; wherein the target video monitoring area is divided into a plurality of rectangular detection areas.

[0079] The present invention obtains the camera video stream (h264, h265 encoding) through the standard onvif protocol (open network video interface), or obtains the video stream through the Wi-Fi and 5G modules configured by the device through the real-time streaming protocol RTSP (Real Time Streaming Protocol, RTSP).

[0080] S2, preprocessing the target video data to extract the target detection frame of each video frame.

[0081] Before preprocessing, first train the target detection model. Collect historical target video data of similar scenes or the same scene in advance to make a data set. After a large amount of long-term training of the target detection model, a model for identifying specific targets (such as the reconstructed YOLO series detection model) is obtained. Then perform hailo compile to quantify it into a hef file model that can be executed by the device.

[0082] The YOLO (You Only Look Once) series of models is a group of deep learning models for target detection. It processes the target detection task as a whole and directly outputs information such as the target category and location in the image through a single neural network. Currently, there are versions from YOLO1 to YOLO8.

[0083] First, initialize the hailo computing card module and obtain the device ID of the computing card module node. Obtain the computing card network model information (such as the number of input virtual stream channels, maximum input data, number of output virtual stream channels, and maximum output data) based on the device ID and model parameters (such as the height, width, number of channels, number of recognized classes, input network name, and output network name). Then create a task for each video channel, read a frame of video data from the corresponding video channel, write it into the input virtual stream channel created by the computing card, and perform target inference. After the inference is completed, feature data of different scales are obtained through the output virtual stream channel. The feature data is then processed by NMS (Non-Maximum Suppression) to remove overlapping bounding boxes in the target box to ensure that each target is detected only once, thereby improving the accuracy and efficiency of detection. After the above steps, detect each frame of the target video data, obtain all target detection boxes (including the center point coordinates, confidence, and class number of the target box) queues of the target video data, and complete the target detection.

[0084] S3, input the target detection frame of each video frame into the improved multi-target tracking algorithm for trajectory matching and tracking, and obtain the target trajectory queue of all targets in each video frame.

[0085] The present invention is equipped with two trajectory tracking algorithms. The first one is an improved ByteTrack multi-target tracking algorithm, which includes a Kalman filter and a Hungarian algorithm.

[0086] In this embodiment, a Kalman filter is an efficient recursive filter that can estimate the state of a dynamic system from a series of noisy observation data. The Kalman filter is based on the state space representation of a linear dynamic system, assuming that the system state is linear and that both the process noise and the observation noise are Gaussian distributed.

[0087] The Kalman filter continuously corrects the estimated value of the system state through two steps: prediction and update.

[0088] The calculation steps of the Kalman filter are divided into two stages: prediction and update:

[0089] (1) Prediction stage:

[0090] State prediction: Use the results of the previous state prediction and the control input to predict the current state.

[0091] Error prediction: The estimated uncertainty (covariance) of the current state when predicting it.

[0092] (2) Update phase:

[0093] Calculate the Kalman gain: it determines which of the predictions or measurements is more reliable.

[0094] Update state: Use new measurements to update the prediction and get a new state estimate.

[0095] Update Error: The uncertainty (covariance) of the update estimate.

[0096] In this embodiment, the Hungarian Algorithm is a combinatorial optimization algorithm for solving the task allocation problem in polynomial time. The Hungarian Algorithm transforms the efficiency matrix (or cost matrix) by rows or columns to convert it into a non-negative matrix with only one element of 0 in each row and column. Then, the maximum matching problem is solved by finding the augmenting path or the minimum number of lines covering the 0 elements.

[0097] like Figure 2 As shown, the steps of the improved ByteTrack multi-target tracking algorithm are:

[0098] In the first step, the detected object detection frames are classified according to the preset trajectory threshold. Object detection frames with a confidence score greater than the trajectory threshold are classified as high-score detection frames (Dhigh), and those with a confidence score less than the trajectory threshold are classified as low-score detection frames (Dlow). Dhigh is more representative of the real object, while Dlow may contain more noise or false detections.

[0099] Define and initialize an existing track library, match it with the detected target detection frame, and mark it as an unconfirmed track and a confirmed track. The confirmed track is a stable track that the target detection frame successfully matches with the existing track library; the unconfirmed track is a track that the target detection frame does not successfully match with the existing track library.

[0100] In order to improve accuracy and matching efficiency, in this embodiment, the existing trajectory library is all the trajectories that have been detected in the previously acquired target video data (i.e., data before the current video frame) and have been marked, including unconfirmed trajectories and confirmed trajectories. During the first detection and matching, the existing trajectory library is empty, that is, the initialization state of the existing trajectory library is empty.

[0101] In the second step, for those marked as confirmed tracks, a Kalman filter (KF) can be used to predict their positions in the current frame (for confirmed tracks, a Kalman filter is used to predict the position of the target in the current frame, which helps to deal with detection failures due to occlusion, fast movement, etc.).

[0102] Then the first match is performed, the high-score detection frame Dhigh is matched with the confirmed track, the intersection over union (IoU) of the two track frame areas is calculated, and then the Hungarian algorithm is used for optimal matching. If the calculated IoU is less than the set IoU threshold, it means that the match is not successful; if it is greater than the set IoU threshold, it means that the pairing is successful, thus obtaining the first matching result, namely: the first successful matching track (Fmtrack), the first unsuccessful matching track (Futrack), and the first unsuccessful matching target detection frame (Fudetection).

[0103] A Kalman filter is used to update the detection information of Fmtrack (such as position, state, state estimation of the Kalman filter, and covariance matrix).

[0104] The third step is to perform a second match, matching the low-score detection frame Dlow and Futrack. The IoU and Hungarian algorithm are also used, but when comparing IoU, the IoU threshold is adjusted to obtain the second matching result, namely: the second successful matching track (Smtrack), the second unsuccessful matching track (Sutrack), and the second unsuccessful matching target detection frame (Sudetection).

[0105] Then, the detection information of Smtrack is updated (such as position, state, state estimation of Kalman filter and covariance matrix). For Sutrack, it is considered to be a temporarily lost target and is put into the temporarily lost queue. For Sudetection, it is considered to have low confidence and no successfully matched track, so it is discarded and not recorded.

[0106] The fourth step is to perform the third matching, matching Fudetection with the unconfirmed track. The IoU and Hungarian algorithm are also used, but when comparing IoU, the IoU threshold is adjusted again to obtain the third matching result, namely: the track that was successfully matched for the third time (Tmtrack), the track that was not successfully matched for the third time (Tutrack), and the target detection box that was not successfully matched for the third time (Tudetection).

[0107] Then, Tmtrack updates the detection information (such as position, state, Kalman filter state estimation and covariance matrix). For Tutrack, it is considered to be an unconfirmed track, which is an unreliable record in the existing track library. The track is not continuous, so it is directly discarded. For Tudetection, it is considered that they may be new targets, and a new target track ID is created and marked as unconfirmed track as the existing track library data of the subsequent frame, and tracking continues.

[0108] In the fifth step, the successfully matched tracks Fmtrack, Smtrack and Tmtrack are added to the target track queue; the tracks in the temporarily lost queue are judged by time. If these tracks exist in the queue for more than a certain time (such as 30 frames), they are considered to be permanently lost and deleted from the temporarily lost queue set; otherwise, they are added to the target track queue as new tracks.

[0109] All the trajectories in the target trajectory queue are checked for duplicate trajectories. After the check, a new target trajectory queue is obtained to prepare for the next frame. Repeat the above 5 steps until the trajectory tracking of all target detection frames is completed.

[0110] The second algorithm is the improved DeepSort (Deep Simple Online and Realtime Tracking) multi-target tracking algorithm, whose core principle is to use the deep learning model for target detection and combine the Kalman filter for target state prediction and data association to achieve continuous tracking of the target. Figure 3 As shown, the steps are as follows:

[0111] The first step is to initialize a deep neural network (i.e., initial feature model) to extract its appearance features. This model is pre-trained to learn the appearance features of the target detection box, such as color, texture, etc. These features are crucial for distinguishing different targets.

[0112] In the second step, two metrics are used for data association, motion metric (based on Kalman filter prediction) and appearance metric (based on deep feature extraction). In the motion metric process, the Kalman filter is used to predict the position of each trajectory in the existing trajectory library in the current frame, and the distance between these predicted positions and all targets detected in the current frame is calculated (existing distance calculation methods such as Euclidean distance can be used, which is not limited here) to obtain a distance matrix. In the appearance metric process, the similarity of the appearance features between each detected target and the trajectory in the existing trajectory library is calculated (existing similarity calculation methods such as cosine similarity calculation method can be used, which is not limited here), and a similarity feature matrix is ​​obtained.

[0113] Then, the two measurement result matrices are fused to form a comprehensive cost matrix, which is used to evaluate the matching quality between each detection box and the trajectory in the existing trajectory library. In the fusion process, weighted fusion can be performed according to the actual situation, which is not limited here.

[0114] The third step is to use the Hungarian Algorithm for matching. If the cost between the detection box and a track in the existing track library is low (which means a good match), then this track is added to the target track queue and updated. The updated information includes the track's position, speed, and other information.

[0115] If the detection box fails to match, it is considered a new target, and a new track ID is added to the target track queue.

[0116] If a track in the existing track library fails to match any target detection frame, it is considered that the target may have temporarily left the field of view or been blocked. The track will be marked as "unconfirmed" and added to the target track queue; however, if the track does not match a detection frame for several consecutive frames, it is considered that the target has left the field of view and the track is deleted.

[0117] The fourth step is to output the tracking results of all targets in each frame, that is, the target trajectory queue, including their positions, IDs and other related information, in preparation for tracking the next frame. When a new frame arrives, repeat the above steps to achieve continuous trajectory tracking of the target detection frame.

[0118] Of the two trajectory tracking algorithms mentioned above, the improved DeepSORT algorithm not only considers the target's motion trajectory, but also utilizes the target appearance features extracted by the deep learning model, which makes it more capable of target re-identification in complex scenarios. The improved ByteTrack algorithm is more concise, and its efficiency is improved by simplifying the matching strategy and reducing the need for feature extraction. Its matching strategy pays more attention to the speed and direction of the target. In general, DeepSORT is more suitable for application scenarios that require high accuracy, especially when the target appearance information is very important for distinguishing different objects. ByteTrack is more suitable for application scenarios that require real-time performance and computational efficiency, especially when dealing with a large number of fast-moving targets. Which algorithm to choose depends on the specific application requirements and resource constraints.

[0119] S4, matching the rectangular detection areas of all the targets in the target trajectory queue in turn, determining whether the target trajectory passes through each rectangular detection area, and performing statistics according to the rules, thereby obtaining the statistical results of the multi-target trajectories in each rectangular detection area, that is, the multi-target statistical tracking result; wherein the rules are:

[0120] Establish a global queue with the same number as the number of rectangular detection areas, and store the target trajectories passing through the corresponding rectangular detection areas in a one-to-one correspondence;

[0121] According to the current position of each target track in the rectangular detection area, the target track is added to the global queue corresponding to the current rectangular detection area, so as to obtain the statistical results of the target track in each rectangular detection area;

[0122] Two direction counter queues are defined to count the number of target trajectories that start from the two ends of the target video monitoring area and pass through each rectangular detection area in sequence, so as to realize the statistics of bidirectional moving targets in the target video monitoring area.

[0123] According to the trajectory tracking algorithm, a target trajectory queue can be obtained. In the target trajectory queue, the same target detection object is assigned the same ID. The position of the target detection object at a certain moment can be known, based on which further target statistics can be performed.

[0124] In this embodiment, opencv is used to draw a rectangular detection area in the video monitoring area, so that the monitoring area is divided into three areas (it can also be divided into other numbers of areas according to actual conditions). Figure 4 As shown, the left side of the rectangular area is marked as area 1 ( ), the rectangle is marked as area 2 ( ), the right side of the rectangle is marked as area 3 ( ).

[0125] In this embodiment, all target trajectories in the target trajectory queue are matched with a rectangular area to obtain the current position of the target trajectory relative to the rectangular area.

[0126] According to the number of rectangular areas, initialize the global queue set QUEUE={ , , }, two direction counter queues and .

[0127] If a target trajectory is located at position, add it to the global queue 1 ( ), and determine whether the previous frame of this target trajectory is in and If there is, it means that the target trajectory is from arrive At this time, calculator 1 ( ) is accumulated once; if the target trajectory is located at position, then directly join the global queue 2 ( ); If the target trajectory is located at position, then join the global queue 3 ( ), and determine whether the previous frame of the target trajectory passes through area1 and area2. If so, it means that the target trajectory is from arrive , then calculator 2 ( ) is accumulated once.

[0128] According to the above steps, all target tracks are matched with a rectangular area, and finally three global queues are obtained. , , , all target tracks in the queue are checked within a certain period of time (such as 300 frames) to see if the recorded rectangular area has changed. If not, it means that the target track has disappeared from the monitoring range, and the corresponding track record in the queue is cleared.

[0129] Execute the above process repeatedly to obtain the global queue QUEUE set and the counter queue of the statistical result and , in order to achieve two-way counting of moving targets (such as cars and people).

[0130] In another preferred embodiment, opencv is used to overlay the statistical results onto the video, and the video can be previewed and saved to the hard disk. The statistical data can also be written into the database at a certain time interval to facilitate the analysis of the flow of target objects (such as cars and people) in the monitoring area.

[0131] In addition, in another preferred embodiment, the hardware system of the present invention can also be provided with local and cloud dual preview functions, supporting real-time viewing and historical video playback, bringing unprecedented convenient experience to users.

[0132] To summarize, the present invention first trains a target detection model, then uses the target detection model to detect the target object, then uses a multi-target tracking algorithm to calculate the target trajectory, assigns each target object a trajectory ID, and finally uses a rectangular area counting method to perform target object statistics to obtain statistical data.

[0133] Compared with the prior art, the present invention has the following beneficial effects:

[0134] First, the present invention introduces a trajectory classification mechanism to clearly distinguish between "confirmed trajectories" and "unconfirmed trajectories". Confirmed trajectories refer to stable trajectories that have been successfully matched with the detection box and verified by multiple frames. On the contrary, if the detection box cannot find a match in the existing trajectory queue, it is regarded as a possible new target, marked as an unconfirmed trajectory, and temporarily added to the trajectory queue. This dynamic classification not only enhances the algorithm's ability to finely manage trajectories of different stability levels, but also significantly improves matching efficiency and accuracy. Compared with the prior art, the improved ByteTrack multi-target tracking algorithm of the present invention integrates the Kalman filter and the Hungarian algorithm, realizes the precise matching of the target detection box in the video frame with the existing trajectory, and further improves the environmental adaptability and tracking accuracy of the algorithm.

[0135] Secondly, during the matching process, a flexible IoU (intersection over union) comparison threshold adjustment strategy was implemented. Different IoU thresholds were set for different matching stages and target characteristics. In the first match, since high-scoring detection frames and confirmed trajectories are processed, the IoU threshold is set relatively high to ensure the accuracy of the match. Subsequently, in the second match, when faced with low-scoring detection frames (which may have reduced confidence due to occlusion, light changes, etc.) and trajectories that were not matched for the first time, the IoU threshold can be appropriately lowered to increase the possibility of a successful match. Finally, when dealing with unmatched detection frames and unconfirmed trajectories, the IoU threshold can be set between the first and second matches to achieve more detailed matching adjustments. This dynamic threshold adjustment mechanism effectively improves the adaptability of the algorithm in different scenarios and target motion states.

[0136] In addition, the present invention also introduces a loss queue and time judgment mechanism to handle the temporary loss of tracks more intelligently. When the target track disappears temporarily due to occlusion, false detection, etc., the algorithm will add it to the temporary loss queue and make a judgment based on the time factor. This mechanism significantly reduces the accidental deletion of tracks due to external interference, and further improves the robustness and stability of the algorithm.

[0137] The present invention not only optimizes the data processing process and improves statistical accuracy and efficiency, but also injects new vitality and wisdom into the construction and development of intelligent transportation and smart cities.

[0138] Embodiment 2

[0139] like Figure 5 As shown, the second embodiment of the present invention further provides a multi-target detection, tracking and statistics device based on edge computing, including:

[0140] An acquisition module, used to acquire target video data of a target video monitoring area to be detected; wherein the target video monitoring area is divided into a plurality of rectangular detection areas;

[0141] A target detection module, used to pre-process the target video data and extract the target detection frame of each video frame;

[0142] The multi-target tracking module is used to input the target detection frame of each video frame into the improved multi-target tracking algorithm for trajectory matching and tracking, and obtain the target trajectory queue of all targets in each video frame;

[0143] The result statistics module is used to match the rectangular detection areas of all targets in the target trajectory queue in turn, determine whether the target trajectory passes through each rectangular detection area, and perform statistics according to the rules, so as to obtain the statistical results of the multi-target trajectories in each rectangular detection area, that is, the multi-target statistical tracking results; wherein the rules are:

[0144] Establish a global queue with the same number as the number of rectangular detection areas, and store the target trajectories passing through the corresponding rectangular detection areas in a one-to-one correspondence;

[0145] According to the current position of each target track in the rectangular detection area, the target track is added to the global queue corresponding to the current rectangular detection area, so as to obtain the statistical results of the target track in each rectangular detection area;

[0146] Two direction counter queues are defined to count the number of target trajectories that start from the two ends of the target video monitoring area and pass through each rectangular detection area in sequence, so as to realize the statistics of bidirectional moving targets in the target video monitoring area.

[0147] Embodiment 3

[0148] The third embodiment of the present invention also provides a multi-target detection, tracking and statistics device based on edge computing, which includes a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement the multi-target detection, tracking and statistics method based on edge computing as described above.

[0149] Embodiment 4

[0150] The fourth embodiment of the present invention also provides a computer-readable storage medium, on which computer-readable storage medium is stored computer-readable instructions. When the computer-readable instructions are executed by a processor of a device where the computer-readable storage medium is located, the multi-target detection and tracking statistical method based on edge computing as described above is implemented.

[0151] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0152] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0153] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0154] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0155] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0156] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0157] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0158] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-target detection and tracking statistical method based on edge computing, characterized in that: include: S1, obtaining target video data of a target video monitoring area to be detected; wherein the target video monitoring area is divided into a plurality of rectangular detection areas; S2, preprocessing the target video data to extract the target detection frame of each video frame; S3, inputting the target detection frame of each video frame into the improved multi-target tracking algorithm for trajectory matching and tracking, and obtaining the target trajectory queue of all targets in each video frame; wherein the improved multi-target tracking algorithm is an improved ByteTrack multi-target tracking algorithm, including a Kalman filter and a Hungarian algorithm, and S3 specifically includes: Define and initialize an existing trajectory library, match the target detection frame of each video frame with the trajectory of the existing trajectory library, obtain the confirmed trajectory and unconfirmed trajectory of the current video frame, and mark and add them to the existing trajectory library; wherein the confirmed trajectory is the trajectory that the target detection frame successfully matches with the trajectory in the existing trajectory library; the unconfirmed trajectory is the trajectory that the target detection frame does not successfully match with the trajectory in the existing trajectory library; the existing trajectory library contains all the trajectories marked by the previous video frame, and the initialization state of the existing trajectory library is empty; According to the confidence of the target detection frame and a preset trajectory threshold, the target detection frame is divided into a high-score detection frame and a low-score detection frame; The high-score detection frame is matched with the confirmed track for the first time, by calculating the intersection over union (IoU) of the track box area of ​​the high-score detection frame and the confirmed track, and then comparing the IoU with the set IoU threshold, and obtaining the first matching result after optimal matching by the Hungarian algorithm; wherein the first matching result includes: the track Fmtrack that was successfully matched for the first time, the track Futrack that was not successfully matched for the first time, and the target detection frame Fudetection that was not successfully matched for the first time; The low-score detection frame is matched with Futrack for the second time, and the IoU of the two is calculated, the IoU threshold is adjusted, and the Hungarian algorithm is used for matching to obtain the second matching result; wherein the second matching result includes: the track Smtrack that was successfully matched for the second time, the track Sutrack that was not successfully matched for the second time, and the target detection frame Sudetection that was not successfully matched for the second time; Match Fudetection with the unconfirmed track for the third time, calculate the IoU of the two, adjust the IoU threshold, and match them using the Hungarian algorithm to obtain the third matching result; wherein the third matching result includes: the track Tmtrack that was successfully matched for the third time, the track Tutrack that was not successfully matched for the third time, and the target detection box Tudetection that was not successfully matched for the third time; Update the successfully matched tracks Fmtrack, Smtrack and Tmtrack, and add them to the target track queue; Initialize a temporary loss queue and add Sutrack to the temporary loss queue; discard Sudetection and Tutrack; Add the Tudetection to the target track queue and mark it as an unconfirmed track, update the existing track library, and continue to match the next video frame until all video frames in the target video data are completed; In the process of trajectory matching, a Kalman filter is used to predict the target position of the confirmed trajectory in the current video frame; S4, matching the rectangular detection areas of all the targets in the target trajectory queue in turn, determining whether the target trajectory passes through each rectangular detection area, and performing statistics according to the rules, thereby obtaining the statistical results of the multi-target trajectories in each rectangular detection area, that is, the multi-target statistical tracking result; wherein the rules are: Establish a global queue with the same number as the number of rectangular detection areas, and store the target trajectories passing through the corresponding rectangular detection areas in a one-to-one correspondence; According to the current position of each target track in the rectangular detection area, the target track is added to the global queue corresponding to the rectangular detection area, so as to obtain the statistical results of the target track in each rectangular detection area; Two direction counter queues are defined to count the number of target trajectories that start from the two ends of the target video monitoring area and pass through each rectangular detection area in sequence, so as to realize the statistics of bidirectional moving targets in the target video monitoring area.

2. According to the multi-target detection and tracking statistical method based on edge computing in claim 1, it is characterized in that ,Preprocessing the target video data includes: The target video data is input into the pre-trained YOLO series target detection model for frame-by-frame detection. After different scale feature processing, non-maximum suppression processing, and bounding box deduplication, the target detection frame of each video frame is obtained; wherein, the target detection frame includes the center point coordinates, confidence level and class number of the target frame.

3. According to the multi-target detection and tracking statistical method based on edge computing in claim 2, it is characterized in that When updating the successfully matched trajectory, the Kalman filter is used to predict and update the trajectory information; wherein the trajectory information includes the target ID, target position, target state, the state estimation of the Kalman filter and the covariance matrix of the Kalman filter.

4. A multi-target detection and tracking statistical method based on edge computing according to any one of claims 1 to 3, characterized in that , and also includes: performing time judgment on each trajectory in the temporarily lost queue, if the trajectory reappears within a preset time frame and is successfully matched, then adding it to the target trajectory queue; if the trajectory still does not appear after exceeding the preset time, then the trajectory is considered to be a permanently lost trajectory and is deleted from the temporarily lost queue.

5. According to the multi-target detection and tracking statistical method based on edge computing of claim 4, it is characterized in that , the S4 is specifically: The target video monitoring area is divided into m rectangular detection areas, which are divided into , ,..., ; Initialize the global queue set QUEUE={ , ,..., }; wherein the dimension of the global queue set QUEUE is the same as the number of the rectangular detection areas, and is used to store the target tracks passing through the corresponding rectangular detection areas, that is, Represents the first global queue, used to store target trajectory; Represents the mth global queue, used to store target trajectory; Define two direction counter queues and ; To count the target from The direction passes through each rectangular detection area in turn and is currently in The number of target trajectories at the location; To count the target from The direction passes through each rectangular detection area in turn and is currently in The number of target trajectories at the location; All target tracks in the target track queue are matched with the rectangular detection area in turn to obtain the current position of each target track in the rectangular detection area; if the current target track is in , then add the current target trajectory to ;in, , They represent the i-th rectangular detection area and the global queue respectively; If the current target trajectory is in , then add the current target trajectory to , and determine whether the target of the current target trajectory exists in other queues of QUEUE. If so, then The value increases by 1; otherwise The value remains unchanged; if the current target trajectory is in , then add the current target trajectory to , and determine whether the target of the current target trajectory exists in other queues of QUEUE. If so, then The value increases by 1; otherwise The value remains unchanged; until all target trajectories in the target trajectory queue are matched, and the global queue QUEUE set and the direction counter queue are obtained by statistics. and , in order to achieve statistics of two-way movement goals.

6. According to the multi-target detection and tracking statistical method based on edge computing of claim 5, it is characterized in that , also includes: regularly detecting the position change of the rectangular detection area where the target track in the global queue set QUEUE is located. If the current detected target track exists in the current rectangular detection area for more than a preset time, it is considered to be an invalid target track, and the target track is cleared from the global queue QUEUE to reduce the situation of track misdetection.

7. A multi-target detection, tracking and statistics device based on edge computing, characterized in that: include: An acquisition module, used to acquire target video data of a target video monitoring area to be detected; wherein the target video monitoring area is divided into a plurality of rectangular detection areas; A target detection module, used to pre-process the target video data and extract the target detection frame of each video frame; The multi-target tracking module is used to input the target detection frame of each video frame into the improved multi-target tracking algorithm for trajectory matching and tracking, and obtain the target trajectory queue of all targets in each video frame; wherein, the improved multi-target tracking algorithm is an improved ByteTrack multi-target tracking algorithm, which includes a Kalman filter and a Hungarian algorithm, specifically including: Define and initialize an existing trajectory library, match the target detection frame of each video frame with the trajectory of the existing trajectory library, obtain the confirmed trajectory and unconfirmed trajectory of the current video frame, and mark and add them to the existing trajectory library; wherein the confirmed trajectory is the trajectory that the target detection frame successfully matches with the trajectory in the existing trajectory library; the unconfirmed trajectory is the trajectory that the target detection frame does not successfully match with the trajectory in the existing trajectory library; the existing trajectory library contains all the trajectories marked by the previous video frame, and the initialization state of the existing trajectory library is empty; According to the confidence of the target detection frame and a preset trajectory threshold, the target detection frame is divided into a high-score detection frame and a low-score detection frame; The high-score detection frame is matched with the confirmed track for the first time, by calculating the intersection over union (IoU) of the track box area of ​​the high-score detection frame and the confirmed track, and then comparing the IoU with the set IoU threshold, and obtaining the first matching result after optimal matching by the Hungarian algorithm; wherein the first matching result includes: the track Fmtrack that was successfully matched for the first time, the track Futrack that was not successfully matched for the first time, and the target detection frame Fudetection that was not successfully matched for the first time; The low-score detection frame is matched with Futrack for the second time, and the IoU of the two is calculated, the IoU threshold is adjusted, and the Hungarian algorithm is used for matching to obtain the second matching result; wherein the second matching result includes: the track Smtrack that was successfully matched for the second time, the track Sutrack that was not successfully matched for the second time, and the target detection frame Sudetection that was not successfully matched for the second time; Match Fudetection with the unconfirmed track for the third time, calculate the IoU of the two, adjust the IoU threshold, and match them using the Hungarian algorithm to obtain the third matching result; wherein the third matching result includes: the track Tmtrack that was successfully matched for the third time, the track Tutrack that was not successfully matched for the third time, and the target detection box Tudetection that was not successfully matched for the third time; Update the successfully matched tracks Fmtrack, Smtrack and Tmtrack, and add them to the target track queue; Initialize a temporary loss queue and add Sutrack to the temporary loss queue; discard Sudetection and Tutrack; Add the Tudetection to the target track queue and mark it as an unconfirmed track, update the existing track library, and continue to match the next video frame until all video frames in the target video data are completed; In the process of trajectory matching, a Kalman filter is used to predict the target position of the confirmed trajectory in the current video frame; The result statistics module is used to match the rectangular detection areas of all targets in the target trajectory queue in turn, determine whether the target trajectory passes through each rectangular detection area, and perform statistics according to the rules, so as to obtain the statistical results of the multi-target trajectories in each rectangular detection area, that is, the multi-target statistical tracking results; wherein the rules are: Establish a global queue with the same number as the number of rectangular detection areas, and store the target trajectories passing through the corresponding rectangular detection areas in a one-to-one correspondence; According to the current position of each target track in the rectangular detection area, the target track is added to the global queue corresponding to the rectangular detection area, so as to obtain the statistical results of the target track in each rectangular detection area; Two direction counter queues are defined to count the number of target trajectories that start from the two ends of the target video monitoring area and pass through each rectangular detection area in sequence, so as to realize the statistics of bidirectional moving targets in the target video monitoring area.

Images