Multi-target tracking detection method and system based on cascade matching and trajectory confirmation

Abstract

The invention provides a multi-target tracking detection method and system based on cascade matching and trajectory confirmation, and relates to the technical field of digital video monitoring and tracking, and the method comprises the steps: obtaining a personnel construction video, and decomposing a video frame into a plurality of images; identifying a target in the current frame of image based on an improved YOLOv5 network, obtaining a target detection frame, and obtaining a prediction trajectory frame of the current frame through a Kalman filtering algorithm based on the trajectory frame of the previous frame; performing multi-target tracking on the target detection frame and the prediction trajectory frame of the current frame by using a DeepSort algorithm, respectively obtaining motion features and appearance features of the target detection frame and the prediction trajectory frame, performing cascade matching on the motion features and the appearance features of the target detection frame and the prediction trajectory frame by using a Hungary algorithm, and performing matching optimization by using an IOU matching algorithm; and carrying out fusion updating on the successfully matched target detection frame and prediction trajectory frame by using a Kalman filtering algorithm, and replacing the original prediction trajectory frame to obtain a tracking result.

Description

Technical Field

[0001] The present disclosure relates to the technical field of digital video surveillance and tracking, and in particular to a multi-target tracking and detection method and system based on cascade matching and trajectory confirmation. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] In transportation construction projects, bridge projects are increasing. Compared with highway and railway construction, bridge projects have certain complexity and danger, and there are more unsafe factors. Especially in the construction of viaducts, the construction methods are cross-changed, there are many high-altitude operations, the construction is difficult, and it is easy to interfere with the surrounding environment. These are the concentrated manifestations of the high risk of viaduct construction. In order to ensure the safe and orderly construction of viaducts, it is necessary to monitor and study the safety of personnel in viaduct construction in real time. Digital monitoring system is a system that uses digital technology and network communication technology to monitor and manage the site. This system usually includes components such as sensors, data acquisition equipment, data processing and analysis software, network communication equipment, etc., which can collect, transmit, process and display monitoring data in real time, helping users to conduct remote monitoring and real-time decision-making. During the construction of viaducts, it can help administrators and users realize remote monitoring, intelligent analysis and decision support, and improve the safety and reliability of personnel construction.

[0004] With the development of technologies such as the Internet of Things, big data, and artificial intelligence, the limitations of traditional digital monitoring functions have become increasingly exposed, mainly in terms of monitoring accuracy and video security. During the construction of viaducts, workers face many safety issues. As a manager, it is urgent to do a good job in on-site safety construction monitoring. Traditional digital monitoring video networks can realize basic alarm functions. When there is a fire, intrusion by outsiders, equipment failure, etc. on site, the alarm will be triggered in time, and there will be a central voice intercom or broadcast prompt, but the system is not perfect for monitoring the unsafe behavior of internal personnel.

[0005] Early machine learning detection algorithms learned shallow features from video data and achieved significant results in detecting abnormal video behavior through methods such as normal modeling. Methods such as random forests, Bayesian networks, Markov models, and support vector machines are used to understand and identify target behaviors, but these methods rely on preprocessing and handcrafted features, require a lot of time and resources to process, and cannot be well extended to different data sets, showing poor performance in practical applications. In addition, when the digital surveillance network is uploaded to the cloud, the traditional back-end digital port authentication method is vulnerable to attacks, and the resulting illegal intrusion will threaten the confidential security content. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a multi-target tracking and detection method and system based on cascade matching and trajectory confirmation, uses an improved YOLOv5 model for target positioning, and combines the deepsort algorithm to achieve multi-target tracking, continuously annotate target coordinates, dynamically monitor personnel behavior, and bind back-end devices to achieve dynamic monitoring and early warning.

[0007] According to some embodiments, the present disclosure adopts the following technical solutions:

[0008] The multi-target tracking detection method based on cascade matching and trajectory confirmation includes:

[0009] Obtain the construction video of the personnel and decompose the video frame into multiple images;

[0010] Based on the improved YOLOv5 network, the target in the current frame image is identified, the target detection frame is obtained, and the predicted trajectory frame of the current frame is obtained through the Kalman filter algorithm based on the trajectory frame of the previous frame;

[0011] The DeepSort algorithm is used to track multiple targets on the target detection frame and the predicted trajectory frame of the current frame, and the motion features and appearance features of the target detection frame and the predicted trajectory frame are obtained respectively. The Hungarian algorithm is used to perform cascade matching on the motion features and appearance features of the two, and the IOU matching algorithm is used to optimize the matching.

[0012] The successfully matched target detection frame and predicted trajectory frame are then fused and updated using the Kalman filter algorithm to replace the original predicted trajectory frame to obtain the tracking result.

[0013] According to some embodiments, the present disclosure adopts the following technical solutions:

[0014] The multi-target tracking and detection system based on cascade matching and trajectory confirmation includes:

[0015] A data acquisition module is used to acquire the construction video of personnel and decompose the video frame into multiple images;

[0016] The recognition module is used to identify the target in the current frame image based on the improved YOLOv5 network, obtain the target detection frame, and obtain the predicted trajectory frame of the current frame through the Kalman filter algorithm based on the trajectory frame of the previous frame;

[0017] The multi-target tracking module is used to track the target detection frame and the predicted trajectory frame of the current frame using the DeepSort algorithm, obtain the motion features and appearance features of the target detection frame and the predicted trajectory frame respectively, and use the Hungarian algorithm to cascade match the motion features and appearance features of the two, and use the IOU matching algorithm to optimize the matching;

[0018] The successfully matched target detection frame and predicted trajectory frame are then fused and updated using the Kalman filter algorithm to replace the original predicted trajectory frame to obtain the tracking result.

[0019] According to some embodiments, the present disclosure adopts the following technical solutions:

[0020] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the multi-target tracking and detection method based on cascade matching and trajectory confirmation is implemented.

[0021] According to some embodiments, the present disclosure adopts the following technical solutions:

[0022] An electronic device comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device implements the multi-target tracking and detection method based on cascade matching and trajectory confirmation.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The disclosed multi-target tracking detection method based on cascade matching and trajectory confirmation realizes target detection through YOLOv5 to determine the target coordinates; uses the deepsort algorithm to realize target tracking and continuously annotate the target coordinates. The video is decomposed into multiple images, and the improved YOLOv5 algorithm is used to perform target detection and execute frame by frame. It can be seen that the target tracking box moves with the target. When the worker's behavior appears in the module-specified state, it is connected to the Ruixiang Cloud intelligent alarm platform Cloud Alert, and the platform administrator identity information is called to realize SMS alarm. The integrated alarm connects the system to the platform and can receive alarms issued by the monitoring platform.

[0025] The disclosed multi-target tracking detection method based on cascade matching and trajectory confirmation applies the multi-target tracking DeepSort algorithm to the key figures of target detection, and improves the tracking accuracy through cascade matching and new trajectory confirmation. The trajectory information is divided into a confirmed state and an unconfirmed state. The new trajectory information is in an unconfirmed state and must be matched with the target detection frame for a certain number of consecutive times before it can be converted to a confirmed state. The confirmed trajectory information must be continuously mismatched with the target detection frame for a certain number of times before it will be deleted. Finally, in view of the shortcomings of the IOU matching algorithm, the CIOU matching algorithm is introduced to improve the accuracy of the matching of the detection frame and the prediction frame, and effectively deal with the occlusion and overlap problems in the target tracking process.

[0026] The disclosed multi-target tracking and detection method based on cascade matching and trajectory confirmation uses artificial intelligence algorithms to dynamically monitor personnel behavior before video integration, and binds back-end equipment to achieve dynamic monitoring and early warning. After video integration, it is post-processed and an invisible tag is added to the surveillance video as the basis for system access authentication, which strengthens the security characteristics of digital video in the form of a key. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0028] Figure 1 A key target person detection model based on improved YOLOv5 in an embodiment of the present disclosure;

[0029] Figure 2 It is the key target person detection process based on improved YOLOv5;

[0030] Figure 3 A key target person tracking model for cascade matching in an embodiment of the present disclosure;

[0031] Figure 4 It is the key target person tracking process of the cascade matching of the embodiment of the present disclosure;

[0032] Figure 5 The present invention is a method application flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0034] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0035] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0036] Example 1

[0037] In one embodiment of the present disclosure, a multi-target tracking and detection method based on cascade matching and trajectory confirmation is provided, and an artificial intelligence algorithm is used before video integration to realize multi-target tracking and detection, including:

[0038] Obtain the construction video of the personnel and decompose the video frame into multiple images;

[0039] Based on the improved YOLOv5 network, the target in the current frame image is identified, the target detection frame is obtained, and the predicted trajectory frame of the current frame is obtained through the Kalman filter algorithm based on the trajectory frame of the previous frame;

[0040] The DeepSort algorithm is used to track multiple targets on the target detection frame and the predicted trajectory frame of the current frame, and the motion features and appearance features of the target detection frame and the predicted trajectory frame are obtained respectively. The Hungarian algorithm is used to perform cascade matching on the motion features and appearance features of the two, and the IOU matching algorithm is used to optimize the matching.

[0041] The successfully matched target detection frame and predicted trajectory frame are then fused and updated using the Kalman filter algorithm to replace the original predicted trajectory frame to obtain the tracking result.

[0042] As an embodiment, the specific implementation process of the multi-target tracking detection method based on cascade matching and trajectory confirmation disclosed in the present invention includes:

[0043] Step 1: Obtain the construction video of the personnel and decompose the video frame into multiple images;

[0044] Specifically, opencv obtains the total number of video frames, sets a for loop, and reads images frame by frame.

[0045] Step 2: Identify the target in the current frame image based on the improved YOLOv5 network and obtain the target detection frame;

[0046] Specifically, based on the multiple images decomposed into the video, the improved YOLOv5 algorithm is used to perform target detection frame by frame, and the target detection box can be obtained to move with the target.

[0047] Among them, the YOLOv5 model structure mainly includes:

[0048] Input: The Head network of YOLOv5 consists of three different output layers, which are responsible for detecting large, medium and small scale targets respectively.

[0049] Backbone network: YOLOv5 uses CSPDarknet53 as its backbone network, which has strong feature extraction capabilities and computational efficiency.

[0050] Neck network: YOLOv5 uses an FPN network that can fuse information from different feature map levels.

[0051] Output: Loss function. YOLOv5 uses the Focal Loss loss function, which can alleviate the problem of class imbalance in target detection and improve the performance of the model. Non-maximum suppression (NMS). After outputting the result, YOLOv5 will perform NMS processing on the overlapping target boxes to obtain the final detection result.

[0052] Activation function, YOLOv5 uses the Mish activation function, which is an activation function that replaces ReLU and can improve the performance of the model.

[0053] At present, one-stage target detection algorithms such as YOLOv5 have the advantages of speed and accuracy, but face problems such as target misdetection, target missed detection, and target loss. The present invention mainly solves the above problems by optimizing YOLOv5 and DeepSort, and realizes efficient detection and tracking of key target characters in special scenes. YOLOv5 is improved from three aspects: attention mechanism, loss function, and feature pyramid network.

[0054] The improved method is: insert the CBAM attention mechanism module after the three effective feature layers 256, 512, and 1024 output by the YOLOv5 backbone network, respectively, to achieve effective extraction of key features. Secondly, considering the problem that the key target samples obtained in actual scenes are small and difficult to classify, a focal loss function that is more suitable for the one-stage target detection algorithm is used at the target confidence of the YOLO Head to replace the original BCE Loss, which is beneficial to improve the detection accuracy of difficult-to-classify samples. Finally, the weighted bidirectional feature pyramid network structure is used to replace the original feature fusion network FPN+PANet to achieve multi-scale fusion to make full use of effective features and improve detection accuracy. The key target person detection model based on the improved YOLOv5 is as follows: Figure 1 The key target detection process is as follows Figure 2 shown.

[0055] (1) Introduction of attention mechanism

[0056] Attention mechanism refers to the method of focusing on important areas and ignoring irrelevant parts in computer vision. Attention mechanisms can generally be divided into four categories: channel attention tells the network what to pay attention to, spatial attention tells the network where to pay attention, temporal attention tells the network when to pay attention, and branch attention tells the network what to pay attention to. This content mainly uses CBAM channel and spatial attention mechanism, which is a convolutional attention module. CBAM includes two sub-modules, CAM and SAM. Channel attention comes first and spatial attention comes later. After inputting the feature map, it first enters the channel attention, and obtains the feature value of each channel through global average pooling (GAP) and global maximum pooling (GMP). Then, the attention weight of each channel is calculated through the multi-layer perceptron (MLP), and the weight value is normalized using the Sigmoid function. Finally, the normalized attention weight is multiplied by the original feature map channel by channel to complete the channel attention calibration of the original feature.

[0057] (2) Loss function optimization training model

[0058] After dividing the video frames into training set and test set, the loss function is used to train the model to improve the model performance. FocalLoss is a loss function for difficult samples. Its core idea is to adjust the weight of each sample in the loss function according to the difficulty of the sample. When a sample is easy to classify, the corresponding weight is small; when a sample is difficult to classify, the corresponding weight is large.

[0059] (3) Feature pyramid optimization network model

[0060] The PANet structure is introduced, which adds a bottom-up path based on the top-down structure of FPN, supplements the positioning information ignored by FPN, and passes the strong positioning features of the underlying information to the upper layer. The basic training of the network model is completed, and the video test frames can be input into the network model for prediction.

[0061] Step 3: Based on the trajectory frame of the previous frame, the predicted trajectory frame of the current frame is obtained through the Kalman filter algorithm;

[0062] Specifically, the Kalman filter algorithm is used to predict and update the target motion trajectory. The prediction is based on the system's state equation and uses the trajectory estimate at the previous moment to infer the prior estimate of the trajectory at the current moment and the corresponding prior estimate of the error covariance; the update is to use the trajectory observation value at the current moment to correct the prior estimate of the trajectory obtained in the prediction stage to obtain the optimal estimate of the trajectory, that is, the trajectory box at the current moment.

[0063] Kalman filtering is an algorithm based on the linear system state equation, which can optimally estimate the system state through the observation data of the system input and output. Regardless of whether there is uncertain information in the dynamic system, Kalman filtering can help predict the next state of the system, and make reasonable treatment of various interferences. Through this method, the real signal estimate with the minimum error can be obtained in an average sense. The Kalman filter algorithm can be divided into two steps: prediction and update. The prediction is based on the state equation of the system, using the state estimate of the previous moment to deduce the prior estimate of the state variable at the current moment and the corresponding error covariance prior estimate. The prediction process can be regarded as an estimate of the current state. The formula is as follows:

[0064]

[0065] Update is to use the observation value at the current moment to correct the prior estimate obtained in the prediction stage to obtain the optimal estimate of the state variable, that is, the posterior estimate at the current moment. The formula is as follows:

[0066]

[0067] in and They represent the posterior estimates at time k-1 and time k respectively, that is, the optimal estimate obtained after updating the observation value at the previous moment. is the prior estimate at time k, which is the estimate at the current moment obtained by predicting the optimal estimate at the previous moment, i.e., the result of the prediction equation. k-1 and P k denote the posterior estimated covariance at time k-1 and time k respectively, and is the prior estimated covariance at time k, which is used to measure the accuracy of the estimated value at the current moment. H is used to convert state variables into measurement values, that is, to link state and observation. k Represents the observation value, which is used as the input of the filter to update and correct the state estimate. k It is the gain matrix of the filter, which is used to weigh the difference between the predicted value and the observed value. A is the state transfer matrix, which is used to describe the relationship between the motion law and state of the target at different time points. Q is the covariance matrix of the system process, which is used to describe the error between the state transfer matrix and the actual process, that is, the uncertainty of the system model. R is the measurement noise covariance matrix, which is used to describe the variance of the measurement error. B converts the input signal into a state matrix.

[0068] Step 4: Use the DeepSort algorithm to perform multi-target tracking on the target detection frame and predicted trajectory frame of the current frame, obtain the motion features and appearance features of the target detection frame and predicted trajectory frame respectively, and use the Hungarian algorithm to perform cascade matching on the motion features and appearance features of the two, and use the IOU matching algorithm to optimize the matching;

[0069] Specifically, the multi-target tracking DeepSort algorithm is applied to the key figures of target detection to improve the tracking accuracy through cascade matching and new trajectory confirmation. The trajectory information is divided into confirmed state and unconfirmed state. The new trajectory information is in the unconfirmed state and must match the target detection frame for a certain number of consecutive times before it can be converted to the confirmed state. The confirmed trajectory information must be continuously mismatched with the target detection frame for a certain number of times before it will be deleted.

[0070] First, the Kalman filter algorithm is used to predict and update the target motion trajectory. Prediction is to use the state estimate of the previous moment to infer the prior estimate of the state variable at the current moment and the corresponding error covariance prior estimate according to the state equation of the system. The prediction process can be regarded as an estimate of the current state. Update is to use the observation value at the current moment to correct the prior estimate obtained in the prediction stage to obtain the optimal estimate of the state variable, that is, the posterior estimate at the current moment. DeepSort uses a standard Kalman filter with uniform motion and linear observation model, and introduces an 8-dimensional vector to represent the position information and motion information of the character, thereby achieving more accurate position tracking and reducing the possibility of target misdetection. At the same time, for each tracking trajectory k, it is necessary to count the number of frames ak that have passed since the last successful association with the measurement. The number of frames increases continuously during the prediction period of the Kalman filter, but when the trajectory is successfully associated with the measurement again, ak will be reset to 0, and when ak exceeds the pre-defined maximum value, the trajectory will be deleted. If a target detection cannot be successfully matched with any existing trajectory, a new trajectory information will be reinitialized and marked as uncertain.

[0071] In the next three frames, if it still cannot be successfully associated with any measurement target, the track will be deleted.

[0072] The Hungarian algorithm is applied to solve the association matching problem between the Kalman filter prediction frame and the target detection frame. First, the ResNet101 network is used to extract the motion features and appearance features of the detected object, and the cost matrix is ​​calculated by combining the motion features and appearance features. The cosine distance is used to measure the similarity between the two target features. Finally, the CIOU matching algorithm is introduced to complete the matching of the detection frame and the prediction frame, which effectively deals with the occlusion and overlap problems in the target tracking process. The specific key target person tracking model is as follows Figure 3 The key target person tracking flow chart is as follows Figure 4 shown.

[0073] The CIOU matching algorithm can measure the degree of overlap between the predicted box and the true box. The higher the value, the more accurate the model prediction, and vice versa, the worse the model performance.

[0074]

[0075] Where A is the predicted box, B is the real box, and C is the minimum circumscribed rectangular area. When A and B have no overlapping parts, the smaller the GIOU value is, the greater the distance between the two elements is. When the two boxes completely overlap, the GIOU value is 1.

[0076] As an embodiment, the application process of the multi-target tracking detection method based on cascade matching and trajectory confirmation disclosed in the present invention is:

[0077] The improved YOLOv5 is used as the detector to identify the current frame image in the tracking video to obtain the detection frame. Then, based on the Kalman filter algorithm, the predicted trajectory frame of the current frame is obtained according to the trajectory of the previous frame. The improved deep feature extraction network is used to re-identify the pedestrian features. The detection frame and the prediction frame are combined with motion features and appearance features using the Hungarian algorithm for association matching. At the same time, the IOU matching algorithm is optimized after cascade matching. Finally, the successfully matched detection frame and prediction frame are fused and updated using the Kalman filter to replace the original prediction frame to obtain the tracking result.

[0078] According to the tracking results, the dangerous behaviors of workers are integrated, and the network model is trained according to the types of behaviors to solve the classification problem. Two valid behaviors are initially stipulated, namely: climbing over and not wearing a safety helmet. The video key frame is pre-processed according to the valid behaviors, mainly setting the image enhancement effect. After the enhancement processing, the model is pre-trained, and the model uses a CNN network. After deep neural network learning, target recognition is achieved after video integration, and the trained classification model is input for each target to obtain warning information. Distinguish whether it belongs to behavior A or behavior B. When behaviors AB occur, the digital video surveillance network will automatically connect to the Ruixiang Cloud intelligent alarm platform Cloud Alert, and the back-end device will generate warning information.

[0079] Video integration to the cloud is vulnerable to illegal attacks, which can interfere with the normal construction process of workers in serious cases, so a secure network environment is also necessary. For massive video data, the unique noise generated by the video is used for security construction. The fingerprint noise is hidden in the video sequence in the form of a specific password (video generated noise: PRNU) through reversible information hiding technology, which can prevent illegal attacks on video integrators.

[0080] Specifically, the image is analyzed frame by frame before video integration. First, the integer wavelet transform technology is applied to extract the high-frequency part, which can represent the fingerprint information characteristics. The fingerprint information characteristics are suppressed, and an image that is unrelated to the chip fingerprint of the source device is output. Then the suppressed PRNU is embedded into the output image of the previous step to obtain reversibility. The embedding method adopts a reversible information hiding algorithm based on CDM. Spread spectrum sequences play an important role in CDM-based communication systems. Generally, the Walsh Hadamard matrix is ​​used to derive the spread sequence. It consists of "1" or "-1", and every two rows or columns are orthogonal to each other. Secret information can be embedded based on the orthogonality of the spread sequence. A major advantage of reversible information hiding technology is that the embedded information and the original information can be restored without loss. Therefore, when the back-end device requests access, we can recover the embedded information (PRNU) and perform a PCE matching algorithm with the video integrated fingerprint information. It is stipulated that if the PCE value is ≥50, it is considered to be related, and access is allowed, which further organizes the attack of illegal devices.

[0081] The surveillance video information suppressed by fingerprint information is integrated into the network cloud and deployed in a safe environment. Key personnel are tracked through target detection and multi-target tracking. When the video worker exhibits two specified valid behaviors, the behavior module is called to generate a pool code to pre-classify dangerous behaviors. At the same time, the early warning platform access is activated and the early warning information is sent to the manager's mobile phone for the manager to make smart decisions.

[0082] Example 2

[0083] In one embodiment of the present disclosure, a multi-target tracking and detection system based on cascade matching and trajectory confirmation is provided, comprising:

[0084] A data acquisition module is used to acquire the construction video of personnel and decompose the video frame into multiple images;

[0085] The recognition module is used to identify the target in the current frame image based on the improved YOLOv5 network, obtain the target detection frame, and obtain the predicted trajectory frame of the current frame through the Kalman filter algorithm based on the trajectory frame of the previous frame;

[0086] The multi-target tracking module is used to track the target detection frame and the predicted trajectory frame of the current frame using the DeepSort algorithm, obtain the motion features and appearance features of the target detection frame and the predicted trajectory frame respectively, and use the Hungarian algorithm to cascade match the motion features and appearance features of the two, and use the IOU matching algorithm to optimize the matching;

[0087] The successfully matched target detection frame and predicted trajectory frame are then fused and updated using the Kalman filter algorithm to replace the original predicted trajectory frame to obtain the tracking result.

[0088] Example 3

[0089] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the multi-target tracking and detection method based on cascade matching and trajectory confirmation is implemented.

[0090] Example 4

[0091] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the multi-target tracking and detection method based on cascade matching and trajectory confirmation.

[0092] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0094] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A multi-target tracking and detection method based on cascade matching and trajectory confirmation, characterized in that: include: Obtain the construction video of the personnel and decompose the video frame into multiple images; Based on the improved YOLOv5 network, the target in the current frame image is identified, the target detection frame is obtained, and the predicted trajectory frame of the current frame is obtained through the Kalman filter algorithm based on the trajectory frame of the previous frame; The DeepSort algorithm is used to track multiple targets on the target detection frame and the predicted trajectory frame of the current frame, and the motion features and appearance features of the target detection frame and the predicted trajectory frame are obtained respectively. The Hungarian algorithm is used to perform cascade matching on the motion features and appearance features of the two, and the IOU matching algorithm is used to optimize the matching. The successfully matched target detection frame and predicted trajectory frame are then fused and updated using the Kalman filter algorithm to replace the original predicted trajectory frame to obtain the tracking result.

2. The multi-target tracking and detection method based on cascade matching and trajectory confirmation as claimed in claim 1, characterized in that: The CBAM attention mechanism is introduced behind the backbone network feature layer of the improved YOLOv5 network to extract key features, and the focus loss function of the target detection algorithm is used at the target confidence of the YOLO Head to replace the original BCELoss loss function.

3. The multi-target tracking and detection method based on cascade matching and trajectory confirmation as claimed in claim 2, characterized in that: The weighted bidirectional feature pyramid network structure is used to replace the original feature fusion network FPN+PANet to achieve multi-scale fusion, identify pedestrians, determine the coordinates of the target, and form a target detection frame.

4. The multi-target tracking and detection method based on cascade matching and trajectory confirmation as claimed in claim 1, characterized in that: The multi-target tracking DeepSort algorithm is applied to the target detection frame, and tracking is achieved through cascade matching and new trajectory confirmation. The trajectory information is divided into confirmed and unconfirmed states. The new trajectory information is in the unconfirmed state and must match the target detection frame continuously for a set number of times before it can be converted to the confirmed state. The trajectory information in the confirmed state must mismatch the target detection frame continuously for a set number of times before it will be deleted.

5. The multi-target tracking and detection method based on cascade matching and trajectory confirmation as claimed in claim 1, characterized in that: The Kalman filter algorithm is used to predict and update the target motion trajectory. The prediction is based on the system's state equation and uses the trajectory estimate at the previous moment to infer the prior estimate of the trajectory at the current moment and the corresponding prior estimate of the error covariance; the update is to use the trajectory observation value at the current moment to correct the prior estimate of the trajectory obtained in the prediction stage to obtain the optimal estimate of the trajectory, that is, the trajectory frame at the current moment.

6. The multi-target tracking and detection method based on cascade matching and trajectory confirmation as claimed in claim 1, characterized in that: The DeepSort algorithm uses a standard Kalman filter with uniform motion and linear observation models, and introduces an 8-dimensional vector to represent the position and motion information of the character. For each tracking trajectory k, it is necessary to count the number of frames ak that have passed since the last successful association with the measurement. The number of frames continues to increase during the prediction period of the Kalman filter. When the trajectory is successfully associated with the measurement again, ak will be reset to 0, and when ak exceeds the pre-defined maximum value, the trajectory will be deleted.

7. The multi-target tracking and detection method based on cascade matching and trajectory confirmation as claimed in claim 1, characterized in that: The Hungarian algorithm is applied to perform association matching between the Kalman filter prediction box and the target detection box. The ResNet101 network is used to extract the motion features and appearance features of the detected object. The motion features and appearance features are combined to calculate the cost matrix. The cosine distance is used to measure the similarity between the two target features. The CIOU matching algorithm is introduced to achieve the optimal matching.

8. A multi-target tracking and detection system based on cascade matching and trajectory confirmation, characterized in that: include: A data acquisition module is used to acquire the construction video of personnel and decompose the video frame into multiple images; The recognition module is used to identify the target in the current frame image based on the improved YOLOv5 network, obtain the target detection frame, and obtain the predicted trajectory frame of the current frame through the Kalman filter algorithm based on the trajectory frame of the previous frame; The multi-target tracking module is used to track the target detection frame and the predicted trajectory frame of the current frame using the DeepSort algorithm, obtain the motion features and appearance features of the target detection frame and the predicted trajectory frame respectively, and use the Hungarian algorithm to cascade match the motion features and appearance features of the two, and use the IOU matching algorithm to optimize the matching; The successfully matched target detection frame and predicted trajectory frame are then fused and updated using the Kalman filter algorithm to replace the original predicted trajectory frame to obtain the tracking result.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by the processor, the multi-target tracking and detection method based on cascade matching and trajectory confirmation as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes the multi-target tracking and detection method based on cascade matching and trajectory confirmation as described in any one of claims 1 to 7.

Images