Multi-target detection tracking system and method based on deep learning

Through a multi-object detection and tracking system combined with deep learning and Kalman filtering algorithm, the detection difficulty problem caused by target occlusion in the AGV car interactive environment is solved, and more accurate and stable trajectory matching is achieved, which enhances the performance and stability of the system.

CN119131358AActive Publication Date: 2024-12-13CNBM RES INST FOR AUTOMATION OF LIGHT IND CO LTD

Patent Information

Application Number
CN202411190784.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-13
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

In the AGV car interactive operation environment, the multi-object detection and tracking system is prone to increase detection difficulty due to target occlusion, and low confidence targets are ignored, resulting in trajectory interruption and loss, affecting the stability and accuracy of tracking.

Method used

A multi-object detection and tracking system based on deep learning is adopted, combined with Kalman filtering algorithm and four-stage matching algorithm, through high-score and low-score matching, BIoU motion similarity measurement is used to ensure the accuracy and continuity of trajectory matching.

Benefits of technology

It improves the accuracy of object detection and tracking, reduces missed and missed detection, enhances the stability and performance of the system, can effectively deal with occlusion and noise interference, and maintains the consistency of the trajectory.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119131358A_ABST
    Figure CN119131358A_ABST
Patent Text Reader

Abstract

The invention provides a multi-target detection tracking system and method based on deep learning, and the system comprises an acquisition module which is used for carrying out the camera shooting of an AGV trolley, and collecting image information; the detection module is used for detecting a target in the acquired image information by using a deep learning model and extracting appearance features of the plurality of AGVs; the tracking module is used for predicting the movement tracks of the tracks of the multiple AGVs at the last moment at the new moment by using a Kalman filtering algorithm, and finally performing correlation calculation by using a four-stage matching algorithm to complete multi-target tracking; according to the output module, the system outputs and displays a tracking result, optimization processing is carried out on specific conditions in each stage from high-confidence object matching to low-confidence object matching to historical position matching and unconfirmed track matching, and compared with traditional optimization tracking carried out on high-confidence object matching singly, the tracking efficiency is improved, and the tracking efficiency is improved. Therefore, the matching accuracy can be improved, and the situations of missing detection and false detection can be reduced to the greatest extent.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of target tracking technology, and in particular to a multi-target detection and tracking system based on deep learning and a method thereof. Background Art

[0002] In the application of AGV, the multi-target detection and tracking system based on deep learning can greatly improve the autonomous navigation and handling capabilities of the AGV in complex environments. The multi-target detection and tracking system based on deep learning is a complex and efficient computer vision system that combines deep learning and target detection and tracking algorithms to achieve accurate detection and tracking of multiple targets in continuous video frames. The following are some core concepts and basic principles of the system. First, target detection is one of the key steps of the system. By using deep learning target detection algorithms, the system can detect all targets of interest in each frame and obtain their location information. These algorithms extract image features through convolutional neural networks to identify the location and shape of the target objects. Secondly, feature extraction is another important link. For each detected target, the system extracts its visual features (such as color, texture, shape, etc.) or motion features (such as speed, acceleration, etc.). These features play a key role in the subsequent tracking process and help match and identify the same target in continuous frames.

[0003] In the existing multi-target detection and tracking system, especially in the environment involving multiple AGVs interacting with each other, mutual occlusion does occur frequently, which makes target detection more difficult. In particular, the confidence score of the occluded target will be significantly reduced. Therefore, when the traditional tracking algorithm that only relies on high-confidence detection frames is used again, it is easy to ignore these low-confidence detection objects, resulting in the interruption or even loss of the target trajectory, which in turn affects the stability and accuracy of tracking.

[0004] Therefore, it is necessary to provide a new multi-target detection and tracking system based on deep learning and its method to solve the above technical problems. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a multi-target detection and tracking system and method based on deep learning.

[0006] The multi-target detection and tracking system and method based on deep learning provided by the present invention include: A multi-target detection and tracking system based on deep learning, comprising: Acquisition module: This module is used to take pictures of the AGV and collect image information; Detection module: Use deep learning models to detect targets in collected image information and extract appearance features of multiple AGVs; Tracking module: collects the appearance features of the AGV in the detection module, uses the Kalman filter algorithm to predict the movement trajectories of multiple AGVs at the new moment, and finally uses the processing unit to perform correlation calculations to complete multi-target tracking; Output module: The system collects the multi-target tracking results in the tracking module and outputs and displays the tracking results.

[0007] Furthermore, the data association method is a BYTE data association method, and its specific use process is as follows: S01: First, each detection frame is divided into two categories: high-score frame and low-score frame, and matched twice; S02: The first match is performed using the high-resolution frame and the previous tracking track, and the second match is performed using the low-resolution frame and the tracking track that was not matched with the high-resolution frame in the first match.

[0008] Furthermore, the specific steps of extracting appearance features of multiple AGVs in the detection module are as follows: S001: Receive image information collected by the acquisition module, annotate the collected images, mark the position and bounding box of the AGV, and form a data set for training the YOLO model; S002: Use the labeled data set to train the selected YOLO model, and adjust the model parameters and optimizer settings to enable the model to accurately identify the AGV car; S003: Input the image to be detected into the trained model. The model will output the position and confidence of the AGV in the image. The overlapping bounding boxes are removed by the non-maximum suppression method to obtain the final detection result. S004: Based on target detection, a pre-trained convolutional neural network can be used as a feature extractor to extract the appearance features of the AGV.

[0009] Furthermore, the device for acquiring images in the acquisition module is a camera.

[0010] Furthermore, the tracking result outputted in the output module is in the form of a picture.

[0011] Furthermore, the processing unit performs association calculation to complete the steps of multi-target tracking as follows: S1: The AGV driving time period is divided into the trajectory set at time t-1, which is divided into tracking trajectory, unconfirmed trajectory, and lost trajectory. The tracking trajectory and the lost trajectory are predicted by Kalman filtering to obtain the predicted trajectory position at time t; S2: High-confidence object matching: Based on the BYTE data association method, the predicted position information of the AGV at time t is matched with the high-score detected object with high confidence; S3: For the track that failed to match the high-confidence object, match it with the low-confidence detection object, where the track that failed to match is matched with the low-score detection object as a low-confidence object; S4: Historical position matching: For the track that failed in the first two matches, the historical position is matched with the detected object. The historical position is the position of the AGV in the t-1 period. S5: Unconfirmed trajectory matching: for high-confidence objects among the detected objects that also failed historical position matching, they are matched with the unconfirmed trajectory; S6: Based on the results of the above four successfully matched trajectories, the system updates the trajectory set of the AGV at time t. The successfully matched trajectories will be updated according to the latest position and status information; the unsuccessfully matched trajectories are marked as lost or removed from the current tracking set; at the same time, newly detected objects that have not yet been associated will be added to the unconfirmed trajectory set.

[0012] Furthermore, the two matches in S2 and S3, which use high-confidence object matching and low-confidence detection object matching, adopt the BIoU motion similarity metric. The BIoU motion similarity metric can more accurately capture the contour changes of the moving target by emphasizing boundary matching, thereby improving the accuracy of moving target detection and tracking.

[0013] The method of multi-target detection based on deep learning is applied to the above-mentioned multi-target detection and tracking system based on deep learning, and includes the following steps: S0001: First, the acquisition module takes a picture of the AGV and collects the image information of the AGV; S0002: Receive image information collected by the acquisition module, annotate the collected images, mark the position and bounding box of the AGV, and form a data set for training the YOLO model. Use the annotated data set to train the selected YOLO model. Adjust the model parameters and optimizer settings to enable the model to accurately identify the AGV. Input the image to be detected into the trained model. The model will output the position and confidence of the AGV in the image. Remove the overlapping bounding boxes through the non-maximum suppression method to obtain the final detection result. On the basis of target detection, the pre-trained convolutional neural network can be used as a feature extractor to extract the appearance features of the AGV. S0003: Then, in the tracking module, by receiving the appearance feature data of the AGV, the Kalman filter algorithm is used to predict the motion trajectories of multiple AGVs at the previous moment and at the new moment, and finally the four-stage matching algorithm is used to perform association calculation to complete multi-target tracking; S0004: Display the tracking result through the output module, and the tracking result is presented in the form of a picture.

[0014] Compared with the related art, the multi-target detection and tracking system and method based on deep learning provided by the present invention have the following beneficial effects: The present invention predicts the motion trajectories of multiple AGVs at the new moment by using the Kalman filter algorithm, and finally uses the four-stage matching algorithm to perform association calculation to complete multi-target tracking. When used specifically, the time period of the AGV is divided into the t-1 moment trajectory set, which is divided into tracking trajectory, unconfirmed trajectory, and lost trajectory. For the trajectory that fails to match the high-confidence object, it is matched with the low-confidence detection object, wherein the failed matching trajectory is matched with the low-score detection object for the low-confidence object, for the trajectory that fails to match the first two times, it is matched with the detection object for the historical position, and for the high-confidence object in the detection object that also fails to match the historical position, it is matched with the unconfirmed trajectory. According to the results of the above four successfully matched trajectories, the system updates the trajectory set of the AGV at time t, and the successfully matched trajectory will be updated according to the latest position and status information; The four-stage matching algorithm can comprehensively and finely handle trajectory matching problems in different situations, from high-confidence object matching to low-confidence object matching, to historical position matching, and unconfirmed trajectory matching. Each stage is optimized for specific situations. Compared with the traditional method of only optimizing and tracking high-confidence object matching, this can not only improve the matching accuracy, but also minimize missed detections and false detections. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The overall block diagram of the multi-target detection and tracking system based on deep learning provided by the present invention; Figure 2 A flowchart of a four-stage matching algorithm for a multi-target detection and tracking system based on deep learning provided by the present invention; Figure 3 Flowchart of the multi-target detection and tracking method based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0017] Please refer to Figure 1 , Figure 2, Figure 3 ,in, Figure 1 The overall block diagram of the multi-target detection and tracking system based on deep learning provided by the present invention; Figure 2 A flowchart of a four-stage matching algorithm for a multi-target detection and tracking system based on deep learning provided by the present invention; Figure 3 Flowchart of the multi-target detection and tracking method based on deep learning provided by the present invention.

[0018] In the specific implementation process, Figure 1-Figure 3 As shown, a multi-target detection and tracking system based on deep learning includes: Acquisition module: This module is used to take pictures of the AGV and collect image information; Detection module: Use deep learning models to detect targets in collected image information and extract appearance features of multiple AGVs; Tracking module: collects the appearance features of the AGV in the detection module, uses the Kalman filter algorithm to predict the movement trajectories of multiple AGVs at the new moment, and finally uses the processing unit to perform correlation calculations to complete multi-target tracking; Output module: The system collects the multi-target tracking results in the tracking module and outputs and displays the tracking results; The processing unit performs association calculation and completes multi-target tracking in a four-stage matching algorithm. The specific steps are as follows: S1: The AGV driving time period is divided into the trajectory set at time t-1, which is divided into tracking trajectory, unconfirmed trajectory, and lost trajectory. The tracking trajectory and the lost trajectory are predicted by Kalman filtering to obtain the predicted trajectory position at time t; S2: High-confidence object matching: Based on the BYTE data association method, the predicted position information of the AGV at time t is matched with the high-score detected object with high confidence; S3: For the track that failed to match the high-confidence object, match it with the low-confidence detection object, where the track that failed to match is matched with the low-score detection object as a low-confidence object; S4: Historical position matching: For the track that failed in the first two matches, the historical position is matched with the detected object. The historical position is the position of the AGV in time period t. In the specific implementation process, if the first two matches fail, the system will try to use historical position information for matching. The historical position is the known position of the AGV within the time period t. They may come from previous sensor data or trajectory records. By comparing the historical position with the current position of the detected object, the system can further narrow the matching range and improve the matching accuracy. S5: Unconfirmed trajectory matching: for high-confidence objects among the detected objects that also failed historical position matching, they are matched with the unconfirmed trajectory; In the specific implementation process, for those high-confidence detection objects that have not been matched in the historical position matching, the system matches them with unconfirmed tracks. Unconfirmed tracks are those newly detected tracks that have not been matched with any known tracks. By matching with unconfirmed tracks, the system may be able to discover new AGVs that have not been noticed before; S6: Based on the results of the above four matching stages, the system updates the trajectory set at time t. The successfully matched trajectories will be updated according to the latest position and status information; unsuccessfully matched trajectories may be marked as lost or removed from the current tracking set; at the same time, newly detected and unassociated objects will be added to the unconfirmed trajectory set; The specific process of the Kalman filter algorithm mentioned above is as follows, where the state vector D of the Kalman filter is ,in, and is the center coordinate of the target box; is the aspect ratio of the target box; is the length of the target box; Respectively represent the corresponding parameters speed, is the lateral acceleration and longitudinal acceleration of the target; Let X = , Y= , Z = , then the target information of the AGV is expressed as:

[0019] The prediction process of the Kalman filter algorithm mentioned above is:

[0020] in, is the predicted value of the position information of the AGV in the previous frame, is the optimal estimate of the position information of the AGV, Z is the state transfer matrix of the AGV, B is the control input matrix, P is the estimated covariance matrix of the AGV, and Q is the state transfer covariance matrix of the AGV. The state transfer matrix mentioned above describes how the system state is transferred from the previous moment to the current moment. The estimated covariance matrix is ​​used to quantify the uncertainty of the AGV state estimation. This matrix contains the information of the covariance between the state variables of the AGV, which describes the accuracy of the state estimation and the correlation between the state variables. The state transfer covariance matrix describes the uncertainty of the predicted state of the AGV. This matrix contains the information of the covariance between the state variables of the AGV, which is used to estimate the uncertainty of the state variables and the correlation between them. The Kalman filter algorithm predicts the target position information of the current frame based on the target information of the previous frame, and compares it with the target position information output by the detection link. The application of the Kalman filter algorithm can significantly improve the accuracy and stability of target tracking. It can predict the motion state at a new moment based on the trajectory information of the AGV car at the previous moment, effectively deal with interference factors such as occlusion and noise that may occur during the target movement, reduce the loss of the target, and further improve the tracking accuracy; The specific use process of the BYTE data association method is as follows: S01: First, each detection frame is divided into two categories: high-score frames and low-score frames, and two matches are performed; S02: The first match is performed using the high-resolution frame and the previous tracking track, and the second match is performed using the low-resolution frame and the tracking track that was not matched with the high-resolution frame in the first match; The above process is particularly helpful in cases where the object is occluded, by using the position overlap between the detection frame and the trajectory to dig out the real object from the low-score frame, thereby maintaining the continuity of the trajectory; The specific steps for extracting appearance features of multiple AGVs in the detection module are as follows: S001: Receive image information collected by the acquisition module, annotate the collected images, mark the position and bounding box of the AGV, and form a data set for training the YOLO model; S002: Use the labeled data set to train the selected YOLO model, and adjust the model parameters and optimizer settings to enable the model to accurately identify the AGV car; S003: Input the image to be detected into the trained model. The model will output the position and confidence of the AGV in the image. The overlapping bounding boxes are removed by the non-maximum suppression method to obtain the final detection result. S004: Based on target detection, a pre-trained convolutional neural network can be used as a feature extractor to extract the appearance features of the AGV.

[0021] The device for collecting images in the acquisition module is a camera, and the tracking results output in the output module are in the form of pictures. The two matches in S2 and S3 use high-confidence object matching and low-confidence detection object matching, which adopt the BIoU motion similarity measurement; Among them, the BIoU motion similarity measurement method is as follows: First, determine the boundary lines of the predicted bounding box and the true bounding box, and calculate the intersection length between the two boundary lines. This can be achieved by finding the overlapping part of the two boundary lines and calculating their lengths. Then, calculate the total length of the boundary lines of the predicted bounding box and the true bounding box. This can be achieved by adding the lengths of the two boundary lines. Finally, divide the boundary intersection length by the boundary union length to obtain the BIoU value. The higher the BIoU value, the higher the boundary matching degree between the predicted bounding box and the true bounding box, that is, the higher the similarity. Through the BIoU motion similarity measurement method, since BIoU pays special attention to the boundary matching degree between the predicted bounding box and the true bounding box, in the AGV car motion scene, the boundary of the AGV car often carries important information, such as the shape and posture of the AGV car. Therefore, by emphasizing boundary matching, BIoU can more accurately capture the contour changes of the moving target, thereby improving the accuracy of moving target detection and tracking. The BIoU motion similarity measurement method described above is usually the IoU measurement method in traditional measurement methods, and its calculation method is IoU = intersection area / (prediction area + real area - intersection area), where the intersection area refers to the area of ​​the intersection of the prediction result and the real result, the prediction area refers to the area of ​​the prediction result, and the real area refers to the area of ​​the real result. By adopting the BIoU motion similarity measurement method, BIoU not only considers the overlap between the target bounding boxes, but also incorporates the motion information of the AGV car, so that BIoU can more comprehensively evaluate the similarity of the target between consecutive frames, especially when the AGV car moves faster or there are multiple AGV car targets in the scene, BIoU can provide more accurate measurement results, which helps to reduce the loss and false detection of AGV cars during tracking, and further improve the performance of the entire tracking system; It should be noted that the specific training method of the above YOLO model is as follows: First, you need to prepare a dataset for training. The dataset should contain a large number of annotated images, in which the AGV in each image is accurately annotated. Usually, the dataset is divided into training set, validation set and test set to verify the performance of the model during the training process and finally evaluate the accuracy of the model. Install the necessary deep learning framework and dependent libraries, select the appropriate YOLO version, and the YOLOv7 version is selected here. Then configure the parameters of the model, such as learning rate, batch size, number of iterations, etc., and use the training set to train the model. During the training process, the model will learn how to identify the target objects in the image and optimize its internal parameters to improve the recognition accuracy. During the training process, the validation set is used to evaluate the performance of the model. According to the verification results, the parameters or structure of the model can be adjusted to optimize its performance. This process may require multiple iterations until the performance of the model on the validation set is satisfactory. Finally, the trained model is tested and evaluated using the test set. The test set is independent of the training set and validation set, and is used to finally evaluate the generalization ability and accuracy of the model. In the specific implementation process, the convolutional neural network process used above is as follows: the image is first input for preprocessing to convert the image into a standard size; the preprocessed image is input into the encoder to extract features and generate a high-resolution feature map; the high-resolution feature map passes through the target detection branch network and the appearance feature extraction branch network to generate the appearance features of the AGV car; The method of multi-target detection based on deep learning is applied to the multi-target detection and tracking system based on deep learning mentioned above, and includes the following steps: S0001: First, the acquisition module takes a picture of the AGV and collects the image information of the AGV; S0002: Receive image information collected by the acquisition module, annotate the collected images, mark the position and bounding box of the AGV, and form a data set for training the YOLO model. Use the annotated data set to train the selected YOLO model. Adjust the model parameters and optimizer settings to enable the model to accurately identify the AGV. Input the image to be detected into the trained model. The model will output the position and confidence of the AGV in the image. Remove the overlapping bounding boxes through the non-maximum suppression method to obtain the final detection result. On the basis of target detection, the pre-trained convolutional neural network can be used as a feature extractor to extract the appearance features of the AGV. S0003: Then, in the tracking module, by receiving the appearance feature data of the AGV, the Kalman filter algorithm is used to predict the motion trajectories of multiple AGVs at the previous moment and at the new moment, and finally the four-stage matching algorithm is used to perform association calculation to complete multi-target tracking; S0004: Display the tracking result through the output module, and the tracking result is presented in the form of a picture; The method of removing overlapping bounding boxes by using the non-maximum suppression method in step S0002 is specifically implemented as follows: A1: First, sort all detected AGV bounding boxes according to their confidence scores (from large to small) to ensure that bounding boxes with higher confidence scores are processed first in subsequent processing.

[0022] A2: Select the bounding box with the highest confidence from the sorted list and use it as the bounding box currently being processed; A3: Calculate the intersection over union (IoU) of the currently selected bounding box and the remaining bounding boxes in the list. IoU is an indicator that measures the degree of overlap between two bounding boxes. It ranges from 0 to 1. The higher the IoU, the greater the degree of overlap between the two bounding boxes.

[0023] A4: If the IoU value between the currently selected bounding box and other bounding boxes in the list is greater than the preset threshold, these overlapping bounding boxes are removed from the list. This means that if two bounding boxes overlap too much, only the one with the highest confidence is retained; A5: Continue to select the bounding box with the second highest confidence from the remaining bounding boxes, and repeat steps 3 and 4 until all bounding boxes are traversed; Through the above-mentioned non-maximum suppression method, each target is usually detected only once, which greatly reduces redundant detection frames, improves the accuracy and efficiency of AGV detection, and ensures that in the post-processing stage of AGV detection, only the relatively optimal bounding box is output as the final detection result, thereby reducing the possibility of false detection and repeated detection.

[0024] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0025] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended to include all changes within the meaning and scope of the equivalent elements of the claims, and any figure mark in the claims should not be regarded as limiting the claims involved.

[0026] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A multi-target detection and tracking system based on deep learning, It is characterized in that include: Acquisition module: This module is used to take pictures of the AGV and collect image information; Detection module: Use deep learning models to detect targets in collected image information and extract appearance features of multiple AGVs; Tracking module: collects the appearance features of the AGV in the detection module, uses the Kalman filter algorithm to predict the movement trajectories of multiple AGVs at the new moment, and finally uses the processing unit to perform correlation calculations to complete multi-target tracking; Output module: The system collects the multi-target tracking results in the tracking module and outputs and displays the tracking results.

2. The multi-target detection and tracking system based on deep learning according to claim 1, It is characterized in that The processing unit performs association calculation to complete the steps of multi-target tracking as follows: S1: Divide the AGV driving time period into the trajectory set at time t-1 into tracking trajectory, unconfirmed trajectory, and lost trajectory, and use the prediction algorithm to obtain the predicted trajectory position at time t; S2: High-confidence object matching: Through data association, the predicted position information of the AGV at time t is matched with the high-score detected object with high confidence; S3: For the track that failed to match the high-confidence object, match it with the low-confidence detection object, where the track that failed to match is matched with the low-score detection object for low-confidence object matching; S4: Historical position matching: For the track that failed in the first two matches, the historical position is matched with the detected object. The historical position is the position of the AGV in the t-1 period. S5: Unconfirmed trajectory matching: for high-confidence objects among the detected objects that also failed historical position matching, they are matched with the unconfirmed trajectory; S6: Based on the results of the above four successfully matched trajectories, the system updates the trajectory set of the AGV at time t, and the successfully matched trajectory will be updated according to the latest position and status information.

3. The multi-target detection and tracking system based on deep learning according to claim 2, It is characterized in that The specific steps of extracting appearance features of multiple AGVs in the detection module are as follows: S001: Receive image information collected by the acquisition module, annotate the collected images, mark the position and bounding box of the AGV, form a data set, use the annotated data set to train the selected model, and adjust the model parameters and optimizer settings; S002: Input the image to be detected into the trained model, remove the overlapping bounding boxes by the non-maximum suppression method, and obtain the final detection result; S003: Based on target detection, a feature extractor can be used to extract appearance features.

4. The multi-target detection and tracking system based on deep learning according to claim 3, It is characterized in that The device for collecting images in the collection module is a camera.

5. The multi-target detection and tracking system based on deep learning according to claim 4, It is characterized in that The tracking result outputted in the output module is in the form of a picture.

6. The multi-target detection and tracking system based on deep learning according to claim 5, It is characterized in that The two matches in S2 and S3 use a motion similarity measurement algorithm to match high-confidence objects and low-confidence detection objects.

7. A method for multi-target detection based on deep learning, applied to any of the multi-target detection and tracking systems based on deep learning in any of the above items 1 to 6. It is characterized in that The steps include: S0001: First, the acquisition module takes a picture of the AGV and collects the image information of the AGV; S0002: Receive image information collected by the acquisition module, annotate the collected images, mark the position and bounding box of the AGV, form a data set, use the annotated data set to train the selected model, adjust the model parameters and optimizer settings, so that the model can accurately identify the AGV, input the image to be detected into the trained model, the model will output the position and confidence of the AGV in the image, remove the overlapping bounding boxes through the non-maximum suppression method, and obtain the final detection result; S0003: Then, in the tracking module, by receiving the appearance feature data of the AGV car, the Kalman filter algorithm is used to predict the movement trajectory of multiple AGV cars at the new moment based on the trajectory of the previous moment. Finally, the four-stage matching algorithm is used to perform association calculation to complete multi-target tracking. The tracking result is presented in the form of a picture, and the tracking result is output through the output module.

8. Multi-target detection method based on deep learning, It is characterized in that In S0002, the overlapping bounding boxes are removed by the non-maximum suppression method to obtain the final detection result. Based on the target detection, the feature extractor can be used to extract the appearance features of the AGV car.

Citation Information

Patent Citations

  • Unmanned aerial vehicle multi-target tracking method based on ByteTrack

    CN116630376A

  • Vehicle multi-target tracking method suitable for highway tunnel monitoring scene

    CN117173218A

  • Lightweight multi-pedestrian target tracking method in scene with similar target appearance

    CN117333917A

  • Target tracking matching method and device, equipment and medium

    CN117372477A

  • Multi-target tracking method based on Kalman filtering and correlation matching

    CN117649430A

Cited By

  • Portable nail fold microcirculation leukocyte real-time noninvasive monitoring and counting method and device based on deep learning

    CN120298304A

  • Monitoring video analysis method and system for tobacco industry

    CN122067187A