Target detection tracking method, system and device and storage medium

Abstract

The embodiment of the invention relates to the field of trajectory tracking, and discloses a target detection tracking method, system and device and a storage medium, and the method comprises the steps: obtaining a plurality of image frames in a video or image sequence; performing detection by using a target detection model to obtain feature detection information of each image frame; inputting the feature detection information into a trajectory tracking model for analysis and processing, and establishing and tracking a motion trajectory of the target object; wherein the method specifically comprises the following steps of: associating the target object by utilizing the frame similarity among the plurality of bounding boxes; and establishing and tracking a motion track of the target object based on an association result. According to the method disclosed by the invention, the technical problems that the tracking object is difficult to accurately detect and the calculation time is relatively long and relatively large computing power resources need to be occupied are solved, the computing power can be reasonably distributed, the balance of high-precision detection and real-time monitoring is realized, the target object is quickly sensed in a complex shielding environment, and the target object detection and tracking effect is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of trajectory tracking, and in particular to a target detection and tracking method, system, device and storage medium. Background Art

[0002] Transportation is a basic, leading and strategic industry in the national economy. It is an important service industry and an important part of the modern economic system. As pedestrians are an important part of the transportation system, it is very important to accurately detect and count pedestrians. Pedestrian detection and tracking has a wide range of applications in many scenarios.

[0003] Currently, there is a target detection algorithm YOLOv5 (You Only Look Once version 5) for identifying and locating pedestrians in videos or images. Common multi-target tracking (MOT) algorithms include FairMOT. The ordinary MOT model jointly optimizes the two tasks of target detection and target matching. However, these two tasks compete with each other in the actual training or detection process. Previous methods regard feature matching as a secondary task, and its accuracy is seriously affected by the detection task. However, the current detection and tracking algorithms have the following problems:

[0004] (1) General algorithms have difficulty detecting and tracking pedestrians when occlusion is severe. In some scenes, occlusion is very severe. General algorithms can detect and track some objects with a small amount of occlusion, but their performance is average when a large amount of occlusion occurs.

[0005] (2) It is difficult to balance high-precision detection and real-time detection. When striving for higher detection accuracy, model inference often takes more time and occupies a portion of computing power; when pursuing faster detection speed, it is inevitable to compromise on detection accuracy or use more computing power resources. How to balance the two to meet actual needs deserves special attention. Summary of the invention

[0006] The purpose of the present invention is to at least provide a target detection and tracking method, system, device and storage medium, which can at least solve the technical problem that it is difficult to accurately detect and track the object and the calculation time is long and requires a large amount of computing power resources. At least it can achieve a reasonable allocation of computing power, achieve a balance between high-precision detection and real-time monitoring, and quickly perceive the target object in a complex occlusion environment, thereby improving the target object detection and tracking effect.

[0007] To solve the above technical problems, at least one embodiment of the present application provides a target detection and tracking method, including:

[0008] Acquire multiple image frames in a video or image sequence;

[0009] Detecting the target object in each of the image frames by using the target detection model, and obtaining feature detection information of each of the image frames; the feature detection information includes a bounding box of the target object;

[0010] Inputting feature detection information of each of the multiple image frames into a trajectory tracking model for analysis and processing frame by frame, and establishing and tracking the motion trajectory of the target object; wherein establishing and tracking the motion trajectory of the target object includes: associating the target objects in the multiple image frames using the frame similarity between the boundary boxes of multiple objects in the multiple image frames; and establishing and tracking the motion trajectory of the target object based on the result of the association.

[0011] At least one embodiment of the present application further provides a target detection and tracking system, including:

[0012] An image acquisition module, used for acquiring multiple image frames in a video or image sequence;

[0013] A target detection module connected to the image acquisition module is used to detect the target object in each of the image frames by frame using the target detection model to obtain feature detection information of each of the image frames; the feature detection information includes a bounding box of the target object; and

[0014] The target tracking module connected to the target detection module is used to input the feature detection information of each of the multiple image frames into the trajectory tracking model for frame analysis and processing, and establish and track the motion trajectory of the target object; wherein, establishing and tracking the motion trajectory of the target object includes: using the box similarity between the boundary boxes of multiple objects in the multiple image frames to associate the target objects in the multiple image frames; and establishing and tracking the motion trajectory of the target object based on the result of the association.

[0015] At least one embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned target detection and tracking method.

[0016] At least one embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned target detection and tracking method when executed by a processor.

[0017] The target detection and tracking method, system, device and storage medium provided by the embodiments of the present application, by forming an upstream and downstream cascade relationship between target detection and target tracking, can reasonably allocate computing power when faced with a large amount of video data, and achieve a balance between high-precision detection and real-time monitoring; at the same time, the use of frame similarity can well track fast-moving targets, and combined with the feature monitoring information of target monitoring, it can suppress useless information, thereby quickly perceiving the target object in a complex occlusion environment and improving the target object detection and tracking effect.

[0018] In some optional embodiments, the target detection model includes a convolutional neural network model based on a CBAM attention module; the attention module includes a channel attention unit and a spatial attention unit;

[0019] The detecting of the target object in each of the image frames by using the target detection model to obtain feature detection information of each of the image frames includes:

[0020] Input the image frame into the convolutional coding part of the convolutional neural network model for detection to obtain a convolutional coding feature map containing the target object;

[0021] Input the convolutional coding feature map into the channel attention unit detection of the CBAM attention module to obtain a channel enhanced convolutional coding feature map containing the target object;

[0022] The channel enhanced convolutional coding feature map is input into the spatial attention unit of the CBAM attention module for detection to obtain feature detection information of the image frame.

[0023] In some optional embodiments, it also includes:

[0024] The DIoU-NMS algorithm is used to train and optimize the convolutional neural network model; the DIoU-NMS algorithm includes:

[0025]

[0026] Where b gt represents the center point of the envelope with the highest detection score, b represents the center point of the envelope to be compared, ρ(·) represents the Euclidean distance between the two center points, and c represents the diagonal distance of the minimum circumscribed rectangles of the two envelope boxes.

[0027] In some optional embodiments, the image frame includes a plurality of target objects to be tracked, and the feature detection information of each image frame is input into the trajectory tracking model for analysis and processing frame by frame to establish and track the motion trajectory of the target object, including:

[0028] Extracting differential features of feature detection information of the image frame using a feature matching structure to distinguish feature detection information of different target objects;

[0029] The trajectory tracking model is used to analyze and process the feature detection information of different target objects, and the motion trajectories of different target objects are established respectively.

[0030] In some optional embodiments, the trajectory tracking model includes a trajectory tracking model constructed based on the Hungarian model; the step of inputting the feature detection information of each of the image frames into the trajectory tracking model for analysis and processing includes:

[0031] Solving the feature detection information through a Kalman filter to obtain a cost matrix of the target object;

[0032] The cost matrix is ​​processed using the Hungarian algorithm to establish and track the motion trajectory of the target object.

[0033] In some optional embodiments, associating the target objects in the multiple image frames by using the box similarity between the bounding boxes of the multiple objects in the multiple image frames; and establishing and tracking the motion trajectory of the target object based on the result of the association, comprises:

[0034] According to the motion trajectory information, the similarity of the visual features, and the consistency of the position and scale, the bounding boxes of the same target object in each image frame are determined to perform box similarity, and the box similarity result is obtained;

[0035] Associating the same target object in the different image frames according to the frame similarity result to obtain an association result of the target object;

[0036] A motion trajectory of the target object is established and tracked based on the result of the association.

[0037] In some optional embodiments, the measurement formula of the frame similarity includes:

[0038] S=αS IoU +βS motion +γS appearance

[0039] In the formula, S IoU is the IoU metric; S motion is the similarity of motion trajectories; S appearance For appearance similarity.

[0040] In some optional embodiments, the loss function of the trajectory tracking model training includes:

[0041]

[0042] In the formula, N represents the number of target objects in the image, K represents the number of all ID numbers in the training set, and L i (k) is the one-hot encoding representation of the GT class label. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0044] Figure 1 This is a process of a target detection and tracking method provided by an embodiment of the present application. Figure 1 ;

[0045] Figure 2 This is a process of a target detection and tracking method provided by an embodiment of the present application. Figure 2 ;

[0046] Figure 3 is a schematic diagram of a CBAM model provided by an embodiment of the present application;

[0047] Figure 4 An embodiment of the present application provides Figure 1 The process of step 102 Figure 1 ;

[0048] Figure 5 is a schematic diagram of the structure of a target detection model provided by an embodiment of the present application;

[0049] Figure 6 An embodiment of the present application provides Figure 1 The process of step 103 in Figure 1 ;

[0050] Figure 7 This is a process of a target detection and tracking method provided by an embodiment of the present application. Figure 3 ;

[0051] Figure 8 is a structural schematic diagram of a trajectory tracking model based on Hungarian matching provided by an embodiment of the present application;

[0052] Fig. 9 An embodiment of the present application provides Figure 1 The process of step 103 Figure 2 ;

[0053] Fig.10 is a structural schematic diagram of a target detection and tracking system diagram provided by another embodiment of the present application;

[0054] Fig.11is a structural schematic diagram of an electronic device provided by another embodiment of the present application;

[0055] Fig.12 It is a schematic diagram of the structure of a computer-readable storage medium provided by another embodiment of the present application. DETAILED DESCRIPTION

[0056] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0057] In order to facilitate understanding of the embodiments of the present application, relevant content about pedestrian detection is first introduced here.

[0058] Transportation is a basic, leading and strategic industry in the national economy. It is an important service industry and an important part of the modern economic system. Today, my country is promoting the integrated, intelligent and green development of comprehensive transportation. As pedestrians are an important part of the transportation system, it is very important to accurately detect and count pedestrians. Pedestrian detection and tracking are widely used in many scenarios, such as passenger flow statistics analysis in public transportation places such as railway stations and subway stations, and identification and tracking of people who have abnormally intruded on railways. It can greatly reduce the workload of human work, improve work efficiency and management efficiency, and achieve more scientific management.

[0059] Currently, there is a target detection algorithm YOLOv5 (You Only Look Once version 5) for identifying and locating pedestrians in videos or images. It is the fifth version of the YOLO (You Only Look Once) algorithm, which is improved on the basis of YOLOv4. Its advantage is that it can run at high speed and performs well on different image resolutions. It consists of four parts: input end, backbone network layer, intermediate layer and prediction layer. The input end mainly completes basic image processing tasks such as Mosaic data enhancement, adaptive anchor box calculation and adaptive image scaling. The backbone network layer is the backbone network, and the Focus structure and CSP structure are added. It mainly extracts different levels of features of the input image through deep convolution operations. The Focus structure mainly compresses and integrates the image at the input end to obtain the image feature map, and the Bottle net CSP structure improves the model accuracy and speed by reducing the number of network model parameters. The intermediate layer uses the feature map pyramid network (FPN) and the path aggregation structure (PAN) to further strengthen the feature fusion of the backbone network layer to extract different levels of image features and improve the detection performance. The prediction layer makes predictions and uses weighted NMS to screen multiple target boxes to obtain the optimal target box, mainly predicting targets of different sizes in feature maps of different levels.

[0060] However, in some scenes, people occlusion often occurs. Some of them are caused by the high density of people, and some are caused by irrelevant objects in complex scenes. These occlusion phenomena seriously affect the effect of existing pedestrian detection and tracking algorithms. The detection accuracy is low and cannot meet the actual detection and tracking needs. Therefore, pedestrian tracking detection algorithms have emerged to solve the occlusion problem.

[0061] Common multi-object tracking (MOT) algorithms include FairMOT. The ordinary MOT model jointly optimizes the two tasks of object detection and object matching. However, these two tasks compete with each other in the actual training or detection process. Previous methods regard feature matching as a secondary task, and its accuracy is seriously affected by the detection task. FairMOT treats the two tasks of object detection and object matching equally, solving this unequal task distribution problem. It consists of three parts: the backbone network, the object detection branch, and the re-identification branch. The network uses ResNet-34 as the backbone, and the enhanced version of deep layer aggregation (DLA) is applied to it to fuse multiple layers of features. In the detection branch, three parallel head networks are attached to DLA-34, which are used to estimate heat maps, object center offsets, and bounding box sizes, respectively. The implementation process of each head network is: 3×3 convolution (256 channels) is applied to the output features of DLA-34, and then a 1×1 convolution layer generates the final result. The implementation of the re-identification branch is to apply a convolution layer with 128 kernels on the backbone features to extract the re-identification features at each position. FairMOT alleviates the impact of occlusion on the algorithm performance to a certain extent, but its performance is still poor in scenes with severe occlusion.

[0062] In summary, there are two main technical problems in the prior art:

[0063] (1) General algorithms have difficulty detecting and tracking pedestrians when occlusion is severe. In some scenes, occlusion is very severe. General algorithms can detect and track some objects with a small amount of occlusion, but their performance is average when a large amount of occlusion occurs.

[0064] (2) It is difficult to balance high-precision detection and real-time detection. When striving for higher detection accuracy, model inference often takes more time and occupies a portion of computing power; when pursuing faster detection speed, it is inevitable to compromise on detection accuracy or use more computing power resources. How to balance the two to meet actual needs deserves special attention.

[0065] In order to solve the above-mentioned technical problems that it is difficult to accurately detect and track objects and the calculation time is long and requires a large amount of computing power resources, the present invention proposes a target detection and tracking method. The implementation details of the target detection and tracking method of this embodiment are specifically described below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0066] Embodiment 1:

[0067] The target detection and tracking method of this embodiment can be applied to electronic devices with communication, computing and data storage capabilities. The specific process can be as follows: Figure 1 and Figure 2 As shown, steps 101-103 are included, specifically including:

[0068] Step 101, acquiring multiple image frames in a video or image sequence;

[0069] Specifically, the video or image sequence may be captured by an image detection device, such as a camera, so as to capture and record the target object to form a video or image sequence.

[0070] In some examples, the target detection and tracking method is used to track pedestrians, that is, the feature information in the image frame is a pedestrian. The image frame includes multiple target objects to be tracked. It is understandable that there may be one pedestrian or multiple different pedestrians in one image frame, and each pedestrian is detected and tracked.

[0071] Step 102, using a target detection model to detect the target object in each of the image frames frame by frame, to obtain feature detection information of each of the image frames; the feature detection information includes a bounding box of the target object.

[0072] Specifically, in this embodiment, in pedestrian detection, the CBAM module is also introduced to suppress useless information in the scene, focusing on personnel detection, and helping the model to quickly perceive pedestrians to be detected in complex situations. The target detection model includes a convolutional neural network model based on the CBAM attention module; the attention module includes a channel attention unit and a spatial attention unit, such as Figure 3 As shown in the figure, the CBAM attention module is used to calculate the attention weights of the given input feature map along the spatial dimension and the channel dimension, and then multiply the calculated weight coefficient with the input feature map to adaptively adjust the input features.

[0073] In some examples, step 102, such as Figure 4 As shown, including:

[0074] Step 1021: input the image frame into the convolutional coding part of the convolutional neural network model for detection to obtain a convolutional coding feature map containing the target object;

[0075] Step 1022: input the convolutional coding feature map into the channel attention unit of the CBAM attention module for detection, and obtain a channel enhanced convolutional coding feature map containing the target object;

[0076] Step 1023: input the channel enhanced convolutional coding feature map into the spatial attention unit of the CBAM attention module for detection to obtain feature detection information of the image frame.

[0077] In this embodiment, the network structure of the target detection model including the convolutional neural network model based on the CBAM attention module is as follows: Figure 5As shown in the figure, based on the original YOLOv5 network structure, the CSP1_1, CSP1_2, CSP1_3 in the backbone network and the CSP2_1 and CBL modules in the neck network are replaced with CBAM modules. The CBAM module uses the channel relationship of features to output the channel attention map: first, the spatial size of the input feature map is compressed to effectively calculate the channel attention; second, in order to gather spatial information, the module performs average pooling and maximum pooling operations on the input feature map; then the obtained average features and spatial features are sent to a weight sharing network composed of multi-layer perceptrons to generate a channel attention map; spatial attention is a supplement to channel attention: first, average set and maximum set operations are performed along the channel axis, and they are concatenated to generate feature descriptions, and convolution layers are applied to the concatenated feature descriptions to supplement the generation of spatial attention maps. After obtaining the two attention maps, the model summarizes the channel information of the feature map through pooling operations, and then the obtained features are concatenated and convolved to produce the final output attention image.

[0078] In some embodiments, in order to further accurately suppress the detection frame of redundant detection, in this embodiment, DIoU_nms is introduced into the target detection model, that is, the DIoU-NMS algorithm is used to train and optimize the convolutional neural network model; the DIoU-NMS algorithm includes:

[0079]

[0080] Where bgt represents the center point of the envelope with the highest detection score, b represents the center point of the envelope to be compared, ρ(·) represents the Euclidean distance between the two center points, and c represents the diagonal distance of the minimum circumscribed rectangles of the two envelopes.

[0081] The DIoU-NMS algorithm not only focuses on the overlapping area between the predicted bounding boxes, but also considers the center distance between the two boxes, so as to accurately suppress redundant detection and improve the detection accuracy.

[0082] Step 103, inputting feature detection information of each of the multiple image frames into a trajectory tracking model for analysis and processing, and establishing and tracking the motion trajectory of the target object;

[0083] Specifically, establishing and tracking the motion trajectory of the target object includes: associating the target objects in the multiple image frames using the box similarity between the bounding boxes of multiple objects in the multiple image frames; and establishing and tracking the motion trajectory of the target object based on the result of the association.

[0084] In some examples, since the image frame includes multiple target objects to be tracked, Figure 6As shown, step 103 includes:

[0085] Step 1031: extracting differential features of feature detection information of the image frame using a feature matching structure to distinguish feature detection information of different target objects;

[0086] Step 1032: Analyze and process the feature detection information of each different target object using the trajectory tracking model, and establish motion trajectories for each different target object.

[0087] By combining the detection results of the target detection model, the characteristic information of different target objects can be distinguished, so that the corresponding motion trajectories can be established for different target objects respectively, and the detection and tracking of multiple pedestrians can be realized.

[0088] In some examples, in order to improve the accuracy of tracking the motion trajectory of the target object, the step uses the box similarity between the bounding boxes of the multiple objects in the multiple image frames to associate the target objects in the multiple image frames; based on the result of the association, establish and track the motion trajectory of the target object, such as Figure 7 As shown, including:

[0089] Step 201, determining the bounding boxes of the same target object in each image frame according to the motion trajectory information, the similarity of the visual features, and the consistency of the position and scale, and performing box similarity to obtain a box similarity result;

[0090] Step 202: Associating the same target object in the different image frames according to the frame similarity result to obtain an association result of the target object;

[0091] Step 203: Establish and track the motion trajectory of the target object based on the association result.

[0092] Among them, the measurement formula of box similarity includes:

[0093] S=αS IoU +βS motion +γS appearance

[0094] In the formula, S IoU is the IoU metric; S motion is the similarity of motion trajectories; S appearanee For appearance similarity.

[0095] By introducing box similarity to measure the similarity between each bounding box, constraints are added to reduce the loss of tracking due to rapid movement of pedestrians. In the box similarity distance, the motion trajectory information, visual feature similarity, and position and scale consistency are combined to measure the box similarity. This combination of pedestrian motion information, appearance information, and position information can effectively improve the tracking robustness in intense motion scenes.

[0096] In some embodiments, Figure 8 As shown, the trajectory tracking model includes a trajectory tracking model constructed based on the Hungarian Fig. 9 As shown, step 103 includes:

[0097] Step 1033: solving the feature detection information through a Kalman filter to obtain a cost matrix of the target object;

[0098] Step 1034: Use the Hungarian algorithm to process the cost matrix, and establish and track the motion trajectory of the target object.

[0099] In this embodiment, in pedestrian tracking, if Figure 8 The figure shows a trajectory tracking model based on Hungarian matching, which can treat detection and re-identification tasks equally. The trajectory tracking model first extracts the differential features that can distinguish different objects through the feature matching structure, and then uses this as a basis to establish trajectories for different objects, thereby realizing motion trajectory tracking of different target objects.

[0100] Among them, when training the trajectory tracking model, the following loss function is used:

[0101]

[0102] In the formula, N represents the number of target objects in the image, K represents the number of all ID numbers in the training set, and L i (k) is the one-hot encoding representation of the GT class label.

[0103] In this embodiment, the target object is a pedestrian. A convolution layer with 128 convolution kernels is used to extract the difference features from the features extracted by the backbone network. In the actual training process, a classification task is used to enable the model to learn matching features. All pedestrians with the same identity in the training set are regarded as the same class. For each detection box in the image, First, the detector is used to calculate the center point Then, the feature matching branch is used to extract pedestrian matching features, and the similarity between boxes is measured according to the box similarity distance. The fully connected layer and softmax operation are used to map it to the class distribution vector P = {p(k), k∈(1,k)} to obtain the predicted category (ID number) possibility distribution of the object.

[0104] Furthermore, after obtaining the location information and matching information of the target object, the cost matrix is ​​solved by using Kalman filtering. The specific implementation process is as follows:

[0105] (1) Status prediction:

[0106]

[0107] in, is the state prediction value, is the optimal estimate of the state, F k is the state transfer matrix, B k is the control input matrix, u k is the control vector, is the covariance between the true value and the predicted value, P k is the covariance between the true value and the optimal estimate, Q k is the covariance matrix of the process noise.

[0108] (2) Observation and prediction:

[0109]

[0110] in, is the observed value prediction of the state matrix, H k is the state observation matrix, S k is the sum of variances, R k is the covariance matrix of the measurement noise.

[0111] (3) Kalman gain:

[0112]

[0113] Among them, K k is the Kalman gain.

[0114] (4) Status update:

[0115]

[0116] Among them, z k is the observation of the state matrix, and I is the unit matrix.

[0117] The cost matrix reflects the degree of association between each position and each match, that is, the continuity of the tracking. The output of the Kalman filter can provide additional information in this process, which helps to calculate the cost matrix more accurately. The Hungarian algorithm is used to process the cost matrix. Through the Hungarian algorithm, the system can effectively achieve a coherent match of the pedestrian trajectory at each time step, thereby establishing a stable tracking trajectory.

[0118] In this embodiment, by forming an upstream and downstream cascade relationship between target detection and target tracking, it is possible to reasonably allocate computing power when facing a large amount of video data, and achieve a balance between high-precision detection and real-time monitoring; at the same time, the use of frame similarity can well track fast-moving targets, and combined with the characteristic monitoring information of target monitoring, it can suppress useless information, so as to quickly perceive the target object in a complex occluded environment and improve the detection and tracking effect of the target object. Among them, the model improves the original YOLOv5 network and proposes a CBAM-YOLOv5 network: by adding high-performance convolution to improve the inference speed and enhance the perception of complex shapes; by introducing the CBAM module to enhance the focus on pedestrians and suppress useless information, it helps the model to quickly perceive pedestrian objects in a complex occluded environment; at the same time, the DIoU_nms algorithm is used in the post-processing part, and the model converges quickly and the results obtained are more reasonable and effective. The frame similarity distance introduced in the tracking task can well track fast-moving pedestrians without loss.

[0119] Embodiment 2:

[0120] Another embodiment of the present application relates to a target detection and tracking system. The implementation details of the target detection and tracking system of this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for the implementation of this solution. The schematic diagram of the target detection and tracking system of this embodiment can be as follows Fig.10 As shown, it includes an image acquisition module 801, a target detection module 802 and a target tracking module 803; wherein the image acquisition module 801, the target detection module 802 and the target tracking module 803 perform a detection and tracking process of a pedestrian as shown in FIG. Figure 2 shown.

[0121] The image acquisition module 801 is used to acquire multiple image frames in a video or image sequence;

[0122] The target detection module 802 is connected to the image acquisition module 801, and is used to detect the target object in each of the image frames by frame using the target detection model to obtain feature detection information of each of the image frames; the feature detection information includes the bounding box of the target object; and

[0123] The target tracking module 803 is connected to the target detection module 802, and is used to input the feature detection information of each image frame into the trajectory tracking model for analysis and processing, and establish and track the motion trajectory of the target object; wherein, tracking the motion trajectory of the target object includes: using the box similarity between the boundary boxes of the same target object in multiple image frames to associate the target object, and obtain the motion trajectory tracking result of the target object.

[0124] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0125] Embodiment three:

[0126] Another embodiment of the present application relates to an electronic device, such as Fig.11 As shown, it includes: at least one processor 901; and a memory 902 that is communicatively connected to the at least one processor 901; wherein the memory 902 stores instructions that can be executed by the at least one processor 901, and the instructions are executed by the at least one processor 901 so that the at least one processor 901 can execute the target detection and tracking method in the above-mentioned embodiments.

[0127] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0128] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0129] Embodiment 4:

[0130] Another embodiment of the present application relates to a computer-readable storage medium, such as Fig.12 A computer program 31 is stored. When the computer program 31 is executed by a processor, the above method embodiment is implemented.

[0131] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0132] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A target detection and tracking method, characterized in that: include: Acquire multiple image frames in a video or image sequence; Detecting the target object in each of the image frames frame by frame using the target detection model to obtain feature detection information of each of the image frames; The feature detection information includes a bounding box of the target object; Inputting feature detection information of each of the multiple image frames into a trajectory tracking model for analysis and processing frame by frame, and establishing and tracking the motion trajectory of the target object; wherein establishing and tracking the motion trajectory of the target object includes: associating the target objects in the multiple image frames using the frame similarity between the boundary boxes of multiple objects in the multiple image frames; and establishing and tracking the motion trajectory of the target object based on the result of the association.

2. The target detection and tracking method according to claim 1, characterized in that: The target detection model includes a convolutional neural network model based on a CBAM attention module; the attention module includes a channel attention unit and a spatial attention unit; The detecting of the target object in each of the image frames by using the target detection model to obtain feature detection information of each of the image frames includes: Input the image frame into the convolutional coding part of the convolutional neural network model for detection to obtain a convolutional coding feature map containing the target object; Input the convolutional coding feature map into the channel attention unit detection of the CBAM attention module to obtain a channel enhanced convolutional coding feature map containing the target object; The channel enhanced convolutional coding feature map is input into the spatial attention unit of the CBAM attention module for detection to obtain feature detection information of the image frame.

3. The target detection and tracking method according to claim 2, characterized in that: Also includes: The DIoU-NMS algorithm is used to train and optimize the convolutional neural network model; The DIoU-NMS algorithm includes: Where b gt represents the center point of the envelope with the highest detection score, b represents the center point of the envelope to be compared, ρ(·) represents the Euclidean distance between the two center points, and c represents the diagonal distance of the minimum circumscribed rectangles of the two envelope boxes.

4. The target detection and tracking method according to claim 1, characterized in that: The image frame includes a plurality of target objects to be tracked, and the feature detection information of each of the image frames is input into the trajectory tracking model for analysis and processing, and the motion trajectory of the target object is established and tracked, including: Extracting differential features of feature detection information of the image frame using a feature matching structure to distinguish feature detection information of different target objects; The trajectory tracking model is used to analyze and process the feature detection information of different target objects, and the motion trajectories of different target objects are established respectively.

5. The target detection and tracking method according to claim 1, characterized in that: The trajectory tracking model includes a trajectory tracking model constructed based on the Hungarian method; the feature detection information of each image frame is input into the trajectory tracking model frame by frame for analysis and processing, including: Solving the feature detection information through a Kalman filter to obtain a cost matrix of the target object; The cost matrix is ​​processed using the Hungarian algorithm to establish and track the motion trajectory of the target object.

6. The target detection and tracking method according to claim 1, characterized in that: Associating the target objects in the multiple image frames by utilizing the box similarity between the bounding boxes of the multiple objects in the multiple image frames; Establishing and tracking the motion trajectory of the target object based on the result of the association includes: According to the motion trajectory information, the similarity of the visual features, and the consistency of the position and scale, the bounding boxes of the same target object in each image frame are determined to perform box similarity, and the box similarity result is obtained; Associating the same target object in the different image frames according to the frame similarity result to obtain an association result of the target object; A motion trajectory of the target object is established and tracked based on the result of the association.

7. The target detection and tracking method according to claim 6, characterized in that: The measurement formula of the frame similarity includes: S=αSIoU+βSmotion+γSappearance Where, SIoU is the IoU metric; Smotion is the similarity of motion trajectories; Sappearance is the distance between the two moving parts. e For appearance similarity.

8. A target detection and tracking method according to claim 1, characterized in that: The loss function of the trajectory tracking model training includes: In the formula, N represents the number of target objects in the image, K represents the number of all ID numbers in the training set, and L i (k) is the one-hot encoding representation of the GT class label.

9. A target detection and tracking system, characterized in that: include: An image acquisition module, used for acquiring multiple image frames in a video or image sequence; The target detection module connected to the image acquisition module is used to detect the target object in each image frame by frame using the target detection model to obtain feature detection information of each image frame; The feature detection information includes a bounding box of the target object; as well as, The target tracking module connected to the target detection module is used to input the feature detection information of each of the multiple image frames into the trajectory tracking model for frame analysis and processing, and establish and track the motion trajectory of the target object; wherein, establishing and tracking the motion trajectory of the target object includes: using the box similarity between the boundary boxes of multiple objects in the multiple image frames to associate the target objects in the multiple image frames; and establishing and tracking the motion trajectory of the target object based on the result of the association.

10. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the target detection and tracking method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the target detection and tracking method according to any one of claims 1 to 8 is implemented.

Images