Target tracking method, device, equipment and medium

Abstract

The present application discloses a target tracking method, apparatus, device and medium, which relates to the field of computer vision technology, including: using a detection and segmentation model to obtain current detection information and current contours of all objects in a current frame to be detected image; the current detection information includes the detection frame position information and detection frame confidence of the object; extracting the foreground features of the object from the current detection information and current contours; obtaining the current frame tracking frame position information of the object based on the current interactive filtering model predicted based on historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame in the historical frame; for objects whose detection frame confidence is within a first preset range, matching is performed based on the historical frame observation information and the foreground features of the object, as well as the current frame detection frame position information and the current frame tracking frame position information, so as to achieve target tracking and improve the accuracy of target tracking.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a target tracking method, device, equipment and medium. Background Art

[0002] Object tracking is an important research area in computer vision. Its primary task is to continuously track individual target objects in a video sequence and predict their motion trajectories. Object tracking is valuable in many practical applications, such as intelligent surveillance, autonomous driving, and drone navigation. Traditional machine learning-based multi-target tracking methods rely primarily on handcrafted features and simple models, such as Gaussian mixture models (GMMs) and Kalman filters (KFs). While these methods improve tracking performance to some extent, their performance remains limited due to limitations in feature design and model complexity.

[0003] With the development of deep learning technology, especially the emergence of convolutional neural networks (CNNs), object tracking has achieved significant breakthroughs. CNNs have powerful feature extraction capabilities and can automatically learn features useful for object tracking. Furthermore, the emergence of attention mechanisms such as the Transformer has provided new insights for multi-object tracking. By introducing attention mechanisms, the model can pay more attention to information related to the current target, thereby improving tracking performance. In recent years, multi-object tracking methods based on deep learning have achieved significant improvements in speed, accuracy, and robustness, and have been widely used in various industries. However, in practical applications, the use cases of object tracking are often complex, resulting in low object tracking accuracy.

[0004] From the above, it can be seen that how to improve the accuracy of target tracking is a problem to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a target tracking method, device, equipment and medium to improve the accuracy of target tracking. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a target tracking method, comprising:

[0007] After receiving the current frame image to be detected, the detection segmentation model is used to obtain the current detection information and current contours of all objects in the current frame image to be detected; wherein the current detection information includes the detection frame position information and the detection frame confidence of the object;

[0008] extracting a foreground feature of the object from the current detection information and the current contour;

[0009] Obtaining the current frame tracking frame position information of the object predicted based on the current interactive filtering model according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame of the historical frame;

[0010] For objects whose detection frame confidence is within a first preset range, matching is performed based on historical frame observation information and the foreground features of the object, current frame detection frame position information, and current frame tracking frame position information to achieve target tracking.

[0011] Optionally, the acquiring current detection information and current contours of all objects in the image to be detected in the current frame by using the detection segmentation model includes:

[0012] Obtaining current initial detection information and current initial contours of all objects in the image to be detected in the current frame using a detection and segmentation model;

[0013] The current initial detection information and the current initial contour are filtered to obtain current detection information and a current contour of the object.

[0014] Optionally, filtering the current initial detection information and the current initial contour to obtain the current detection information and the current contour of the object includes:

[0015] If the size of the current initial contour is larger than the size of the detection box in the corresponding current initial detection information or the difference between the size of the detection box in the current initial detection information and the size of the corresponding current initial contour is greater than a preset threshold, the current initial contour and the corresponding current initial detection information are eliminated to obtain the current detection information and current contour of the object.

[0016] Optionally, matching the historical frame observation information with the foreground features of the object, the current frame detection frame position information, and the current frame tracking frame position information to achieve target tracking includes:

[0017] Determining a position matching matrix of the current contour of the object based on an intersection-over-union ratio of the current detection frame position information and the current frame tracking frame position information, and determining a first candidate matching result set of the object according to the position matching matrix;

[0018] Determining a feature matching matrix of the object based on foreground features of the current contour and foreground features of the object in historical frame observation information, and determining a second candidate matching result set of the object according to the feature matching matrix;

[0019] A target matching result is determined based on the first candidate matching result set and the second candidate matching result set to achieve target tracking of the object.

[0020] Optionally, the target tracking method further includes:

[0021] For objects whose detection frame confidence is within a second preset range, the Euclidean distance between the detection frame position information of the object and the tracking frame position information in the historical frame observation information is calculated, and the current frame detection information of the object is matched with the tracking frame position information in the historical frame observation information according to the Euclidean distance to achieve target tracking.

[0022] Optionally, matching the current frame detection information of the object with tracking frame position information in historical frame observation information according to the Euclidean distance to achieve target tracking includes:

[0023] Filtering the tracking frame position information of the historical frame observation information according to the Euclidean distance to obtain the current filtered tracking frame position information that matches the current frame detection information;

[0024] Obtaining a first size change value between a detection frame in the current frame detection information and a detection frame in the previous frame detection information, and a second size change value between a tracking frame in the current filtered tracking frame position information and a tracking frame in the previous filtered tracking frame position information; wherein the previous filtered tracking frame position information is tracking frame position information that matches the previous frame detection information;

[0025] Based on the size relationship between the first size change value, the second size change value and a preset size change threshold, the current frame detection information of the object is matched with the tracking frame position information in the historical frame observation information to achieve target tracking.

[0026] Optionally, the target tracking method further includes:

[0027] Storing the current frame tracking frame position information of the object for which target tracking has not been completed;

[0028] The current frame tracking frame position information of the object for which target tracking is completed is input into the current interactive filtering model to implement parameter update.

[0029] In a second aspect, the present application discloses a target tracking device, comprising:

[0030] A detection module is configured to receive the current frame image to be detected and obtain current detection information and current contours of all objects in the current frame image to be detected using a detection segmentation model; wherein the current detection information includes the detection frame position information and detection frame confidence of the object;

[0031] an extraction module, configured to extract foreground features of the object from the current detection information and the current contour;

[0032] A prediction module is configured to obtain the current frame tracking frame position information of the object predicted based on the current interactive filtering model according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame in the historical frame;

[0033] A tracking module is used to match the object whose detection frame confidence is within a first preset range based on historical frame observation information and the foreground features of the object, the current frame detection frame position information and the current frame tracking frame position information to achieve target tracking.

[0034] In a third aspect, the present application discloses an electronic device, comprising:

[0035] Memory, used to store computer programs;

[0036] A processor is used to execute the computer program to implement the steps of the target tracking method disclosed above.

[0037] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the target tracking method disclosed above are implemented.

[0038] It can be seen that after receiving the current frame image to be detected, the present application uses the detection segmentation model to obtain the current detection information and current contours of all objects in the current frame image to be detected; wherein, the current detection information includes the detection frame position information and detection frame confidence of the object; the foreground features of the object are extracted from the current detection information and the current contour; the current frame tracking frame position information of the object predicted based on the current interactive filtering model is obtained according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame in the historical frame; for objects whose detection frame confidence is within the first preset range, matching is performed based on the historical frame observation information and the foreground features of the object, the current frame detection frame position information and the current frame tracking frame position information to achieve target tracking. It can be seen that the present application uses the detection segmentation model to obtain the current detection information and current contour of the image to be detected in the current frame, and extracts the foreground features of the object from the current detection information and the current contour. On the one hand, if the detection frame in the current detection information is blocked, it avoids the inaccuracy of extracting features only from the current detection information. On the other hand, it can avoid background interference, that is, the extracted foreground features are more accurate, which provides a strong guarantee for the subsequent matching of the tracking frame position information and the detection information. Furthermore, the current interactive filtering model of the present application predicts more accurate current frame tracking frame position information based on historical frame observation information, and matches the current frame tracking frame position information with the current detection information based on features and positions to screen out the current frame tracking frame position information that matches the current detection information as the tracking result of the object, which can improve the accuracy of target tracking and complete target tracking in real time to meet the real-time requirements in the application scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0040] Figure 1 A flow chart of a target tracking method disclosed in this application;

[0041] Figure 2 A flow chart of a specific target tracking method disclosed in this application;

[0042] Figure 3 This is a flowchart of another specific target tracking method disclosed in this application;

[0043] Figure 4A specific confidence diagram disclosed in this application;

[0044] Figure 5 This is a schematic structural diagram of a target tracking device disclosed in this application;

[0045] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] With the development of deep learning technology, especially convolutional neural networks, object tracking has achieved significant breakthroughs. CNNs possess powerful feature extraction capabilities, automatically learning features useful for object tracking. Furthermore, the emergence of attention mechanisms, such as the Transformer, has provided new insights for multi-object tracking. By introducing attention mechanisms, the model can focus more on information related to the current target, thereby improving tracking performance. In recent years, deep learning-based multi-object tracking methods have achieved significant improvements in speed, accuracy, and robustness, and have been widely used across various industries. However, in practical applications, object tracking scenarios are often complex, resulting in limited accuracy.

[0048] To this end, this application provides a target tracking solution to improve the accuracy of target tracking.

[0049] See also Figure 1 As shown, the embodiment of the present application discloses a target tracking method, including:

[0050] Step S11: After receiving the current frame image to be detected, the detection segmentation model is used to obtain the current detection information and current contours of all objects in the current frame image to be detected; wherein the current detection information includes the detection frame position information and detection frame confidence of the object.

[0051] Receive the current frame to be detected in the video to be detected. That is, in this embodiment, target tracking is performed frame by frame, and the tracking results are displayed in real time. The current frame to be detected is input into the detection and segmentation model, and the current detection information and current contour of the current frame to be detected output by the model are obtained, wherein the current detection information includes the detection frame position information and the detection frame confidence of the object. It is understood that the number of output current detection information and current contours can be multiple. That is, if there are multiple objects in the current frame to be detected, the current detection information and current contours outputted are also multiple.

[0052] A detection and segmentation model uses computer vision techniques to automatically detect and segment objects in images or videos. It simultaneously performs object detection and semantic segmentation, meaning it detects the location of objects in an image and segments them pixel-wise. Such a model typically consists of two main components: a detector and a segmenter. The detector is responsible for locating and identifying objects in an image, generating bounding boxes for each object and assigning them a class label. The segmenter is responsible for pixel-wise segmentation of the detected objects, classifying each object's pixels into different classes. A detection and segmentation model can be trained using transfer learning from multiple object detection and segmentation models. The choice of detection and segmentation model can be flexible depending on the scenario. It can be based on computer vision methods such as threshold segmentation, region growing, and edge detection. These methods perform segmentation based on image color, grayscale values, and edges. It can also be based on deep learning methods such as fully convolutional networks and U-Net. These models typically use stacked multi-layer convolutional neural networks to achieve pixel-wise segmentation and extract high-level semantic information from the image. Deep learning-based detection and segmentation models can simultaneously meet the tasks of object detection and segmentation. Common end-to-end detection and segmentation models include Mask R-CNN (Mask Region-based Convolutional Networks), SOLO (Segmenting Objects by Locations), and the YOLO (You Only Look Once) series. They are widely used in target recognition, image analysis, and other fields, and have achieved excellent results on many challenging datasets.

[0053] For example, using YOLOv8 as the detection and segmentation model, based on the pre-trained model, collect pedestrian data sets for annotation. The annotation format is as follows:

[0054] (label,pt1x,pt1y,pt2x,pt2y,pt3x,pt3y,pt4x,pt4y,…);

[0055] Among them, label represents the category, ptx is the ratio of the horizontal coordinate of each key point in the current contour to the width, and pty is the ratio of the vertical coordinate of each key point in the current contour to the height. The key points correspond to the foreground of different objects. The model finally outputs the current contour, category, and current detection information including the position information of the detection box and the confidence of the detection box of the current frame to be detected. Among them, the category represents the category of the target object, such as human, cat, dog, etc., and the confidence of the detection box represents the reliability of the current detection box, for example Figure 2 A specific confidence diagram is shown, in which the target objects are people, and there are multiple people. The current detection information and current contour of each target object are output, and the corresponding detection frame confidence is marked on the current detection frame corresponding to each object. The detection frame can also be divided into preset high-confidence detection frames and preset low-confidence detection frames according to the level of confidence. For example, the confidence of 0.6 to 1 is determined as the first preset range, and the confidence of 0.3 to 0.6 is determined as the second preset range. It can be understood that if the confidence of the detection frame corresponding to the object is lower than 0.3, it indicates that the reliability of the detection frame is very low, so the detection frame and contour with a confidence lower than 0.3 can be eliminated.

[0056] In this embodiment, the use of the detection segmentation model to obtain the current detection information and current contours of all objects in the current frame image to be detected includes: using the detection segmentation model to obtain the current initial detection information and current initial contours of all objects in the current frame image to be detected; filtering the current initial detection information and the current initial contours to obtain the current detection information and current contours of the objects. After using the detection segmentation model to obtain the current initial detection information and current initial contours of the current frame image to be detected, it is necessary to filter them to filter out inaccurate detection information and contours to obtain more accurate current detection information and current contours of the current frame image to be detected, thereby providing a strong guarantee for subsequent target tracking based on the current detection information and current contours of the current frame image to be detected.

[0057] In this embodiment, the filtering of the current initial detection information and the current initial contour to obtain the current detection information and the current contour of the object includes: if the size of the current initial contour is larger than the size of the detection frame in the corresponding current initial detection information or the difference between the size of the detection frame in the current initial detection information and the size of the corresponding current initial contour is larger than a preset threshold, then the current initial contour and the corresponding current initial detection information are eliminated to obtain the current detection information and the current contour of the object. It is understandable that if there are multiple targets in the current frame to be detected, the number of current initial detection information and current initial contours will both be multiple. Because there may be cases where the target is occluded or the segmented contour is inaccurate, it is necessary to filter the current initial detection information and the current initial contour. For example, if the current initial contour A is larger than the current initial detection information A, then the current initial contour A and the current initial detection information A are eliminated; if the difference between the current initial detection information B and the current initial contour B is larger than the preset threshold, that is, the current initial contour B is much smaller than the current initial detection information B, then the current initial contour B and the current initial detection information B are eliminated.

[0058] Step S12: extracting foreground features of the object from the current detection information and the current contour.

[0059] In this embodiment, the current detection information and current contour of the current frame image to be detected output by the detection segmentation model are respectively used as inputs of the feature extraction model, and the feature extraction model outputs the foreground features of the object. It can be understood that if there are multiple objects in the current frame image to be detected, the number of foreground features is also multiple.

[0060] Feature extraction models are used to extract and represent useful features from raw data. Specifically, they extract foreground features from the current target detection bounding box and the current target outline. By extracting key features, the data dimension can be reduced and patterns and structures within the data can be better expressed. Common feature extraction models include VGG16 (Visual Geometry Group-16 layers), ResNet (Residual Network), DenseNet (Dense Convolutional Network), and MobileNet (Mobile Neural Network). Training feature models relies on extensive computing resources and data. Person re-identification datasets are typically used for training. These datasets feature diverse target poses and expressions, varying lighting conditions, and varying observation angles. Therefore, the extracted features are suitable for scenarios with multiple moving targets. Feature extraction models typically consist of two parts: feature selection and feature transformation. Feature selection refers to selecting the most informative features relevant to the task from the raw data. This can be achieved through filtering, packaging, or embedding methods. Filtering methods sort and select features based on the correlation between them and the target variable, such as mutual information and correlation coefficients. Packaging methods select the best feature subset by training and evaluating the model on a given feature subset. Embedding methods embed feature selection into the model training process, such as L1 regularization and decision tree pruning. Feature transformation refers to transforming or mapping selected features to better represent the data. Common feature transformation methods include principal component analysis (PCA), linear discriminant analysis (LDA), and kernel principal component analysis (KPCA). These methods can project the original features into a new space to better distinguish or represent the data. In this embodiment, the ResNet34 network can be used as the feature extraction model to output a 256-dimensional feature vector. The training set uses Market1501, and the map (average precision) value of the model on the test set is 0.933, the Rank-1 value is 0.983, and the Rank-5 value is 0.996.

[0061] Step S13: Obtain the current frame tracking frame position information of the object predicted based on the current interactive filtering model according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame in the historical frame.

[0062] The interactive modeling method (IMM) predicts the position of the tracking box in the current frame based on historical frame observations, i.e., the posterior position information corresponding to the image to be detected in the current frame. The IMM stores motion models, including uniform motion, accelerated motion, and turning motion models. It is primarily used to estimate and compensate for the motion of moving objects in an image sequence, thereby reducing image noise and blur caused by motion and improving image quality. The detected moving object is modeled, then estimated based on historical motion information to obtain motion information such as direction and speed. This can be achieved using methods such as optical flow and Kalman filtering. Motion prediction filtering techniques have applications in many fields, such as video surveillance, robotic vision, and medical image processing. Interactive filtering is used to model pedestrian motion. Multiple centralized models are used to estimate the target state. The results of these models are then fused in a specific manner, using the maximum likelihood value to represent the model estimation effect. Normalization can better integrate different information, resulting in a more adaptable estimation model. Each model has its own corresponding state equation and a unified measurement equation:

[0063] x i+1 =f i (x i );

[0064] z k =h i (x i，k );

[0065] In the formula, x i is the state of the target of the i-th model at the previous moment, z k It is the observed value of the target at the current moment. According to the linear Kalman filter and nonlinear Kalman filter algorithm, the estimated state of the target after each model update is obtained and the covariance matrix

[0066] According to the updated confidence u i，k The models can be fused to obtain the output target state and covariance matrix of the IMM filtering algorithm:

[0067]

[0068]

[0069] Step S14: For objects whose detection frame confidence is within a first preset range, matching is performed based on historical frame observation information and the foreground features of the object, current frame detection frame position information, and current frame tracking frame position information to achieve target tracking.

[0070] The current detection information, current outline, foreground features, and the current frame tracking frame position information are input into the tracker. It can be understood that the current frame tracking frame position information includes the tracking frame position information of the current frame to be detected, as predicted by the current interactive filtering model, and the object identification information. The core goal of the tracking algorithm is to estimate the position of the object in each frame based on its motion information in the image sequence and match it with the actual position to achieve object tracking. Multi-target tracking requires matching multiple moving targets.

[0071] In this embodiment, a matching condition can be set. For example, based on the current contour and foreground features, the current frame tracking frame position information that matches the current detection information A is screened out from each current frame tracking frame position information to select the current frame tracking frame position information that matches the current detection information A as current frame tracking frame position information A5, the current frame tracking frame position information that matches the current detection information B as current frame tracking frame position information B1, and the current frame tracking frame position information that matches the current detection information C as current frame tracking frame position information C3. Among them, the current frame tracking frame position information A5 matches the corresponding current detection information A for five consecutive frames, the current frame tracking frame position information B1 matches the corresponding current detection information B for the first frame, and the current frame tracking frame position information C3 matches the corresponding current detection information C for three consecutive frames. When the observation information matches the corresponding observation information for three or more consecutive frames, the matching condition is met. Therefore, only the current frame tracking frame position information A5 and the current frame tracking frame position information C3 are the observation information for achieving target tracking. However, the current frame tracking frame position information B1 that does not meet the matching condition is not deleted for subsequent determination of the number of frames for which the observation information continuously matches.

[0072] When the tracking frame position information and the current detection information continuously match, the tracking frame can be displayed on the display interface to achieve target tracking, making it convenient for the user to observe based on the tracking frame.

[0073] This embodiment further includes: storing the current frame tracking frame position information of objects that have not yet completed target tracking; and inputting the current frame tracking frame position information of objects that have completed target tracking into the current interactive filtering model to achieve parameter updates. On the one hand, during target tracking, not all current frame tracking frame position information can be continuously matched with the current detection information. That is, there may be current frame tracking frame position information of some objects that have not yet completed target tracking. This current frame tracking frame position information is saved for subsequent target tracking. It can be understood that when the next frame observation information is obtained, the current frame tracking frame position information of the objects that have not yet completed target tracking becomes part of the historical frame observation information. On the other hand, during the target tracking process, the parameters of the interactive filtering model are continuously updated based on the current frame tracking frame position information of the objects that have completed target tracking, such as updating the mean and variance of each motion model. In other words, the current interactive filtering model is used every time observation information is predicted. In other words, each time observation information is predicted, an interactive filtering model with more optimized parameters is used to improve prediction accuracy.

[0074] It can be seen that after receiving the current frame image to be detected, the present application uses the detection segmentation model to obtain the current detection information and current contours of all objects in the current frame image to be detected; wherein, the current detection information includes the detection frame position information and detection frame confidence of the object; the foreground features of the object are extracted from the current detection information and the current contour; the current frame tracking frame position information of the object predicted based on the current interactive filtering model is obtained according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame in the historical frame; for objects whose detection frame confidence is within the first preset range, matching is performed based on the historical frame observation information and the foreground features of the object, the current frame detection frame position information and the current frame tracking frame position information to achieve target tracking. It can be seen that the present application uses the detection segmentation model to obtain the current detection information and current contour of the image to be detected in the current frame, and extracts the foreground features of the object from the current detection information and the current contour. On the one hand, if the detection frame in the current detection information is blocked, it avoids the inaccuracy of extracting features only from the current detection information. On the other hand, it can avoid background interference, that is, the extracted foreground features are more accurate, which provides a strong guarantee for the subsequent matching of the tracking frame position information and the detection information. Furthermore, the current interactive filtering model of the present application predicts more accurate current frame tracking frame position information based on historical frame observation information, and matches the current frame tracking frame position information with the current detection information based on features and positions to screen out the current frame tracking frame position information that matches the current detection information as the tracking result of the object, which can improve the accuracy of target tracking and complete target tracking in real time to meet the real-time requirements in the application scenario.

[0075] See also Figure 3 As shown, the embodiment of the present application discloses a specific target tracking method, including:

[0076] Step S21: After receiving the current frame image to be detected, the detection segmentation model is used to obtain the current detection information and current contours of all objects in the current frame image to be detected; wherein the current detection information includes the detection frame position information and detection frame confidence of the object.

[0077] Step S22: extracting foreground features of the object from the current detection information and the current contour.

[0078] Step S23: Obtain the current frame tracking frame position information of the object predicted based on the current interactive filtering model according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame in the historical frame.

[0079] Step S24: For the object whose detection frame confidence is within the first preset range, the position matching matrix of the current contour of the object is determined based on the intersection-over-union ratio of the current detection frame position information and the current frame tracking frame position information, and the first alternative matching result set of the object is determined based on the position matching matrix; the feature matching matrix of the object is determined based on the foreground features of the current contour and the foreground features of the object in the historical frame observation information, and the second alternative matching result set of the object is determined based on the feature matching matrix.

[0080] It can be understood that there may be multiple current detection information, current contours, and foreground features, among which there may be a detection frame with a higher confidence, that is, the detection frame confidence is within the first preset range, and the first target object whose detection frame confidence is within the first preset range is determined, and the segmentation contour IOU (Intersection over Union) matching is used as the similarity measure, that is, the intersection over union ratio between the current detection frame position information of the first target object and the intersection over union ratio of the current frame tracking frame position information is calculated, and the position matching matrix of the first target object is obtained based on the intersection over union ratio, and the first alternative matching result set of the first target object is determined according to the position matching matrix; based on the foreground features of the current contour and the foreground features of the object in the historical frame observation information, the feature matching matrix of the first target object is determined, and the second alternative matching result set of the first target object is determined according to the feature matching matrix.

[0081] Step S25: determining a target matching result based on the first candidate matching result set and the second candidate matching result set to achieve target tracking of the object.

[0082] The target matching result of the first target object is determined based on the first alternative matching result set and the second alternative matching result set, and the tracking frames in these target matching results are displayed to complete target tracking. That is, the Hungarian algorithm is used for matching based on similarity, and the unmatched detection frame position information, foreground features, tracking frame position information and matching results are retained to facilitate subsequent target tracking of the next frame of the image to be detected.

[0083] It can be seen from this that the present application matches the current frame tracking box position information and the current detection information from the two aspects of features and position. That is to say, the first alternative matching result set is screened out based on the position aspect, and the second alternative matching result set is screened out based on the features aspect. The target matching result determined based on the first alternative matching result set and the second alternative matching result set is more accurate, so as to achieve more accurate target tracking.

[0084] See also Figure 4 As shown, the embodiment of the present application discloses a specific target tracking method, including:

[0085] Step S31: After receiving the current frame image to be detected, the detection segmentation model is used to obtain the current detection information and current contours of all objects in the current frame image to be detected; wherein the current detection information includes the detection frame position information and detection frame confidence of the object.

[0086] Step S32: extracting foreground features of the object from the current detection information and the current contour.

[0087] Step S33: Obtain the current frame tracking frame position information of the object predicted based on the current interactive filtering model according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame in the historical frame.

[0088] Step S34: For objects whose detection frame confidence is within the first preset range, matching is performed based on historical frame observation information and the foreground features of the object, current frame detection frame position information, and current frame tracking frame position information to achieve target tracking.

[0089] Step S35: For the object whose detection frame confidence is within the second preset range, the Euclidean distance between the detection frame position information of the object and the tracking frame position information in the historical frame observation information is calculated, and the current frame detection information of the object is matched with the tracking frame position information in the historical frame observation information according to the Euclidean distance to achieve target tracking.

[0090] In this embodiment, the matching of the current frame detection information of the object with the tracking frame position information in the historical frame observation information according to the Euclidean distance to achieve target tracking includes: filtering out current filtered tracking frame position information that matches the current frame detection information from the tracking frame position information in the historical frame observation information according to the Euclidean distance; obtaining a first size change value between the detection frame in the current frame detection information and the detection frame in the previous frame detection information, and a second size change value between the current filtered tracking frame position information and the tracking frame in the previous filtered tracking frame position information; wherein the previous filtered tracking frame position information is the tracking frame position information that matches the previous frame detection information; and matching the current frame detection information of the object with the tracking frame position information in the historical frame observation information based on a size relationship between the first size change value, the second size change value, and a preset size change threshold to achieve target tracking.

[0091] A second target object whose detection frame confidence is within a second preset range is determined from all objects, and the Euclidean distance between the detection frame position information corresponding to the second target object and the tracking frame position information in the historical frame observation information is calculated. A first round of screening is performed based on the Euclidean distance, that is, the tracking frame position information with a larger Euclidean distance is eliminated, and the tracking frame position information with a smaller Euclidean distance is retained, that is, the currently filtered tracking frame position information that matches the current frame detection information is screened out from the tracking frame position information in the historical frame observation information based on the Euclidean distance.

[0092] A first size change value between a detection frame in the current frame detection information and a detection frame in the previous frame detection information, and a second size change value between the current filtered tracking frame position information and the previous filtered tracking frame position information are obtained. It can be understood that the current frame detection information and the previous frame detection information here are both detection information of the same object in different frames, and the current filtered tracking frame position information and the previous filtered tracking frame position information are also tracking frame position information of the same object in different frames, and the previous filtered tracking frame position information matches the previous frame detection information; then a second round of screening is performed, that is, based on the first size change value, the second size change value and a preset size change threshold, tracking frame position information that matches the current detection information is determined from the current filtered tracking frame position information, and then the tracking frame position information is displayed to complete the target tracking of the object. Set a size change threshold to filter out sudden changes in the detection frame size in the detection information and the tracking frame size in the observation information. For example, if the first size change value and the second size change value are not greater than the preset size change threshold, the current filtered tracking frame position information will be displayed. If the first size change value and the second size change value are greater than the preset size change threshold, it means that the current filtered tracking frame position information is at the image boundary, which means that the target has disappeared, and the corresponding tracking frame position information should be removed in time.

[0093] It can be seen that in this application, for objects with low confidence in the detection frame, the first round of screening of the tracking frame position information is performed based on the Euclidean distance. Furthermore, a second round of screening is performed based on the changes in the detection frame size in the detection information and the changes in the tracking frame size in the observation information. This can filter out sudden changes in the detection frame and the tracking frame, making the final matching result more accurate and improving the robustness and accuracy of target tracking.

[0094] See also Figure 5 As shown, the embodiment of the present application discloses a target tracking device, comprising:

[0095] The detection module 11 is configured to receive the current frame image to be detected and obtain the current detection information and current contours of all objects in the current frame image to be detected using the detection segmentation model; wherein the current detection information includes the detection frame position information and detection frame confidence of the object;

[0096] An extraction module 12, configured to extract foreground features of the object from the current detection information and the current contour;

[0097] A prediction module 13 is configured to obtain the current frame tracking frame position information of the object predicted based on the current interactive filtering model according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information, and matching results of the object in each frame in the historical frame;

[0098] The tracking module 14 is used to match the object whose detection frame confidence is within the first preset range based on the historical frame observation information and the foreground features of the object, the current frame detection frame position information and the current frame tracking frame position information to achieve target tracking.

[0099] It can be seen that after receiving the current frame image to be detected, the present application uses the detection segmentation model to obtain the current detection information and current contours of all objects in the current frame image to be detected; wherein, the current detection information includes the detection frame position information and detection frame confidence of the object; the foreground features of the object are extracted from the current detection information and the current contour; the current frame tracking frame position information of the object predicted based on the current interactive filtering model is obtained according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame in the historical frame; for objects whose detection frame confidence is within the first preset range, matching is performed based on the historical frame observation information and the foreground features of the object, the current frame detection frame position information and the current frame tracking frame position information to achieve target tracking. It can be seen that the present application uses the detection segmentation model to obtain the current detection information and current contour of the image to be detected in the current frame, and extracts the foreground features of the object from the current detection information and the current contour. On the one hand, if the detection frame in the current detection information is blocked, it avoids the inaccuracy of extracting features only from the current detection information. On the other hand, it can avoid background interference, that is, the extracted foreground features are more accurate, which provides a strong guarantee for the subsequent matching of the tracking frame position information and the detection information. Furthermore, the current interactive filtering model of the present application predicts more accurate current frame tracking frame position information based on historical frame observation information, and matches the current frame tracking frame position information with the current detection information based on features and positions to screen out the current frame tracking frame position information that matches the current detection information as the tracking result of the object, which can improve the accuracy of target tracking and complete target tracking in real time to meet the real-time requirements in the application scenario.

[0100] Furthermore, an embodiment of the present application also provides an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram should not be considered as any limitation to the scope of application of the present application.

[0101] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Specifically, the device may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the target tracking method performed by the electronic device disclosed in any of the aforementioned embodiments.

[0102] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0103] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0104] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0105] The operating system 221 is used to manage and control the various hardware devices and computer programs 222 on the electronic device, so as to enable the processor 21 to calculate and process the massive data 223 in the memory 22. It can be Windows, Unix, Linux, etc. In addition to including computer programs that can be used to implement the target tracking method performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs that can be used to perform other specific tasks. In addition to including data transmitted by external devices received by the electronic device, the data 223 can also include data collected by its own input and output interface 25.

[0106] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the target tracking method disclosed above is implemented. The specific steps of this method can be referred to the corresponding contents disclosed in the aforementioned embodiments and will not be repeated here.

[0107] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0108] Professionals may further appreciate that the units and algorithmic steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable EPROM (Erasable Programmable Read Only Memory), electrically erasable programmable EEPROM (Electrically Erasable Programmable read only memory), registers, hard disk, removable disk, CD-ROM (CoMP23037365act Disc Read-Only Memory), or any other form of storage medium known in the technical field.

[0109] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0110] The above is a detailed introduction to the target tracking method, device, equipment and medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A target tracking method, characterized in that: include: After receiving the current frame image to be detected, the detection segmentation model is used to obtain the current detection information and current contours of all objects in the current frame image to be detected; wherein the current detection information includes the detection frame position information and the detection frame confidence of the object; extracting a foreground feature of the object from the current detection information and the current contour; Obtaining the current frame tracking frame position information of the object predicted based on the current interactive filtering model according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame of the historical frame; For an object whose detection frame confidence is within a first preset range, matching is performed based on historical frame observation information and the foreground features of the object, current frame detection frame position information, and current frame tracking frame position information to achieve target tracking; The matching of the historical frame observation information with the foreground features of the object, the current frame detection frame position information, and the current frame tracking frame position information to achieve target tracking includes: Based on the intersection-and-union ratio of the current detection frame position information and the current frame tracking frame position information, the position matching matrix of the current contour of the object is determined, and the first alternative matching result set of the object is determined based on the position matching matrix; based on the foreground features of the current contour and the foreground features of the object in the historical frame observation information, the feature matching matrix of the object is determined, and the second alternative matching result set of the object is determined based on the feature matching matrix; based on the first alternative matching result set and the second alternative matching result set, the target matching result is determined to achieve target tracking of the object.

2. The target tracking method according to claim 1, characterized in that The method of using the detection and segmentation model to obtain current detection information and current contours of all objects in the image to be detected in the current frame includes: Obtaining current initial detection information and current initial contours of all objects in the image to be detected in the current frame using a detection and segmentation model; The current initial detection information and the current initial contour are filtered to obtain current detection information and a current contour of the object.

3. The target tracking method according to claim 2, characterized in that The filtering of the current initial detection information and the current initial contour to obtain the current detection information and the current contour of the object includes: If the size of the current initial contour is larger than the size of the detection box in the corresponding current initial detection information or the difference between the size of the detection box in the current initial detection information and the size of the corresponding current initial contour is greater than a preset threshold, the current initial contour and the corresponding current initial detection information are eliminated to obtain the current detection information and current contour of the object.

4. The target tracking method according to claim 1, wherein: Also includes: For an object whose detection frame confidence is within a second preset range, the Euclidean distance between the detection frame position information of the object and the tracking frame position information in the historical frame observation information is calculated, and the current detection information of the object is matched with the tracking frame position information in the historical frame observation information according to the Euclidean distance to achieve target tracking.

5. The target tracking method according to claim 4, characterized in that: The matching of the current detection information of the object with the tracking frame position information in the historical frame observation information according to the Euclidean distance to achieve target tracking includes: Filtering the tracking frame position information of the historical frame observation information according to the Euclidean distance to obtain the current filtered tracking frame position information that matches the current detection information; Obtaining a first size change value between a detection frame in the current detection information and a detection frame in the previous frame detection information, and a second size change value between the tracking frame position information after the current filtering and the tracking frame position information after the previous filtering; wherein the previous tracking frame position information after the previous filtering is the tracking frame position information that matches the detection information of the previous frame; Based on the magnitude relationship between the first size change value, the second size change value and a preset size change threshold, the current detection information of the object is matched with the tracking frame position information in the historical frame observation information to achieve target tracking.

6. The target tracking method according to any one of claims 1 to 5, characterized in that: Also includes: Storing the current frame tracking frame position information of the object for which target tracking has not been completed; The current frame tracking frame position information of the object for which target tracking is completed is input into the current interactive filtering model to implement parameter update.

7. A target tracking device, characterized in that: include: A detection module is configured to receive the current frame image to be detected and obtain current detection information and current contours of all objects in the current frame image to be detected using a detection segmentation model; wherein the current detection information includes the detection frame position information and detection frame confidence of the object; an extraction module, configured to extract foreground features of the object from the current detection information and the current contour; A prediction module is configured to obtain the current frame tracking frame position information of the object predicted based on the current interactive filtering model according to the historical frame observation information; the historical frame observation information includes the detection frame position information, foreground features, tracking frame position information and matching results of the object in each frame in the historical frame; A tracking module is configured to match, for an object whose detection frame confidence is within a first preset range, the foreground features of the object based on historical frame observation information, the position information of the current frame detection frame, and the position information of the current frame tracking frame, so as to achieve target tracking; The tracking module is specifically used to: Based on the intersection-and-union ratio of the current detection frame position information and the current frame tracking frame position information, the position matching matrix of the current contour of the object is determined, and the first alternative matching result set of the object is determined based on the position matching matrix; based on the foreground features of the current contour and the foreground features of the object in the historical frame observation information, the feature matching matrix of the object is determined, and the second alternative matching result set of the object is determined based on the feature matching matrix; based on the first alternative matching result set and the second alternative matching result set, the target matching result is determined to achieve target tracking of the object.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the target tracking method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, the steps of the target tracking method according to any one of claims 1 to 6 are implemented.

Images