Target tracking method, device, computer equipment and storage medium

Abstract

The present application relates to a target tracking method, device, computer equipment and storage medium. The method comprises: performing target detection on the current frame image, obtaining the current frame feature map and target detection result corresponding to the current frame image; segmenting the current frame feature map based on the target detection result, obtaining each current frame feature map block corresponding to the current frame feature map; performing feature normalization processing on each current frame feature map block, obtaining the current frame target feature corresponding to the current frame image; obtaining the previous frame target feature corresponding to the previous frame image of the current frame image; performing feature matching on the current frame target feature and the previous frame target feature, and obtaining the target tracking result of the current frame image according to the feature matching result. The use of this method can improve the target tracking effect.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a target tracking method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of computer technology, the theory and technology of artificial intelligence have become increasingly mature, and its application areas are constantly expanding, including robotics, speech recognition, image recognition, natural language processing, and expert systems. Target tracking is one of the hot topics in computer vision research. Target tracking refers to the detection, extraction, recognition, and tracking of moving targets in image sequences to obtain the motion parameters of the moving targets and achieve behavioral understanding of the moving targets. It has broad application prospects in many fields such as military guidance, video surveillance, robot visual navigation, human-computer interaction, and medical diagnosis.

[0003] Currently, traditional target tracking processes mostly use the center point, length, and width of the detection box in the detection results for target tracking. However, the center point, length, and width of the detection box are highly variable, resulting in limited target tracking effectiveness. Summary of the Invention

[0004] Therefore, it is necessary to provide a target tracking method, apparatus, computer equipment, and storage medium that can improve the target tracking effect in response to the above-mentioned technical problems.

[0005] A target tracking method, the method comprising:

[0006] Perform object detection on the current frame image to obtain the current frame feature map and object detection results.

[0007] Based on the target detection results, the current frame feature map is segmented to obtain each current frame feature map block corresponding to the current frame feature map;

[0008] Each current frame feature map block is normalized to obtain the current frame target features corresponding to the current frame image.

[0009] Obtain the target features of the previous frame corresponding to the current frame image;

[0010] The target features in the current frame are matched with the target features in the previous frame, and the target tracking result of the current frame image is obtained based on the feature matching result.

[0011] In one embodiment, the current frame feature map is segmented based on the target detection result to obtain each current frame feature map block corresponding to the current frame feature map, including:

[0012] Determine the geometric relationship between the current frame feature map and the current frame image;

[0013] The target detection results are scaled based on geometric relationships to obtain the scaled results corresponding to the target detection results.

[0014] The current frame feature map is segmented according to the scaling result to obtain the current frame feature map blocks corresponding to the current frame feature map.

[0015] In one embodiment, feature normalization processing is performed on each current frame feature map block to obtain the current frame target features corresponding to the current frame image, including:

[0016] Each current frame feature map block is segmented to obtain a feature map sub-block;

[0017] Each feature map sub-block is subjected to max pooling to obtain the sub-block features corresponding to each feature map sub-block.

[0018] The features of each sub-block are concatenated to obtain the target features of the current frame image.

[0019] In one embodiment, feature matching is performed between the target features of the current frame and the target features of the previous frame, and the target tracking result of the current frame image is obtained based on the feature matching result, including:

[0020] Determine the feature distance between the target features in the current frame and the target features in the previous frame;

[0021] Feature matching is performed based on feature distance using a matching algorithm to obtain the feature matching results;

[0022] The target tracking result of the current frame image is obtained based on the feature matching result.

[0023] In one embodiment, obtaining the target tracking result of the current frame image based on the feature matching result includes:

[0024] Determine the successfully matched and unmatched items from the feature matching results;

[0025] Obtain the target detection results for successfully matched and unmatched items respectively;

[0026] The target detection results corresponding to the successfully matched and unsuccessfully matched items are used as the target tracking results of the current frame image.

[0027] In one embodiment, the target detection results corresponding to the successfully matched items and the unsuccessfully matched items are used as the target tracking results of the current frame image, including:

[0028] The target detection results corresponding to the successfully matched items and the unsuccessfully matched items are filtered to obtain the filtered target detection results.

[0029] The filtered target detection result is used as the target tracking result of the current frame image.

[0030] In one embodiment, performing object detection on the current frame image to obtain the current frame feature map and object detection result corresponding to the current frame image includes:

[0031] Obtain the current frame image and query the pre-trained traffic light detection model;

[0032] The traffic light detection model is used to detect traffic lights in the current frame image to obtain target detection results including the traffic light detection boxes in the current frame image.

[0033] Obtain the current frame feature map corresponding to the current frame image during traffic light detection.

[0034] A target tracking device, the device comprising:

[0035] The target detection module is used to perform target detection on the current frame image and obtain the current frame feature map and target detection results corresponding to the current frame image;

[0036] The feature map segmentation module is used to segment the current frame feature map based on the target detection result, and obtain each current frame feature map block corresponding to the current frame feature map;

[0037] The current frame feature acquisition module is used to perform feature normalization processing on each current frame feature patch to obtain the current frame target features corresponding to the current frame image;

[0038] The previous frame feature acquisition module is used to acquire the target features of the previous frame corresponding to the previous frame image of the current frame image;

[0039] The tracking result acquisition module is used to perform feature matching between the target features of the current frame and the target features of the previous frame, and obtain the target tracking result of the current frame image based on the feature matching result.

[0040] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0041] Perform object detection on the current frame image to obtain the current frame feature map and object detection results.

[0042] Based on the target detection results, the current frame feature map is segmented to obtain each current frame feature map block corresponding to the current frame feature map;

[0043] Each current frame feature map block is normalized to obtain the current frame target features corresponding to the current frame image.

[0044] Obtain the target features of the previous frame corresponding to the current frame image;

[0045] The target features in the current frame are matched with the target features in the previous frame, and the target tracking result of the current frame image is obtained based on the feature matching result.

[0046] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0047] Perform object detection on the current frame image to obtain the current frame feature map and object detection results.

[0048] Based on the target detection results, the current frame feature map is segmented to obtain each current frame feature map block corresponding to the current frame feature map;

[0049] Each current frame feature map block is normalized to obtain the current frame target features corresponding to the current frame image.

[0050] Obtain the target features of the previous frame corresponding to the current frame image;

[0051] The target features in the current frame are matched with the target features in the previous frame, and the target tracking result of the current frame image is obtained based on the feature matching result.

[0052] The aforementioned target tracking method, apparatus, computer equipment, and storage medium, based on the target detection results of the current frame image, segment the current frame feature map corresponding to the current frame image, perform feature normalization processing on each current frame feature map block corresponding to the obtained current frame feature map to obtain the current frame target features corresponding to the current frame image, perform feature matching between the current frame target features and the previous frame target features, and obtain the target tracking result of the current frame image based on the feature matching result. In the target tracking process, the target features obtained by performing feature normalization processing on each feature map block corresponding to the feature map in the image are used as the target tracking features. Target features possess rich information and strong robustness, which can effectively improve the target tracking effect. Attached Figure Description

[0053] Figure 1 This is a diagram illustrating the application environment of a target tracking method in one embodiment.

[0054] Figure 2 This is a flowchart illustrating a target tracking method in one embodiment;

[0055] Figure 3 This is a schematic diagram of the feature map segmentation process in one embodiment;

[0056] Figure 4 This is a structural block diagram of a target tracking device in one embodiment;

[0057] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] The target tracking method provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. Terminal 102 acquires the current frame image and sends it to server 104. After receiving the current frame image, server 104 performs target detection on the current frame image, obtaining the current frame feature map and target detection result. Based on the target detection result, server 104 segments the current frame feature map, performs feature normalization on each current frame feature map block, obtains the current frame target feature, and performs feature matching between the current frame target feature and the previous frame target feature. Based on the feature matching result, the target tracking result of the current frame image is obtained. Furthermore, server 104 can also return the obtained target tracking result to terminal 102. In other implementations, the target tracking method can also be implemented directly by terminal 102. For example, after acquiring the current frame image, terminal 102 can directly perform target detection on the current frame image and perform subsequent target tracking processing to obtain the target tracking result.

[0060] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets and portable wearable devices, and the server 104 can be implemented by a standalone server or a server cluster consisting of multiple servers.

[0061] In one embodiment, such as Figure 2 As shown, a target tracking method is provided, which is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0062] Step 202: Perform target detection on the current frame image to obtain the current frame feature map and target detection result corresponding to the current frame image.

[0063] Target tracking, in particular, involves predicting the size and position of a target in subsequent frames given its size and position in the initial frame of a video sequence. The current frame image is the image frame currently acquired that requires target tracking. Target tracking requires continuous detection and tracking of targets in the image, specifically through continuous acquisition of multiple frames. The task of target detection is to find all targets (objects) of interest in an image and determine their position and size; it is one of the core problems in the field of machine vision. Due to the different appearances, shapes, and poses of various objects, coupled with interference from factors such as lighting and occlusion during imaging, target detection has always been one of the most challenging problems in machine vision. Target detection has been widely applied in people's lives, such as in security monitoring and autonomous driving. Target detection of the current frame image can be achieved through target detection algorithms, such as R-CNN (Region Convolutional Neural Networks), Fast R-CNN (Fast Region Convolutional Neural Networks), YOLO (You Only Look Once), Attention Net, and various other target detection algorithms.

[0064] The current frame feature map is the feature map identified by the object detection algorithm from the current frame image during object detection. When the object detection algorithm identifies multiple features, such as when performing object detection on the current frame image using a convolutional neural network model, the convolutional neural network model has multiple layers, and each layer can output different feature maps. The object detection result is the detection result obtained from the object detection process, which can specifically include the target object identified from the current frame.

[0065] Specifically, the terminal sends the acquired current frame image to the server. The server performs object detection on the received current frame image. This can be done using a preset object detection algorithm, such as a deep learning model. The server obtains the current frame feature map acquired by the deep learning model during object detection. When the deep learning model has multiple layers, the feature map output by the layer with richer features can be selected as the current frame feature map. The object detection result is then obtained from the output of the deep learning model.

[0066] Step 204: Based on the target detection results, the current frame feature map is segmented to obtain each current frame feature map block corresponding to the current frame feature map.

[0067] The current frame feature map patch is obtained by segmenting the current frame feature map, and the size of each feature map patch corresponds to the size of the target detection result. Specifically, after obtaining the current frame feature map and the target detection result corresponding to the current frame image, the server segments the current frame feature map based on the target detection result. For example, the current frame feature map can be segmented according to the detection boxes of each target object in the target detection result to obtain current frame feature map patches of the same size as the detection boxes.

[0068] Step 206: Perform feature normalization processing on each current frame feature map block to obtain the current frame target features corresponding to the current frame image.

[0069] Feature normalization can fuse the features of each current frame feature patch to obtain the current frame target features that can characterize the features of the current frame image. The current frame target features are obtained by performing feature normalization on each current frame feature patch. They carry the features of each current frame feature patch, have rich feature information, and can accurately express the features of the current frame image. Target tracking based on the current frame target features can improve the target tracking effect.

[0070] Specifically, after obtaining each current frame feature map block corresponding to the current frame feature map, the server performs feature normalization processing on each current frame feature map block. Specifically, the server can perform region of interest pooling processing on each current frame feature map block to obtain the pooled features of each current frame feature map block, and then concatenate the pooled features of each current frame feature map block to obtain the current frame target features corresponding to the current frame image.

[0071] Step 208: Obtain the target features of the previous frame corresponding to the previous frame image of the current frame image.

[0072] The previous frame image is the adjacent frame image acquired before the current frame image. Target objects in the previous frame image are tracked. The target features of the previous frame image represent its characteristics and are used for target tracking. In practical applications, the target features of the previous frame image are extracted from the previous frame image in the same way as the target features of the current frame image. Specifically, the server can acquire the previous frame image of the current frame image, perform target detection on the previous frame image, obtain the corresponding previous frame feature map and target detection results, segment the previous frame feature map based on the target detection results, obtain each corresponding previous frame feature map patch, and perform feature normalization processing on each previous frame feature map patch to obtain the target features corresponding to the previous frame image.

[0073] Step 210: Perform feature matching between the target features of the current frame and the target features of the previous frame, and obtain the target tracking result of the current frame image based on the feature matching result.

[0074] After obtaining the target features of the current frame (representing the characteristics of the current image) and the target features of the previous frame (representing the characteristics of the previous image), the server performs feature matching between the current and previous frame target features. Specifically, feature matching can be based on the Hungarian algorithm. The feature matching result yields the target tracking result for the current frame image. In practical applications, after obtaining the target tracking result for the current frame image, the server can continue to acquire the next frame image and continue target tracking processing based on the target tracking method.

[0075] In the aforementioned target tracking method, based on the target detection result of the current frame image, the current frame feature map corresponding to the current frame image is segmented. The feature maps corresponding to each current frame feature map are then normalized to obtain the target features of the current frame image. These target features are then matched with the target features of the previous frame, and the target tracking result of the current frame image is obtained based on the feature matching result. In the target tracking process, the target features obtained by normalizing the feature maps corresponding to each feature map in the image are used as the target tracking features. These target features possess rich information and strong robustness, effectively improving the target tracking performance.

[0076] In one embodiment, such as Figure 3 As shown, the processing steps for feature map segmentation, namely, segmenting the current frame feature map based on the target detection result to obtain the current frame feature map blocks corresponding to the current frame feature map, include:

[0077] Step 302: Determine the geometric relationship between the current frame feature map and the current frame image.

[0078] The geometric relationship refers to the size relationship between the current frame feature map and the current frame image. Specifically, it can be a geometric ratio, such as the ratio of the current frame feature map to the current frame image. The current frame feature map is the feature map generated during the target detection process on the current frame image. The image size of the current frame feature map also changes during target detection. By comparing the current frame feature map and the current frame image geometrically, the geometric relationship between them can be determined, such as determining their image ratio.

[0079] Step 304: Scale the target detection results based on geometric relationships to obtain the scaled results corresponding to the target detection results.

[0080] After obtaining the geometric relationship between the current frame feature map and the current frame image, the server scales the target detection result according to this relationship. For example, if the geometric relationship is the ratio of the current frame feature map to the current frame image, the target detection result can be scaled according to this ratio to obtain the scaled result. The size of the target detection result corresponds to the current frame image, while the scaled result corresponds to the current frame feature map.

[0081] Step 306: Segment the current frame feature map according to the scaling result to obtain each current frame feature map block corresponding to the current frame feature map.

[0082] After obtaining the scaling result corresponding to the target detection result, the server segments the current frame feature map according to the scaling result. For example, it can segment the current frame feature map according to the scaling detection box of each target object in the scaling result to obtain each current frame feature map block with the same size as the scaling detection box.

[0083] In this embodiment, the target detection result is scaled according to the geometric relationship between the current frame feature map and the current frame image. The current frame feature map is then segmented according to the scaling result corresponding to the target detection result. This allows the current frame feature map to be segmented according to the geometric relationship, thereby obtaining each current frame feature map block carrying rich feature information. This enables target tracking through each current frame feature map block, improving the target tracking effect.

[0084] In one embodiment, the current frame feature map blocks are subjected to feature normalization processing to obtain the current frame target features corresponding to the current frame image, including: segmenting each current frame feature map block to obtain each feature map sub-block; performing max pooling processing on each feature map sub-block to obtain the sub-block features corresponding to each feature map sub-block; and concatenating the sub-block features to obtain the current frame target features corresponding to the current frame image.

[0085] In this embodiment, feature normalization can be performed on each current frame feature map patch using Region of Interest (ROI) pooling to obtain the target features corresponding to the current frame image. ROI pooling is a type of pooling layer, specifically for ROIs (Regions of Interest). Its characteristic is that the input feature map size is not fixed, but the output feature map size is fixed. ROI pooling can significantly accelerate training and testing, and improve detection accuracy.

[0086] Specifically, after obtaining the feature map blocks of each current frame, the server segments each current frame feature map block, such as by dividing each current frame feature map into equal 4*4 segments to obtain feature map sub-blocks. For each feature map sub-block, the server performs max pooling, that is, the maximum pixel value in each feature map sub-block is used as the pixel value of the corresponding feature map sub-block. The pixel value of the obtained feature map sub-block is the sub-block feature of the feature map sub-block. The server concatenates the features of each sub-block to obtain the target feature of the current frame image. In a specific implementation, the server can take the maximum value of each feature map sub-block as the sub-block feature corresponding to each feature map sub-block. The server concatenates the sub-block features corresponding to each feature map sub-block according to the channel and segmentation range to obtain a one-dimensional feature vector, and uses this one-dimensional feature vector as the target feature of the current frame image.

[0087] In this embodiment, after segmenting the feature map of the current frame, max pooling is performed on each sub-block of the segmented feature map, and the sub-block features corresponding to each sub-block are concatenated to obtain the target features of the current frame image. This ensures that the target features of the current frame corresponding to different detection box sizes in the target detection result have the same feature dimension, and the number of target features of the current frame is the same as the number of detection boxes in the target detection result. This allows the target features of the current frame image to be accurately represented by the target features of the current frame, which is beneficial to improving the target tracking effect.

[0088] In one embodiment, feature matching is performed between the target features of the current frame and the target features of the previous frame, and the target tracking result of the current frame image is obtained based on the feature matching result. This includes: determining the feature distance between the target features of the current frame and the target features of the previous frame; performing feature matching based on the feature distance using a matching algorithm to obtain the feature matching result; and obtaining the target tracking result of the current frame image based on the feature matching result.

[0089] After obtaining the target features of the current frame and the previous frame, the server performs feature matching between them. Specifically, the server determines the feature distance between the target features of the current frame and the target features of the previous frame. In practice, this can be achieved by calculating the cosine distance between the target features of the current frame and the target features of the previous frame. After obtaining the feature distance, the server performs feature matching using a preset matching algorithm, such as the Hungarian algorithm, to obtain the feature matching result. The Hungarian algorithm is a combinatorial optimization algorithm that solves the task allocation problem in polynomial time. The Hungarian algorithm can be used to solve matching problems in bipartite graphs, such as feature matching between target features of the current frame and the target features of the previous frame. In practice, when performing feature matching using the Hungarian algorithm, a distance threshold can be set. If the sum of the feature matching distances exceeds this threshold, the feature matching is considered unsuccessful.

[0090] After obtaining the feature matching result, the server determines the target tracking result of the current frame image based on the feature matching result. For example, the target detection result of the current frame image can be matched with the target detection result of the previous frame image based on the feature matching result, thereby realizing target tracking processing.

[0091] In this embodiment, a matching algorithm is used to perform feature matching on the feature distance between the target features in the current frame and the target features in the previous frame. Based on the feature matching result, the target tracking result of the current frame image is determined. This allows the target detection result of the current frame image to correspond with the target detection result of the previous frame image, thereby achieving target tracking processing and improving the target tracking effect.

[0092] In one embodiment, obtaining the target tracking result of the current frame image based on the feature matching result includes: determining the successfully matched items and the unsuccessfully matched items from the feature matching result; obtaining the target detection results corresponding to the successfully matched items and the unsuccessfully matched items respectively; and using the target detection results corresponding to the successfully matched items and the unsuccessfully matched items as the target tracking result of the current frame image.

[0093] When determining the target tracking result for the current frame image, the server identifies successfully matched and unmatched items from the feature matching results. Specifically, the server can divide the feature matching results into successfully matched and unmatched items. The server then determines the target detection results corresponding to each successfully matched and unmatched item. A successfully matched item indicates that the target was detected in both the previous and current frames, allowing tracking to continue. An unmatched item indicates that the target was not detected in either the previous or current frame; that is, the target may have existed in the previous frame but not in the current frame, suggesting a possible missed detection. In this case, the target detection result needs to be added from the previous frame to the current frame for continued target tracking. Conversely, if the target detection result exists in the current frame but not in the previous frame, it indicates a newly added target object, requiring tracking to continue.

[0094] Furthermore, the server uses the target detection results corresponding to the successfully matched and unmatched items as the target tracking results of the current frame image. That is, the target tracking results of the current frame image include both the target detection results that successfully matched with the previous frame image and the target detection results that did not successfully match with the previous frame image. This preserves the target detection results of the current frame image and the previous frame image, which can prevent missed detections and improve the target tracking effect.

[0095] In one embodiment, using the target detection results corresponding to the successfully matched items and the unsuccessfully matched items as the target tracking results of the current frame image includes: filtering the target detection results corresponding to the successfully matched items and the unsuccessfully matched items to obtain filtered target detection results; and using the filtered target detection results as the target tracking results of the current frame image.

[0096] After obtaining the target detection results for both successful and unsuccessful matches, the server filters these results. Specifically, a Kalman filter is applied to both sets of results to smooth the target detection and improve its performance. The server then uses the filtered target detection results as the target tracking result for the current frame. By filtering the target detection results for both successful and unsuccessful matches, noise reduction and smoothing are achieved, which helps to smooth the target tracking position and improves the target tracking effect.

[0097] In one embodiment, performing target detection on the current frame image to obtain the current frame feature map and target detection result corresponding to the current frame image includes: acquiring the current frame image and querying a pre-trained traffic light detection model; performing traffic light detection on the current frame image through the traffic light detection model to obtain target detection results including the traffic light detection box in the current frame image; and acquiring the current frame feature map corresponding to the current frame image during the traffic light detection process.

[0098] In this embodiment, the target object for target tracking is a traffic light, such as a traffic light on a road. When performing target detection processing on the current frame image, the server acquires the current frame image, which is an image captured of the traffic light on the road. The server queries a pre-trained traffic light detection model, which can be a deep learning network model pre-trained based on a deep learning algorithm. The server performs traffic light detection on the current frame image through the traffic light detection model. Specifically, the current frame image can be input into the traffic light detection model to obtain the target detection result output by the traffic light detection model, which includes the traffic light detection box in the current frame image. The target detection result includes the traffic light detection box, which is the detection box within the range of the traffic light in the current frame image. On the other hand, the server acquires the current frame feature map corresponding to the current frame image during the traffic light detection process. Specifically, the server can acquire the feature map of the traffic light detection model during the traffic light detection process of the current frame image, as the current frame feature map corresponding to the current frame image.

[0099] In this embodiment, a pre-trained traffic light detection model is used to detect traffic lights in the current frame image, and the current frame feature map is obtained during the traffic light detection process of the traffic light detection model. This allows for target tracking through the current frame feature map carrying rich feature information, thereby improving the traffic light tracking effect.

[0100] In one embodiment, the target tracking method provided in this application is applied to target tracking of traffic lights, specifically in the traffic light tracking process of autonomous driving. In autonomous driving, most existing traffic light detection technologies use the two-dimensional attributes of the detection results, such as the center point, length, and width of the detection box, as input features for tracking, and then perform Kalman filtering to update the tracking target. However, because this method uses a single feature, the tracking effect is not ideal, resulting in discontinuous or abrupt detection results when a frame fails to detect a target in visual target detection, leading to instability in the overall traffic light detection results. The target tracking method provided in this application can compensate for false detections and missed detections, improving the traffic light tracking effect.

[0101] Specifically, after the server obtains the current frame image, it performs traffic light detection on the current frame image using a deep learning model for visual images. The detection result is represented as a rectangular box in the current frame image. Since the current frame feature map (i.e., the deep learning feature map) has a geometrically proportional relationship with the input image, the server scales the detection box results proportionally, where ratio = feature map scale / input image scale. The server projects the scaled rectangular boxes from the current frame image onto the deep learning feature map and uses the scaled rectangular boxes to crop the deep learning feature map, obtaining a series of corresponding current frame feature map patches. The number of current frame feature map patches is C (channels) x H (height) x W (width). The server performs ROI Pooling on the cropped current frame feature map patches, that is, it cuts each current frame feature map patch into a 4x4 proportional segment, takes the maximum value of the segment within each channel, and concatenates the obtained maximum values ​​according to the channel and segment range to form a one-dimensional feature vector. At this time, the feature vectors of all cropped current frame feature map patches have the same dimension, and the number of current frame feature vectors is equal to the number of deep learning detection boxes. The server uses the feature vector of the current frame image as the feature for traffic light tracking.

[0102] Further, the server obtains the feature vector corresponding to the previous frame image. Using cosine values ​​as the distance scale, the server calculates the cosine distance between each feature vector in the previous and current frames, obtaining a distance matrix. The server then performs matching processing on this distance matrix using the Hungarian algorithm and sets a distance threshold to determine the matching relationship of each feature vector. Specifically, a match is considered successful when the sum of the matching distances is less than the threshold, and unsuccessful when the sum is greater. The Hungarian algorithm is used to process the distance matrix, yielding feature matching results, including a matching matrix, a previous frame unmatched matrix, and a current frame unmatched matrix. The matching matrix includes the traffic light detection boxes corresponding to successfully matched feature vectors in the current and previous frames. The previous frame unmatched matrix includes traffic light detection boxes corresponding to feature vectors present in the previous frame but not in the current frame, and the current frame unmatched matrix includes traffic light detection boxes corresponding to feature vectors present in the current frame but not in the previous frame.

[0103] Furthermore, the server inputs the feature vectors of the current frame image from the matching matrix into the Kalman filter, updating the parameters in the Kalman filter. Since there is a one-to-one correspondence between the feature vectors and the Kalman filter, the server retains the traffic light detection boxes corresponding to the successfully matched feature vectors in the matching matrix for continued traffic light tracking. The server inputs the feature vectors of the unmatched matrix in the current frame into the Kalman filter as initial values, thus initializing the tracking of the traffic light detection boxes corresponding to the unmatched matrix in the current frame. Additionally, the server extracts the bounding boxes corresponding to the feature vectors of the unmatched matrix in the previous frame and projects them into the current frame based on the prediction results of the Kalman filter, thereby adding the traffic light detection boxes corresponding to the unmatched matrix in the previous frame to the current frame image for continued traffic light tracking.

[0104] In this embodiment, high-dimensional deep learning features are used as the features for traffic light tracking. Deep learning features carry rich feature information, enhancing the robustness of target tracking and improving the tracking matching effect. At the same time, adding the results detected in the previous frame but not detected in the current frame to the current frame for continued tracking can improve the missed detection recall rate, further enhancing the traffic light tracking effect.

[0105] It should be understood that, although Figure 2-3 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2-3 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0106] In one embodiment, such as Figure 4 As shown, a target tracking device 400 is provided, including: a target detection module 402, a feature map segmentation module 404, a current frame feature acquisition module 406, a previous frame feature acquisition module 408, and a tracking result acquisition module 410, wherein:

[0107] The target detection module 402 is used to perform target detection on the current frame image and obtain the current frame feature map and target detection result corresponding to the current frame image;

[0108] The feature map segmentation module 404 is used to segment the current frame feature map based on the target detection result to obtain each current frame feature map block corresponding to the current frame feature map;

[0109] The current frame feature acquisition module 406 is used to perform feature normalization processing on each current frame feature patch to obtain the current frame target features corresponding to the current frame image.

[0110] The previous frame feature acquisition module 408 is used to acquire the target features of the previous frame corresponding to the previous frame image of the current frame image;

[0111] The tracking result acquisition module 410 is used to perform feature matching between the target features of the current frame and the target features of the previous frame, and obtain the target tracking result of the current frame image based on the feature matching result.

[0112] In one embodiment, the feature map segmentation module 404 includes a geometric relationship determination module, a scaling processing module, and a segmentation processing module; wherein: the geometric relationship determination module is used to determine the geometric relationship between the current frame feature map and the current frame image; the scaling processing module is used to scale the target detection result based on the geometric relationship to obtain the scaled result corresponding to the target detection result; and the segmentation processing module is used to segment the current frame feature map according to the scaling result to obtain each current frame feature map block corresponding to the current frame feature map.

[0113] In one embodiment, the current frame feature acquisition module 406 includes a tile segmentation module, a pooling processing module, and a stitching module; wherein: the tile segmentation module is used to segment each current frame feature tile to obtain each feature map sub-block; the pooling processing module is used to perform max pooling processing on each feature map sub-block to obtain the sub-block features corresponding to each feature map sub-block; and the stitching module is used to stitch the sub-block features to obtain the current frame target features corresponding to the current frame image.

[0114] In one embodiment, the tracking result acquisition module 410 includes a feature distance determination module, a feature matching module, and a matching result processing module; wherein: the feature distance determination module is used to determine the feature distance between the target features of the current frame and the target features of the previous frame; the feature matching module is used to perform feature matching on the feature distance based on the matching algorithm to obtain the feature matching result; and the matching result processing module is used to obtain the target tracking result of the current frame image based on the feature matching result.

[0115] In one embodiment, the matching result processing module includes a matching item determination module, a matching item analysis module, and a matching item processing module; wherein: the matching item determination module is used to determine successfully matched items and unsuccessfully matched items from the feature matching results; the matching item analysis module is used to obtain the target detection results corresponding to the successfully matched items and unsuccessfully matched items respectively; and the matching item processing module is used to use the target detection results corresponding to the successfully matched items and unsuccessfully matched items respectively as the target tracking results of the current frame image.

[0116] In one embodiment, the matching item processing module is further configured to filter the target detection results corresponding to the successfully matched items and the unsuccessfully matched items to obtain the filtered target detection results; and use the filtered target detection results as the target tracking results of the current frame image.

[0117] In one embodiment, the target detection module 402 includes a data acquisition module, a traffic light detection module, and a feature map acquisition module; wherein: the data acquisition module is used to acquire the current frame image and query a pre-trained traffic light detection model; the traffic light detection module is used to perform traffic light detection on the current frame image through the traffic light detection model to obtain target detection results including the traffic light detection box in the current frame image; the feature map acquisition module is used to acquire the current frame feature map corresponding to the current frame image during the traffic light detection process.

[0118] Specific limitations regarding the target tracking device can be found in the limitations of the target tracking method described above, and will not be repeated here. Each module in the aforementioned target tracking device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the operations corresponding to each module.

[0119] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores target detection algorithm data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a target tracking method.

[0120] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0121] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0122] Perform object detection on the current frame image to obtain the current frame feature map and object detection results.

[0123] Based on the target detection results, the current frame feature map is segmented to obtain each current frame feature map block corresponding to the current frame feature map;

[0124] Each current frame feature map block is normalized to obtain the current frame target features corresponding to the current frame image.

[0125] Obtain the target features of the previous frame corresponding to the current frame image;

[0126] The target features in the current frame are matched with the target features in the previous frame, and the target tracking result of the current frame image is obtained based on the feature matching result.

[0127] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the geometric relationship between the current frame feature map and the current frame image; scaling the target detection result based on the geometric relationship to obtain the scaling result corresponding to the target detection result; and segmenting the current frame feature map according to the scaling result to obtain each current frame feature map block corresponding to the current frame feature map.

[0128] In one embodiment, when the processor executes the computer program, it further performs the following steps: segmenting each current frame feature map block to obtain each feature map sub-block; performing max pooling on each feature map sub-block to obtain the sub-block features corresponding to each feature map sub-block; and concatenating the sub-block features to obtain the current frame target features corresponding to the current frame image.

[0129] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining the feature distance between the target features of the current frame and the target features of the previous frame; performing feature matching based on the feature distance using a matching algorithm to obtain the feature matching result; and obtaining the target tracking result of the current frame image based on the feature matching result.

[0130] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining successfully matched items and unsuccessfully matched items from the feature matching results; obtaining the target detection results corresponding to the successfully matched items and unsuccessfully matched items respectively; and using the target detection results corresponding to the successfully matched items and unsuccessfully matched items as the target tracking results of the current frame image.

[0131] In one embodiment, when the processor executes the computer program, it further performs the following steps: filtering the target detection results corresponding to the successfully matched items and the unsuccessfully matched items to obtain the filtered target detection results; and using the filtered target detection results as the target tracking results of the current frame image.

[0132] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring the current frame image and querying the pre-trained traffic light detection model; performing traffic light detection on the current frame image through the traffic light detection model to obtain the target detection result including the traffic light detection box in the current frame image; and acquiring the current frame feature map corresponding to the current frame image during the traffic light detection process.

[0133] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0134] Perform object detection on the current frame image to obtain the current frame feature map and object detection results.

[0135] Based on the target detection results, the current frame feature map is segmented to obtain each current frame feature map block corresponding to the current frame feature map;

[0136] Each current frame feature map block is normalized to obtain the current frame target features corresponding to the current frame image.

[0137] Obtain the target features of the previous frame corresponding to the current frame image;

[0138] The target features in the current frame are matched with the target features in the previous frame, and the target tracking result of the current frame image is obtained based on the feature matching result.

[0139] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the geometric relationship between the current frame feature map and the current frame image; scaling the target detection result based on the geometric relationship to obtain the scaled result corresponding to the target detection result; and segmenting the current frame feature map according to the scaling result to obtain each current frame feature map block corresponding to the current frame feature map.

[0140] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: segmenting each current frame feature map block to obtain each feature map sub-block; performing max pooling on each feature map sub-block to obtain the sub-block features corresponding to each feature map sub-block; and concatenating the sub-block features to obtain the current frame target features corresponding to the current frame image.

[0141] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the feature distance between the target features of the current frame and the target features of the previous frame; performing feature matching based on the feature distance using a matching algorithm to obtain the feature matching result; and obtaining the target tracking result of the current frame image based on the feature matching result.

[0142] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining successfully matched items and unsuccessfully matched items from the feature matching results; obtaining the target detection results corresponding to the successfully matched items and unsuccessfully matched items respectively; and using the target detection results corresponding to the successfully matched items and unsuccessfully matched items as the target tracking results of the current frame image.

[0143] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: filtering the target detection results corresponding to the successfully matched items and the unsuccessfully matched items to obtain the filtered target detection results; and using the filtered target detection results as the target tracking results of the current frame image.

[0144] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring the current frame image and querying the pre-trained traffic light detection model; performing traffic light detection on the current frame image using the traffic light detection model to obtain the target detection result including the traffic light detection box in the current frame image; and acquiring the current frame feature map corresponding to the current frame image during the traffic light detection process.

[0145] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0146] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A target tracking method, characterized in that: The method comprises: Performing target detection on the current frame image to obtain a current frame feature map and a target detection result corresponding to the current frame image; Determine a geometric relationship between the current frame feature map and the current frame image; Scaling the target detection result based on the geometric relationship to obtain a scaling result corresponding to the target detection result; Segmenting the current frame feature map according to the scaling result to obtain current frame feature map blocks corresponding to the current frame feature map; Performing feature normalization processing on each of the current frame feature blocks to obtain the current frame target features corresponding to the current frame image; Obtaining a previous frame target feature corresponding to a previous frame image of the current frame image; The target features of the current frame are matched with the target features of the previous frame, and the target tracking result of the current frame image is obtained according to the feature matching result.

2. The method according to claim 1, characterized in that The step of performing feature normalization processing on each of the current frame feature blocks to obtain the current frame target features corresponding to the current frame image includes: Segmenting each of the current frame feature image blocks to obtain feature image sub-blocks; Performing maximum pooling processing on each feature map sub-block respectively to obtain sub-block features corresponding to each feature map sub-block; The sub-block features are spliced ​​together to obtain the current frame target features corresponding to the current frame image.

3. The method according to claim 1, characterized in that The performing feature matching on the target feature of the current frame and the target feature of the previous frame, and obtaining the target tracking result of the current frame image according to the feature matching result, comprises: Determine a feature distance between the current frame target feature and the previous frame target feature; Performing feature matching on the feature distance based on a matching algorithm to obtain a feature matching result; The target tracking result of the current frame image is obtained according to the feature matching result.

4. The method according to claim 3, characterized in that The obtaining the target tracking result of the current frame image according to the feature matching result includes: Determine successfully matched items and unsuccessfully matched items from the feature matching results; Obtain target detection results corresponding to the successfully matched items and the unsuccessfully matched items respectively; The target detection results corresponding to the successfully matched items and the unsuccessfully matched items are respectively used as the target tracking results of the current frame image.

5. The method according to claim 4, characterized in that The using the target detection results corresponding to the successfully matched items and the unsuccessfully matched items respectively as the target tracking result of the current frame image includes: Filtering the target detection results corresponding to the successfully matched items and the unsuccessfully matched items to obtain filtered target detection results; The filtered target detection result is used as the target tracking result of the current frame image.

6. The method according to any one of claims 1 to 5, characterized in that: The performing target detection on the current frame image to obtain the current frame feature map and target detection result corresponding to the current frame image comprises: Get the current frame image and query the pre-trained traffic light detection model; Performing signal light detection on the current frame image by using the signal light detection model to obtain a target detection result including a signal light detection frame in the current frame image; Obtain a current frame feature map corresponding to the current frame image during the traffic light detection process.

7. A target tracking device, characterized in that: The device comprises: The target detection module is used to perform target detection on the current frame image and obtain the current frame feature map and target detection result corresponding to the current frame image; A feature map segmentation module is used to determine the geometric relationship between the current frame feature map and the current frame image; scale the target detection result based on the geometric relationship to obtain a scaled result corresponding to the target detection result; segment the current frame feature map according to the scaled result to obtain each current frame feature map block corresponding to the current frame feature map; A current frame feature acquisition module is used to perform feature normalization processing on each of the current frame feature blocks to obtain the current frame target feature corresponding to the current frame image; A previous frame feature acquisition module, used to acquire the previous frame target feature corresponding to the previous frame image of the current frame image; The tracking result acquisition module is used to perform feature matching on the target features of the current frame and the target features of the previous frame, and obtain the target tracking result of the current frame image according to the feature matching result.

8. The device according to claim 7, characterized in that The current frame feature acquisition module includes: A block segmentation module, used for segmenting each of the current frame feature blocks to obtain each feature map sub-block; A pooling processing module, used to perform maximum pooling processing on each feature map sub-block respectively to obtain sub-block features corresponding to each feature map sub-block; The splicing module is used to splice the sub-block features to obtain the current frame target features corresponding to the current frame image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images