Target tracking method, device, electronic device, and computer-readable storage medium

Abstract

The present invention provides a target tracking method, device, electronic device, and computer-readable storage medium, wherein the target tracking method includes: performing target detection in the current frame of a video to be processed to obtain a first overall position area and a first local position area of ​​the target to be processed; based on the tracking result of the target to be processed in a reference frame, predicting a second overall position area and a second local position area of ​​the target to be processed in the current frame; wherein the second local position area corresponds to the first local position area, and the reference frame includes a forward frame of the current frame in the video to be processed; correlating the first overall position area, the first local position area, the second overall position area, and the second local position area to obtain a tracking result of the target to be processed. This method is applicable to multi-angle scenes and improves the accuracy and stability of target tracking.

Description

Technical Field

[0001] The present invention relates to the field of video image processing technology, and in particular to a target tracking method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Current low-position video tracking technology typically uses a detector trained on multiple samples in conjunction with a high-frame-rate single-target correlation filter tracker for target detection and tracking. Vehicle targets in low-position video tracking systems experience significant variations in scale and angle with distance, making detection and tracking relatively challenging. Furthermore, lightweight low-position video tracking systems require high algorithm frame rates and a fast, time-efficient solution. Existing technology is not well-suited for low-position, multi-angle scenarios and needs improvement. Summary of the Invention

[0003] The present invention provides a target tracking method, device, electronic device, and computer-readable storage medium. The method is applicable to multi-angle scenes and improves the accuracy and stability of target tracking.

[0004] To solve the above technical problems, the first technical solution provided by the present invention is: to provide a target tracking method, comprising: performing target detection on the current frame in the video to be processed to obtain a first overall position area of ​​the target to be processed and a first local position area within the first overall position area; based on the tracking result of the target to be processed in the reference frame, predicting the second overall position area of ​​the target to be processed in the current frame and the second local position area within the second overall position area; wherein, the second local position area corresponds to the first local position area, and the reference frame includes the forward frame of the current frame in the video to be processed; associating the first overall position area, the first local position area, the second overall position area and the second local position area to obtain the tracking result of the target to be processed.

[0005] Among them, the step of performing target detection on the current frame in the video to be processed to obtain the first overall position area of ​​the target to be processed and the first local position area within the first overall position area includes: performing target detection on the current frame based on the trained target detection model to obtain the first overall position area and the first local position area of ​​the target to be processed; the target detection model is trained using training samples of the overall position area marked with historical targets and the local position area within the overall position area.

[0006] Among them, the step of predicting the second overall position area of ​​the target to be processed in the current frame and the second local position area within the second overall position area based on the tracking results of the target to be processed in the reference frame includes: based on the tracking results of the target to be processed in the reference frame, using a filtering algorithm to predict the first overall center point coordinates corresponding to the second overall position area, and the first local center point coordinates corresponding to the second local position area; based on the first overall center point coordinates and the first local center point coordinates, using a regression network model to predict the second overall position area and the second local position area, and the second overall position area and the second local position area are associated.

[0007] Among them, the step of using a regression network model to predict the second overall position area and the second local position area based on the first overall center point coordinates and the first local center point coordinates includes: determining a first eigenvector based on the first overall center point coordinates and the first local center point coordinates, the first eigenvector representing the positional relationship between the second overall position area and the second local position area; and using the regression network model to predict the second overall position area and the second local position area based on the first eigenvector.

[0008] Among them, the step of predicting the second overall position area and the second local position area in the current frame based on the first feature vector using the regression network model includes: based on the first feature vector, using the regression network model to obtain the size of the second overall position area and the size of the second local position area; based on the size of the second overall position area and the coordinates of the first overall center point, determining the second overall position area; and based on the size of the second local position area and the coordinates of the first local center point, determining the second local position area.

[0009] Among them, the step of determining the second overall position area based on the size of the second overall position area and the coordinates of the first overall center point; and determining the second local position area based on the size of the second local position area and the coordinates of the first local center point, further includes: using the second overall position area and the second local position area to adjust the parameters of the filtering algorithm.

[0010] Among them, the step of associating the first overall position area, the first local position area, the second overall position area and the second local position area to obtain the tracking result of the target to be processed includes: associating the first overall position area with the second overall position area to obtain the tracking result of the target to be processed; associating the first local position area with the second local position area to obtain the tracking result of the target to be processed.

[0011] Among them, the step of associating the first overall position area, the first local position area, the second overall position area and the second local position area to obtain the tracking result of the target to be processed includes: in response to the intersection-and-union ratio of the first overall position area and the second overall position area being greater than a first threshold, associating the first overall position area with the second overall position area; the step of associating the first local position area with the second local position area to obtain the tracking result of the target to be processed includes: in response to the intersection-and-union ratio of the first local position area and the second local position area being greater than a second threshold, associating the first local position area with the second local position area.

[0012] Among them, the step of associating the first overall position area, the first local position area, the second overall position area and the second local position area to obtain the tracking result of the target to be processed also includes: in response to the intersection and union ratio of the first overall position area and the second overall position area being not greater than a first threshold, or in response to the intersection and union ratio of the first local position area and the second local position area being not greater than a second threshold, based on the inclusion relationship between the first local position area and the first overall position area, associating the first local position area and the first overall position area.

[0013] Among them, the step of associating the first local location area and the first overall location area based on the inclusion relationship between the first local location area and the first overall location area includes: in response to the first local location area being within the first overall location area, determining that the first local location area has an inclusion relationship with the first overall location area; determining a second eigenvector based on the second overall center point coordinates corresponding to the first local location area with the inclusion relationship and the second local center point coordinates corresponding to the first overall location area, the second eigenvector characterizing the positional relationship between the first overall location area and the first local location area; based on the second eigenvector, obtaining the third local location area and the third overall location area of ​​the target to be processed using a regression network model; calculating the first variance of the first overall location area and the third overall location area, calculating the second variance of the first local location area and the third local location area, and obtaining the variance sum based on the first variance and the second variance; and associating the first local location area with the first overall location area based on the variance sum.

[0014] The step of associating the first local location area with the first overall location area based on the sum of the variances includes associating the first local location area with the first overall location area in response to the sum of the variances being less than a third threshold.

[0015] Among them, the step of using a regression network model to predict the second overall location area and the second local location area based on the first overall center point coordinates and the first local center point coordinates previously includes: obtaining a second training sample, the second training sample is marked with the overall location area, local location area and association information of the overall location area and the local location area of ​​the historical target; determining the overall center point coordinates corresponding to the overall location area based on the overall location area, and determining the local center point coordinates corresponding to the local location area based on the local location area; obtaining a third eigenvector based on the overall center point coordinates and the local center point coordinates, the third eigenvector representing the positional relationship between the overall location area and the local location area of ​​the historical target; using the second eigenvector, the overall location information of the target, and the local location information of the target to train the initial model, thereby obtaining the regression network model.

[0016] In order to solve the above technical problems, the second technical solution provided by the present invention is: to provide a target tracking device, the target tracking device comprising: a detection module, used to perform target detection on the current frame in the video to be processed, and obtain a first overall position area of ​​the target to be processed and a first local position area within the first overall position area; a prediction module, used to predict the second overall position area of ​​the target to be processed in the current frame and the second local position area within the second overall position area based on the tracking result of the target to be processed in the reference frame, and the second local position area corresponds to the first local position area, and the reference frame is the forward frame of the current frame in the video to be processed; an association module, used to associate the first overall position area, the first local position area, the second overall position area and the second local position area, and thereby obtain the tracking result of the target to be processed.

[0017] To solve the above technical problems, the third technical solution provided by the present invention is: to provide an electronic device, comprising: a memory and a processor, wherein the memory stores program instructions, and the processor calls the program instructions from the memory to execute any of the above methods.

[0018] In order to solve the above technical problems, the fourth technical solution provided by the present invention is: providing a computer-readable storage medium storing a program file, which can be executed to implement any of the above methods.

[0019] The beneficial effects of the present invention are different from those of the prior art. The present invention performs target detection on the current frame of the video to be processed, obtains a first overall position area of ​​the target to be processed and a first local position area within the first overall position area, predicts a second overall position area of ​​the target to be processed in the current frame and a second local position area within the second overall position area based on the tracking result of the target to be processed in the reference frame, associates the first overall position area, the first local position area, the second overall position area, and the second local position area, and obtains the tracking result of the target to be processed. This method can associate different detection results, is applicable to multi-angle scenes, and improves the accuracy and stability of target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0021] Figure 1A schematic flow chart of an embodiment of a target tracking method of the present invention;

[0022] Figure 2 for Figure 1 A flow chart of an embodiment of step S12;

[0023] Figure 3 for Figure 2 A schematic flow chart of an embodiment of step S22;

[0024] Figure 4 A schematic diagram of the overall location area and local location area of ​​the vehicle;

[0025] Figure 5 This is a flow chart of another embodiment of the target tracking method of the present application;

[0026] Figure 6 and Figure 7 for Figure 1 A flow chart of an embodiment of step S13;

[0027] Figure 8 Schematic diagram of a flow chart of an embodiment of a method for training a regression network model of the present invention;

[0028] Figure 9 A schematic structural diagram of an embodiment of a target tracking device of the present invention;

[0029] Figure 10 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention;

[0030] Figure 11 It is a structural diagram of an embodiment of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0031] Prior art offers a multi-scale tracking method based on geometric estimation. This method, based on the principle of camera imaging, estimates the actual distance of a target from the camera. Using this distance, it estimates the change in the target's scale, thereby narrowing the range of the target scale variable. This approach employs multiple target scale tracking algorithms. By employing geometric estimation methods to estimate the target size and combining them with corresponding geometric relationships, this significantly reduces the computational complexity of the multiple target scale tracking algorithms, ensuring the algorithm's frame rate, improving its real-time performance, and achieving accurate and rapid target tracking. However, this method calculates the scale based on the target's height using the camera's imaging distance. Different target distances can lead to significant discrepancies in the corresponding scale estimates, resulting in significant differences in predicted structures in low-level scenes and when the target is close. The present application provides a target tracking method that is time-efficient and provides more stable and accurate detection and tracking. The following, in conjunction with the accompanying drawings, provides a clear and complete description of the technical solutions in the embodiments of this application. It should be understood that the described embodiments represent only a portion of the embodiments of this application, and not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of this application without inventive effort are intended to fall within the scope of protection of this application.

[0032] See Figure 1 , which is a flow chart of a first embodiment of the target tracking method of the present invention, specifically includes:

[0033] Step S11: performing target detection on the current frame in the video to be processed to obtain a first overall position region of the target to be processed and a first local position region within the first overall position region.

[0034] Specifically, an object detection model is used to perform object detection on the current frame of the video to be processed, obtaining a first overall location region of the object to be processed and a first local location region within the first overall location region. The object detection model is a traditional model capable of detecting objects, such as a deep neural network (DNN) model. The video to be processed can be composed of temporally continuous or discontinuous image frames.

[0035] In one embodiment, the first overall position area of ​​the target to be processed is an area including the entire target to be processed, and the first local position area is an area of ​​a certain part of the target to be processed. Specifically, taking vehicle target tracking as an example, the first overall position area can be the position area of ​​the vehicle's circumscribed rectangular frame, and the first local position area can be the position area of ​​the vehicle's front, rear, and other parts. Figure 4In another embodiment, taking the tracking of a person as an example, the first overall position area may be the position area of ​​the person's circumscribed rectangular frame, and the first local position area may be the position area of ​​the person's head, body, legs, and other parts.

[0036] Furthermore, it is necessary to train and acquire a target detection model, and use the trained target detection model to perform target detection on the current frame. Specifically, the training samples for training the target detection model are annotated with the overall location area of ​​the historical target and the local location area within the overall location area.

[0037] For conventional detection with a fixed camera perspective, a single target bounding box cannot well express the relationship between the target coordinate position and the target height and width. Taking a vehicle as an example, the coordinates of the same vehicle center point are very different when the vehicle is parked upright and sideways. Therefore, this application introduces a vehicle local position area to regress the height and width of the vehicle at a certain point, thereby improving reliability.

[0038] Step S12: Based on the tracking result of the target to be processed in the reference frame, predict a second overall position area of ​​the target to be processed in the current frame and a second local position area within the second overall position area.

[0039] Specifically, using the reference frame as a reference, based on the tracking structure of the target to be processed in the reference frame, a second overall position area of ​​the target to be processed in the current frame and a second local position area within the second overall position area are predicted. The second local position area corresponds to the first local position area. For example, if the first local position area is the front area of ​​the vehicle, the second local position area is also the front area of ​​the vehicle. The reference frame includes a previous frame of the current frame in the video to be processed. In one embodiment, the previous frame is the previous frame of the current frame. In another embodiment, the previous frame is the nth frame before the current frame.

[0040] In one embodiment, please combine Figure 2 , step S12 includes:

[0041] Step S21: Based on the tracking result of the target to be processed in the reference frame, a filtering algorithm is used to predict the coordinates of the first overall center point corresponding to the second overall position area and the coordinates of the first local center point corresponding to the second local position area.

[0042] Specifically, the tracking result of the target to be processed in the reference frame includes the first overall position area, the second overall position area, and the first local position area and the second local position area of ​​the target to be processed. There is also an association relationship between the first overall position area, the second overall position area, and the first local position area and the second local position area. The first overall center coordinates of the second overall position area in the current frame are predicted based on the first overall position area and the second overall position area in the reference frame; and the first local position area and the second local position area in the reference frame are used to predict the first local center point coordinates corresponding to the second local position area in the current frame.

[0043] Step S22: Based on the first overall center point coordinates and the first local center point coordinates, a regression network model is used to predict the second overall location area and the second local location area, and the second overall location area and the second local location area are associated.

[0044] Conventional tracking is not well adapted to scale changes, especially in video scenes where the angles and sizes of vehicles vary greatly. The common method of using pyramid features to traverse scales is not very time-consuming and effective. Therefore, this method uses the method of regressing the target height and width to assist tracking, achieving better tracking results with less time.

[0045] For details, please combine Figure 3 , step S22 specifically includes:

[0046] Step S31: Based on the first global center point coordinates and the first local center point coordinates, a first eigenvector is determined, where the first eigenvector represents a positional relationship between the second global position area and the second local position area.

[0047] Specifically, let the first eigenvector be X, and the first eigenvector X is:

[0048]

[0049] Among them, (x bi ,y bi ) is the first overall center coordinate, (x hi ,y hi ) are the first local center coordinates.

[0050] Step S32: Based on the first feature vector, the second overall location area and the second local location area are predicted using the regression network model.

[0051] Specifically, based on the first eigenvector X, the regression network model is used to obtain the size of the second overall location region and the size of the second local location region. Specifically, the first eigenvector X is input into the regression network model to obtain the size of the second overall location region and the size of the second local location region. The size of the second overall location region includes the width and height of the second overall location region, and the size of the second local location region includes the width and height of the second local location region.

[0052] Based on the size of the second overall position area and the coordinates of the first overall center point, the second overall position area is determined; and based on the size of the second local position area and the coordinates of the first local center point, the second local position area is determined. Specifically, if the height and width of the second overall position area are known, and the coordinates of the first overall center point corresponding to the second overall position area are known, the position coordinates of each vertex of the second overall position area can be determined based on the size of the second overall position area and the coordinates of the first overall center point, thereby determining the second overall position area. In addition, if the height and width of the second local position area are known, and the coordinates of the first local center point corresponding to the second local position area are known, the position coordinates of each vertex of the second local position area can be determined based on the size of the second local position area and the coordinates of the first local center point, thereby determining the second local position area.

[0053] In this embodiment, after finally obtaining the second overall position area and the second local position area, the parameters of the filtering algorithm are adjusted based on the second overall position area and the second local position area so that the next time the result of predicting the N+1 frame based on the N frame is predicted, it can be more accurate.

[0054] In this application, further training is required to obtain a regression network model. For details, please refer to Figure 8 , the methods for training regression network models include:

[0055] Step S81: Acquire a second training sample, where the second training sample is annotated with the overall location area, the local location area, and association information between the overall location area and the local location area of ​​the historical target.

[0056] Step S82: determining the overall center point coordinates corresponding to the overall location area based on the overall location area, and determining the local center point coordinates corresponding to the local location area based on the local location area.

[0057] Specifically, since the overall location area and local location area of ​​the historical target in the second training sample are known, the overall center point coordinates corresponding to the overall location area and the local center point coordinates corresponding to the local location area are also known. Assume that the overall center point coordinates corresponding to the overall location area are The coordinates of the local center point corresponding to the local position area are Assume that the width and height of the overall position area are The width and height of the local position area are

[0058] Building a regression function

[0059]

[0060] Step S83: Based on the overall center point coordinates and the local center point coordinates, a third feature vector is obtained, where the third feature vector represents the positional relationship between the overall location area and the local location area of ​​the historical target.

[0061] Further, a third eigenvector is obtained based on the global center coordinates and the local center coordinates. Specifically, a eigenvector is extracted based on the global center coordinates and the local center coordinates to obtain the third eigenvector;

[0062]

[0063] The third feature vector represents the positional relationship between the overall location area and the local location area of ​​the historical target.

[0064] In one embodiment, it is assumed that the third eigenvector X is:

[0065]

[0066] When there is no corresponding value or the calculation result cannot be obtained, X is assumed to be 0. Then substitute formula (3) into formula (2) to obtain:

[0067]

[0068] Substituting formula (4) into the above formula (1), we get:

[0069]

[0070] The third eigenvector X can be calculated based on formula (5).

[0071] Step S84: using the second feature vector, the overall position information of the target, and the local position information of the target to train the initial model, thereby obtaining the regression network model.

[0072] Specifically, the second eigenvector, the overall position information of the target, and the local position information of the target are input into the initial model to obtain a regression network model.

[0073] Specifically, the width and height of the overall position area are The width and height of the local position area are for Substituting Y into the above formula (5), we finally have Y = w T X+b.

[0074] The specific training process is the same as the existing technology, for example, using mean square error to calculate the loss and iterating samples to obtain the regression network.

[0075] Step S13: Associating the first overall location area, the first local location area, the second overall location area, and the second local location area to obtain a tracking result of the target to be processed.

[0076] Specifically, the first overall position area and the first local position area output by the target detection model are associated with the second overall position area and the second local position area output by the regression network model, thereby obtaining the tracking result of the target in the current frame.

[0077] In a specific embodiment, the first overall position area of ​​the target can be associated with the second overall position area of ​​the target to obtain the tracking result of the target in the current frame. Figure 6 Step S61: Calculate the intersection-and-union ratio of the first overall position region and the second overall position region. Step S62: In response to the intersection-and-union ratio of the first overall position region and the second overall position region being greater than a first threshold, associate the first overall position information of the target with the second overall position information of the target to obtain a tracking result of the target in the current frame.

[0078] In another embodiment, the first local position area of ​​the target can be associated with the second local position area of ​​the target to obtain the tracking result of the target in the current frame. Figure 6 , step S63: calculating the intersection-and-union ratio of the first local position area and the second local position area, step S64: in response to the intersection-and-union ratio of the first local position area and the second local position area being greater than the second threshold, associating the first local position area of ​​the target with the second local position area of ​​the target to obtain the tracking result of the target in the current frame.

[0079] In yet another embodiment, on the one hand, the first overall position area of ​​the target is associated with the second overall position area of ​​the target, and on the other hand, the first local position area of ​​the target is associated with the second local position area of ​​the target to obtain the tracking result of the target in the current frame. Specifically, in response to the intersection-and-union ratio of the first overall position area and the second overall position area being greater than a first threshold, the first overall position area of ​​the target is associated with the second overall position area of ​​the target; in response to the intersection-and-union ratio of the first local position area and the second local position area being greater than a second threshold, the first local position area of ​​the target is associated with the second local position area of ​​the target, and finally the tracking result of the target in the current frame is obtained. It should be noted that the first threshold and the second threshold can be the same or different, and are not specifically limited.

[0080] In another embodiment, in response to the intersection-and-union ratio of the first overall location area of ​​the target and the second overall location area of ​​the target being not greater than a first threshold, or in response to the intersection-and-union ratio of the first local location area and the second local location area being not greater than a second threshold, the first local location area and the first overall location area are associated based on the inclusion relationship between the first local location area and the first overall location area.

[0081] Specifically, please combine Figure 7 , step S71: in response to the intersection-and-union ratio of the first overall position area and the second overall position area being no greater than a first threshold, and in response to the first local position area being within the first overall position area, it is determined that the first local position area and the first overall position area have an inclusion relationship. At this time, step S72 is executed: based on the second overall center point coordinates corresponding to the first local position area with the inclusion relationship, and the second local center point coordinates corresponding to the first overall position area, a second feature vector is determined, the second feature vector representing the positional relationship between the first overall position area and the first local position area. The calculation method of the second feature vector is the same as the calculation method of the first feature vector and the second feature vector mentioned above, and will not be repeated here.

[0082] Step S73: Based on the second eigenvector, the regression network model is used to obtain the third local position area and the third overall position area of ​​the target to be processed, that is, the second eigenvector is input into the regression network model to obtain the third local position area and the third overall position area of ​​the target to be processed.

[0083] Step S74: Calculate a first variance between the first overall location area and the third overall location area, calculate a second variance between the first local location area and the third local location area, and obtain a sum of variances based on the first and second variances. Associate the first local location area with the first overall location area based on the sum of variances. Specifically, in response to the sum of variances being less than a third threshold, associate the first local location area with the first overall location area.

[0084] Because target detection and tracking use conventional detectors to obtain vehicle position information in images, this can result in inaccurate results. Furthermore, vehicle tracking requires determining whether the current frame is the same vehicle as the previous frame. Therefore, this application associates the target detection results of the current frame with the tracking results of the current frame based on a reference frame to improve the accuracy and reliability of target tracking.

[0085] Specifically, such as Figure 5 As shown, the present application uses an adaptive size tracking algorithm of a regression network model to make predictions based on the input of N frames and the processing results of N-frames to obtain prediction results. Then, based on the target association of the regression network model, the prediction results are target-associated with the target detection results of N frames, and finally the processing results of N frames are obtained. The processing results of N frames are used to process the N+1 frame. The method of processing the N+1 frame using the processing results of N frames is the same as the method of processing the N-frame using the N-frame, and will not be repeated here.

[0086] For conventional detection with a fixed camera perspective, the target tracking method of this application uses a single vehicle bounding box to poorly represent the relationship between the vehicle's coordinate position and its height and width. For example, for the same vehicle center coordinates, the height and width differ significantly when the vehicle is parked upright or sideways. Therefore, this solution uses a localized area of ​​the vehicle, such as the vehicle's face, to regress the vehicle's height and width at a specific point (specifically, to calculate the dimensions) to improve reliability. Furthermore, the tracking template's traversal of changes from the conventional overall scale ratio is specifically focused on height and width, providing greater accuracy and speed.

[0087] Furthermore, during target tracking, global and local correlation is required to confirm that both belong to the same vehicle. Specifically, the method of this application reuses a regression network based on intersection-over-union correlation. The sample annotations of the regression network reflect the target association relationship. This solution determines the association relationship by comparing the regression value with the actual difference, which increases reliability.

[0088] See Figure 9 , is a structural diagram of an embodiment of a target tracking device of the present invention, which specifically includes: a detection module 41, a prediction module 42 and an association module 43.

[0089] The detection module 41 is configured to perform target detection on a current frame in the video to be processed, and obtain a first overall position region of the target to be processed and a first local position region within the first overall position region.

[0090] In one embodiment, the detection module 41 performs target detection on the current frame based on the trained target detection model to obtain the first overall position area and the first local position area of ​​the target to be processed; the target detection model is trained using training samples of the overall position area marked with historical targets and the local position area within the overall position area.

[0091] Among them, the prediction module 42 is used to predict the second overall position area of ​​the target to be processed in the current frame and the second local position area within the second overall position area based on the tracking results of the target to be processed in the reference frame; wherein, the second local position area corresponds to the first local position area, and the reference frame includes the forward frame of the current frame in the video to be processed.

[0092] In one embodiment, the prediction module 42 uses a filtering algorithm to predict the first overall center point coordinates corresponding to the second overall position area and the first local center point coordinates corresponding to the second local position area based on the tracking results of the target to be processed in the reference frame; based on the first overall center point coordinates and the first local center point coordinates, the second overall position area and the second local position area are predicted using a regression network model, and the second overall position area and the second local position area are associated.

[0093] In one embodiment, the prediction module 42 determines a first eigenvector based on the first overall center point coordinates and the first local center point coordinates, where the first eigenvector represents the positional relationship between the second overall position area and the second local position area; based on the first eigenvector, the second overall position area and the second local position area are predicted using the regression network model.

[0094] In one embodiment, the prediction module 42 obtains the size of the second overall position area and the size of the second local position area based on the first feature vector using the regression network model; determines the second overall position area based on the size of the second overall position area and the coordinates of the first overall center point; and determines the second local position area based on the size of the second local position area and the coordinates of the first local center point.

[0095] In one embodiment, the prediction module 42 uses the second global location area and the second local location area to adjust the parameters of the filtering algorithm.

[0096] The associating module 43 is configured to associate the first overall location area, the first local location area, the second overall location area, and the second local location area, thereby obtaining a tracking result of the target to be processed.

[0097] In one embodiment, the association module 43 is used to associate the first overall location area with the second overall location area to obtain the tracking result of the target to be processed; and associate the first local location area with the second local location area to obtain the tracking result of the target to be processed.

[0098] In one embodiment, the association module 43 associates the first overall location area with the second overall location area when the intersection-and-union ratio of the first overall location area and the second overall location area is greater than a first threshold. Alternatively, the association module 43 associates the first local location area with the second local location area when the intersection-and-union ratio of the first local location area and the second local location area is greater than a second threshold.

[0099] In one embodiment, when the intersection-and-union ratio (IOR) of the first overall location area and the second overall location area is not greater than a first threshold, or when the IOR of the first local location area and the second local location area is not greater than a second threshold, the association module 43 associates the first local location area with the first overall location area based on the inclusion relationship between the first local location area and the first overall location area. Specifically, when the first local location area is within the first overall location area, it is determined that the first local location area and the first overall location area have an inclusion relationship; based on the coordinates of the second overall center point corresponding to the first local location area with the inclusion relationship and the coordinates of the second local center point corresponding to the first overall location area, a second eigenvector is determined, the second eigenvector representing the positional relationship between the first overall location area and the first local location area; based on the second eigenvector, a third local location area and a third overall location area of ​​the target to be processed are obtained using a regression network model; a first variance between the first overall location area and the third overall location area is calculated, a second variance between the first local location area and the third local location area is calculated, and a variance sum is obtained based on the first variance and the second variance; and the association module 43 associates the first local location area with the first overall location area based on the variance sum. Specifically, in response to the sum of the variances being smaller than a third threshold, the associating module 43 associates the first local location area with the first global location area.

[0100] In one embodiment, the prediction module 42 is further used to obtain a second training sample, wherein the second training sample is annotated with the overall location area, the local location area, and the association information between the overall location area and the local location area of ​​the historical target. Based on the overall location area, the overall center point coordinates corresponding to the overall location area are determined, and based on the local location area, the local center point coordinates corresponding to the local location area are determined. Based on the overall center point coordinates and the local center point coordinates, a third eigenvector is obtained, and the third eigenvector represents the positional relationship between the overall location area and the local location area of ​​the historical target. The initial model is trained using the second eigenvector, the overall location information of the target, and the local location information of the target to obtain the regression network model.

[0101] Because target detection and tracking use conventional detectors to obtain vehicle position information in images, this can result in inaccurate results. Furthermore, vehicle tracking requires determining whether the current frame is the same vehicle as the previous frame. Therefore, this application associates the target detection results of the current frame with the tracking results of the current frame based on a reference frame to improve the accuracy and reliability of target tracking.

[0102] For conventional detection with a fixed camera perspective, the target tracking device of this application uses a single vehicle bounding box that cannot accurately represent the relationship between the vehicle's coordinate position and its height and width. For example, for the same vehicle center coordinates, the height and width differ significantly when the vehicle is parked upright or sideways. Therefore, this solution uses a localized area of ​​the vehicle, such as the vehicle's face, to regress the vehicle's height and width at a specific point (specifically, to calculate the dimensions) to improve reliability. Furthermore, the tracking template's traversal of changes from the conventional overall scale ratio is specifically specified to the height and width values, providing greater accuracy and speed.

[0103] Furthermore, during target tracking, global and local correlation is required to confirm that both belong to the same vehicle. Specifically, the method of this application reuses a regression network based on intersection-over-union correlation. The sample annotations of the regression network reflect the target association relationship. This solution determines the association relationship by comparing the regression value with the actual difference, which increases reliability.

[0104] See Figure 10 , is a structural diagram of an electronic device according to an embodiment of the present invention, the electronic device includes a memory 202 and a processor 201 connected to each other.

[0105] The memory 202 is used to store program instructions for implementing any of the above-mentioned device methods.

[0106] The processor 201 is configured to execute program instructions stored in the memory 202 .

[0107] The processor 201 may also be referred to as a CPU (Central Processing Unit). The processor 201 may be an integrated circuit chip having signal processing capabilities. The processor 201 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.

[0108] The memory 202 can be a memory stick, a TF card, etc., which can store all the information in the electronic device of the device, including the input raw data, computer programs, intermediate operation results and final operation results are all stored in the memory. It stores and retrieves information according to the location specified by the controller. Only with memory can the electronic device have a memory function and ensure normal operation. The memory of the electronic device can be divided into main memory (internal memory) and auxiliary memory (external memory) according to its use, and there is also a classification method of dividing it into external memory and internal memory. External memory is usually a magnetic medium or an optical disk, etc., which can store information for a long time. Memory refers to the storage component on the motherboard, which is used to store the data and programs currently being executed, but is only used to temporarily store programs and data. If the power is turned off or the power is cut off, the data will be lost.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0112] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, system server, or network device, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application.

[0113] See also Figure 11 , which is a structural diagram of the computer-readable storage medium of the present invention. The storage medium of the present application stores a program file 203 that can implement all the above methods, wherein the program file 203 can be stored in the above storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the various implementation methods of the present application. The aforementioned storage device includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0114] The above are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A target tracking method, characterized in that: include: Performing target detection on a current frame in the video to be processed to obtain a first overall position region of the target to be processed and a first local position region within the first overall position region; Based on the tracking result of the target to be processed in the reference frame, predicting a second overall position area and a second local position area of ​​the target to be processed in the current frame; wherein the second local position area corresponds to the first local position area, and the reference frame includes a forward frame of the current frame in the video to be processed; Associating the first overall location area, the first local location area, the second overall location area, and the second local location area to obtain a tracking result of the target to be processed; The step of associating the first overall location area, the first local location area, the second overall location area, and the second local location area to obtain the tracking result of the target to be processed includes: In response to the intersection-and-union ratio of the first overall position area and the second overall position area being not greater than a first threshold, or in response to the intersection-and-union ratio of the first local position area and the second local position area being not greater than a second threshold, based on the inclusion relationship between the first local position area and the first overall position area, the first local position area and the first overall position area are associated to confirm whether the targets in the reference frame and the current frame are the same target.

2. The tracking method according to claim 1, characterized in that: The step of performing target detection on the current frame in the video to be processed to obtain a first overall position area of ​​the target to be processed and a first local position area within the first overall position area includes: Target detection is performed on the current frame based on the trained target detection model to obtain the first overall position area and the first local position area of ​​the target to be processed; the target detection model is trained using training samples of the overall position area marked with historical targets and the local position area within the overall position area.

3. The tracking method according to claim 1, wherein: The step of predicting a second overall position area of ​​the target to be processed in the current frame and a second local position area within the second overall position area based on the tracking result of the target to be processed in the reference frame includes: Based on the tracking result of the target to be processed in the reference frame, using a filtering algorithm to predict the coordinates of the first overall center point corresponding to the second overall position area and the coordinates of the first local center point corresponding to the second local position area; Based on the first overall center point coordinates and the first local center point coordinates, the second overall location area and the second local location area are predicted using a regression network model, and the second overall location area and the second local location area are associated.

4. The tracking method according to claim 3, characterized in that: The step of predicting the second overall location area and the second local location area using a regression network model based on the first overall center point coordinates and the first local center point coordinates includes: Determine a first feature vector based on the first overall center point coordinates and the first local center point coordinates, where the first feature vector represents a positional relationship between the second overall position area and the second local position area; Based on the first feature vector, the second overall location area and the second local location area are predicted using the regression network model.

5. The tracking method according to claim 4, characterized in that: The step of predicting the second overall position region and the second local position region in the current frame using the regression network model based on the first feature vector includes: Based on the first feature vector, using the regression network model to obtain the size of the second overall location area and the size of the second local location area; The second overall location area is determined based on the size of the second overall location area and the first overall center point coordinates; and the second local location area is determined based on the size of the second local location area and the first local center point coordinates.

6. The tracking method according to claim 5, characterized in that: determining the second overall position area based on the size of the second overall position area and the coordinates of the first overall center point; and determining the second local location area based on the size of the second local location area and the coordinates of the first local center point, and then further comprising: The second overall location area and the second local location area are used to adjust parameters of the filtering algorithm.

7. The tracking method according to claim 3, characterized in that: The step of associating the first overall location area, the first local location area, the second overall location area, and the second local location area to obtain the tracking result of the target to be processed includes: Associating the first overall location area with the second overall location area to obtain a tracking result of the target to be processed; The first local location area is associated with the second local location area to obtain a tracking result of the target to be processed.

8. The tracking method according to claim 7, characterized in that: The step of associating the first overall location area, the first local location area, the second overall location area, and the second local location area to obtain the tracking result of the target to be processed includes: In response to an intersection-over-union ratio of the first overall location area to the second overall location area being greater than a first threshold, associating the first overall location area with the second overall location area; The step of associating the first local location area with the second local location area to obtain the tracking result of the target to be processed includes: In response to an intersection-over-union ratio of the first local location area to the second local location area being greater than a second threshold, the first local location area is associated with the second local location area.

9. The tracking method according to claim 3, characterized in that: The step of associating the first local location area with the first overall location area based on the inclusion relationship between the first local location area and the first overall location area includes: In response to the first local location area being within the first overall location area, determining that the first local location area and the first overall location area have a containment relationship; determining a second feature vector based on the coordinates of a second overall center point corresponding to the first local location area having the containment relationship and the coordinates of the second local center point corresponding to the first overall location area, the second feature vector representing the positional relationship between the first overall location area and the first local location area; Based on the second eigenvector, a regression network model is used to obtain a third local location area and a third overall location area of ​​the target to be processed; Calculating a first variance between the first overall location area and the third overall location area, calculating a second variance between the first local location area and the third local location area, and obtaining a variance sum based on the first variance and the second variance; The first local location area is associated with the first global location area based on the sum of the variances.

10. The tracking method according to claim 9, characterized in that: The step of associating the first local location area with the first global location area based on the variance comprises: In response to the sum of the variances being less than a third threshold, the first local location area is associated with the first global location area.

11. The tracking method according to claim 9, characterized in that: The step of predicting the second overall location area and the second local location area using a regression network model based on the first overall center point coordinates and the first local center point coordinates previously includes: Acquire a second training sample, where the second training sample is annotated with an overall location area, a local location area, and association information between the overall location area and the local location area of ​​the historical target; Determining the coordinates of the overall center point corresponding to the overall location area based on the overall location area, and determining the coordinates of the local center point corresponding to the local location area based on the local location area; Based on the overall center point coordinates and the local center point coordinates, a third eigenvector is obtained, wherein the third eigenvector represents a positional relationship between the overall location area and the local location area of ​​the historical target; The initial model is trained using the second eigenvector, the overall position information of the target, and the local position information of the target to obtain the regression network model.

12. A target tracking device, characterized in that: include: A detection module, configured to perform target detection on a current frame in the video to be processed, and obtain a first overall position region of the target to be processed and a first local position region within the first overall position region; a prediction module, configured to predict, based on a tracking result of the target to be processed in a reference frame, a second overall position region of the target to be processed in the current frame and a second local position region within the second overall position region, wherein the second local position region corresponds to the first local position region, and the reference frame is a forward frame of the current frame in the video to be processed; an associating module, configured to associate the first overall location area, the first local location area, the second overall location area, and the second local location area, thereby obtaining a tracking result of the target to be processed; It is used to respond to the intersection-and-union ratio of the first overall position area and the second overall position area being not greater than a first threshold, or responding to the intersection-and-union ratio of the first local position area and the second local position area being not greater than a second threshold, and based on the inclusion relationship between the first local position area and the first overall position area, to associate the first local position area and the first overall position area to confirm whether the targets in the reference frame and the current frame are the same target.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the target tracking method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that A program file is stored, and the program file can be executed by a processor to implement the target tracking method according to any one of claims 1 to 11.

Images