Target tracking method, device, terminal and storage medium

Abstract

The present disclosure is about a target tracking method, device, terminal and storage medium, wherein the method includes: obtaining multiple detection images after each set time; inputting the first detection image at the current moment and the corresponding second detection image at the previous moment into a set deep learning network model to determine the position information, re-identification features and position offset information of the tracking target in the first detection image; configuring a track identifier for each tracking target corresponding to all the first detection images at the current moment according to all the second position information corresponding to the previous moment and all the first position information, all the first position offset information and all the first re-identification features corresponding to the current moment. This method can improve the performance of the entire target tracking method and provide conditions for real-time and online multi-target tracking across detection modules.

Description

Technical Field

[0001] The present disclosure relates to the field of terminal technology, and in particular to a target tracking method, device, terminal and storage medium. Background Art

[0002] With the rapid development of intelligent transportation systems, cross-camera multi-target tracking has become one of the important computer vision tasks. The purpose of cross-camera multi-target tracking is to locate and identify vehicles in a multi-camera system.

[0003] Generally speaking, most cross-camera multi-object tracking systems consist of two key modules, namely, single-camera multi-object tracking and cross-camera multi-object tracking. The single-camera multi-object tracking algorithm aims to track the vehicle trajectory within a single camera, while the cross-camera multi-object tracking aims to re-identify the trajectory of the same vehicle in multiple cameras through vehicle re-identification technology. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a target tracking method, device, terminal and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a target tracking method is provided, which is applied to a terminal, and the method includes:

[0006] After each set time period, a plurality of detection images are acquired, wherein the plurality of detection images are acquired by detection by a plurality of detection modules, and the plurality of detection images correspond one to one to the plurality of detection modules;

[0007] Inputting a first detection image at a current moment and a corresponding second detection image at a previous moment into a set deep learning network model, determining position information, re-identification features, and position offset information of a tracking target in the first detection image, wherein the duration between the current moment and the previous moment is equal to the set duration, and the position offset information represents an offset distance of the corresponding tracking target from the previous moment to the current moment;

[0008] According to all the second position information corresponding to the previous moment, and all the first position information, all the first position offset information and all the first re-identification features corresponding to the current moment, a trajectory identifier is configured for each tracking target corresponding to all the first detection images at the current moment.

[0009] Optionally, configuring a track identifier for each tracking target corresponding to all the first detection images at the current moment according to all the second position information corresponding to the previous moment, and all the first position information, all the first position offset information, and all the first re-identification features corresponding to the current moment, includes:

[0010] Determine, according to the first position information and the first position offset information of the tracking target in the first detection image, the predicted position information of the tracking target detected by the target detection module at the previous moment, wherein the target detection module is a detection module used to detect the first detection image;

[0011] Generate a cost matrix according to all predicted position information and all second position information; wherein the number of rows and the number of columns of the cost matrix are respectively equal to the number of predicted position information and the number of second position information, and the elements in the cost matrix are the distances between the positions represented by the corresponding predicted position information and the second position information;

[0012] Determining an optimal solution for the cost matrix;

[0013] Determine a feature pair according to each of the optimal solutions, wherein the feature pair includes a set of first-recognition features and historical features corresponding to the same tracking target;

[0014] The historical features and the first-recognition features in each feature pair are subjected to feature fusion processing to obtain fused features, and the trajectory identifier of the historical features in the feature pair is determined as the trajectory identifier of the fused features, and the first-recognition features are updated to the fused features.

[0015] Optionally, determining the optimal solution of the cost matrix includes:

[0016] A greedy algorithm is used to determine the optimal solution of the cost matrix.

[0017] Optionally, after determining the trajectory identifier of the historical feature in the feature pair as the trajectory identifier of the fused feature and updating the first re-recognition feature to the fused feature, the method includes:

[0018] According to the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target;

[0019] The same track identifier is configured for all first-recognition features of the same feature group.

[0020] Optionally, the step of determining, according to a hierarchical clustering algorithm, first-recognition features corresponding to the same tracking target among all first-recognition features as a feature group corresponding to the tracking target comprises:

[0021] According to the set distance threshold and the Euclidean distance between any two first-recognition features, and the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target.

[0022] Optionally, configuring the same track identifier for all first-recognition features of the same feature group includes:

[0023] If there is a first-recognition feature with a trajectory identifier configured in the feature group, then the trajectory identifier is determined as the trajectory identifier of all first-recognition features in the feature group; and / or,

[0024] If there is no first-recognition feature with a trajectory identifier configured in the feature group, a new trajectory identifier is configured for all first-recognition features of the feature group.

[0025] Optionally, after configuring the same trajectory identifier for all first re-identification features of the same feature group, the method includes:

[0026] All the first-level identification features in the same feature group are subjected to feature fusion processing to obtain weighted fused features, and this feature is added to the historical feature pool as the new historical feature of the tracking target corresponding to this feature group. At the same time, if the tracking target has an original historical feature in the historical feature pool, the original historical feature is deleted.

[0027] Optionally, after configuring the same trajectory identifier for all first re-identification features of the same feature group, the method includes:

[0028] Determine the similarity between each first-recognition feature and each historical feature from the first-recognition features of the undetermined trajectory identifier;

[0029] If the similarity is greater than a set similarity threshold, the trajectory identifier corresponding to the corresponding historical feature is determined as the trajectory identifier corresponding to the corresponding first-recognition feature; and / or, if the similarity is less than or equal to the set similarity threshold, a new trajectory identifier is created for the corresponding first-recognition feature.

[0030] Optionally, after determining the trajectory identifier corresponding to the corresponding historical feature as the trajectory identifier corresponding to the corresponding first-recognition feature, the method includes: performing feature fusion processing on the historical feature and the first-recognition feature to obtain a fused new feature, and updating the historical feature in the historical feature pool to the new feature;

[0031] and / or,

[0032] After creating a new trajectory identifier for the corresponding first re-recognition feature, the method includes: adding the first re-recognition feature to a historical feature pool as a new historical feature.

[0033] Optionally, the setting of the deep learning network model is determined by the following method:

[0034] Acquire multiple training sample pairs, each of which includes an input sample and an output sample, wherein the input sample includes a first sample image and a second sample image detected by the same detection module at the set detection interval, and the output sample is the position information, re-identification features, and position offset information of the tracking target in the first sample image, wherein the first sample image is detected at a later moment;

[0035] The original neural network model is trained using the multiple training sample pairs to obtain the set deep learning network model.

[0036] According to a second aspect of an embodiment of the present disclosure, a target tracking device is provided, which is applied to a terminal, and the device includes:

[0037] An acquisition module, used for acquiring a plurality of detection images after a set time period, wherein the plurality of detection images are detected by a plurality of detection modules, and the plurality of detection images correspond to the plurality of detection modules one by one;

[0038] A determination module, used to input a first detection image at a current moment and a corresponding second detection image at a previous moment into a set deep learning network model, and determine the position information, re-identification features, and position offset information of the tracking target in the first detection image, wherein the duration between the current moment and the previous moment is equal to the set duration, and the position offset information represents the offset distance of the corresponding tracking target from the previous moment to the current moment;

[0039] It is also used to configure a trajectory identifier for each tracking target corresponding to all the first detection images at the current moment based on all the second position information corresponding to the previous moment, and all the first position information, all the first position offset information and all the first re-identification features corresponding to the current moment.

[0040] Optionally, the determining module is used to:

[0041] Determine, according to the first position information and the first position offset information of the tracking target in the first detection image, the predicted position information of the tracking target detected by the target detection module at the previous moment, wherein the target detection module is a detection module used to detect the first detection image;

[0042] Generate a cost matrix according to all predicted position information and all second position information; wherein the number of rows and the number of columns of the cost matrix are respectively equal to the number of predicted position information and the number of second position information, and the elements in the cost matrix are the distances between the positions represented by the corresponding predicted position information and the second position information;

[0043] Determining an optimal solution for the cost matrix;

[0044] Determine a feature pair according to each of the optimal solutions, wherein the feature pair includes a set of first-recognition features and historical features corresponding to the same tracking target;

[0045] The historical features and the first-recognition features in each feature pair are subjected to feature fusion processing to obtain fused features, and the trajectory identifier of the historical features in the feature pair is determined as the trajectory identifier of the fused features, and the first-recognition features are updated to the fused features.

[0046] Optionally, the determining module is used to:

[0047] A greedy algorithm is used to determine the optimal solution of the cost matrix.

[0048] Optionally, the determining module is used to:

[0049] According to the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target;

[0050] The same track identifier is configured for all first-recognition features of the same feature group.

[0051] Optionally, the determining module is used to:

[0052] According to the set distance threshold and the Euclidean distance between any two first-recognition features, and the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target.

[0053] Optionally, the determining module is used to:

[0054] If there is a first-recognition feature with a trajectory identifier configured in the feature group, then the trajectory identifier is determined as the trajectory identifier of all first-recognition features in the feature group; and / or,

[0055] If there is no first-recognition feature with a trajectory identifier configured in the feature group, a new trajectory identifier is configured for all first-recognition features of the feature group.

[0056] Optionally, the device includes an updating module, wherein the updating module is configured to:

[0057] All the first-level identification features in the same feature group are subjected to feature fusion processing to obtain weighted fused features, and this feature is added to the historical feature pool as the new historical feature of the tracking target corresponding to this feature group. At the same time, if the tracking target has an original historical feature in the historical feature pool, the original historical feature is deleted.

[0058] Optionally, the determining module is used to:

[0059] Determine the similarity between each first-recognition feature and each historical feature from the first-recognition features of the undetermined trajectory identifier;

[0060] If the similarity is greater than a set similarity threshold, the trajectory identifier corresponding to the corresponding historical feature is determined as the trajectory identifier corresponding to the corresponding first-recognition feature; and / or, if the similarity is less than or equal to the set similarity threshold, a new trajectory identifier is created for the corresponding first-recognition feature.

[0061] Optionally, the device includes an updating module, wherein the updating module is configured to:

[0062] After determining the trajectory identifier corresponding to the corresponding historical feature as the trajectory identifier corresponding to the corresponding first-recognition feature, performing feature fusion processing on the historical feature and the first-recognition feature to obtain a fused new feature, and updating the historical feature in the historical feature pool to the new feature;

[0063] and / or,

[0064] After creating a new track mark for the corresponding first-recognition feature, the first-recognition feature is added to the historical feature pool as a new historical feature.

[0065] According to a third aspect of an embodiment of the present disclosure, a terminal is provided, the terminal including:

[0066] processor;

[0067] a memory for storing instructions executable by the processor;

[0068] The processor is configured to execute the method as described in any one of the first aspects.

[0069] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute a method as described in any one of the first aspects.

[0070] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: In this method, the same deep learning network model is used to determine the position information, re-identification features and position offset information of the tracking target in each detection image detected at the current moment, which can improve the efficiency of the entire target tracking method, and since the position offset information (characterizing the offset distance of the corresponding tracking target from the previous moment to the current moment) can be directly obtained, the performance of the entire target tracking method can be improved, which provides conditions for real-time and online multi-target tracking across detection modules, and can better meet the user experience.

[0071] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0073] Figure 1 The figure is a flow chart of a target tracking method according to an exemplary embodiment.

[0074] Figure 2 The figure is a block diagram of a target tracking device according to an exemplary embodiment.

[0075] Figure 3 is a block diagram of a terminal according to an exemplary embodiment. DETAILED DESCRIPTION

[0076] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0077] The present disclosure provides a target tracking method, which is applied to a terminal. In this method, the same deep learning network model is used to determine the position information, re-identification features and position offset information of the tracking target in each detection image detected at the current moment, which can improve the efficiency of the entire target tracking method, and because the position offset information (characterizing the offset distance of the corresponding tracking target from the previous moment to the current moment) can be directly obtained, the performance of the entire target tracking method can be improved, which provides conditions for real-time, online multi-target tracking across detection modules, and can better meet the user experience.

[0078] In an exemplary embodiment, a target tracking method is provided, which is applied to a terminal. The terminal may be a car, a server, a computer, etc. Figure 1 As shown, this method includes:

[0079] S110, determining a plurality of detection images after each set time period;

[0080] S120, inputting the first detection image at the current moment and the corresponding second detection image at the previous moment into a set deep learning network model to determine the position information, re-identification features and position offset information of the tracking target in the first detection image;

[0081] S130, configuring a track identifier for each tracking target corresponding to all the first detection images at the current moment according to all the second position information corresponding to the previous moment, and all the first position information, all the first position offset information, and all the first re-identification features corresponding to the current moment.

[0082] In step S110, multiple detection images are detected by multiple detection modules, and multiple detection images correspond to multiple detection modules one by one. Multiple detection modules refer to that the number of detection modules can be two or more than two. The detection module can include a camera. The detection module can be arranged at the terminal or can be electrically connected to the terminal. After the detection module detects the detection image, the detection image can be transmitted to the processor of the terminal so that the processor of the terminal obtains the detection image.

[0083] After the position of each detection module is fixed, the corresponding area where the image can be detected can be recorded as the preset detection area of ​​this detection module. For example, if the terminal is a car, multiple cameras, such as four cameras, can be set on the car to detect the images of the following directions of the car: front, left front, left rear and rear.

[0084] It should be noted that the position of the detection module is adjustable, so that the preset detection area can be adjusted to better meet the user's usage needs.

[0085] The set duration may refer to the duration of a detection cycle, and the set duration may be set in a terminal or in a detection module.

[0086] When the set time is set in the terminal, it can be set before the terminal leaves the factory, or it can be set after the terminal leaves the factory, and it can be modified later. When the set time is set in the detection module, its setting method is the same as its setting method in the terminal, and it will not be repeated here.

[0087] The set duration can be determined according to actual conditions. The better the data processing capability of the terminal, the shorter the set duration can be. The worse the data processing capability of the terminal, the longer the set duration can be.

[0088] For example, the duration between two adjacent frames may be determined as the set duration. In this case, the terminal may obtain each frame of image detected by the image detection module.

[0089] For another example, the duration between ten adjacent frames may be determined as the set duration. In this case, the terminal may use the image of the first frame as the detection image of the current detection cycle, and then use the image of the eleventh frame as the detection image of the next detection cycle.

[0090] In this step S110, every set time period, each detection module can transmit the detection image detected at the current moment to the processor of the terminal, so that the processor obtains the detection image detected by the corresponding detection module at the current moment.

[0091] It should be noted that, within each detection cycle (ie, every set time period), all detection modules perform an image detection on the corresponding preset detection area at the same time to obtain multiple detection images at this moment.

[0092] Example 1,

[0093] The terminal is a car, and three detection modules are arranged on the car, and the detection modules are cameras. The detection cycles of the three cameras are the same, and the detection cycle is the duration of adjacent frames, that is, the set duration is the duration of adjacent frames. The three cameras are respectively recorded as the first camera, the second camera, and the third camera. The preset detection area of ​​the first camera is recorded as the first area, the preset detection area of ​​the second camera is recorded as the second area, and the preset detection area of ​​the third camera is recorded as the third area.

[0094] After starting the target tracking of the car, the first camera, the second camera and the third camera are started at the same time to detect images of the first area, the second area and the third area respectively.

[0095] In this example, the first camera transmits each frame of detected images to the processor of the car, so that the processor obtains the detection image of the first area detected by the first camera in each detection cycle. The second camera transmits each frame of detected images to the processor of the car, so that the processor obtains the detection image of the second area detected by the second camera in each detection cycle. The third camera transmits each frame of detected images to the processor of the car, so that the processor obtains the detection image of the third area detected by the third camera in each detection cycle. In this way, the processor of the car obtains the detection image of each preset detection area in each detection cycle.

[0096] In step S120, the duration between the current moment and the previous moment is equal to the set duration, the current moment refers to the moment of detecting an image in the current detection cycle, and the previous moment refers to the moment of detecting an image in the previous detection cycle. The detection image obtained at the current moment is recorded as the first detection image, and the detection image obtained at the previous moment is recorded as the second detection image.

[0097] In this step, the detection images of adjacent detection cycles (i.e., the first detection image and the second detection image) can be input into the set deep learning network model, and the set deep learning network model can process the above detection images and output the position information, re-identification features and position offset information corresponding to the tracking target in each detection image of the current detection cycle (i.e., the current moment). Among them, the detection image may include more than one target image of the tracking target, and the tracking target in the detection image refers to the tracking target corresponding to all the target images in the detection image. The position information, re-identification features and position offset information of the tracking target refer to the position information, re-identification features and position offset information corresponding to the target image corresponding to this tracking target. The position offset information characterizes the offset distance of the corresponding tracking target from the previous moment to the current moment. The tracking target can be a pedestrian, a car, a bicycle, an electric car, an animal, etc., which is not limited here.

[0098] Among them, setting the deep learning network model can be determined by the following methods:

[0099] S100, obtaining multiple training sample pairs;

[0100] S200. Training the original neural network model through the multiple training samples to obtain the set deep learning network model.

[0101] In step S100, each training sample pair includes an input sample and an output sample. The input sample includes a first sample image detected by the same detection module at a first moment, and a second sample image detected at a second moment. The first moment and the second moment are separated by a set time, and the first moment is located after the second moment. It can be seen that the first sample image is the image detected at the later moment of the set time.

[0102] The output sample is the position information, re-identification features and position offset information of the tracking target in the first sample image.

[0103] It should be noted that the input samples of each training sample pair are detected by the same detection module. However, the input samples in different training sample pairs can be detected by different detection modules. For example, multiple training sample pairs can be divided into three parts, the input samples in the first part of the training sample pairs are all detected by the first detection module, the input samples in the second part of the training samples are all detected by the second detection module, and the input samples in the third part of the training samples are all detected by the third detection module.

[0104] In step S200, the original neural network model may be resdcn18, wherein resdcn18 is formed by adding dcn (deformable convolutional network) to resnet18.

[0105] The deep learning network model may include a backbone network and an additional detection head, a re-identification feature head, and a tracking offset head. The full name of reid is re-identification, which means re-identification.

[0106] The detection head is used to determine the position information, which may include the center coordinates of the tracked target and the size information of the tracked target. For example, the position information can be recorded as B = (cx, cy, w, h). Among them, cx and cy represent the center position information of the tracked target, which is the center coordinate of the tracked target, w represents the lateral size of the tracked target (also called the length size), and h represents the longitudinal size of the tracked target (also called the height size). The center coordinate offset head is used to determine the position offset information, which may be the center coordinate offset, and the center coordinate offset can be recorded as D = (dx, dy). For the training process of the network branch of the position information and the center coordinate offset, please refer to CenterTrack.

[0107] The re-identification feature head is used to determine the re-identification feature, which can be denoted as F. In the training of the re-identification network branch, each tracking target can be divided into independent categories and supervised using the cross entropy loss function; a total of 140 epochs are trained (an epoch is the process of training all training samples once), the batch size (the number of samples selected for one training) is 32, the initial learning rate is set to 1.25e-4, and the learning rate is decayed to 1 / 10 of the original in the 90th and 120th epochs respectively. The adam optimizer can be used to update the network parameters.

[0108] Example 2,

[0109] The terminal in this example 2 is the same as the terminal in example 1.

[0110] In this example 2, the detection image of the first camera at the current moment is the detection image of the first camera in the current frame. This detection image can be recorded as the first detection image. The detection image of the first camera at the previous moment is the detection image of the first camera in the previous frame. This detection image can be recorded as the second detection image.

[0111] The first detection image and the second detection image are input into the set deep learning network model, and the set deep learning network model can output the position information B1, the re-identification feature F1 and the center coordinate offset D1 of each tracking target in the first detection image, so that the processor determines the position information B1, the re-identification feature F1 and the center coordinate offset D1 of the tracking target detected by the first camera.

[0112] Similarly, the car's processor can determine the position information B2, re-identification feature F2 and center coordinate offset D2 of the tracking target detected by the second camera, and the position information B3, re-identification feature F3 and center coordinate offset D3 of the tracking target detected by the third camera.

[0113] As a result, the car's processor determines the position information, re-identification features, and center coordinate offset of the tracking target in each first detection image at the current moment.

[0114] In step S130, the terminal may determine the position information, re-identification features, and position offset information of each tracked target in each detection image at the current moment every set time period.

[0115] In this step, the last moment refers to the image detection moment corresponding to the last detection cycle. The detection image detected at the last moment is recorded as the second detection image. The position information, re-identification feature and position offset information of each tracked target in the detection image of the last detection cycle are recorded as the second position information, the second re-identification feature and the second position offset information respectively. The image of the tracked target in the second detection image can be recorded as the second target image.

[0116] The current moment refers to the image detection moment corresponding to the current detection cycle. The detection image detected at the current moment is recorded as the first detection image. The position information, re-identification feature and position offset information of each tracked target in the detection image of the current detection cycle are recorded as the first position information, the first re-identification feature and the first position offset information respectively. The image of the tracked target in the first detection image can be recorded as the first target image.

[0117] This step can determine whether the same detection module detects the same tracking target at the previous moment and the current moment, and whether all detection modules detect the same tracking target at the current moment, based on all the second position information, all the first position information, all the first re-identification features, and all the first position offset information, and then configure the same track identifier for the first re-identification features corresponding to the same tracking target. Based on this, the same track identifier can be configured for the same tracking target detected by multiple detection modules to better achieve multi-target tracking across cameras.

[0118] Among them, when determining whether the same detection module detects the same tracking target at the previous moment and the current moment, the relevant technology generally uses the Kalman filter algorithm and the Hungarian algorithm, but this method has limited performance and poor accuracy in the tracking target at a turning scene.

[0119] In the method disclosed in the present invention, since the deep learning network model is set to directly determine the position offset information of the tracking target from the previous moment to the current moment, the position information of the tracking target at the previous moment can be predicted directly based on the position offset information and the first position information of the same tracking target. Therefore, this method can achieve better target tracking performance within the same detection module without using the Kalman filter algorithm to predict the position information at the previous moment, and can improve the reliability of target tracking.

[0120] In this method, the same deep learning network model is used to determine the position information, re-identification features and position offset information of each tracked target in each detection image detected at the current moment, which can improve the efficiency of the entire target tracking method. In addition, since the position offset information (characterizing the offset distance of the tracked target from the previous moment to the current moment) can be obtained directly, the performance of the entire target tracking method can be improved, which provides conditions for real-time and online multi-target tracking across detection modules, and can better meet the user experience.

[0121] In an exemplary embodiment, a target tracking method is provided, which is applied to a terminal. In this method, according to all the second position information corresponding to the previous moment, and all the first position information, all the first position offset information and all the first re-identification features corresponding to the current moment, a track identifier is configured for each tracking target corresponding to all the first detection images at the current moment, which may include:

[0122] S210, determining predicted position information of the tracked target detected by the target detection module at the last moment according to the first position information and the first position offset information of the tracked target in the first detection image, wherein the target detection module is a detection module used to detect the first detection image;

[0123] S220, generating a cost matrix according to all predicted position information and all second position information;

[0124] S230, determining the optimal solution of the cost matrix;

[0125] S240, determining a feature pair according to an optimal solution of the cost matrix, where the feature pair includes a set of first-level recognition features and historical features corresponding to the same tracking target;

[0126] S250, performing feature fusion processing on the historical features and the first-recognition features in each feature pair to obtain fused features, and determining the trajectory identifier of the historical features in this feature pair as the trajectory identifier of the fused features, and updating the first-recognition features to the fused features.

[0127] In step S210, generally, each detection module can detect a first detection image at the current moment, and based on the set deep learning network model, the first position information, first position offset information and first re-identification feature of the tracking target in each first detection image can be determined. Among them, for each detection module, the predicted position information of the tracking target at the previous moment can be determined based on the first position information and first position offset information of the same tracking target.

[0128] The first position information can generally be recorded as B = (cx, cy, w, h), where cx and cy represent the center position information of the tracking target, which is the center coordinate of the tracking target, w represents the horizontal size (also called the length size) of the tracking target, and h represents the vertical size (also called the height size) of the tracking target. The first position offset information is generally the center coordinate offset, which can be recorded as D = (dx, dy). Based on the above first position information and first position offset information, the corresponding predicted position information Y = (cx-dx, cy-dy) can be determined.

[0129] Example 1,

[0130] The terminal is a car, and two detection modules are arranged on the car, and the detection modules are cameras. The detection cycles of the two cameras are the same, and the detection cycle is the duration of adjacent frames. The two cameras are respectively recorded as the first camera and the second camera. After starting the target tracking of the car, the first camera and the second camera are started at the same time to detect the images of the first area and the second area respectively based on the detection cycle. The first area is the preset detection area of ​​the first camera, and the second area is the preset detection area of ​​the second camera.

[0131] The detection image detected by the first camera at the current moment includes target images of two tracking targets, which are respectively recorded as target image M1 of one tracking target and target image M2 of the other tracking target. The detection image detected by the second camera at the current moment also includes target images of two tracking targets, which are respectively recorded as target image M3 of one tracking target and target image M4 of the other tracking target.

[0132] In this example, according to the first position information B corresponding to the target image M1 M1 and the first center coordinate offset D M1 , the predicted position information Y of the tracking target corresponding to the target image M1 at the previous moment can be determined M1 Similarly, the predicted position information Y of the tracking target corresponding to the target image M2 at the previous moment can be determined. M2 , the predicted position information Y of the tracking target corresponding to the target image M3 at the previous moment M3 And the predicted position information Y of the tracking target corresponding to the target image M4 at the previous moment M4 .

[0133] In step S220, the number of rows and columns of the cost matrix are equal to the number of predicted position information and the number of second position information, respectively, and the elements in the cost matrix are the distances between the positions represented by the corresponding predicted position information and the second position information. It should be noted that the distance between the predicted position information and the position represented by the second position information that do not belong to the same detection module is set to infinity, that is, the corresponding elements in the cost matrix are set to infinity.

[0134] Example 2,

[0135] This example 2 is configured the same as the above example 1. The entire predicted position information includes the predicted position information Y M1 , predicted location information Y M2 , predicted location information Y M3 and predicted position information Y M4 The entire second location information includes the second location information B N1 , second location information B N2 , second location information B N3 and second position information B N4 Among them, the predicted position information Y M1 , predicted location information Y M2 , second location information B N1 and second position information B N2 Corresponding to the first camera, predict the position information Y M3 , predicted location information Y M4 Target image M1, second position information B N3 and second position information BN4 Corresponding to the second camera.

[0136] The number of rows in the cost matrix is ​​the number of predicted position information, that is, the number of rows is 4, corresponding to the predicted position information Y M1 , predicted location information Y M2 , predicted location information Y M3 and predicted position information Y M4 The number of columns in the cost matrix is ​​the number of the second position information, that is, the number of columns is 4, corresponding to the second position information B N1 , second location information B N2 , second location information B N3 and second position information B N4 .

[0137] This cost matrix may include 16 elements, among which the predicted position information Y M1 and the second location information B N1 Both correspond to the first camera, so the predicted position information Y M1 The corresponding row and the second position information B N1 The element at the cross position of the corresponding column is, the predicted position information Y M1 The represented position and the second position information B N1 The distance between the represented locations, based on the predicted location information Y M1 and second position information B N1 Calculated. Predicted position information Y M1 and the second location information B N3 Corresponding to the first camera and the second camera respectively, therefore, the predicted position information Y M1 The corresponding row and the second position information B N3 The elements at the intersection of the corresponding columns are set to infinity.

[0138] The method for determining other elements can refer to the above method and will not be described in detail here.

[0139] In step S230, the cost matrix may be solved by using a Hungarian algorithm or a greedy algorithm to determine an optimal solution of the cost matrix.

[0140] Among them, the greedy algorithm (also known as the greedy algorithm) means that when solving a problem, it always makes the best choice at the moment. In other words, it does not consider the overall optimal solution, and the algorithm obtains a local optimal solution in a certain sense.

[0141] It should be noted that in the cost matrix, the distance between the predicted position information and the second position information representation position that do not belong to the same detection module is set to infinity, that is, the elements corresponding to the predicted position information and the second position information that do not belong to the same detection module in the cost matrix are set to infinity. Therefore, when determining the optimal solution of the cost matrix, the optimal solution will not point to the elements corresponding to the predicted position information and the second position information that do not belong to the same detection module, but will only point to the elements corresponding to the predicted position information and the second position information that belong to the same detection module.

[0142] Example 3,

[0143] The cost matrix consists of four elements, which are element S M1-N1 Element S M1-N3 Element S M3-N1 and element S M3-N3 Among them, the element S M1-N1 Represents the predicted position information Y M1 The represented position and the second position information B N1 The distance between the positions represented by the element S M1-N3 Represents the predicted position information Y M1 The represented position and the second position information B N3 The distance between the positions represented by the element S M3-N1 Represents the predicted position information Y M3 The represented position and the second position information B N1 The distance between the positions represented by the element S M3-N3 Represents the predicted position information Y M3 The represented position and the second position information B N3 The distance between the characterized locations.

[0144] Predicted location information Y M1 and second position information B N1 Corresponding to the first camera. Predicted position information Y M3 The represented position and the second position information B N3 Corresponding to the second camera. It can be seen that the element S M1-N3 and element S M3-N1 Is infinite.

[0145] In this example, when determining the optimal solution based on the greedy algorithm, due to the element S M1-N3 and element S M3-N1 is infinite, so the optimal solution will not point to element S M1-N3 and element S M3-N1 , can only point to element S M1-N1 and element S M3-N3 .

[0146] It should be noted that when calculating the optimal solution of the cost matrix based on the greedy algorithm, zero optimal solution may be determined, or one or more optimal solutions may be determined.

[0147] In step S240, after the optimal solution of the cost matrix is ​​determined, it can be considered that the predicted position information and the second position information corresponding to each optimal solution correspond to the same tracking target, that is, the first position information corresponding to the predicted position information and the second position information correspond to the same tracking target. Therefore, the first re-identification feature corresponding to the first position information and the historical feature corresponding to the second position information in the historical feature pool can be determined as a feature pair, which corresponds to the same tracking target.

[0148] It should be noted that the historical features in the historical feature pool are also re-identification features. The first batch of historical features can be obtained from the network or other terminals, or they can be re-identification features corresponding to all tracking targets detected in the first detection cycle, or they can be determined by other means, which are not limited here. After a set period of time, the historical feature pool can be updated based on the detected detection image. Alternatively, if the historical features of the network or other terminals are updated, the historical feature pool of this terminal is updated synchronously. Of course, the historical feature pool can also be updated in other ways, which are not limited here.

[0149] Among them, each historical feature corresponds to a tracking target. After the feature pair is determined according to the optimal solution, the tracking target corresponding to the historical feature in this feature pair is the same tracking target corresponding to the first recognition feature and the historical feature in the feature pair.

[0150] Example 4,

[0151] This example 4 has the same settings as the above example 3. It is assumed that the optimal solution determined in this example 4 is element S M1-N1 , then the predicted position information Y M1 The corresponding first re-identification feature and the second location information B N1 The corresponding historical features are re-identification features belonging to the same tracking target, and then the first re-identification feature and the historical features are determined as a feature pair, and the tracking target corresponding to the historical features of this feature pair is the tracking target corresponding to the feature pair.

[0152] In step S250, the historical features in each feature pair may be fused with the first re-recognition features to obtain a fused feature, which may be recorded as the first feature. The corresponding historical features and the first re-recognition features may be fused using a moving weighted average method to obtain the fused first feature. Of course, feature fusion may also be achieved in other ways, which are not limited here.

[0153] After determining the first feature corresponding to the feature pair, the trajectory identifier of the historical feature of this feature pair can be determined as the trajectory identifier of this first feature, and the first re-identification feature in this feature pair can be updated to the first feature, that is, the first feature is determined as the new first re-identification feature of the tracked target in the corresponding first detection image.

[0154] Example 5,

[0155] The setting of Example 5 is the same as that of Example 1. M1 , and the historical feature F in the historical feature pool 00 , determined as a feature pair. Historical feature F 00 The corresponding track identifier is trackid. The first identification feature F M1 The first feature after feature fusion with the historical feature F0 is F 10 .

[0156] In this example, the track ID can be identified as track id and the first feature is F 10 The trajectory is marked, and the first re-recognition feature F corresponding to the target image M1 is M1 Update the first feature F 10 That is, when the target is tracked later, the first re-recognition feature corresponding to the target image M1 is F 10 , whose track identifier is track id.

[0157] In this method, there is no need to use the Kalman filtering algorithm. Instead, the first position offset information is determined directly by setting a deep learning network model. The first recognition feature and the historical feature corresponding to the same tracking target can be accurately and efficiently determined. Then, the fused first feature is obtained based on the first recognition feature and the historical feature, and the first recognition feature is updated to the first feature. The trajectory identifier of the historical feature is determined as the trajectory identifier of the new first recognition feature (i.e., the first feature), thereby realizing the association between the tracking target detected at the current moment and the previous moment of the same detection module, realizing continuous and reliable tracking of the tracking target, and being able to achieve better multi-target tracking performance within the same detection module, providing conditions for real-time and online multi-target tracking across detection modules, and better satisfying the user experience.

[0158] It should be noted that when the historical features in the historical feature pool are determined based on detection images of multiple detection modules, when the detection image is acquired for the first time, there are no historical features in the historical feature pool, and there is no tracking target detected at the previous moment. Therefore, there is no need to execute the steps in this embodiment.

[0159] In an exemplary embodiment, a target tracking method is provided, which is applied to a terminal. In the method, after determining the trajectory identifier of the historical feature in the feature pair as the trajectory identifier of the fused feature, and updating the first re-recognition feature to the fused feature, the method may include:

[0160] S310, according to a hierarchical clustering algorithm, determining the first-recognition features corresponding to the same tracking target among all the first-recognition features as a feature group corresponding to the tracking target;

[0161] S320: configuring the same trajectory identifier for all first-recognition features of the same feature group.

[0162] In step S310, the basic idea of ​​the hierarchical clustering algorithm is to calculate the similarity between nodes by a certain similarity measure, sort them from high to low by similarity, and gradually reconnect the nodes. In this step, each first-recognition feature can be used as a node.

[0163] In this step, based on the hierarchical clustering algorithm, all first-recognition features corresponding to the same tracking target can be determined as a feature group to complete accurate clustering of the first-recognition features. It should be noted that each feature group can generally include more than one first-recognition feature.

[0164] Among them, when clustering is performed using a hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features may be determined as the feature group corresponding to the tracking target according to the set distance threshold and the Euclidean distance between any two first-recognition features. That is, according to the set distance threshold and the Euclidean distance between any two first-recognition features and the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features may be determined as the feature group corresponding to the tracking target.

[0165] In each layer of the hierarchical clustering algorithm, similarity is judged based on the Euclidean distance and the set distance threshold. Studies have shown that when determining the first-level identification features corresponding to the same tracking target, the clustering results based on the Euclidean distance are more effective and more accurate.

[0166] The set distance threshold may be set before the terminal leaves the factory, or may be set after the terminal leaves the factory, and the set distance threshold may be modified later to better meet different needs of users.

[0167] Example 1,

[0168] Set the distance threshold to D0.

[0169] There are three first-recognition features, namely, first-recognition feature F1, first-recognition feature F2, and first-recognition feature F3.

[0170] The Euclidean distance between the first-recognition feature F1 and the first-recognition feature F2 is recorded as D1, the Euclidean distance between the first-recognition feature F1 and the first-recognition feature F3 is recorded as F2, and the Euclidean distance between the second-recognition feature F2 and the third-recognition feature F3 is recorded as D3.

[0171] When determining whether the first re-identification feature F1 and the first re-identification feature F2 correspond to the same tracking target, the size of D1 and D0 can be determined. If D1 is smaller than D0, it is considered that the first re-identification feature F1 and the second re-identification feature F2 have a high similarity, and it is considered that the first re-identification feature F1 and the first re-identification feature F2 correspond to the same tracking target. Otherwise, it is considered that the first re-identification feature F1 and the first re-identification feature F2 do not correspond to the same tracking target.

[0172] The method of determining whether the first re-identification feature F1 and the first re-identification feature F3 correspond to the same tracking target, and the method of determining whether the second re-identification feature F2 and the first re-identification feature F3 correspond to the same tracking target are similar to the above-mentioned method of determining whether the first re-identification feature F1 and the first re-identification feature F2 correspond to the same tracking target, and are not described in detail here.

[0173] When it is necessary to determine whether the first re-identification feature F1, the first re-identification feature F2, and the first re-identification feature F3 correspond to the same tracking target, the average value of D1, D2, and D3 may be determined first, and then the size of the average value and D0 may be determined. If the average value is less than D0, it is considered that the first re-identification feature F1, the first re-identification feature F2, and the first re-identification feature F3 correspond to the same tracking target. Otherwise, it is considered that the first re-identification feature F1, the first re-identification feature F2, and the first re-identification feature F3 do not correspond to the same tracking target.

[0174] In this method, when the first-fold recognition features corresponding to the same tracking target are determined from all the first-fold recognition features through a hierarchical clustering algorithm, the similarity between the first-fold recognition features is judged based on the Euclidean distance, and the obtained clustering result is better, which can improve the accuracy of clustering, further improve the target tracking effect, and enhance the user experience.

[0175] It should be noted that this method can also determine the first-recognition features corresponding to the same tracking target in other ways, which are not limited here. For example, the first-recognition features corresponding to the same tracking target can be determined based on the cosine distance (or cosine similarity) of any two first-recognition features.

[0176] However, when determining the first-level identification features corresponding to the same tracking target, the clustering result based on the Euclidean distance is better and more accurate.

[0177] In step S420, the first re-identification feature of each feature group corresponds to the same tracking target, that is, the first re-identification feature detected by different detection modules included in each feature group. Therefore, the same trajectory identifier can be configured for each feature group, that is, the same trajectory identifier is configured for each first re-identification feature in the same feature group to achieve tracking of the tracking target. Among them, the first re-identification feature in the feature group can be the first re-identification feature determined based on the set deep learning network model, or it can be the updated first re-identification feature (i.e., the first feature).

[0178] Example 2,

[0179] Three feature groups are determined according to the hierarchical clustering algorithm, which are respectively recorded as the first feature group, the second feature group and the third feature group. The first feature group includes two first-recognition features of the first tracking target, the second feature group includes four first-recognition features of the second tracking target, and the third feature group includes three first-recognition features of the third tracking target.

[0180] In this example, the two first re-identification features of the first feature group can be configured with track identifier track id1, the four first re-identification features of the second feature group can be configured with track identifier track id2, and the three first re-identification features of the third feature group can be configured with track identifier track id3. Among them, track identifier track id1 is the track identifier of the first tracking target, track identifier track id2 is the track identifier of the second tracking target, and track identifier track id3 is the track identifier of the third tracking target. In this example, the above three tracking targets can be tracked through the above three track identifiers.

[0181] In this method, different detection modules may detect the target image of the same tracking target at the same time, thereby determining the re-identification features of the same tracking target. The first re-identification features corresponding to different detection modules can be clustered through a hierarchical clustering algorithm, and then the same trajectory identifier is configured for more than one first re-identification features corresponding to the same tracking target, thereby realizing the association of tracking targets detected by multiple detection modules at the current moment, so as to realize the tracking of multiple targets by multiple detection modules, further improving the user experience.

[0182] It should be noted that when the historical features in the historical feature pool are determined based on the detection images of multiple detection modules, when the detection image is acquired for the first time, there are no historical features in the historical feature pool, and there is no tracking target detected at the previous moment. Therefore, there is no need to perform the association between the tracking target detected by the same detection module at the current moment and the previous moment. Instead, the steps of this embodiment can be directly executed, that is, the hierarchical clustering algorithm can be directly used for clustering.

[0183] In an exemplary embodiment, a target tracking method is provided, which is applied to a terminal. In the method, configuring the same trajectory identifier for all first-recognition features of the same feature group may include:

[0184] S510, determining whether there is a first-level identification feature that has been configured with a trajectory identifier in the feature group; if the determination result is yes, executing step S520; otherwise, executing step S530;

[0185] S520, determining the trajectory identifier configured for the first-recognition feature as the trajectory identifier of all first-recognition features in the feature group;

[0186] S530: configure new trajectory identifiers for all first-recognition features of this feature group.

[0187] In this method, the first re-identification features in each feature group correspond to the same tracking target. Therefore, it is necessary to configure the same track identifier for all first re-identification features in the same feature group. However, the feature group may include an updated first re-identification feature, that is, a first feature determined based on a feature pair, which has been configured with a track identifier corresponding to the corresponding historical feature. Therefore, it is necessary to first determine whether there is a first re-identification feature with a track identifier configured in the feature group.

[0188] If there is a first-recognition feature in the feature group that has been configured with a trajectory identifier, that is, at least one first-recognition feature in the feature group is configured with a trajectory identifier, then this trajectory identifier can be determined as the trajectory identifier of all first-recognition features in this feature group, that is, this trajectory identifier can be determined as the trajectory identifier of the tracking target corresponding to this feature group, so that the same trajectory identifier is configured for the same tracking target to better achieve target tracking.

[0189] If there is no first-recognition feature with a configured trajectory identifier in the feature group, that is, no first-recognition feature in the feature group is configured with a trajectory identifier, a new trajectory identifier can be configured for all first-recognition features in this feature group, that is, a new trajectory identifier can be configured for the tracking target corresponding to this feature group, so that the same trajectory identifier is configured for the same tracking target to better achieve target tracking.

[0190] Example 1,

[0191] The terminal is a car, and two detection modules are arranged on the car, and the detection modules are cameras. The detection cycles of the two cameras are the same, and the detection cycle is the duration of adjacent frames. The two cameras are respectively recorded as the first camera and the second camera. After starting the target tracking of the car, the first camera and the second camera are started at the same time to detect the images of the first area and the second area respectively based on the detection cycle. The first area is the preset detection area of ​​the first camera, and the second area is the preset detection area of ​​the second camera.

[0192] The detection image detected by the first camera at the current moment includes target images of two tracking targets, namely, target image M1 of the first tracking target and target image M2 of the second tracking target. The detection image detected by the second camera at the current moment also includes target images of two tracking targets, namely, target image M3 of the first tracking target and target image M4 of the second tracking target.

[0193] Among them, the first recognition feature F corresponding to the target image M1 M1 , the first re-recognition feature F corresponding to the target image M2 M2 , the first re-recognition feature F corresponding to the target image M3 M3 , the first re-recognition feature F corresponding to the target image M4 M4 The historical feature pool includes historical feature F 00 , which corresponds to the first tracking target.

[0194] In this example, the first re-recognition feature F corresponding to the target image M1 M1 , and the historical feature F in the historical feature pool 00 , determined as a feature pair. Historical feature F 00 The corresponding track identifier is track id0. The first identification feature F M1 and historical characteristics F 00 The first feature after feature fusion is F 10 Then, the track can be identified as track id0 and the first feature can be determined as F 10 The trajectory is marked, and the first re-recognition feature F corresponding to the target image M1 is M1 Update the first feature F 10 That is, when the target is tracked later, the first re-recognition feature corresponding to the target image M1 is F 10 , whose track ID is trackid0.

[0195] In this example, when the hierarchical clustering algorithm is used for clustering, the first recognition feature F 10 and the first recognition feature F M3is determined as a feature group, recorded as the first feature group, corresponding to the first tracking target. M2 and the first recognition feature F M4 It is determined as a feature group, recorded as the second feature group, corresponding to the second tracking target.

[0196] Among them, the first re-identification feature F in the first feature group 10 Track ID track id0 has been configured, so track ID track id0 can be determined as the track ID of the first tracking target, that is, the first re-identification feature F in the first feature group. M3 The track identifier track id0 is configured. The two first-recognition features in the second feature group are not configured with track identifiers. Therefore, the first-recognition feature F in the second feature group can be M2 and the first recognition feature F M4 A new track identifier track id1 is configured, that is, the track identifier track id1 is determined as the track identifier of the second tracking target.

[0197] In this example, the first tracking target can be tracked based on the track identifier track id0, and the second tracking target can be tracked based on the track identifier track id1.

[0198] In this method, if the first-recognition features in the feature group are not configured with trajectory identifiers, it means that the terminal has not detected the tracking targets corresponding to these first-recognition features before. Therefore, a new trajectory identifier can be configured for this tracking target, that is, a new trajectory identifier can be configured for these first-recognition features to facilitate the subsequent tracking of this tracking target. If there is a first-recognition feature in the feature group that has been configured with a trajectory identifier based on the historical features of the tracking target, this trajectory identifier can still be used as the trajectory identifier of this tracking target, that is, this trajectory identifier is directly determined as the trajectory identifier corresponding to all the first-recognition features in this feature group, so as to facilitate the continuous tracking of the same tracking target. This method can realize the accurate tracking of multiple targets of multiple detection modules, further improving the user experience.

[0199] In an exemplary embodiment, a target tracking method is provided, which is applied to a terminal. In the method, after configuring the same trajectory identifier for all first-recognition features of the same feature group, the method may include:

[0200] S610, perform feature fusion processing on all first-recognition features in the same feature group to obtain a fused feature, and add the feature to the historical feature pool as a new historical feature of the tracking target corresponding to the feature group. At the same time, if the tracking target has an original historical feature in the historical feature pool, delete the original historical feature.

[0201] In this method, after completing the association between the tracking targets detected by the same detection module at the current moment and the previous moment, and the association between the tracking targets detected by different detection modules at the current moment, the historical feature pool can be updated based on all the first-level recognition features to facilitate subsequent continuous target tracking.

[0202] Among them, all the first re-identification features of the same feature group can be subjected to feature fusion processing, and the method of feature fusion can be a moving weighted fusion method. After the feature fusion of all the first re-identification features of the same feature group, the fused feature is obtained, which can be recorded as the second feature. Then, this second feature can be added to the historical feature pool as the re-identification feature of the tracking target corresponding to this feature group. At the next moment (i.e., the moment after the current moment is set for a certain period of time), this re-identification feature can participate in the process of the target tracking method as a historical feature in the historical feature pool.

[0203] In addition, if the tracking target corresponding to the second feature already has a historical feature in the historical feature pool (recorded as the original historical feature), the original historical feature can be deleted when the second feature is added to the historical feature pool to ensure that the same tracking target only corresponds to one re-identification feature in the historical feature pool, thereby better improving the reliability of target tracking.

[0204] In this method, the historical feature pool is continuously updated according to the first re-identification feature corresponding to the current moment, and it is ensured that the same tracking target in the historical feature pool corresponds to only one re-identification feature, which can better improve the reliability of target tracking and further enhance the user experience.

[0205] In an exemplary embodiment, a target tracking method is provided, which is applied to a terminal. In the method, after configuring the same trajectory identifier for all first-recognition features of the same feature group, the method may include:

[0206] S710, determining the similarity between each first-recognition feature and each historical feature from the first-recognition features of the undetermined trajectory identifier;

[0207] S720, determine whether the similarity is greater than a set similarity threshold; if the determination result is yes, execute step S730; otherwise, execute step S740;

[0208] S730, determining the trajectory identifier corresponding to the corresponding historical feature as the trajectory identifier corresponding to the corresponding first-level recognition feature;

[0209] S740: Create a new track identifier for the corresponding first-recognition feature.

[0210] Among the multiple detection modules, at the current moment, only one detection module may have a detection image including a target image of one or more tracking targets. The above tracking target may not have been detected at the previous moment, but there are historical features corresponding to the tracking target in the historical feature pool, for example, the tracking target was detected at an earlier moment than the previous moment.

[0211] In the above case, it is necessary to determine whether there is a first-recognition feature with a high similarity to the historical feature among the first-recognition features of the undetermined trajectory identifier.

[0212] Among them, the similarity between each first-recognition feature and each historical feature can be determined from the first-recognition features of the undetermined track identification, and then based on the similarity and the set similarity threshold, it is determined whether there is a first-recognition feature corresponding to the same tracking target as the historical feature.

[0213] If the similarity is greater than the set similarity threshold, the trajectory identifier corresponding to the corresponding historical feature is determined as the trajectory identifier of the first re-identification feature, so as to configure the same trajectory identifier for the same tracking target. If the similarity is less than or equal to the set similarity threshold, it means that the corresponding first re-identification feature corresponds to the newly detected tracking target, and a new trajectory identifier can be directly configured for the corresponding first re-identification feature to better ensure the reliability of target tracking.

[0214] The similarity threshold may be set before or after the terminal leaves the factory, and the similarity threshold may be modified later to better meet different needs of users.

[0215] Among them, after step S730, this method may include: performing feature fusion processing on this historical feature and this first-recognition feature to obtain a fused new feature (which can be recorded as a third feature), and updating this historical feature in the historical feature pool to this new feature.

[0216] That is, if the similarity is greater than the set similarity threshold, the trajectory identifier corresponding to the corresponding historical feature is determined as the trajectory identifier of the first-recognition feature. Then, this historical feature can be fused with the first-recognition feature to obtain a fused new feature (which can be recorded as the third feature), and this historical feature in the historical feature pool is updated to this new feature.

[0217] Among them, the feature fusion method can adopt the moving weighted average fusion method, or other methods, which will not be described in detail here. In this method, by using the third feature of the corresponding tracking target to replace the original historical feature, the re-identification feature of the same tracking target can be updated in time, so that its re-identification feature can better characterize the tracking target, further improving the reliability of the target tracking method.

[0218] After step S740 , the method may include: adding the first-recognition feature to the historical feature pool as a new historical feature.

[0219] That is, if the similarity is less than or equal to the set similarity threshold, it means that the corresponding first-recognition feature corresponds to the newly detected tracking target, and a new trajectory identifier can be directly configured for the corresponding first-recognition feature. Then, this first-recognition feature can be added to the historical feature pool as a new historical feature.

[0220] Among them, after a new tracking target is detected at the current moment, a new trajectory identifier is configured for this tracking target to identify the trajectory segment of this tracking target, and the corresponding first-level identification feature is added to the historical feature pool to supplement the historical features in the historical feature pool, which can facilitate subsequent target tracking and further improve the reliability of target tracking.

[0221] Example 1,

[0222] The terminal is a car, and three detection modules are arranged on the car, and the detection modules are cameras. The three cameras have the same detection cycle, and the detection cycle is the duration of adjacent frames. The three cameras are respectively recorded as the first camera, the second camera, and the third camera. After starting the target tracking of the car, the first camera, the second camera, and the third camera can be started at the same time to detect the images of the first area, the second area, and the second area respectively based on the detection cycle. The first area is the preset detection area of ​​the first camera, the second area is the preset detection area of ​​the second camera, and the third area is the preset detection area of ​​the third camera.

[0223] The detection image detected by the first camera at the current moment includes target images of two tracking targets, namely, target image M1 of the first tracking target and target image M2 of the second tracking target. The detection image detected by the second camera at the current moment includes target images of two tracking targets, namely, target image M3 of the first tracking target and target image M4 of the fourth tracking target. The detection image detected by the third camera at the current moment includes target images of three tracking targets, namely, target image M5 of the third tracking target, target image M6 of the fourth tracking target and target image M7 of the fifth tracking target.

[0224] Among them, the first recognition feature F corresponding to the target image M1 M1 , the first re-recognition feature F corresponding to the target image M2 M2 , the first re-recognition feature F corresponding to the target image M3 M3 , the first re-recognition feature F corresponding to the target image M4 M4 , the first re-recognition feature F corresponding to the target image M5M5 , the first re-recognition feature F corresponding to the target image M6 M6 , the first re-recognition feature F corresponding to the target image M7 M7 .

[0225] The first camera detected the first tracking target at the last moment, and the second camera detected the second tracking target at the last moment. The third tracking target was detected at an earlier moment than the last moment. The fourth tracking target and the fifth tracking target have never been detected.

[0226] The historical feature pool includes the historical feature F corresponding to the first tracking target 01 , corresponding to the historical feature F of the second tracking target 02 and the historical feature F corresponding to the third tracking target 03 Among them, the historical feature F 01 Corresponding track identifier trackid1, historical feature F 02 Corresponding track identifier track id2, historical feature F 03 The corresponding track identifier is track id3.

[0227] In this example, when associating the tracking target of the same camera at the current moment with that at the previous moment, the first recognition feature F M1 and historical feature F in the historical feature pool 01 , determined as a feature pair, recorded as the first feature pair; and the re-identification feature F M2 and historical feature F in the historical feature pool 02 , determined as a feature pair, recorded as the second feature pair.

[0228] Among them, the first recognition feature F M1 and historical characteristics F 00 The first feature after feature fusion is F 10 Then, the track can be identified as track id1 and the first feature can be determined as F 10 The trajectory is marked, and the first re-recognition feature F corresponding to the target image M1 is M1 Update the first feature F 10 That is, the first recognition feature corresponding to the target image M1 is F 10 , its track identifier is track id1, and the track identifier corresponding to the first tracking target is still track id1.

[0229] The first recognition feature F M2 and historical characteristics F 02 The first feature after feature fusion is F 20 Then, the track can be identified as track id2 and the first feature can be determined as F20 The trajectory is marked, and the first re-recognition feature F corresponding to the target image M2 is M2 Update the first feature F 20 That is, the first recognition feature corresponding to the target image M2 is F 20 , its track identifier is track id2, and the track identifier corresponding to the second tracking target is still track id2.

[0230] When different cameras detect the tracking target at the current moment, the first recognition feature F 10 and the first recognition feature F M3 is determined as a feature group, recorded as the first feature group, corresponding to the first tracking target. M4 and the first recognition feature F M6 It is determined as a feature group, recorded as the second feature group, corresponding to the fourth tracking target.

[0231] Among them, the first re-identification feature F in the first feature group 10 The track identifier track id1 has been configured, so the track identifier track id1 can be determined as the track identifier of the first tracking target, that is, the first re-identification feature F in the first feature group. M3 The track identifier track id1 is configured. The two first-recognition features in the second feature group are not configured with track identifiers. Therefore, the first-recognition feature F in the second feature group can be M4 and the first recognition feature F M6 A new track identifier track id4 is configured, that is, the track identifier track id4 is determined as the track identifier of the second tracking target.

[0232] Then, determine the first recognition feature F M5 The similarity with each historical feature and determine the first recognition feature F M7 Similarity to each historical feature.

[0233] Among them, the first recognition feature F can be determined M5 With historical characteristics F 03 If the similarity is greater than the set similarity threshold, the track ID3 can be determined as the first recognition feature F M5 The track identifier of the third tracking target is still track id3. And the first identification feature F can be determined M7 If the similarity with any historical feature is less than or equal to the set similarity threshold, it can be the first recognition feature F. M7A new track identifier track id5 is created. The track identifier track id5 is the track identifier corresponding to the fifth tracking target.

[0234] After determining the trajectory identifier corresponding to each first-recognition feature, the new historical features corresponding to each detected tracking target can be re-determined. The first-recognition feature F can be fused using the moving weighted average method. 10 and the first recognition feature F M3 , get the fused new feature, and use the new feature as the new historical feature F corresponding to the first tracking target 001 The first recognition feature can be F 20 Determine the new historical feature F corresponding to the second tracking target 002 The first recognition feature F can be fused using the moving weighted average method M5 With historical characteristics F 03 , get the fused new feature, and use the new feature as the new historical feature F corresponding to the third tracking target 003 The first recognition feature F can be fused using the moving weighted average method m4 and the first recognition feature F M6 , get the fused new feature, and use the new feature as the historical feature F corresponding to the fourth tracking target 004 The first identification feature F M7 Directly identified as the historical feature F of the fifth tracking target 005 .

[0235] Finally, the original historical features F in the historical feature pool can be 01 、Historical characteristics 02 and historical characteristics F 03 Delete, then add the historical feature F corresponding to the first tracking target 001 , the historical feature F corresponding to the second tracking target 002 , the historical feature F corresponding to the third tracking target 003 , the historical feature F corresponding to the fourth tracking target 004 The historical feature F corresponding to the fifth tracking target 005 .

[0236] At the next moment, target tracking is performed based on the above historical feature pool newly determined at the current moment, so as to reliably and in real time track multiple targets across cameras and improve the user experience.

[0237] In this method, a trajectory identifier can be configured for each first-recognition feature detected at the current moment, and the trajectory identifiers corresponding to the first-recognition features corresponding to the same tracking target are the same, so as to more accurately realize real-time tracking of multiple targets across cameras, further improving the user experience.

[0238] In an exemplary embodiment, a target tracking device is provided, which is applied to a terminal. The device is used to implement the above target tracking method, wherein reference Figure 2 As shown, the device includes an acquisition module 101 and a determination module 102. In the process of implementing the above method, the device

[0239] An acquisition module 101 is used to acquire a plurality of detection images after a set time period, wherein the plurality of detection images are detected by a plurality of detection modules, and the plurality of detection images correspond to the plurality of detection modules one by one;

[0240] The determination module 102 is used to input the first detection image at the current moment and the corresponding second detection image at the previous moment into a set deep learning network model to determine the position information, re-identification features and position offset information of the tracking target in the first detection image, wherein the duration between the current moment and the previous moment is equal to the set duration, and the position offset information represents the offset distance of the corresponding tracking target from the previous moment to the current moment;

[0241] It is also used to configure a trajectory identifier for each tracking target corresponding to all the first detection images at the current moment based on all the second position information corresponding to the previous moment, and all the first position information, all the first position offset information and all the first re-identification features corresponding to the current moment.

[0242] In an exemplary embodiment, a target tracking device is provided, which is applied to a terminal. Figure 2 As shown, in the device, the determination module 102 is used to:

[0243] Determine, according to the first position information and the first position offset information of the tracking target in the first detection image, the predicted position information of the tracking target detected by the target detection module at the last moment, wherein the target detection module is a detection module used to detect the first detection image;

[0244] Generate a cost matrix according to all predicted position information and all second position information; wherein the number of rows and the number of columns of the cost matrix are respectively equal to the number of predicted position information and the number of second position information, and the elements in the cost matrix are the distances between the positions represented by the corresponding predicted position information and the second position information;

[0245] Determine the optimal solution of the cost matrix;

[0246] According to each optimal solution, a feature pair is determined, where the feature pair includes a set of first-recognition features and historical features corresponding to the same tracking target;

[0247] The historical features and the first-recognition features in each feature pair are subjected to feature fusion processing to obtain fused features, and the trajectory identifier of the historical features in this feature pair is determined as the trajectory identifier of the fused features, and the first-recognition features are updated to the fused features.

[0248] In an exemplary embodiment, a target tracking device is provided, which is applied to a terminal. Figure 2 As shown, in the device, the determination module 102 is used to:

[0249] Use a greedy algorithm to determine the optimal solution for the cost matrix.

[0250] In an exemplary embodiment, a target tracking device is provided, which is applied to a terminal. In the device, a determination module is used to:

[0251] According to the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target;

[0252] The same track identifier is configured for all first-recognition features of the same feature group.

[0253] In an exemplary embodiment, a target tracking device is provided, which is applied to a terminal. Figure 2 As shown, in the device, the determination module 102 is used to:

[0254] According to the set distance threshold and the Euclidean distance between any two first-recognition features and the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target.

[0255] In an exemplary embodiment, a target tracking device is provided, which is applied to a terminal. Figure 2 As shown, in the device, the determination module 102 is used to:

[0256] If there is a first-recognition feature with a trajectory identifier configured in the feature group, then the trajectory identifier is determined as the trajectory identifier of all first-recognition features in the feature group; and / or,

[0257] If there is no first-recognition feature with a trajectory identifier configured in the feature group, a new trajectory identifier is configured for all first-recognition features of the feature group.

[0258] In an exemplary embodiment, a target tracking device is provided, which is applied to a terminal. Figure 2 As shown, the device may include an updating module 103, wherein the updating module 103 is used to:

[0259] All the first-level identification features in the same feature group are subjected to feature fusion processing to obtain weighted fused features, and this feature is added to the historical feature pool as the new historical feature of the tracking target corresponding to this feature group. At the same time, if the tracking target has an original historical feature in the historical feature pool, the original historical feature is deleted.

[0260] In an exemplary embodiment, a target tracking device is provided, which is applied to a terminal. Figure 2 As shown, in the device, the determination module 102 is used to:

[0261] Determine the similarity between each first-recognition feature and each historical feature from the first-recognition features of the undetermined trajectory identifier;

[0262] If the similarity is greater than a set similarity threshold, the trajectory identifier corresponding to the corresponding historical feature is determined as the trajectory identifier corresponding to the corresponding first-recognition feature; and / or, if the similarity is less than or equal to the set similarity threshold, a new trajectory identifier is created for the corresponding first-recognition feature.

[0263] In an exemplary embodiment, a target tracking device is provided, which is applied to a terminal. Figure 2 As shown, the device may include an updating module 103, wherein the updating module 103 is used to:

[0264] After determining the trajectory identifier corresponding to the corresponding historical feature as the trajectory identifier corresponding to the corresponding first-recognition feature, perform feature fusion processing on the historical feature and the first-recognition feature to obtain a fused new feature, and update the historical feature in the historical feature pool to the new feature;

[0265] and / or,

[0266] After creating a new trajectory identifier for the corresponding first-recognition feature, the first-recognition feature is added to the historical feature pool as a new historical feature.

[0267] In an exemplary embodiment, a terminal is provided, and the terminal may be a car, a server, a computer, etc.

[0268] refer to Figure 3 As shown, terminal 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0269] The processing component 402 generally controls the overall operation of the terminal 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0270] The memory 404 is configured to store various types of data to support operations at the terminal 400. Examples of such data include instructions for any application or method operating on the terminal 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 may be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0271] Power component 406 provides power to various components of terminal 400. Power component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to terminal 400.

[0272] The multimedia component 408 includes a screen that provides an output interface between the terminal 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera module and / or a rear camera module. When the terminal 400 is in an operating mode, such as a shooting mode or a video mode, the front camera module and / or the rear camera module may receive external multimedia data. Each front camera module and the rear camera module may be a fixed optical lens system or have a focal length and optical zoom capability.

[0273] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), and when the terminal 400 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 404 or sent via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0274] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0275] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the terminal 400. For example, the sensor assembly 414 can detect the open / closed state of the terminal 400, the relative positioning of the components, such as the display and keypad of the terminal 400, and the sensor assembly 414 can also detect the position change of the terminal 400 or a component of the terminal 400, the presence or absence of contact between the user and the terminal 400, the orientation or acceleration / deceleration of the terminal 400, and the temperature change of the terminal 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, this sensor assembly 414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0276] The communication component 416 is configured to facilitate wired or wireless communication between the terminal 400 and other terminals. The terminal 700 can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0277] In an exemplary embodiment, terminal 400 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing terminals (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned methods.

[0278] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, and the instructions can be executed by a processor 420 of a terminal 400 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage terminal, etc. When the instructions in the storage medium are executed by the processor of the terminal, the terminal is enabled to perform the method shown in the above embodiment.

[0279] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0280] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0281] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A target tracking method, applied to a terminal, characterized in that: The method comprises: After each set time period, a plurality of detection images are acquired, wherein the plurality of detection images are acquired by detection by a plurality of detection modules, and the plurality of detection images correspond one to one to the plurality of detection modules; Inputting a first detection image at a current moment and a corresponding second detection image at a previous moment into a set deep learning network model, determining position information, re-identification features, and position offset information of a tracking target in the first detection image, wherein the duration between the current moment and the previous moment is equal to the set duration, and the position offset information represents an offset distance of the corresponding tracking target from the previous moment to the current moment; According to all the second position information corresponding to the previous moment, and all the first position information, all the first position offset information, and all the first re-identification features corresponding to the current moment, a trajectory identifier is configured for each tracking target corresponding to all the first detection images at the current moment; The configuring a track identifier for each tracking target corresponding to all the first detection images at the current moment according to all the second position information corresponding to the previous moment, and all the first position information, all the first position offset information, and all the first re-identification features corresponding to the current moment, includes: Determining predicted position information of the tracked target detected by the target detection module at the previous moment according to a difference between first position information and first position offset information of the tracked target in the first detection image, wherein the target detection module is a detection module used to detect the first detection image; Generate a cost matrix according to all predicted position information and all second position information; wherein the number of rows and the number of columns of the cost matrix are respectively equal to the number of predicted position information and the number of second position information, and the elements in the cost matrix are the distances between the positions represented by the corresponding predicted position information and the second position information; Determining an optimal solution for the cost matrix; Determine a feature pair according to each of the optimal solutions, wherein the feature pair includes a set of first-recognition features and historical features corresponding to the same tracking target; The historical features and the first-recognition features in each feature pair are subjected to feature fusion processing to obtain fused features, and the trajectory identifier of the historical features in the feature pair is determined as the trajectory identifier of the fused features, and the first-recognition features are updated to the fused features.

2. The method according to claim 1, characterized in that The determining of the optimal solution of the cost matrix comprises: A greedy algorithm is used to determine the optimal solution of the cost matrix.

3. The method according to claim 1, characterized in that After determining the trajectory identifier of the historical feature in the feature pair as the trajectory identifier of the fused feature, and updating the first re-recognition feature to the fused feature, the method includes: According to the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target; The same track identifier is configured for all first-recognition features of the same feature group.

4. The method according to claim 3, characterized in that The step of determining, according to the hierarchical clustering algorithm, first-recognition features corresponding to the same tracking target from among all first-recognition features as a feature group corresponding to the tracking target comprises: According to the set distance threshold and the Euclidean distance between any two first-recognition features, and the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target.

5. The method according to claim 3, characterized in that: The configuring the same trajectory identifier for all first-recognition features of the same feature group includes: If there is a first-recognition feature with a trajectory identifier configured in the feature group, then the trajectory identifier is determined as the trajectory identifier of all first-recognition features in the feature group; and / or, If there is no first-recognition feature with a trajectory identifier configured in the feature group, a new trajectory identifier is configured for all first-recognition features of the feature group.

6. The method according to claim 3, characterized in that After configuring the same trajectory identifier for all first-recognition features of the same feature group, the method includes: All the first-level identification features in the same feature group are subjected to feature fusion processing to obtain weighted fused features, and this feature is added to the historical feature pool as the new historical feature of the tracking target corresponding to this feature group. At the same time, if the tracking target has an original historical feature in the historical feature pool, the original historical feature is deleted.

7. The method according to claim 3, characterized in that After configuring the same trajectory identifier for all first-recognition features of the same feature group, the method includes: Determine the similarity between each first-recognition feature and each historical feature from the first-recognition features of the undetermined trajectory identifier; If the similarity is greater than a set similarity threshold, the trajectory identifier corresponding to the corresponding historical feature is determined as the trajectory identifier corresponding to the corresponding first-recognition feature; and / or, if the similarity is less than or equal to the set similarity threshold, a new trajectory identifier is created for the corresponding first-recognition feature.

8. The method according to claim 7, characterized in that After determining the trajectory identifier corresponding to the corresponding historical feature as the trajectory identifier corresponding to the corresponding first-recognition feature, the method includes: performing feature fusion processing on the historical feature and the first-recognition feature to obtain a fused new feature, and updating the historical feature in the historical feature pool to the new feature; and / or, After creating a new trajectory identifier for the corresponding first re-recognition feature, the method includes: adding the first re-recognition feature to a historical feature pool as a new historical feature.

9. The method according to any one of claims 1 to 8, characterized in that: The setting of the deep learning network model is determined by the following method: Acquire multiple training sample pairs, each of which includes an input sample and an output sample, wherein the input sample includes a first sample image and a second sample image detected by the same detection module at the set detection interval, and the output sample is the position information, re-identification features, and position offset information of the tracked target in the first sample image, wherein the first sample image is an image detected at a later moment; The original neural network model is trained using the multiple training sample pairs to obtain the set deep learning network model.

10. A target tracking device, applied to a terminal, characterized in that: The device comprises: An acquisition module, used for acquiring a plurality of detection images after a set time period, wherein the plurality of detection images are detected by a plurality of detection modules, and the plurality of detection images correspond to the plurality of detection modules one by one; A determination module, used to input a first detection image at a current moment and a corresponding second detection image at a previous moment into a set deep learning network model, and determine the position information, re-identification features, and position offset information of the tracking target in the first detection image, wherein the duration between the current moment and the previous moment is equal to the set duration, and the position offset information represents the offset distance of the corresponding tracking target from the previous moment to the current moment; It is also used to configure a track identifier for each tracking target corresponding to all the first detection images at the current moment according to all the second position information corresponding to the previous moment, and all the first position information, all the first position offset information and all the first re-identification features corresponding to the current moment; The determining module is used to: Determining predicted position information of the tracked target detected by the target detection module at the previous moment according to a difference between first position information and first position offset information of the tracked target in the first detection image, wherein the target detection module is a detection module used to detect the first detection image; Generate a cost matrix according to all predicted position information and all second position information; wherein the number of rows and the number of columns of the cost matrix are respectively equal to the number of predicted position information and the number of second position information, and the elements in the cost matrix are the distances between the positions represented by the corresponding predicted position information and the second position information; Determining an optimal solution for the cost matrix; Determine a feature pair according to each of the optimal solutions, wherein the feature pair includes a set of first-recognition features and historical features corresponding to the same tracking target; The historical features and the first-recognition features in each feature pair are subjected to feature fusion processing to obtain fused features, and the trajectory identifier of the historical features in the feature pair is determined as the trajectory identifier of the fused features, and the first-recognition features are updated to the fused features.

11. The device according to claim 10, characterized in that The determining module is used to: A greedy algorithm is used to determine the optimal solution of the cost matrix.

12. The device according to claim 10, characterized in that The determining module is used to: According to the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target; The same track identifier is configured for all first-recognition features of the same feature group.

13. The device according to claim 12, characterized in that The determining module is used to: According to the set distance threshold and the Euclidean distance between any two first-recognition features, and the hierarchical clustering algorithm, the first-recognition features corresponding to the same tracking target among all the first-recognition features are determined as the feature group corresponding to the tracking target.

14. The device according to claim 12, characterized in that The determining module is used to: If there is a first-recognition feature with a trajectory identifier configured in the feature group, then the trajectory identifier is determined as the trajectory identifier of all first-recognition features in the feature group; and / or, If there is no first-recognition feature with a trajectory identifier configured in the feature group, a new trajectory identifier is configured for all first-recognition features of the feature group.

15. The device according to claim 12, characterized in that The device comprises an updating module, wherein the updating module is configured to: All the first-level identification features in the same feature group are subjected to feature fusion processing to obtain weighted fused features, and this feature is added to the historical feature pool as the new historical feature of the tracking target corresponding to this feature group. At the same time, if the tracking target has an original historical feature in the historical feature pool, the original historical feature is deleted.

16. The device according to claim 12, characterized in that The determining module is used to: Determine the similarity between each first-recognition feature and each historical feature from the first-recognition features of the undetermined trajectory identifier; If the similarity is greater than a set similarity threshold, the trajectory identifier corresponding to the corresponding historical feature is determined as the trajectory identifier corresponding to the corresponding first-recognition feature; and / or, if the similarity is less than or equal to the set similarity threshold, a new trajectory identifier is created for the corresponding first-recognition feature.

17. The device according to claim 16, characterized in that The device comprises an updating module, wherein the updating module is configured to: After determining the trajectory identifier corresponding to the corresponding historical feature as the trajectory identifier corresponding to the corresponding first-recognition feature, performing feature fusion processing on the historical feature and the first-recognition feature to obtain a fused new feature, and updating the historical feature in the historical feature pool to the new feature; and / or, After creating a new track mark for the corresponding first-recognition feature, the first-recognition feature is added to the historical feature pool as a new historical feature.

18. A terminal, characterized in that: The terminal comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the method according to any one of claims 1 to 9.

19. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the method according to any one of claims 1 to 9.

Images