Method and device for identifying track of target object in multiple videos

Abstract

The invention relates to the field of computer vision, in particular to a method and a device for identifying a track of a target object in multiple videos. The method mainly comprises the following steps: performing feature calculation on image frame data of all videos by using a feature extraction network to obtain a feature vector of each sample in the videos; calculating Euclidean distances of all the feature vectors, and calculating corresponding mutual neighbor distances according to the Euclidean distances; based on the mutual neighbor distance, performing clustering analysis by using at least two distance threshold parameters to obtain a cluster sample and an outlier sample; and training the feature extraction network based on the cluster sample and the outlier sample, and using the trained feature extraction network to identify the trajectory of the target object. According to the method, the sample similarity relationship can be better measured, and high-accuracy cross-domain re-identification is realized.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method and device for identifying target object trajectories in multiple videos. Background Art

[0002] In a camera cluster, the videos shot by different cameras have the characteristics of crossing time and space. In order to realize the trajectory tracking of different objects such as pedestrians, vehicles and ships across cameras, it is necessary to first extract features from the data (query) such as the image or video clip of the target object under a certain camera, and then find the trajectory of the target object in the data (gallery) of other cameras in the cluster based on the extracted features. The above method of tracking the trajectory of the same target object in multiple camera data is called re-identification technology.

[0003] In order to ensure that the feature extraction network has a high recognition accuracy in camera clusters in different scenarios, unsupervised video re-identification technology can be used. Instead of labeling the data, a large amount of data obtained from different camera clusters is directly used to guide the training of the feature extraction model. Existing unsupervised video re-identification technologies use graph matching, metric learning, or combined with camera pre-information to guide training, but the above methods all associate samples at the local level, making it difficult to correctly grasp the true association relationship between samples. The trained feature extraction model is applicable to small-scale data sets, but not to large-scale data sets or real scenes, and does not consider noise issues.

[0004] In view of this, how to overcome the defects of the existing technology and solve the problem that the existing video re-identification technology is not suitable for large-scale data sets is a problem to be solved in this technical field. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention solves the problem that the existing video re-identification technology is not suitable for large-scale data sets.

[0006] The embodiment of the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for identifying the trajectory of a target object in multiple videos, specifically: using a feature extraction network to perform feature calculations on image frame data of all videos to obtain a feature vector for each sample in the video; calculating the Euclidean distance of all feature vectors, and calculating the corresponding mutual neighbor distance based on the Euclidean distance; based on the mutual neighbor distance, using at least two distance threshold parameters to perform cluster analysis to obtain cluster samples and outlier samples; based on the cluster samples and outlier samples, training the feature extraction network, and using the trained feature extraction network to identify the trajectory of the target object.

[0008] Preferably, the calculation of the corresponding mutual neighbor distance based on the Euclidean distance specifically includes: taking each sample as the current sample in turn, and obtaining a mutual neighbor set of specified order according to the Euclidean distance; for each pair of mutual neighbor samples in the mutual neighbor set, obtaining a pair of mutual neighbor vectors between the mutual neighbor sample and the current sample, and obtaining the larger vector and the smaller vector in a pair of mutual neighbor vectors; and calculating the mutual neighbor distance of the pair of mutual neighbor samples based on the larger vector and the smaller vector of all samples.

[0009] Preferably, the cluster analysis is performed based on the mutual neighbor distance using at least two distance threshold parameters to obtain cluster samples and outlier samples, specifically including: increasing the distance threshold parameter from a first threshold to a second threshold, obtaining discrete samples on the second threshold based on the mutual neighbor distance, and calculating the corresponding isolation score; reducing the distance threshold parameter from the first threshold to a third threshold, obtaining discrete samples on the third threshold based on the mutual neighbor distance, and calculating the corresponding compactness score; and reducing βg to a value less than the isolation score and β j Discrete samples with a smaller compactness score are regarded as cluster samples, and other discrete samples are regarded as outlier samples.

[0010] Preferably, the calculation of the corresponding isolation score specifically includes:

[0011] The isolation score is calculated using:

[0012]

[0013] Among them, φ g (x) is the isolation score; x is a discrete sample; T m (x) is the discrete sample cluster when the first threshold is used as the distance threshold parameter; T h (x) is a discrete sample cluster when the second threshold is used as the distance threshold parameter; |·| represents the number of samples in the set.

[0014] Preferably, the calculating of the corresponding compactness score specifically includes:

[0015] The compactness score is calculated using:

[0016]

[0017] Among them, φ k (x) is the compactness score; x is a discrete sample; T m (x) is the discrete sample cluster when the first threshold is used as the distance threshold parameter; T l (x) is a discrete sample cluster when the third threshold is used as the distance threshold parameter; |·| represents the number of samples in the set.

[0018] Preferably, the discrete samples whose βg is smaller than the isolation score and βj is smaller than the compactness score are taken as cluster samples, specifically including:

[0019] The obtained cluster samples and outlier samples are expressed as follows:

[0020]

[0021] in, is the mutual neighbor distance between every two samples The matrix is ​​composed of N, the total number of samples; F level-clister Indicates that the samples are clustered according to the method provided by the present invention; Indicates clustering with the first threshold, Indicates clustering based on the second threshold. Indicates clustering with the third threshold; represents the pth cluster obtained by hierarchical clustering, represents the qth outlier sample obtained; p is the total number of clusters, and q is the total number of outlier samples.

[0022] Preferably, the training of the feature extraction network based on cluster samples and outlier samples specifically includes: taking the sample feature mean of the cluster samples as the proxy sample of the sample, and taking the feature vector of the outlier sample itself as the proxy sample of the sample; updating the feature vectors in all videos by means of sliding average, and updating the proxy samples according to the updated feature vectors; calculating the cluster contrast learning loss of the feature extraction network according to the updated proxy samples, and training the feature extraction network based on the cluster contrast learning loss.

[0023] Preferably, the step of calculating the cluster contrast learning loss of the feature extraction network according to the updated proxy samples specifically includes:

[0024] The cluster contrastive learning loss is calculated using:

[0025]

[0026] Among them, τ∈(0,1) is the scale parameter used to balance the probability distribution, N T is the number of samples, is the feature vector of the i-th proxy sample, is the feature vector of the jth proxy sample, is the feature vector of the kth proxy sample, |·| represents the length of the calculated feature vector, and exp represents the exponential function.

[0027] In a second aspect, the present invention provides a device for identifying the trajectory of a target object in multiple videos, specifically comprising: comprising at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for identifying the trajectory of a target object in multiple videos in the first aspect.

[0028] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method provided in the first aspect.

[0029] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory, and executing the method provided in the first aspect.

[0030] According to a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer or a processor, the computer or the processor executes the method provided in the first aspect.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows: in view of the shortcomings of the existing unsupervised video re-identification technology in feature learning, the present invention designs a cluster contrast learning method to guide the training of the feature extraction model; adopts the mutual neighbor distance instead of the Euclidean distance to better measure the sample similarity relationship; and adopts hierarchical clustering to optimize the cluster distribution to achieve high-accuracy cross-domain re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 A flow chart of a method for identifying target object trajectories in multiple videos provided by an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of the structure of a feature extraction model used in an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the structure of a non-local attention model in a feature extraction model used in an embodiment of the present invention;

[0036] Figure 4 A flow chart of another method for identifying target object trajectories in multiple videos provided by an embodiment of the present invention;

[0037] Figure 5 A flow chart of another method for identifying target object trajectories in multiple videos provided by an embodiment of the present invention;

[0038] Figure 6 A flow chart of another method for identifying target object trajectories in multiple videos provided by an embodiment of the present invention;

[0039] Figure 7 A schematic diagram of the structure of a device for identifying target object trajectories in multiple videos provided by an embodiment of the present invention;

[0040] The reference numerals are as follows:

[0041] 11: processor; 12: memory. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0043] The present invention is an architecture of a specific functional system, so the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

[0044] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0045] Embodiment 1:

[0046] In existing contrastive learning methods, local level sample association is usually used, that is, through instance contrastive learning, neighbor samples of anchor samples are found between different cameras, and the distance relationship between anchor samples, neighbor samples and non-neighbor samples is constrained. However, instance contrastive learning implements constraints between samples, and its constraint range only focuses on local areas, making it difficult to grasp data distribution from a global perspective; in addition, there may be conflicts between constraints between different anchor samples.

[0047] The original samples are regarded as discrete samples. Through cluster analysis, discrete samples with similar features can be divided into a set, which is called a discrete sample cluster (referred to as a cluster); if there are multiple discrete samples in a cluster, each discrete sample in the cluster is a cluster sample; if there is only one discrete sample in a cluster, this discrete sample is an outlier sample. In this embodiment, instance contrast learning is improved to cluster contrast learning. Cluster contrast learning can upgrade the samples participating in training from single instances to clusters, greatly improving the effectiveness of training, and constraining clusters and clusters, and cluster samples and outlier samples. Based on cluster contrast learning, a training algorithm for feature extraction network is proposed to achieve high-accuracy re-identification in cross-domain large-scale data scenarios, solving the problem that existing video re-identification technology is not suitable for large-scale data sets.

[0048] like Figure 1 As shown, the method for identifying target object trajectories in multiple videos provided by the embodiment of the present invention has the following specific steps:

[0049] Step 101: Use a feature extraction network to perform feature calculations on image frame data of all videos to obtain a feature vector for each sample in the video.

[0050] When performing video re-identification, the moving objects in the video are used as samples, and the samples include the target object and other objects. The feature extraction network is used to extract features from the video image frame data of multiple cameras to obtain the feature vector of the sample.

[0051] In actual implementation, a corresponding feature extraction network can be constructed according to the actual scenario needs. For example, a convolutional neural network embedded with a non-local attention module can be used as a feature extraction network. Figure 2 As shown in Figure 1, the convolutional neural network uses the ResNet50 network, whose deep structure enables it to learn complex features from data, thereby achieving high-precision image classification and target detection; among them, the non-local attention module embedded in the ResNet50 network, namely Figure 2 The Non-local module in the example is as follows: Figure 3 shown.

[0052] Step 102: Calculate the Euclidean distance of all feature vectors, and calculate the corresponding mutual neighbor distance based on the Euclidean distance.

[0053] In order to complete cluster contrast learning, it is first necessary to cluster the samples according to the distance between the sample feature vectors to obtain cluster samples and outlier samples after clustering.

[0054] Due to the complexity of multi-camera video data in spatial and temporal dimensions, the classic Euclidean distance cannot effectively measure the distance relationship between samples, which ultimately leads to a very chaotic cluster distribution of sample features. Therefore, in this embodiment, the mutual neighbor distance is used instead of the Euclidean distance to measure the distance between samples. The mutual neighbor distance is a criterion for distance measurement based on neighbor similarity, which reflects the degree of similarity between two samples by the degree of overlap of their common neighbors. In actual implementation, the Euclidean distance of all feature vectors can be calculated first, and the mutual neighbor distance can be calculated based on the Euclidean distance.

[0055] Step 103: Based on the mutual neighbor distance, at least two distance threshold parameters are used to perform cluster analysis to obtain cluster samples and outlier samples.

[0056] Unsupervised video re-identification technology does not have labels that can be used as guidance like supervised technology, but uses a clustering algorithm to obtain labels in unlabeled data, and the labels obtained by clustering convert unsupervised into supervised. Although the mutual neighbor distance can better measure the distance relationship between samples, making the clustering result more reliable, the traditional clustering algorithm still cannot effectively solve the problem of false positive samples mixed in the cluster. Therefore, a hierarchical clustering algorithm is used for clustering in this embodiment. Hierarchical clustering can use different clustering thresholds to cluster samples into clusters of different density levels, and then use the two indicators of cluster isolation and cluster tightness to eliminate unreliable samples distributed at the edge of the cluster, thereby obtaining cluster samples and outlier samples. In specific implementation, cluster analysis can be performed based on the mutual neighbor distance, and loose and tight clustering results can be obtained by changing the clustering threshold to obtain cluster samples and outlier samples.

[0057] Step 104: Train the feature extraction network based on the cluster samples and the outlier samples, and use the trained feature extraction network to identify the trajectory of the target object.

[0058] After the cluster analysis is completed, the feature extraction network can be trained using cluster contrast learning based on the clustering results. After each training is completed, the network parameters and feature parameters of the feature extraction network need to be updated according to the training results, and the process of step 102-step 104 is repeated until the required feature extraction accuracy is achieved. After the feature extraction network training is completed, the trained feature extraction network can be used to identify the target object trajectory in a large number of camera videos through the target object trajectory in a small number of cameras.

[0059] After steps 101 to 104 provided in this embodiment, the trajectory of the target object in the multi-camera video data can be identified, and the trajectory tracking of the target object in the camera cluster can be completed based on the identification results.

[0060] like Figure 4As shown, the corresponding mutual neighbor distance can be calculated according to the Euclidean distance in the following manner.

[0061] Step 201: taking each sample as the current sample in turn, and obtaining a mutual neighbor set of a specified order according to the Euclidean distance.

[0062] Assume that the dataset X = {x i |i=1,2,…,N}, where there are N samples, sample x i The calculation method of the k-order mutual neighbor set is as follows:

[0063] R(x i ,k)={x j |x i ∈C(x j ,k)∧x j ∈C(x i ,k)};(1)

[0064] Among them, R(x i ,k) represents the sample x i The k-order mutual neighbor set, C(x i ,k) represents the distance sample x i The set of the most recent k samples.

[0065] Step 202: For each pair of mutual neighbor samples in the mutual neighbor set, obtain a pair of mutual neighbor vectors between the mutual neighbor sample and the current sample, and obtain a larger vector and a smaller vector in the pair of mutual neighbor vectors.

[0066] Obviously, with C(x i ,k) samples, R(x i ,k) The samples in the set and the sample x i However, in actual scenarios, due to factors such as camera lighting and camera parameters, the differences between samples are too large, so that some true positive samples are not included in R(x i ,k). To do this, we need to i ,k) Relax the restrictions and set x i The nearest neighbor of the neighbor also includes the mutual neighbor set R(x i ,k). The loose k-order mutual neighbor set is calculated as shown in formula (2):

[0067]

[0068] in, Represents sample x i A loose k-order mutual neighbor set, R(x i ,k) represents the sample x i The k-order mutual neighbor set of Represents sample x j of The set of neighbors of order . Sample x j is x i k-order mutual neighbors, that is, x j ∈R(x i ,k), and satisfies The number of samples in the set is greater than R(x i ,k)γ times the number of samples in the set to ensure that x j To improve the reliability of the proposed algorithm and prevent the introduction of too many false positive samples, the larger the γ, the stricter the constraint.

[0069] Therefore, the sample x i and sample x j The calculation method of the mutual neighbor distance between the feature vectors is as follows:

[0070]

[0071] in, Represents sample x i and sample x j The mutual neighbor distance between the feature vectors of , |·| represents the number of samples in the set, Represents sample x i Relaxed set of k-order mutual neighbors.

[0072] Considering the convenience of intersection and union calculation, and the difference in contribution between neighbor samples of different rankings, in this embodiment, the calculation method of k-order neighbor vector is improved, using V = {v i ∈R N |i=1,2,…,N} represents the set of improved mutual neighbor vectors. j is the sample x i k-order mutual neighbor samples, and without considering the difference in neighbor contribution, there is v ij = 1. After considering the differences of neighbor samples with different rankings, v ij The calculation method of is as follows:

[0073]

[0074] Among them, v ij Represents sample x i and sample x j The mutual neighbor vector after considering the neighbor distance, d ij Represents sample x i and sample x j The Euclidean distance of , where e is the natural base.

[0075] Using formula (4), the original hard mutual neighbor vector can be converted into a soft vector, so that the neighbor samples with closer Euclidean distance account for a larger proportion in the mutual neighbor samples. By using the mutual neighbor vector, the calculation of intersection and union can be converted into the problem of summing the maximum and minimum values.

[0077] x t The mutual neighbor vector after considering the neighbor distance, min means taking a smaller value, and max means taking a larger value.

[0078] Step 203: Calculate the mutual neighbor distance of the pair of mutual neighbor samples according to the larger vector and the smaller vector of all samples.

[0079] Substituting formula (5) and formula (6) into formula (3), we can obtain that the mutual neighbor distance is calculated as formula (7):

[0080]

[0081] in, Represents sample x i and sample x j The mutual neighbor distance between the feature vectors of .

[0082] After steps 201 to 203 provided in this embodiment, the mutual neighbor distances between the sample feature vectors can be obtained.

[0083] In general scenarios, the discrete sample representation method obtained based on the mutual neighbor distance is as follows:

[0084]

[0085] in, is the mutual neighbor distance between every two samples The matrix is ​​composed of, N is the total number of samples; Indicates the clustering threshold ρ k Perform clustering; Indicates that the clustering threshold ρ k The i-th discrete sample cluster obtained by the above clustering, m is the total number of discrete sample clusters; Indicates that the clustering threshold ρ k The jth discrete sample obtained by clustering above, n is the total number of discrete samples.

[0086] In actual implementation, each cluster represents a label of a sample. A reliable cluster should be far away from other clusters and discrete outlier samples so that samples with different labels can be accurately distinguished; at the same time, the samples within each reliable cluster should be distributed very densely to reflect that the feature vectors of the samples have a high similarity. However, in the above clustering calculation process, simply relying on the distance between cluster centers, or the distance between cluster centers and outlier samples, it is impossible to effectively judge the isolation of clusters. Ideally, the distance between reliable samples is very small, and slight parameter changes will not affect cluster division. However, unsupervised video re-identification technology lacks labels as guidance, and some samples in the extracted feature distribution may be distributed only at the edge of several clusters. Slight changes in parameters will cause these samples to be incorporated into or excluded from a certain cluster. Therefore, in this embodiment, based on the sensitivity of the clustering algorithm to the distance threshold parameter, hierarchical clustering is used to improve the accuracy of cluster analysis.

[0087] In this embodiment, a variable distance threshold parameter ρ is used to represent the maximum distance between any two samples that can be incorporated into the same cluster. When the ρ value is large, fewer clusters will be generated, but each cluster contains more samples; when the ρ value is small, more clusters will be generated, but each cluster contains fewer samples. In specific implementations, three different distance thresholds of low, medium and high can be used: the first threshold ρ in the middle is m , a larger second threshold ρ h , a smaller third threshold ρ l , to cluster all samples into clusters of different levels, and screen out reliable clusters and unreliable samples based on cluster isolation and cluster compactness as indicators.

[0088] like Figure 5 As shown, cluster analysis can be performed using different distance threshold parameters in the following way to obtain cluster samples and outlier samples.

[0089] Step 301: Increase the distance threshold parameter from a first threshold to a second threshold, obtain discrete samples above the second threshold based on the mutual neighbor distance, and calculate the corresponding isolation score.

[0090] First, at the first threshold ρ m The clustering result is obtained under m Slightly increase to the second threshold ρ h , then according to the first threshold ρ mThe obtained cluster will expand outward to absorb more samples. In extreme cases, several clusters that are close to each other will merge into one. If this expansion causes a cluster to add too many samples, it means that the isolation of the cluster is poor and the isolation score of the cluster samples is also low. The isolation score is calculated as shown in formula (9):

[0091]

[0092] Among them, φ g (x) is the isolation score; x is a discrete sample; T(x) represents the set of discrete samples in the discrete sample cluster except for the discrete sample x, T m (x) is the discrete sample cluster when the first threshold is used as the distance threshold parameter; T h (x) is a discrete sample cluster when the second threshold is used as the distance threshold parameter; |·| represents the number of samples in the set.

[0093] Step 302: reduce the distance threshold parameter from the first threshold to the third threshold, obtain discrete samples on the third threshold based on the mutual neighbor distance, and calculate the corresponding compactness score.

[0094] First, at the first threshold ρ m The clustering result is obtained under m Slightly narrowed to the third threshold ρ l , then according to the first threshold ρ m The obtained cluster will shrink towards the center of the cluster, and some edge samples will be removed from the cluster. In extreme cases, a larger cluster will split into several small clusters. If this contraction causes a cluster to split into very scattered clusters, it means that the compactness of the cluster is poor, and the compactness score of the cluster samples is also low. The compactness score is calculated as shown in formula (10):

[0095]

[0096] Among them, φ j (x) is the compactness score; x is a discrete sample; T(x) represents the set of discrete samples in the discrete sample cluster except for the discrete sample x, T m (x) is the discrete sample cluster when the first threshold is used as the distance threshold parameter, T l (x) is a discrete sample cluster when the third threshold is used as the distance threshold parameter, and |·| represents the number of samples in the set.

[0097] Step 303: β g Less than the isolation score and β j Discrete samples with a smaller compactness score are regarded as cluster samples, and other discrete samples are regarded as outlier samples.

[0098] Finally, satisfying φ g (x)>β g And φ j (x)>β j The sample x is a reliable sample in each cluster, and the label of the corresponding cluster can be extracted through its feature vector, while other samples that do not meet the conditions will be removed from the original cluster and exist as outlier samples.

[0099] In the specific implementation, the obtained cluster samples and outlier samples are expressed as shown in formula (11):

[0100]

[0101] in, is the mutual neighbor distance between every two samples The matrix is ​​composed of N, the total number of samples; F level-clister Indicates that the samples are clustered according to the method provided in this embodiment; Indicates clustering with the first threshold, Indicates clustering based on the second threshold. Indicates clustering with the third threshold; represents the pth cluster obtained by hierarchical clustering, and the discrete samples contained in each cluster are the cluster samples in this cluster; represents the qth outlier sample obtained; p is the total number of clusters, and q is the total number of outlier samples.

[0102] After steps 301 to 303 provided in this embodiment, cluster samples and outlier samples can be obtained through hierarchical clustering.

[0103] After the above cluster analysis, we can get clusters and discrete samples After integration, the cluster samples can be expressed as: {(X1,y1),(X2,y2),…,(X p ,y p )}, the outlier sample can be expressed as: {(x o+1 ,y p+1 ),(x p+2 ,y o+2 ),…,(x p+q ,u p+q )}. Where X represents the video set obtained by all cameras, x represents a single video sequence, y represents the corresponding cluster pseudo label; p represents the number of clusters obtained by clustering, and q represents the number of outlier samples.

[0104] like Figure 6 As shown, the feature extraction network can be trained based on cluster samples and outlier samples in the following manner.

[0105] Step 401: taking the sample feature mean of the cluster sample as the proxy sample of the sample, and taking the feature vector of the outlier sample itself as the proxy sample of the sample.

[0106] For cluster samples, the proxy sample c is the mean of the feature vectors of all cluster samples in a cluster, while for outlier samples, the proxy sample r is its own feature vector. The proxy sample is calculated as shown in formula (12) and formula (13):

[0107]

[0108] r k =G(x k )k=p+1,p+2,…p+q;(13)

[0109] in, represents the number of samples in the set, G(x) represents the feature vector of video x, and G is the feature extractor.

[0110] In actual implementation, the obtained proxy samples can use a memory storage module of the feature extraction network to store features. The size of the storage module is d×(p+q), and each column stores the features of the proxy samples of the corresponding cluster.

[0111] Step 402: The feature vectors in all videos are updated by using a sliding average method, and the proxy samples are updated according to the updated feature vectors.

[0112] All video features are updated using a sliding average method. After each feature update, the features of the proxy sample are updated according to formulas (12) and (13).

[0113] Step 403: Calculate the cluster contrast learning loss of the feature extraction network according to the updated proxy samples, and train the feature extraction network based on the cluster contrast learning loss.

[0114] Cluster contrastive learning focuses on the similarity relationship between the current anchor sample and the proxy sample, constraining the anchor sample to be close to the proxy sample corresponding to the category to which it belongs, and away from proxy samples of other categories. From the perspective of training efficiency, instance contrastive learning needs to compare the feature relationship between the anchor sample and N samples, while cluster contrastive learning only needs to compare the feature relationship between the anchor sample and p+q proxy samples. The loss constraint rises from the local level to the global level, greatly improving the effectiveness of training.

[0115] In actual implementation, the cluster contrast learning loss as shown in formula (14) can be used to guide the training of the feature extraction model:

[0116]

[0117] Among them, τ∈(0,1) is the scale parameter used to balance the probability distribution, N T is the number of samples, is the feature vector of the i-th proxy sample, is the feature vector of the jth proxy sample, is the feature vector of the kth proxy sample, |·| represents the length of the calculated feature vector, and exp represents the exponential function.

[0118] After steps 401 to 403 provided in this embodiment, the training of the feature extraction model can be completed.

[0119] The method for identifying target object trajectories in multiple videos provided in this embodiment proposes a training algorithm based on cluster contrastive learning for the field of unsupervised re-identification. Compared with the existing instance-based training algorithm, it can make more full use of the potential correlation between data structures, give play to the advantages of clusters, extract feature vectors with high discrimination, and achieve recognition accuracy far better than existing methods.

[0120] Embodiment 2:

[0121] Based on the method for identifying target object trajectories in multiple videos provided in the above embodiment 1, the present invention also provides a device for identifying target object trajectories in multiple videos that can be used to implement the above method, such as Figure 7 , which is a schematic diagram of the device architecture of an embodiment of the present invention. The device for identifying target object trajectories in multiple videos of this embodiment includes one or more processors 11 and a memory 12. Figure 7 A processor 11 is taken as an example.

[0122] The processor 11 and the memory 12 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0123] The memory 12 is a non-volatile computer-readable storage medium as a method for identifying the trajectory of target objects in multiple videos, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the method for identifying the trajectory of target objects in multiple videos in Example 1. The processor 11 executes various functional applications and data processing of the device for identifying the trajectory of target objects in multiple videos by running the non-volatile software programs, instructions and modules stored in the memory 12, that is, implementing the method for identifying the trajectory of target objects in multiple videos in Example 1.

[0124] The memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 12 may optionally include a memory remotely arranged relative to the processor 11, and these remote memories may be connected to the processor 11 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0125] The program instructions / modules are stored in the memory 12, and when executed by one or more processors 11, the method for identifying the trajectory of a target object in multiple videos in the above embodiment 1 is executed, for example, the above described Figure 1 , Figure 4 , Figure 5 and Figure 6 The steps shown.

[0126] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, such as Figure 7 A processor 11 in the embodiment may enable the one or more processors to execute the method for identifying target object trajectories in multiple videos in embodiment 1, for example, executing the method described above. Figure 1 , Figures 4 to 6 The steps shown can also be implemented Figure 2 , Figure 3 The various modules and units described.

[0127] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0128] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying target object trajectories in multiple videos, characterized in that: include: Use the feature extraction network to perform feature calculations on the image frame data of all videos to obtain the feature vector of each sample in the video; Calculate the Euclidean distance of all feature vectors, and calculate the corresponding mutual neighbor distance based on the Euclidean distance; Based on the mutual neighbor distance, cluster analysis is performed using at least two distance threshold parameters to obtain cluster samples and outlier samples; The feature extraction network is trained based on cluster samples and outlier samples, and the trajectory of the target object is identified using the trained feature extraction network.

2. The method for identifying target object trajectories in multiple videos provided in claim 1, characterized in that: The calculating of the corresponding mutual neighbor distance according to the Euclidean distance specifically includes: Take each sample as the current sample in turn, and obtain the mutual neighbor set of the specified order according to the Euclidean distance; For each pair of mutual neighbor samples in the mutual neighbor set, obtain a pair of mutual neighbor vectors between the mutual neighbor sample and the current sample, and obtain a larger vector and a smaller vector in the pair of mutual neighbor vectors; According to the larger vector and smaller vector of all samples, the mutual neighbor distance of the pair of mutual neighbor samples is calculated.

3. The method for identifying target object trajectories in multiple videos provided in claim 1, characterized in that: The cluster analysis is performed based on the mutual neighbor distance using at least two distance threshold parameters to obtain cluster samples and outlier samples, specifically including: Increasing the distance threshold parameter from a first threshold to a second threshold, obtaining discrete samples on the second threshold based on the mutual neighbor distance, and calculating the corresponding isolation score; reducing the distance threshold parameter from the first threshold to the third threshold, obtaining discrete samples on the third threshold based on the mutual neighbor distance, and calculating the corresponding compactness score; If βg is less than the isolation score and β j Discrete samples with a smaller compactness score are regarded as cluster samples, and other discrete samples are regarded as outlier samples.

4. The method for identifying target object trajectories in multiple videos provided in claim 3 is characterized in that: The calculation of the corresponding isolation score specifically includes: The isolation score is calculated using: Among them, φ g (x) is the isolation score; x is a discrete sample; T m (x) is the discrete sample cluster when the first threshold is used as the distance threshold parameter; T h (x) is a discrete sample cluster when the second threshold is used as the distance threshold parameter; |·| represents the number of samples in the set.

5. The method for identifying target object trajectories in multiple videos provided in claim 3 is characterized in that: The calculation of the corresponding tightness score specifically includes: The compactness score is calculated using: Among them, φ j (x) is the compactness score; x is a discrete sample; T m (x) is the discrete sample cluster when the first threshold is used as the distance threshold parameter; T l (x) is a discrete sample cluster when the third threshold is used as the distance threshold parameter; |·| represents the number of samples in the set.

6. The method for identifying target object trajectories in multiple videos provided in claim 3, characterized in that: The discrete samples whose βg is smaller than the isolation score and βj is smaller than the compactness score are taken as cluster samples, specifically including: The obtained cluster samples and outlier samples are expressed as follows: in, is the mutual neighbor distance between every two samples The matrix is ​​composed of N, the total number of samples; F level-cluster Indicates that the samples are clustered according to the method provided in claim 3; Indicates clustering with the first threshold, Indicates clustering based on the second threshold, Indicates clustering with the third threshold; represents the pth cluster obtained by hierarchical clustering, represents the qth outlier sample obtained; p is the total number of clusters, and q is the total number of outlier samples.

7. The method for identifying target object trajectories in multiple videos provided by claim 1, characterized in that: The training of the feature extraction network based on cluster samples and outlier samples specifically includes: The sample feature mean of the cluster sample is used as the proxy sample of the sample, and the feature vector of the outlier sample itself is used as the proxy sample of the sample; The feature vectors in all videos are updated using a sliding average method, and the proxy samples are updated based on the updated feature vectors; The cluster contrastive learning loss of the feature extraction network is calculated according to the updated proxy samples, and the feature extraction network is trained based on the cluster contrastive learning loss.

8. The method for identifying target object trajectories in multiple videos provided by claim 7, characterized in that: The calculation of the cluster contrast learning loss of the feature extraction network according to the updated proxy samples specifically includes: The cluster contrastive learning loss is calculated using: Among them, τ∈(0,1) is the scale parameter used to balance the probability distribution, N T is the number of samples, f i t is the feature vector of the i-th proxy sample, f j t is the feature vector of the jth proxy sample, is the feature vector of the kth proxy sample, |·| represents the length of the calculated feature vector, and exp represents the exponential function.

9. A device for identifying target object trajectories in multiple videos, characterized in that: The method comprises at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the method for target object trajectory recognition in multiple videos according to any one of claims 1 to 8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the method for identifying target object trajectories in multiple videos as described in any one of claims 1-8.

Images