Target re-identification method and device based on corrected ranking list

Abstract

The present invention relates to the field of pattern recognition technology, and in particular to a target re-identification method and device based on a correction ranking list, aiming to solve the problem of low accuracy of target re-identification in existing target re-identification methods. The target re-identification method and device based on the correction ranking list replaces the features of gallery set samples with cluster center vectors, considers the distance between the cluster center and the query set to correct the ranking, proposes a distribution ranking list and a correction ranking list, and the correction ranking list can be obtained through the distribution ranking list and the original ranking list. The correction ranking list replaces the original ranking list as the final target re-identification basis, thereby improving the accuracy of target re-identification; in addition, the distribution ranking list and the correction ranking list proposed by the present invention are suitable for large data sets, do not require manual intervention or labeled data, and the method is plug-and-play for previous backbone networks and is suitable for all re-identification tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of pattern recognition, and in particular to a target re-identification method and device based on a corrected ranking list. Background Art

[0002] Object re-identification is a technique that uses visual processing methods to search for specified objects in images and videos. Generally speaking, object re-identification can be considered a retrieval problem. Currently, object re-identification plays an important role in computer vision applications, especially in video surveillance. Due to the differences in cameras, as well as the effects of angles, wear and occlusion, object re-identification has become a challenging topic in computer vision.

[0003] At present, the main object re-identification methods are as follows: (1) object re-identification methods based on representation learning; (2) object re-identification methods based on metrics. Object re-identification methods usually include three stages: feature extraction, distance measurement, and final ranking list. First, the feature extraction part currently mainly uses the ResNet-50 network and its improved deep networks, such as multi-branch networks and attention networks. Among them, the multi-branch network divides an image into multiple perspectives, which can better distinguish different objects under different cameras; the attention network introduces an attention mechanism in the framework of object re-identification to focus on the object area. Usually, researchers use some techniques to improve performance, such as warm-up learning rate, center loss, and label smoothing. Second, the distance metric seeks the consistency of the identity of the target in the image. It is usually measured between the probe and the gallery set by Euclidean distance and Mahalanobis distance. Third, after obtaining the initial ranking list, the final ranking explores the similarity relationship between the probe sample and its top k samples in the gallery set to receive a higher ranking. Since the ranking list does not require manual intervention or labeled data and has achieved significant performance improvements, more and more researchers are interested in re-ranking research. However, the accuracy of existing object re-identification methods is low. Summary of the invention

[0004] The present invention provides a target re-identification method and device based on a correction ranking list, aiming to solve the problem that the existing target re-identification method has low accuracy in target re-identification.

[0005] In order to achieve the above object, in a first aspect, the present invention provides a method for target re-identification based on a corrected ranking list, the method comprising:

[0006] The features of gallery set G and query set Q are extracted through the trained backbone network f(·), where gallery set G = {g i|i=1,2,...,N},query set Q={q j |j=1,2,...,M}, where N is the number of images in the gallery set; M is the number of images in the query set; i and j are subscript indices, each q represents a probe image, and the probe feature of each probe image is f q =f(q), each g represents an image in the gallery set, and the gallery feature of each image in the gallery set is f g =f(g).

[0007] Construct the original ranking list L(q,G) and obtain the probe feature f through the Euclidean distance after feature L2 normalization q and gallery features f g The original distance between q and f g The original distance between two features is:

[0008]

[0009] Among them, z represents the index of the feature; D represents the dimension of the feature;

[0010] Among them, the original ranking list is:

[0011] L(q,G)={g i |d(q,g i )∈d1,d2,...,d k}

[0012] Among them, k represents the k nearest neighbors of q, d(q,g i )<d(q,g i+1 );

[0013] By replacing the features of each gallery set sample with the cluster center vector, the two closest clusters are merged in each iteration, and the distance between the cluster centers is recalculated by the average linkage method, where the distance between the cluster centers is:

[0014]

[0015] Among them, C represents a cluster center; x and y represent C i and C j The feature points in m i and m j Respectively represent C i and C j The size of , dist(·,·) represents the distance function.

[0016] The final clustering result is assigned as a pseudo label to each gallery image, and the same pseudo labels are summed and averaged to obtain the cluster center features:

[0017]

[0018] Among them, c i represents the i-th central feature; N i Indicates that the gallery is concentrated with c i The number of identical labels; y represents the labels of c and gallery image g;

[0019] Construct a distribution ranking list and calculate the distance between the cluster center and the query set to obtain:

[0020] d c (f q ,f c )=||f q -f c ||2

[0021] Among them, f c ={c i |c=1,2,...,i} represents the cluster center feature vector, which is used to store the features in the gallery set; ||·||2 represents the L2 norm; d c (·,·) represents the distance between the cluster center feature and the probe feature;

[0022] By combining the original distance and the distance between the cluster center feature and the query set, the detection samples are moved closer to the corresponding cluster center to obtain the corrected ranking list:

[0023] d * (f q ,f g ,f c )=d(f q ,f g )+λd c (f q ,f c )

[0024] Among them, λ represents an adjustable coefficient, which is used to balance the original ranking list and the distribution ranking list to form a corrected ranking list.

[0025] In the above-mentioned target re-identification method based on the corrected ranking list, optionally, the backbone network uses a ResNet-50 network or an improved network thereof.

[0026] In a second aspect, the present invention provides a target re-identification device based on a corrected ranking list, the device comprising:

[0027] The feature extraction module is used to extract the features of the gallery set G and the query set Q through the trained backbone network f(·), where the gallery set G = {g i |i=1,2,...,N},query set Q={q j |j=1,2,...,M}, where N is the number of images in the gallery set; M is the number of images in the query set; i and j are subscript indices, each q represents a probe image, and the probe feature of each probe image is f q =f(q), each g represents an image in the gallery set, and the gallery feature of each image in the gallery set is f g =f(g);

[0028] The original ranking list construction module is used to construct the original ranking list L(q,G) and obtain the probe feature f through the Euclidean distance after feature L2 normalization. q and gallery features f g The original distance between q and f g The original distance between two features is:

[0029]

[0030] Among them, z represents the index of the feature; D represents the dimension of the feature;

[0031] Among them, the original ranking list is:

[0032] L(q,G)={g i |d(q,g i )∈d1,d2,...,d k}

[0033] Among them, k represents the k nearest neighbors of q, d(q,g i )<d(q,g i+1 );

[0034] The cluster center distance measurement module is used to replace the features of each gallery set sample with the cluster center vector. In each iteration, the two closest clusters are merged and the distance between cluster centers is recalculated by the average link method. The distance between cluster centers is:

[0035]

[0036] Among them, C represents a cluster center; x and y represent C i and C j The feature points in m i and m j Respectively represent C i and C jThe size of , dist(·,·) represents the distance function.

[0037] The cluster center feature acquisition module is used to assign the final clustering result as a pseudo label to each gallery image, and sum and average the same pseudo labels to obtain the cluster center feature:

[0038]

[0039] Among them, c i represents the i-th central feature; N i Indicates that the gallery is concentrated with c i The number of identical labels; y represents the labels of c and gallery image g;

[0040] The distribution ranking list construction module constructs the distribution ranking list and calculates the distance between the cluster center and the query set to obtain:

[0041] d c (f q ,f c )=||f q -f c ||2

[0042] Among them, f c ={c i |c=1,2,...,i} represents the cluster center feature vector, which is used to store the features in the gallery set; ||·||2 represents the L2 norm; d c (·,·) represents the distance between the cluster center feature and the probe feature;

[0043] The correction ranking list construction module combines the original distance and the distance between the cluster center feature and the query set to make the detection sample close to the corresponding cluster center to obtain the correction ranking list:

[0044] d * (f q ,f g ,f c )=d(f q ,f g )+λd c (f q ,f c )

[0045] Among them, λ represents an adjustable coefficient, which is used to balance the original ranking list and the distribution ranking list to form a corrected ranking list.

[0046] In the above-mentioned target re-identification device based on the correction ranking list, optionally, the backbone network uses a ResNet-50 network or an improved network thereof.

[0047] The object re-identification method and device based on the corrected ranking list provided by the present invention replace the features of the gallery set samples with the cluster center vector, consider the distance between the cluster center and the query set to correct the ranking, and propose a distribution ranking list and a correction ranking list. The corrected ranking list can be obtained through the distribution ranking list and the original ranking list. The corrected ranking list replaces the original ranking list as the final basis for object re-identification, thereby improving the accuracy of object re-identification. In addition, the distribution ranking list and the correction ranking list proposed by the present invention are suitable for large data sets, do not require manual intervention or labeled data, and the method is plug-and-play for previous backbone networks and is suitable for all re-identification tasks.

[0048] The construction of the present invention and other objects and advantageous effects thereof will become more apparent through the description of the preferred embodiments in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A schematic diagram of a flow chart of a target re-identification method based on a corrected ranking list provided in an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the structure of a target re-identification device based on a corrected ranking list provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme in the embodiment of the present invention will be described in more detail below in conjunction with the drawings in the preferred embodiments of the present invention. In the drawings, the same or similar reference numerals throughout represent the same or similar parts or parts with the same or similar functions. The described embodiments are part of the embodiments of the present invention, not all of them. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limitations on the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The embodiments of the present invention are described in detail below in conjunction with the drawings.

[0053] like Figure 1 As shown, the present invention provides a target re-identification method based on a corrected ranking list, comprising:

[0054] Step 110: Extract the features of gallery set G and query set Q through the trained backbone network f(·), where gallery set G = {g i |i=1,2,...,N},query set Q={q j |j=1,2,...,M}, where N is the number of images in the gallery set; M is the number of images in the query set; i and j are subscript indices, each q represents a probe image, and the probe feature of each probe image is f q =f(q), each g represents an image in the gallery set, and the gallery feature of each image in the gallery set is f g =f(g).

[0055] It should be noted that the backbone network uses the ResNet-50 network or its improved network. In an achievable way, the first stage of ResNet-50 can be composed of a convolutional layer, a batch normalization (BN) layer, a linear rectifier function (ReLU) and a maximum pooling (MaxPooling) layer. The other four stages contain (3, 4, 6, 3) bottlenecks (BottleNeck) respectively. Each bottleneck (BottleNeck) contains three convolutional layers, and each feature x input to the bottleneck (BottleNeck) has a jump link, so that the output of the bottleneck (BottleNeck) is H(x) = F(x) + x, where F(·) represents some layers in the bottleneck (BottleNeck). Assuming F(x) = 0, so H(x) = x, this allows the network to reduce the number of layers by itself during training, and then the network can adapt to deeper layers. In this way, ResNet-50 contains a total of 1+3*(3+4+6+3)=49 convolutional layers, plus the final fully connected layer, a total of 50 learnable layers.

[0056] Step 120: Construct the original ranking list L(q,G), and obtain the probe feature f by the normalized Euclidean distance of feature L2 q and gallery features f g The original distance between q and f g The original distance between two features is:

[0057]

[0058] Among them, z represents the index of the feature; D represents the dimension of the feature;

[0059] Among them, the original ranking list is:

[0060] L(q,G)={g i |d(q,g i)∈d1,d2,...,d k}

[0061] Among them, k represents the k nearest neighbors of q, d(q,g i )<d(q,g i+1 );

[0062] It should be noted that the original ranking list finds the nearest gallery set samples from the query set and ranks them according to the distance from near to far; the pairwise distance between q and G is obtained through the original ranking list L(q,G). The original ranking list mostly only calculates the distance from the query set sample to the k nearest neighbor samples in the gallery set, but ignores the distribution in the gallery set. It is necessary to correct the distance in L(q,G) to improve the accuracy of target re-identification.

[0063] Step 130: By replacing the features of each gallery set sample with the cluster center vector, in each iteration, the two closest clusters are merged, and the distance between the cluster centers is recalculated by the average linkage method, where the distance between the cluster centers is:

[0064]

[0065] Among them, C represents a cluster center; x and y represent C i and C j The feature points in m i and m j Respectively represent C i and C j The size of , dist(·,·) represents the distance function.

[0066] It should be noted that the original ranking list looks for the nearest gallery set samples from the query set and ranks them according to the distance from near to far. However, it may cause the wrong gallery set samples to be ranked in the top-k. Because when the query set sample is near the decision boundary in the high-dimensional space, its k nearest neighbors usually contain gallery set samples with different identities from the query set sample. At the same time, the distribution information of the gallery set also helps the ranking list. Therefore, by considering the distribution information, the features of each gallery set sample are replaced with the cluster center vector, and each feature is regarded as a separate cluster center, so that the ranking list will only search for samples of the same category.

[0067] Step 140: Assign the final clustering result as a pseudo label to each gallery image, and sum and average the same pseudo labels to obtain the cluster center feature:

[0068]

[0069] Among them, c irepresents the i-th central feature; N i Indicates that the gallery is concentrated with c i The number of identical labels; y represents the labels of c and gallery image g;

[0070] Step 150: Construct a distribution ranking list, calculate the distance between the cluster center and the query set, and obtain:

[0071] d c (f q ,f c )=||f q -f c ||2

[0072] Among them, f c ={c i |c=1,2,...,i} represents the cluster center feature vector, which is used to store the features in the gallery set; ||·||2 represents the L2 norm; d c (·,·) represents the distance between the cluster center feature and the probe feature;

[0073] Step 160: By combining the original distance and the distance between the cluster center feature and the query set, the detection sample is moved closer to the corresponding cluster center to obtain a corrected ranking list:

[0074] d * (f q ,f g ,f c )=d(f q ,f g )+λd c (f q ,f c )

[0075] Among them, λ represents an adjustable coefficient, which is used to balance the original ranking list and the distribution ranking list to form a corrected ranking list.

[0076] It should be noted that the distribution ranking list is combined with the original ranking list to pull the query set samples to the place closest to the cluster center. Being close to the cluster center ensures that the query set and its k nearest neighbors belong to the same identity with a higher probability, making the original ranking lists with the same identity closer.

[0077] The object re-identification method and device based on the corrected ranking list provided by the present invention replace the features of the gallery set samples with the cluster center vector, consider the distance between the cluster center and the query set to correct the ranking, and propose a distribution ranking list and a correction ranking list. The corrected ranking list can be obtained through the distribution ranking list and the original ranking list. The corrected ranking list replaces the original ranking list as the final basis for object re-identification, thereby improving the accuracy of object re-identification. In addition, the distribution ranking list and the correction ranking list proposed by the present invention are suitable for large data sets, do not require manual intervention or labeled data, and the method is plug-and-play for previous backbone networks and is suitable for all re-identification tasks.

[0078] like Figure 2 As shown, the present invention provides a target re-identification device based on a corrected ranking list, comprising:

[0079] The feature extraction module 201 is used to extract features of the gallery set G and the query set Q through the trained backbone network f(·), where the gallery set G = {g i |i=1,2,...,N},query set Q={q j |j=1,2,...,M}, where N is the number of images in the gallery set; M is the number of images in the query set; i and j are subscript indices, each q represents a probe image, and the probe feature of each probe image is f q =f(q), each g represents an image in the gallery set, and the gallery feature of each image in the gallery set is f g =f(g); the backbone network uses the ResNet-50 network or its improved network.

[0080] The original ranking list construction module 202 is used to construct the original ranking list L(q,G) and obtain the probe feature f by the Euclidean distance after the feature L2 normalization. q and gallery features f g The original distance between q and f g The original distance between two features is:

[0081]

[0082] Among them, z represents the index of the feature; D represents the dimension of the feature;

[0083] Among them, the original ranking list is:

[0084] L(q,G)={g i |d(q,g i )∈d1,d2,...,d k}

[0085] Among them, k represents the k nearest neighbors of q, d(q,g i )<d(q,g i+1 );

[0086] The cluster center distance measurement module 203 is used to replace the features of each gallery set sample with the cluster center vector, merge the two closest clusters in each iteration, and recalculate the distance between the cluster centers by the average link method, wherein the distance between the cluster centers is:

[0087]

[0088] Among them, C represents a cluster center; x and y represent C i and C j The feature points in m i and m j Respectively represent C i and C j The size of , dist(·,·) represents the distance function.

[0089] The cluster center feature acquisition module 204 is used to assign the final clustering result as a pseudo label to each gallery image, and perform summation and average operations on the same pseudo labels to obtain the cluster center feature:

[0090]

[0091] Among them, c i represents the i-th central feature; N i Indicates that the gallery is concentrated with c i The number of identical labels; y represents the labels of c and gallery image g;

[0092] The distribution ranking list construction module 205 constructs a distribution ranking list, calculates the distance between the cluster center and the query set, and obtains:

[0093] d c (f q ,f c )=||f q -f c ||2

[0094] Among them, f c ={c i |c=1,2,...,i} represents the cluster center feature vector, which is used to store the features in the gallery set; ||·||2 represents the L2 norm; d c (·,·) represents the distance between the cluster center feature and the probe feature;

[0095] The correction ranking list construction module 206 combines the original distance and the distance between the cluster center feature and the query set to make the detection sample close to the corresponding cluster center to obtain a correction ranking list:

[0096] d * (f q ,f g ,f c )=d(f q ,f g )+λd c (f q ,f c )

[0097] Among them, λ represents an adjustable coefficient, which is used to balance the original ranking list and the distribution ranking list to form a corrected ranking list.

[0098] The target re-identification device based on the correction ranking list provided in the above embodiment and the target re-identification method embodiment based on the correction ranking list belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target re-identification method based on a corrected ranking list, characterized in that: The method comprises: The features of gallery set G and query set Q are extracted through the trained backbone network f(·), where gallery set G = {g i |i=1,2,...,N},query set Q={q j |j=1,2,...,M}, where N is the number of images in the gallery set; M is the number of images in the query set; i and j are subscript indices, each q represents a probe image, and the probe feature of each probe image is f q =f(q), each g represents an image in the gallery set, and the gallery feature of each image in the gallery set is f g =f(g); Construct the original ranking list L(q,G) and obtain the probe feature f through the Euclidean distance after feature L2 normalization q and gallery features f g The original distance between q and f g The original distance between two features is: Among them, z represents the index of the feature; D represents the dimension of the feature; Among them, the original ranking list is: L(q,G)={g i |d(q,g i )∈d1,d2,...,d k } Among them, k represents the k nearest neighbors of q, d(q,g i )<d(q,g i+1 ); By replacing the features of each gallery set sample with the cluster center vector, the two closest clusters are merged in each iteration, and the distance between the cluster centers is recalculated by the average linkage method, where the distance between the cluster centers is: Among them, C represents a cluster center; x and y represent C i and C j The feature points in m i and m j Respectively represent C i and C j The size of , dist(·,·) represents the distance function; The final clustering result is assigned as a pseudo label to each gallery image, and the same pseudo labels are summed and averaged to obtain the cluster center features: Among them, c i represents the i-th central feature; N i Indicates that the gallery is concentrated with c i The number of identical labels; y represents the labels of c and gallery image g; Construct a distribution ranking list and calculate the distance between the cluster center and the query set to obtain: d c (f q ,f c )=||f q -f c ||2 Among them, f c ={c i |c=1,2,...,i} represents the cluster center feature vector, which is used to store the features in the gallery set; ||·||2 represents the L2 norm; d c (·,·) represents the distance between the cluster center feature and the probe feature; By combining the original distance and the distance between the cluster center feature and the query set, the detection samples are moved closer to the corresponding cluster center to obtain the corrected ranking list: d * (f q ,f g ,f c )=d(f q ,f g )+λd c (f q ,f c ) Among them, λ represents an adjustable coefficient, which is used to balance the original ranking list and the distribution ranking list to form a corrected ranking list.

2. The target re-identification method based on the corrected ranking list according to claim 1, characterized in that: The backbone network uses the ResNet-50 network or its improved network.

3. A target re-identification device based on a corrected ranking list, characterized in that: The device comprises: The feature extraction module is used to extract the features of the gallery set G and the query set Q through the trained backbone network f(·), where the gallery set G = {g i |i=1,2,...,N},query set Q={q j |j=1,2,...,M}, where N is the number of images in the gallery set; M is the number of images in the query set; i and j are subscript indices, each q represents a probe image, and the probe feature of each probe image is f q =f(q), each g represents an image in the gallery set, and the gallery feature of each image in the gallery set is f g =f(g); The original ranking list construction module is used to construct the original ranking list L(q,G) and obtain the probe feature f through the Euclidean distance after feature L2 normalization. q and gallery features f g The original distance between q and f g The original distance between two features is: Among them, z represents the index of the feature; D represents the dimension of the feature; Among them, the original ranking list is: L(q,G)={g i |d(q,g i )∈d1,d2,...,d k } Among them, k represents the k nearest neighbors of q, d(q,g i )<d(q,g i+1 ); The cluster center distance measurement module is used to replace the features of each gallery set sample with the cluster center vector. In each iteration, the two closest clusters are merged and the distance between cluster centers is recalculated by the average link method. The distance between cluster centers is: Among them, C represents a cluster center; x and y represent C i and C j The feature points in m i and m j Respectively represent C i and C j The size of , dist(·,·) represents the distance function; The cluster center feature acquisition module is used to assign the final clustering result as a pseudo label to each gallery image, and sum and average the same pseudo labels to obtain the cluster center feature: Among them, c i represents the i-th central feature; N i Indicates that the gallery is concentrated with c i The number of identical labels; y represents the labels of c and gallery image g; The distribution ranking list construction module constructs the distribution ranking list and calculates the distance between the cluster center and the query set to obtain: d c (f q ,f c )=||f q -f c ||2 Among them, f c ={c i |c=1,2,...,i} represents the cluster center feature vector, which is used to store the features in the gallery set; ||·||2 represents the L2 norm; d c (·,·) represents the distance between the cluster center feature and the probe feature; The correction ranking list construction module combines the original distance and the distance between the cluster center feature and the query set to make the detection sample close to the corresponding cluster center to obtain the correction ranking list: d * (f q ,f g ,f c )=d(f q ,f g )+λd c (f q ,f c ) Among them, λ represents an adjustable coefficient, which is used to balance the original ranking list and the distribution ranking list to form a corrected ranking list.

4. The target re-identification device based on the corrected ranking list according to claim 3, characterized in that: The backbone network uses the ResNet-50 network or its improved network.

Images