An unsupervised pedestrian re-identification method fusing relationship features and content features

Abstract

The application discloses an unsupervised pedestrian re-identification method fusing relationship features and content features; the method uses a self-encoder and a graph self-encoder to respectively extract pedestrian image content features and pedestrian image structure content features, and fuses the two, thereby improving the discriminability of the pedestrian image features; the relationship between the pedestrian images is adaptively updated, and the problem of the existence of errors and redundant connections in the pedestrian image structure relationship graph due to the lack of real labels is reduced; finally, the self-encoder and the graph self-encoder are optimized through reconstruction error, and the optimized self-encoder is used to extract the pedestrian image features, so that the unsupervised pedestrian re-identification is realized.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an unsupervised pedestrian re-identification method that integrates relational features and content features. Background Technology

[0002] Currently, with the widespread deployment of video surveillance networks, pedestrian re-identification technology, as a key component of intelligent video surveillance systems, has become a hot research direction in fields such as computer vision and multimedia analysis. The goal of pedestrian re-identification technology is to determine whether different pedestrian images captured by different surveillance cameras share the same identity. This technology can quickly and effectively locate target pedestrians in massive amounts of cross-camera video surveillance data. In recent years, pedestrian re-identification tasks based on surveillance video data have attracted increasing attention, and researchers have developed many effective algorithms to solve this problem. However, the learning process of these algorithms relies on accurate cross-camera pedestrian identity labeling data, which is very costly.

[0003] To address this issue, unsupervised person re-identification methods based on image clustering have been extensively studied in recent years. Most proposed methods gradually utilize unlabeled data by alternately assigning pseudo-labels to pedestrian images and updating the parameters of the person re-identification model. Generally, these methods can be divided into three steps: 1. Feature extraction: Extracting features from pedestrian images in the training dataset using the model; 2. Pedestrian clustering: Using clustering methods such as DBSCAN and k-means, the extracted pedestrian features are clustered into different categories, each category is assigned a pseudo-label, and the model is trained using data with pseudo-labels; 3. Feature storage and updating: During model training, the features of the pedestrian images are updated synchronously as the network parameters change.

[0004] However, these methods typically only consider the features of the pedestrian images themselves and ignore the relationships between the data, thus failing to handle difficult samples caused by factors such as pedestrian clothing and varied postures. Summary of the Invention

[0005] The purpose of this invention is to overcome the problem that existing pedestrian re-identification technologies cannot fully utilize the relationship features and behavior features between pedestrians, resulting in unsatisfactory pedestrian identification results. This invention provides an unsupervised pedestrian re-identification method that integrates relationship features and content features.

[0006] This invention provides an unsupervised pedestrian re-identification method that integrates relational features and content features. The method includes:

[0007] S1: Obtain a pedestrian image dataset from surveillance video; the pedestrian image dataset includes a training set and a test set;

[0008] S2: Extract the initial feature representations of each pedestrian image in the training set, and perform feature normalization on the initial feature representations; calculate the distance similarity between pedestrian images in the training set based on the normalized initial feature representations, and then construct a pedestrian image structure relationship graph;

[0009] S3: Based on the normalized initial feature representation, an autoencoder is used to obtain pedestrian image content features; based on the pedestrian image structure relationship graph, a graph autoencoder is used to obtain pedestrian image structure content features;

[0010] S4: The pedestrian image content features and the pedestrian image structural content features are fused to obtain fused pedestrian image features; the pedestrian image content features, pedestrian image relation features, and connection relationships are reconstructed based on the fused pedestrian image features;

[0011] S5: Calculate the reconstruction loss based on the normalized initial feature representation, the content features, and the relational features; update the pedestrian image structure relationship graph based on the connectivity relationships;

[0012] S6: Optimize the autoencoder and the graph autoencoder based on the reconstruction loss; iterate S3-S6;

[0013] S7: Extract features from pedestrian images in the test set based on the optimized autoencoder, and calculate and sort the features. The sorting result is the pedestrian re-identification result.

[0014] Preferably, in S1, the training set is denoted as: D = {I} i |i=1,2,...,N};I i Let N represent the i-th pedestrian image in the training set, and let N represent the number of pedestrian images. The test set includes a target pedestrian image subset and a candidate pedestrian image subset. In the target pedestrian image subset, there is a one-to-one correspondence between pedestrian identity and pedestrian image. In the candidate pedestrian image subset, a pedestrian identity corresponds to one or more pedestrian images.

[0015] Preferably, in S2, a pre-trained unsupervised feature representation model is used to extract the initial feature representation of each pedestrian image in the training set. The normalized initial feature representation is denoted as: X = {x i |i=1,2,...,N};x i Let N represent the normalized initial feature representation of the i-th pedestrian image in the training set, and let N represent the number of pedestrian images.

[0016] Preferably, in S2, the process of constructing the pedestrian image structure relationship graph includes:

[0017] The distance similarity includes cosine distance and Jaccard distance;

[0018] The formula for calculating cosine distance is:

[0019]

[0020] The formula for calculating the Jaccard distance is:

[0021]

[0022] definition Based on the definition, the Jaccard distance calculation formula is transformed as follows:

[0023]

[0024] R(x i ,k)={x|x∈top k (x i )&x i ∈top k (x)}

[0025] Among them, s i,j x represents the cosine distance between the i-th and j-th pedestrian images in the training set; i x represents the normalized initial feature representation of the i-th pedestrian image in the training set; j d represents the normalized initial feature representation of the j-th pedestrian image in the training set; i,j The top represents the Jaccard distance between the i-th and j-th pedestrian images in the training set. k (x i ) represents the k nearest neighbor image of the i-th pedestrian image in the training set; x represents the k nearest neighbor image that satisfies the condition x∈top k (x i )&x i ∈top k The sample of (x), v i,j It means that x satisfies j ∈R(x i The cosine distance between the i-th and j-th pedestrian images of (k); i ,v j () represents a sample pair;

[0026] The nearest neighbor value is calculated based on cosine distance and Jaccard distance, using the following formula:

[0027] A i,j =λs i,j +(1-λ)d i,j

[0028] Among them, A i,j Let λ represent the nearest neighbor relationship value between the i-th and j-th pedestrian images in the training set, where λ is the first hyperparameter.

[0029] The nearest neighbor relationship value between each pair of pedestrian images in the training set is calculated based on the nearest neighbor relationship value calculation formula. Pedestrian images are used as nodes, and node attributes are normalized initial feature representations. The k nearest neighbor relationships between nodes and other nodes are used as edges, and the nearest neighbor relationship values ​​between each pair of pedestrian images are used as the weights of the edges to construct the pedestrian image structure relationship graph.

[0030] Preferably, in S3,

[0031] The normalized initial feature representation is input into the autoencoder, and the latent space feature representation of the autoencoder is extracted as the pedestrian image content feature; the pedestrian image content feature is denoted as:

[0032] The pedestrian image structure relationship graph is input into the graph autoencoder, and the latent space feature representation of the graph autoencoder is extracted as the pedestrian image structure content feature; the pedestrian image structure content feature is denoted as:

[0033] Where σ() represents a nonlinear activation function; This represents the feature representation of the (l-1)th layer in the autoencoder; This represents the weight parameters of the l-th layer of the autoencoder; This represents the feature representation of the (l-1)th layer in the graph autoencoder; This represents the weight parameters of the l-th layer of the graph autoencoder.

[0034] Preferably, in S4,

[0035] The pedestrian image content features and the pedestrian image structural content features are fused together, and the calculation formula is as follows:

[0036] Z F =αZ AE +(1-α)Z GAE

[0037] Among them, Z F Z represents the pedestrian image features of adaptive fusion; α is the second hyperparameter; AE Z represents the content features of a pedestrian image; GAE Represents the structural content features of pedestrian images;

[0038] An autocorrelation filtering strategy is employed to mine the global relationships between pedestrian images, thereby obtaining the fused pedestrian image features. The calculation formula is as follows:

[0039] Z I =CZ F

[0040] Among them, Z ILet C represent the fused pedestrian image features; C is the autocorrelation filter matrix, calculated as follows:

[0041]

[0042] Among them, C a,b This represents the element in the a-th row and b-th column of the autocorrelation filter matrix; This indicates the similarity between samples a and b after being amplified by an exponential function; This represents the similarity between samples a and q after being amplified by an exponential function; N represents the number of pedestrian images; and q represents the index of all available samples. Represents the transpose of pedestrian image features adaptively fused;

[0043] The fused pedestrian image features are input into an autoencoder to reconstruct the content features of the pedestrian image; the content features are denoted as:

[0044] The fused pedestrian image features are input into a graph autoencoder to reconstruct the relational features of the pedestrian images; the relational features are denoted as:

[0045] Where σ() represents a nonlinear activation function; This represents the feature representation of the (h-1)th layer of the autoencoder; This represents the weight parameters of the h-th layer of the autoencoder; This represents the feature representation of the (h-1)th layer of the graph autoencoder; This represents the weight parameters of the h-th layer of the graph autoencoder;

[0046] The connection relationship is reconstructed based on the fused pedestrian image features, and the calculation formula is as follows:

[0047]

[0048] in, Indicates the connection relationship; sigmoid() is the activation function; Z I Represents the fused pedestrian image features; This represents the transpose of the fused pedestrian image features.

[0049] Preferably, in S5,

[0050] The first reconstruction loss is calculated based on the normalized initial feature representation and the content features, using the following formula:

[0051]

[0052] The second reconstruction loss is calculated based on the normalized initial feature representation and the relational features, using the following formula:

[0053]

[0054] Among them, L sc Indicates the first reconstruction loss; L st The second reconstruction loss is represented by X; X represents the normalized initial feature representation. Indicate content characteristics; Represents relational features; A represents the initial connection relationship; N represents the number of pedestrian images; F represents the calculation of the F-norm;

[0055] The pedestrian image structure graph is updated based on the aforementioned connectivity relationships, and the calculation formula is as follows:

[0056]

[0057] in, γ represents the connection relationship; γ represents the weight parameter; t represents the t-th optimization iteration.

[0058] Preferably, S5 also includes:

[0059] Each normalized initial feature representation is used as a sample. Based on the cosine distance between pedestrian images, the sample pairs are sorted in descending order, and sample pairs with a ranking less than or equal to a first threshold are selected as positive sample pairs, while sample pairs with a ranking greater than a second threshold are selected as negative sample pairs. Sample labels are generated based on this, with the following criteria:

[0060]

[0061] Where, r i,j Represents sample pairs (v) i ,v j The sorting of r pos Indicates the first threshold; r neg Indicates the second threshold;

[0062] A sample set is constructed by selecting sample pairs with sample labels of 0 or 1, and a third reconstruction loss is calculated based on the cosine distance and the sample labels. The calculation formula is as follows:

[0063]

[0064] Among them, L a This represents the third reconstruction loss, and O represents the sample set;

[0065] The first threshold and the second threshold are updated using the following formula:

[0066]

[0067]

[0068] Where, r′ pos Let r′ represent the updated first threshold. neg This represents the updated second threshold, and N represents the number of pedestrian images.

[0069] Preferably, in S6,

[0070] Based on the joint optimization of the autoencoder and graph autoencoder using the first reconstruction loss, the second reconstruction loss, and the third reconstruction loss, the formula is as follows:

[0071] L = L sc +L st +L a +L c

[0072] Where L represents the joint loss; L sc Indicates the first reconstruction loss; L st Indicates the second reconstruction loss; L a Indicates the third reconstruction loss; L c The consistency loss is calculated using the following formula:

[0073]

[0074] Among them, Z AE Z represents the content features of a pedestrian image; GAE It represents the structural content features of pedestrian images.

[0075] Preferably, in S7,

[0076] The target pedestrian image subset includes target pedestrian images; the candidate pedestrian image subset includes candidate pedestrian images.

[0077] The target pedestrian image and the candidate pedestrian image are input into the optimized autoencoder to obtain the feature representation of the target pedestrian image and the feature representation of the candidate pedestrian image, respectively.

[0078] Calculate the distance similarity between the feature representation of the target pedestrian image and the feature representation of the candidate pedestrian images, sort them in descending order, and select the candidate pedestrian images in the subset whose sorting is less than or equal to the third threshold as the pedestrian re-identification result.

[0079] Beneficial effects: This method utilizes an autoencoder and a graph autoencoder to extract pedestrian image content features and pedestrian image structural content features respectively, and then fuses the two to improve the discriminative power of pedestrian image features; it adaptively updates the relationships between pedestrian images, reducing the problem of erroneous and redundant connections in the pedestrian image structural relationship graph due to the lack of real labels; finally, it optimizes the autoencoder and graph autoencoder by reconstruction error, and uses the optimized autoencoder to extract pedestrian image features, achieving unsupervised pedestrian re-identification. Attached Figure Description

[0080] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0081] Figure 1 This is a flowchart of an unsupervised pedestrian re-identification method according to an embodiment of this application. Detailed Implementation

[0082] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0083] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0084] like Figure 1 As shown in the figure, this embodiment provides an unsupervised pedestrian re-identification method that integrates relation features and content features. The method includes:

[0085] S1: Obtain a pedestrian image dataset from surveillance video; the pedestrian image dataset includes a training set and a test set;

[0086] Specifically, the training set is denoted as: D = {I} i |i=1,2,...,N};I i Let N represent the i-th pedestrian image in the training set, and let N represent the number of pedestrian images. The test set includes a target pedestrian image subset and a candidate pedestrian image subset. In the target pedestrian image subset, there is a one-to-one correspondence between pedestrian identity and pedestrian image. In the candidate pedestrian image subset, a pedestrian identity corresponds to one or more pedestrian images.

[0087] S2: Extract the initial feature representations of each pedestrian image in the training set, and perform feature normalization on the initial feature representations; calculate the distance similarity between pedestrian images in the training set based on the normalized initial feature representations, and then construct a pedestrian image structure relationship graph;

[0088] Specifically, a pre-trained unsupervised feature representation model is used to extract initial feature representations for each pedestrian image in the training set. The normalized initial feature representation is denoted as: X = {x i |i=1,2,...,N};x i Let N represent the normalized initial feature representation of the i-th pedestrian image in the training set, and let N represent the number of pedestrian images.

[0089] In this embodiment, the representation model can be any pre-trained unsupervised feature representation model, such as the Bottom-up Clustering model; using a better-performing pre-trained feature extraction model will result in better pedestrian re-identification performance.

[0090] The process of constructing a pedestrian image structure relationship graph includes:

[0091] In this embodiment, the distance similarity includes cosine distance and Jaccard distance; cosine distance calculates the distance between two feature representations, while Jaccard distance considers the k-nearest neighbor relationship between pedestrian images. Combining the two can obtain a more reliable sample relationship.

[0092] The formula for calculating cosine distance is:

[0093]

[0094] The formula for calculating the Jaccard distance is:

[0095]

[0096] definition Based on the definition, the Jaccard distance calculation formula is transformed as follows:

[0097]

[0098] R(x i ,k)={x|x∈top k (x i )&x i ∈top k (x)}

[0099] Among them, s i,j x represents the cosine distance between the i-th and j-th pedestrian images in the training set; i x represents the normalized initial feature representation of the i-th pedestrian image in the training set; jd represents the normalized initial feature representation of the j-th pedestrian image in the training set; i,j The top represents the Jaccard distance between the i-th and j-th pedestrian images in the training set. k (x i ) represents the k nearest neighbor image of the i-th pedestrian image in the training set; x represents the k nearest neighbor image that satisfies the condition x∈top k (x i )&x i ∈top k The sample of (x), v i,j It means that x satisfies j ∈R(x i The cosine distance between the i-th and j-th pedestrian images of (k); i ,v j () represents a sample pair;

[0100] The nearest neighbor value is calculated based on cosine distance and Jaccard distance, using the following formula:

[0101] A i,j =λs i,j +(1-λ)d i,j

[0102] Among them, A i,j Let λ represent the nearest neighbor relationship value between the i-th and j-th pedestrian images in the training set, where λ is the first hyperparameter used to weight the importance of the two distances.

[0103] The nearest neighbor relationship value between each pair of pedestrian images in the training set is calculated based on the nearest neighbor relationship value calculation formula. Pedestrian images are used as nodes, and node attributes are normalized initial feature representations. The k nearest neighbor relationships between nodes and other nodes are used as edges, and the nearest neighbor relationship values ​​between each pair of pedestrian images are used as the weights of the edges to construct the pedestrian image structure relationship graph.

[0104] S3: Based on the normalized initial feature representation, an autoencoder is used to obtain pedestrian image content features; based on the pedestrian image structure relationship graph, a graph autoencoder is used to obtain pedestrian image structure content features;

[0105] Specifically, the normalized initial feature representation is input into the autoencoder, and the latent space feature representation of the autoencoder is extracted as the pedestrian image content feature; the pedestrian image content feature is denoted as:

[0106] The pedestrian image structure relationship graph is input into the graph autoencoder, and the latent space feature representation of the graph autoencoder is extracted as the pedestrian image structure content feature; the pedestrian image structure content feature is denoted as:

[0107] Where σ() represents a nonlinear activation function; This represents the feature representation of the (l-1)th layer in the autoencoder; This represents the weight parameters of the l-th layer of the autoencoder; This represents the feature representation of the (l-1)th layer in the graph autoencoder; This represents the weight parameters of the l-th layer of the graph autoencoder.

[0108] S4: The pedestrian image content features and the pedestrian image structural content features are fused to obtain fused pedestrian image features; the pedestrian image content features, pedestrian image relation features, and connection relationships are reconstructed based on the fused pedestrian image features;

[0109] Specifically, considering only pedestrian image content features or pedestrian image structural content features cannot process difficult sample data. Therefore, the pedestrian image content features and pedestrian image structural content features are fused together, and the calculation formula is as follows:

[0110] Z F =αZ AE +(1-α)Z GAE

[0111] Among them, Z F Z represents the adaptively fused pedestrian image features; α is the second hyperparameter, used to balance the importance of pedestrian image content features and pedestrian image structural content features, which is updated synchronously with optimization to achieve adaptive dynamic fusion of content features and structural content features; AE Z represents the content features of a pedestrian image; GAE Represents the structural content features of pedestrian images;

[0112] An autocorrelation filtering strategy is employed to better uncover the global relationships between pedestrian images, thereby obtaining the fused pedestrian image features. The calculation formula is as follows:

[0113] Z I =CZ F

[0114] Among them, Z I Let C represent the fused pedestrian image features; C is the autocorrelation filter matrix, calculated as follows:

[0115]

[0116] Among them, C a,b This represents the element in the a-th row and b-th column of the autocorrelation filter matrix; This indicates the similarity between samples a and b after being amplified by an exponential function; This represents the similarity between samples a and q after being amplified by an exponential function; N represents the number of pedestrian images; and q represents the index of all available samples. Represents the transpose of pedestrian image features adaptively fused;

[0117] The fused pedestrian image features are input into an autoencoder to reconstruct the content features of the pedestrian image; the content features are denoted as:

[0118] The fused pedestrian image features are input into a graph autoencoder to reconstruct the relational features of the pedestrian images; the relational features are denoted as:

[0119] Where σ() represents a nonlinear activation function; This represents the feature representation of the (h-1)th layer of the autoencoder; This represents the weight parameters of the h-th layer of the autoencoder; This represents the feature representation of the (h-1)th layer of the graph autoencoder; This represents the weight parameters of the h-th layer of the graph autoencoder;

[0120] The connection relationship is reconstructed based on the fused pedestrian image features, and the calculation formula is as follows:

[0121]

[0122] in, Indicates the connection relationship; sigmoid() is the activation function; Z I Represents the fused pedestrian image features; This represents the transpose of the fused pedestrian image features.

[0123] S5: Calculate the reconstruction loss based on the normalized initial feature representation, the content features, and the relational features; update the pedestrian image structure relationship graph based on the connectivity relationships;

[0124] Specifically, the first reconstruction loss is calculated based on the normalized initial feature representation and the content features, using the following formula:

[0125]

[0126] The second reconstruction loss is calculated based on the normalized initial feature representation and the relational features, using the following formula:

[0127]

[0128] Among them, L sc Indicates the first reconstruction loss; L st The second reconstruction loss is represented by X; X represents the normalized initial feature representation. Indicate content characteristics; Represents relational features; A represents the initial connection relationship; N represents the number of pedestrian images; F represents the calculation of the F-norm;

[0129] Considering the lack of true labels for samples in unsupervised pedestrian re-identification tasks, the constructed structural relationship graph will contain a large number of erroneous connections. Therefore, the pedestrian image structural relationship graph is updated using dot product based on the reconstructed connections. In the next optimization iteration, the updated structural relationship graph is used as input to the graph autoencoder to further update and optimize both the autoencoder and the graph autoencoder. The formula for updating the pedestrian image structural relationship graph based on these connections is as follows:

[0130]

[0131] in, γ represents the connection relationship; γ represents the weight parameter, and the initial pedestrian image structure relationship graph is used when γ = 0; t represents the t-th optimization iteration.

[0132] In this embodiment, the reconstructed connections will contain a large number of errors and redundant connections. Directly using the reconstruction errors to optimize the autoencoder and graph autoencoder will degrade their performance. Therefore, it is necessary to select reliable sample connections as supervision signals for optimization. The process is as follows:

[0133] Each normalized initial feature representation is used as a sample. Based on the cosine distance between pedestrian images, the sample pairs are sorted in descending order, and sample pairs with a ranking less than or equal to a first threshold are selected as positive sample pairs, while sample pairs with a ranking greater than a second threshold are selected as negative sample pairs. Sample labels are generated based on this, with the following criteria:

[0134]

[0135] Where, r i,j Represents sample pairs (v) i ,v j The sorting of r pos Indicates the first threshold; r neg Indicates the second threshold;

[0136] A sample set is constructed by selecting sample pairs with sample labels of 0 or 1, and a third reconstruction loss is calculated based on the cosine distance and the sample labels. The calculation formula is as follows:

[0137]

[0138] Among them, L a This represents the third reconstruction loss, and O represents the sample set;

[0139] As training progresses, the number of reliable sample pairs will gradually increase. Therefore, the first threshold and the second threshold are updated based on the following formula:

[0140]

[0141]

[0142] Where, r′ pos Let r′ represent the updated first threshold. neg This represents the updated second threshold, and N represents the number of pedestrian images.

[0143] S6: Optimize the autoencoder and the graph autoencoder based on the reconstruction loss; iterate S3-S6;

[0144] Specifically, the autoencoder and graph autoencoder are jointly optimized based on the first reconstruction loss, the second reconstruction loss, and the third reconstruction loss, as shown in the following formula:

[0145] L = L sc +L st +L a +L c

[0146] Where L represents the joint loss; L sc Indicates the first reconstruction loss; L st Indicates the second reconstruction loss; L a Indicates the third reconstruction loss; L c The consistency loss function is used to ensure the consistency of feature representations learned by the autoencoder and the graph autoencoder. Its calculation formula is as follows:

[0147]

[0148] Among them, Z AE Z represents the content features of a pedestrian image; GAE It represents the structural content features of pedestrian images.

[0149] S7: Extract features from pedestrian images in the test set based on the optimized autoencoder, and calculate and sort the features. The sorting result is the pedestrian re-identification result.

[0150] Specifically, the target pedestrian image subset includes target pedestrian images; the candidate pedestrian image subset includes candidate pedestrian images.

[0151] The target pedestrian image and the candidate pedestrian image are input into the optimized autoencoder to obtain the feature representation of the target pedestrian image and the feature representation of the candidate pedestrian image, respectively.

[0152] Calculate the distance similarity between the feature representation of the target pedestrian image and the feature representation of the candidate pedestrian images, sort them in descending order, and select the candidate pedestrian images in the subset whose sorting is less than or equal to the third threshold as the pedestrian re-identification result.

[0153] In this embodiment, the third threshold is set to 20, but it can also be set to other values ​​according to actual needs.

[0154] The method provided in this embodiment utilizes an autoencoder and a graph autoencoder to extract pedestrian image content features and pedestrian image structural content features respectively, and then fuses the two to improve the discriminative power of pedestrian image features. It proposes an adaptive update strategy for pedestrian nearest neighbor relationships to adaptively update the relationships between pedestrian images, reducing the problem of erroneous and redundant connections in the pedestrian image structural relationship graph caused by the lack of real labels. Finally, it optimizes the autoencoder and graph autoencoder by combining multiple reconstruction errors, and uses the optimized autoencoder to extract pedestrian image features to achieve unsupervised pedestrian re-identification.

[0155] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An unsupervised pedestrian re-identification method that integrates relational features and content features, characterized in that, include: S1: Obtain a pedestrian image dataset from surveillance video; the pedestrian image dataset includes a training set and a test set; S2: Extract the initial feature representations of each pedestrian image in the training set, and perform feature normalization on the initial feature representations; calculate the distance similarity between pedestrian images in the training set based on the normalized initial feature representations, and then construct a pedestrian image structure relationship graph; An unsupervised feature representation model, pre-trained, is used to extract initial feature representations from each pedestrian image in the training set. The normalized initial feature representation is denoted as: ; x i Indicates the first training set i Normalized initial feature representation of Zhang Xingren's image. N Indicates the number of pedestrian images; The nearest neighbor relationship value is calculated based on the nearest neighbor relationship value calculation formula between pairwise pedestrian images in the training set. Pedestrian images are used as nodes, and node attributes are normalized initial feature representations. The nearest neighbor relationship value between nodes and other nodes is then calculated. k The nearest neighbor relationship is used as an edge, and the nearest neighbor relationship value between each pair of pedestrian images is used as the weight of the edge to construct the pedestrian image structure relationship graph; S3: Based on the normalized initial feature representation, an autoencoder is used to obtain pedestrian image content features; Based on the pedestrian image structure relationship graph, a graph autoencoder is used to obtain the pedestrian image structure content features; The normalized initial feature representation is input into the autoencoder, and the latent space feature representation of the autoencoder is extracted as the pedestrian image content feature; the pedestrian image content feature is denoted as: Where σ() represents a nonlinear activation function; Indicates the first in the autoencoder l Feature representation of layer -1; Represents the first autoencoder l Layer weight parameters; S4: The pedestrian image content features and the pedestrian image structural content features are fused to obtain fused pedestrian image features; the calculation formula is: in, Z F Represents pedestrian image features that are adaptively fused; α This is the second hyperparameter; Z AE Indicates the content features of pedestrian images; Z GAE Represents the structural content features of pedestrian images; Based on the fused pedestrian image features, the content features, relational features, and connection relationships of the pedestrian images are reconstructed. S5: Calculate the reconstruction loss based on the normalized initial feature representation, the content features, and the relational features; update the pedestrian image structure relationship graph based on the connectivity relationships; S6: Optimize the autoencoder and the graph autoencoder based on the reconstruction loss; iterate S3-S6; S7: Extract features from pedestrian images in the test set based on the optimized autoencoder, and calculate and sort the features. The sorting result is the pedestrian re-identification result.

2. The unsupervised pedestrian re-identification method based on the fusion of relational features and content features according to claim 1, characterized in that, In S1, the training set is denoted as: ; I i Indicates the first training set i Zhang Xingren's image, N This indicates the number of pedestrian images; the test set includes a subset of target pedestrian images and a subset of candidate pedestrian images. In the target pedestrian image subset, each pedestrian identity corresponds one-to-one with a pedestrian image; In the subset of candidate pedestrian images, a pedestrian identity corresponds to one or more pedestrian images.

3. The unsupervised pedestrian re-identification method based on the fusion of relational features and content features according to claim 2, characterized in that, In S2, the distance similarity includes cosine distance and Jaccard distance; The formula for calculating cosine distance is: The formula for calculating the Jaccard distance is: definition Based on the definition, the Jaccard distance calculation formula is transformed as follows: in, s i,j Indicates the first training set i Zhang Yudi j Cosine distance between images of pedestrians; x i Indicates the first training set i Normalized initial feature representation of Zhang Xingren's image; x j Indicates the first training set j Normalized initial feature representation of Zhang Xingren's image; d i,j Indicates the first training set i Zhang Yudi j Jaccard distance between images of pedestrians; top k ( x i ) indicates the first (in the training set) i Zhang Xingren's image k Nearest neighbor images; Indicates the first training set j Transpose of the normalized initial feature representation of Zhang Xingren's image x This indicates that the condition is met. The sample, v i,j Indicates satisfaction The i Zhang Yudi j The cosine distance between the images of Zhang Xingren; v i , v j () represents a sample pair; The nearest neighbor value is calculated based on cosine distance and Jaccard distance, using the following formula: in, A i,j Indicates the first training set i Zhang Yudi j Nearest neighbor relationship values ​​between images of pedestrians λ This is the first hyperparameter.

4. The unsupervised pedestrian re-identification method based on the fusion of relational features and content features according to claim 3, characterized in that, In S3, The pedestrian image structure relationship graph is input into the graph autoencoder, and the latent space feature representation of the graph autoencoder is extracted as the pedestrian image structure content feature; the pedestrian image structure content feature is denoted as: ; Where σ() represents a nonlinear activation function; A Indicates the initial connection relationship; The graph autoencoder represents the first... l Feature representation of layer -1; The graph autoencoder represents the first... l Layer weight parameters.

5. The unsupervised pedestrian re-identification method based on the fusion of relational features and content features according to claim 4, characterized in that, In S4, An autocorrelation filtering strategy is employed to mine the global relationships between pedestrian images, thereby obtaining the fused pedestrian image features. The calculation formula is as follows: in, Z I Represents the fused pedestrian image features; C The autocorrelation filter matrix is ​​calculated using the following formula: in, C a,b The autocorrelation filter matrix represents the first... a Line number b Column elements; This represents a sample amplified by an exponential function. a and b Similarity; This represents a sample amplified by an exponential function. a and q Similarity; N Indicates the number of pedestrian images; q This represents the index of all available samples; Represents the transpose of pedestrian image features adaptively fused; The fused pedestrian image features are input into an autoencoder to reconstruct the content features of the pedestrian image; the content features are denoted as: ; The fused pedestrian image features are input into a graph autoencoder to reconstruct the relational features of the pedestrian images; the relational features are denoted as: ; Where σ() represents a nonlinear activation function; Indicates the self-encoder's first... h -1 layer feature representation; Represents the first autoencoder h Layer weight parameters; The graph encoder represents the first... h -1 layer feature representation; The graph autoencoder represents the first... h Layer weight parameters; The connection relationship is reconstructed based on the fused pedestrian image features, and the calculation formula is as follows: in, Indicates the connection relationship; sigmoid() is the activation function; Z I Represents the fused pedestrian image features; This represents the transpose of the fused pedestrian image features.

6. The unsupervised pedestrian re-identification method based on the fusion of relational features and content features according to claim 5, characterized in that, In S5, The first reconstruction loss is calculated based on the normalized initial feature representation and the content features, using the following formula: The second reconstruction loss is calculated based on the normalized initial feature representation and the relational features, using the following formula: in, L sc Indicates the first reconstruction loss; L st Indicates the second reconstruction loss; X The initial feature representation after normalization; Indicate content characteristics; Indicate relational characteristics; A Indicates the initial connection relationship; N Indicates the number of pedestrian images; F Indicates calculation F Norm; The pedestrian image structure graph is updated based on the aforementioned connectivity relationships, and the calculation formula is as follows: in, Indicates a connection relationship; γ Indicates the weighting parameter; t Indicates the first t This is the second optimization iteration.

7. The unsupervised pedestrian re-identification method based on the fusion of relational features and content features according to claim 6, characterized in that, S5 also includes: Each normalized initial feature representation is used as a sample. Based on the cosine distance between pedestrian images, the sample pairs are sorted in descending order, and sample pairs with a ranking less than or equal to a first threshold are selected as positive sample pairs, while sample pairs with a ranking greater than a second threshold are selected as negative sample pairs. Sample labels are generated based on this, with the following criteria: in, r i,j Indicates sample pairs ( v i , v j The sorting of ) r pos Indicates the first threshold; r neg Indicates the second threshold; A sample set is constructed by selecting sample pairs with sample labels of 0 or 1, and a third reconstruction loss is calculated based on the cosine distance and the sample labels. The calculation formula is as follows: in, L a Indicates the losses from the third reconstruction. O Represents a sample set; The first threshold and the second threshold are updated using the following formula: in, This represents the updated first threshold. This represents the updated second threshold. N This indicates the number of pedestrian images.

8. The unsupervised pedestrian re-identification method based on the fusion of relational features and content features according to claim 7, characterized in that, In S6, Based on the joint optimization of the autoencoder and graph autoencoder using the first reconstruction loss, the second reconstruction loss, and the third reconstruction loss, the formula is as follows: in, L Indicates joint loss; L sc Indicates the first reconstruction loss; L st Indicates the second reconstruction loss; L a Indicates the losses from the third reconstruction; L c The consistency loss is calculated using the following formula: in, Z AE Indicates the content features of pedestrian images; Z GAE It represents the structural content features of pedestrian images.

9. The unsupervised pedestrian re-identification method based on the fusion of relational features and content features according to claim 8, characterized in that, In S7 The subset of target pedestrian images includes the target pedestrian images; The subset of candidate pedestrian images includes candidate pedestrian images; The target pedestrian image and the candidate pedestrian image are input into the optimized autoencoder to obtain the feature representation of the target pedestrian image and the feature representation of the candidate pedestrian image, respectively. Calculate the distance similarity between the feature representation of the target pedestrian image and the feature representation of the candidate pedestrian images, sort them in descending order, and select the candidate pedestrian images in the subset whose sorting is less than or equal to the third threshold as the pedestrian re-identification result.

Images