A method for person re-identification using attribute features to construct graph convolutional networks

Abstract

This invention relates to the field of image processing technology, specifically to a pedestrian re-identification method that utilizes attribute features to construct a graph convolutional network. The method presented in this invention combines attributes and body parts to construct a graph convolutional network for pedestrian re-identification. It efficiently learns global features through a multi-scale context-aware network, extracts semantic segmentation features of body parts using a body parsing model, and filters and removes attribute nodes with relatively small impact on the recognition results based on attribute relevance, thus constructing a low-dimensional graph network model to extract local features. This invention achieves a more comprehensive pedestrian feature description by fusing global and local feature learning, reducing redundant information between different receptive fields, extracting richer global features of pedestrian images, effectively enhancing visual context information, and performing image filtering based on attribute prediction, effectively reducing the matching range of images in the image library and optimizing the retrieval process.

Description

Technical fields:

[0001] This invention relates to the field of image processing technology, and specifically to a method for pedestrian re-identification that utilizes attribute features to construct a graph convolutional network. Background technology:

[0002] As a key component of automatic target matching and recognition technology in intelligent surveillance networks, pedestrian re-identification methods aim to retrieve target pedestrians from pedestrian image databases collected by multiple cameras. This provides crucial data for intelligent security, pedestrian and vehicle tracking, and other national applications, holding significant importance for building smart and safe cities. However, due to significant differences in lighting, viewing angle, and pose variations in image data collected by different cameras, it is necessary to explore efficient implementation schemes to improve pedestrian re-identification performance.

[0003] With the development of feature learning methods, scholars have proposed a variety of research approaches to improve feature diversity, including the comprehensive use of global and local features, as well as obtaining rich semantic information by defining human attribute features and using attribute information as auxiliary features for network model training.

[0004] To obtain more discriminative features, some research methods utilize new labels formed by combining different attribute labels to construct additional classification loss functions, and combine them with classification losses from multiple pedestrian attribute labels, enabling the feature extraction network to distinguish more person-specific information. Some researchers integrate identity, body part classification, and attribute feature learning into a unified framework, where a global feature network performs global identity classification based on the input image, a local feature network is used for body part detection, and an attribute feature network extracts perceptual regions from attributes to generate attribute attention maps. Considering the correlation between attribute information and body parts, some research methods generate attribute-body part detection operators through attribute detection processing, fusing local feature descriptions with attribute information, which helps handle pose changes and camera angle issues. Since there are significant differences between attribute and identity tasks, the conventional method of directly combining attribute and identity loss functions to learn attributes and identity together is ineffective. Some researchers construct two branches for attribute and identity learning, fusing attribute and identity recognition tasks at the loss level and feature level respectively, and in the attribute branch, combining hierarchical features and feature attention blocks to mine high-level semantic and spatial information respectively. By exploring the relationship between categories and attributes, some scholars have constructed models that simultaneously learn category and attribute features. These models then employ an attribute-category attention mechanism to select more discriminative features, aiding in category and attribute classification. Other scholars have presented pixel-level human region parsing and pose estimation datasets. By labeling images at different granular levels, they have constructed a network model that jointly performs human parsing and pose estimation tasks, fully utilizing detailed pixel-level information and high-level structural information from keypoint locations. Based on a general human parsing structure, some scholars have presented a self-correcting human part segmentation model. This model uses an encoder to extract features from the input image and, based on the decoded features, predicts the human contour in the edge branch and segments the human in the parsing branch. Finally, the outputs of both branches are combined with the features from the encoder. Figure 1 Using it as input to the fusion branch can effectively improve semantic segmentation performance and provide a useful foundation for related research.

[0005] Previous research, based on changes in visual factors, has focused on utilizing information such as body parts and pedestrian identity attributes. By using attribute features as auxiliary information and fusing them with global or local features, more discriminative features can be extracted. However, a common problem with existing technologies is that they do not consider the correlations between attributes, between body parts, and within each attribute. At the same time, fine-grained visual cues are not fully explored, and excessive attribute information can affect the complexity of the network, reduce retrieval efficiency, and result in low performance of typical pedestrian re-identification models. Summary of the Invention:

[0006] This invention aims to provide a pedestrian re-identification method that utilizes attribute features to construct a graph convolutional network, thereby overcoming the problems of high network complexity, low performance of typical pedestrian re-identification models, and low retrieval efficiency in existing technologies.

[0007] To achieve the objectives of this invention, a method for person re-identification using attribute features to construct a graph convolutional network is provided, comprising the following steps:

[0008] Step 1: Construct a multi-scale context-aware network to extract global features and a graph convolutional network to infer local features using pedestrian attributes and body parts:

[0009] In the construction of the multi-scale context-aware network, dilated convolutions with different dilation ratios are used to obtain an enhanced multi-scale receptive field.

[0010] In the construction of the graph convolutional network model: a human body analysis model is used to extract the mask map of human body parts to obtain the visual features of each body part. A search term is embedded to represent the pedestrian attributes. Based on the obtained body part features and the filtered and dimensionality-reduced attribute features, a graph convolutional network is constructed. This graph contains key attribute nodes and body part nodes under the condition of the correlation matrix. Then, the graph convolutional network is used to infer and encode more representative local features.

[0011] Step 2: Optimize image matching processing based on attribute prediction values: Based on the context-aware network and graph convolutional network constructed in Step 1, feature extraction and attribute prediction are performed on the images in the image library. Attribute predictions with high confidence are regarded as reliable predictions for both the query image and the image library images. Then, candidate images in the image library whose reliable attributes are different from those of the query image are deleted.

[0012] In step one above, in the multi-scale context-aware network: the initial convolutional layer uses a 5*5 convolutional kernel to extract the low-level visual features, and starting from the second layer, multi-scale convolutional layers with different dilation ratios are used to obtain image context information.

[0013] The above-mentioned different dilation ratios include three types, which indicate that each convolutional layer has three convolutional kernels, namely convolutional kernels with dilation ratios of 1, 2 and 3.

[0014] The construction of the convolutional network model in step one above includes the following specific methods:

[0015] (1) Method for representing body parts nodes

[0016] The body parts are divided into head, upper body, lower body, and arm regions. Each pedestrian image is segmented into N parts. P Each body part is masked. It means that H kIt is a binary mask of the same size as the pedestrian image, where each mask H... k Scaling to the same size as the feature map F, and then applying L1 normalization, results in H′. k Feature map F∈R W×H×D It has W×H positions, and each position i is associated with a feature vector f. i ∈R D Related. The kth body part The calculation is shown in formula (1):

[0017]

[0018] in It is H′ k The scalar value at position i Projected onto a D-dimensional vector.

[0019] (2) Constructing a graph network model using key attribute nodes and body part nodes: including the optimal selection of key attribute nodes, graph construction, and inference of the graph convolutional neural network.

[0020] ① Optimized selection of attribute nodes

[0021] To preserve key attributes and ensure recognition accuracy, one attribute is removed from each node at a time. The attributes with relatively minor impact are then identified and removed from the graph convolutional network. Based on experimental analysis on the Market-1501 dataset, the following attribute nodes were ultimately removed from the original 27: hat, age, upper body clothing color (including 9 colors), and lower body clothing color (including 8 colors). Eight key attribute nodes with significant impact on the recognition results were retained. These include gender, sleeve length, lower body clothing length, lower body clothing type, hair, backpack, handbag, and other bags.

[0022] ② Graph Construction

[0023] Define the graph as G = (V, E), with nodes V = {V1, V2, ..., V...} N The total number of nodes is N. G =N A +N P , where N A N represents the number of attributes. P The number of body parts. Each node in the graph represents a pedestrian attribute or body part, using a D-dimensional feature vector x. v Initialize it. Attribute word embedding vector It means that N A D represents the number of attributes. W Let be the dimension of the word embedding vector.

[0024] The graph is derived from the correlation matrix. This means that it contains each edge (V) i V j The weights related to E are denoted as M. The correlation matrix M is specifically represented as follows:

[0025]

[0026] in, A matrix representing the correlation between attributes. This represents a matrix showing the correlation between body parts. This represents a correlation matrix between body parts and attributes. This represents the correlation matrix between attributes and body parts.

[0027] ③ Perform inference computation on graph convolutional networks

[0028] A graph is constructed by using body parts and attributes as nodes. Reasoning is then performed on this graph, which is encoded into more representative local features. The obtained features are then connected with global features.

[0029] The construction of the above graph includes the following specific matrix construction:

[0030] a) Correlation matrix between attributes

[0031] Construct a correlation matrix AA between attributes, where elements AA ij This represents the probability that attribute j will appear when attribute i appears, specifically as follows:

[0032]

[0033] Among them, K i L represents the number of times attribute i appears among pedestrians in the training set. ij This indicates the number of times attribute pairs i and j appear together.

[0034] b. Correlation matrix between body parts

[0035] Assuming that in the training set of pedestrians, the body parts of each pedestrian image can be identified, and all elements in the correlation matrix PP between body parts are set to 1, it can be inferred that if body part i is identified, the probability of identifying body part j is 1.

[0036] c. Correlation matrix between body parts and attributes

[0037] For the correlation matrix PA between body parts and attributes, where elements PA ijPA represents the probability of attribute j appearing when body part i is identified. Typically, some attributes are only related to specific body parts; therefore, if attribute j is not attached to body part i when identifying a body part i, PA... ij As shown in formula (4), otherwise as shown in formula (5):

[0038] PA ij =0 (4)

[0039] PA ij =K j (5)

[0040] Among them, K j It represents the percentage of attributes that appear among pedestrians in the training set.

[0041] d. Correlation matrix between attributes and body parts

[0042] For the correlation matrix AP between attributes and body parts, where the elements AP ij This represents the probability that body part i is identified when attribute j appears. Assuming all body parts of a pedestrian are identifiable in the training set, if attribute j is associated with body part i, then AP... ij As shown in formula (6), otherwise as shown in formula (7):

[0043] AP ij =1 (6)

[0044] AP ij =0 (7)

[0045] The specific process of inference computation for the graph convolutional network described in ③ above is as follows:

[0046] Graph convolutional networks define a multi-layer propagation process on a graph G. Specifically, each layer in the graph is represented as a function f(X,M), which, under the influence of the correlation matrix M, propagates through the input nodes. Information is propagated between nodes to update their representations, using L (k) This represents the feature matrix after the input node X is passed to the k-th GCN layer;

[0047] Based on node features The corresponding correlation matrix M is taken as input and transformed through the GCN layer. Each GCN layer is represented as follows:

[0048]

[0049] Where, θ (k) For a specific layer, the trainable weight matrix, This is the normalized version of the correlation matrix M;

[0050] Represented as

[0051]

[0052] Where D is a diagonal matrix of M, It is the identity matrix;

[0053] The input node features X are fed into the k-th stacked GCN layer, and then the output node matrix is ​​obtained. Obtain subset node V a ∈V, where v a ∈V a For attribute nodes, V a The corresponding output node features are stacked together to form

[0054] In step two above, attributes with high confidence are determined by setting a threshold φ. When the confidence score is higher than the threshold, the attribute is considered a reliable attribute. When an attribute is determined to be reliable for both the query image and the image in the library, it is checked whether the two images have the same prediction for the attribute. If not, the image in the library is deleted from the library.

[0055] In step two above, the threshold is set to 0.7.

[0056] Compared with existing technologies, this invention fully utilizes the correlations between attributes and between attributes and body parts. It explores the use of these correlations in the process of obtaining diverse feature representations using body parts and attributes, extracting more discriminative pedestrian features while reducing the retrieval and matching calculation process. The advantages of this invention are:

[0057] 1. This invention uses a multi-scale context-aware network to learn different visual cues, employing dilated convolutions with different dilation ratios to obtain enhanced multi-scale receptive fields. Different contextual information is integrated by embedding feature maps of the current convolutional layer through layer-by-layer convolutional operations. The model selects dilated convolutions with different dilation ratios instead of convolutional filters with different kernel sizes in typical convolutional neural networks to obtain contextual information at different scales. The obtained feature maps are cascaded along the channel axis to form the final output of the current convolutional layer. This reduces redundant information between different receptive fields, extracts richer global features of pedestrian images, and effectively enhances visual contextual information.

[0058] 2. This invention filters out attribute nodes with low correlation based on the correlation between attributes, constructs a low-dimensional graph network structure with the filtered optimized attribute set and body part nodes, and performs inference to encode highly discriminative local features. By filtering and removing attribute nodes with low reliability and minimal impact on the recognition results based on the correlation between attributes and between attributes and pedestrians, the dimensionality of the correlation matrix can be reduced, improving network computational efficiency. Further fusion of global and local features yields a more comprehensive and diverse pedestrian representation, effectively improving pedestrian re-identification performance.

[0059] 3. This invention performs feature extraction and attribute prediction on image libraries based on the designed network structure. Attribute predictions with high confidence are considered reliable predictions for both the query image and the image library images, and candidate images from the image library whose reliable attributes differ from the query image are removed. Based on attribute prediction, most images can be filtered out from the image library images, effectively reducing the matching range and optimizing the retrieval process. Attached image description:

[0060] Figure 1 This is a block diagram illustrating the implementation principle of the present invention. Detailed implementation method:

[0061] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0062] The method of this invention mines identity-sensitive feature information and makes full use of the correlation between attributes and between attributes and body parts. For the specific implementation principle, please refer to [link / reference]. Figure 1 .

[0063] The method of this invention includes two steps: Step 1, extracting global features using a multi-scale network and constructing a graph convolutional network for inference using pedestrian attributes and body parts. In the global feature extraction part, a multi-scale network is first used to obtain global features, and then a human body analysis model is used to extract mask images of human body parts to obtain the visual features of each part. In the graph network model construction part, pedestrian attributes are represented by a lookup term embedding. Based on the obtained body part features and attribute features, a graph network is constructed. This graph contains attribute nodes and body part nodes under the condition of the correlation matrix. A graph convolutional network is used to infer these features and encode them into more representative features, which, together with the global features, serve as pedestrian features. Step 2, optimizing image matching processing based on attribute prediction values.

[0064] This invention presents a pedestrian re-identification method that combines attributes and body parts to construct a graph convolutional network. This method efficiently learns global features through a multi-scale context-aware network, extracts semantic segmentation features of body parts using a body parsing model, and filters and removes attribute nodes with relatively small impact on the recognition results based on attribute relevance. This results in a low-dimensional graph network model that extracts local features. The new method achieves a more comprehensive pedestrian feature description by fusing global and local features, and performs image filtering based on attribute prediction to narrow the matching range.

[0065] Example: A method for person re-identification using attribute features to construct a graph convolutional network, specifically including the following steps:

[0066] Step 1: Construct a multi-scale context-aware network to extract global features and a graph convolutional network to infer local features using pedestrian attributes and body parts. In this step, global features are first obtained through a multi-scale context-aware network. Then, the filtered attributes and body part nodes are used to construct a low-dimensional graph network structure and infer to encode more representative local features.

[0067] (I) In the construction of multi-scale context-aware networks: dilated convolutions with different dilation ratios are used to obtain enhanced multi-scale receptive fields, as described in detail below:

[0068] A multi-scale context-aware network is used to learn different visual cues. The initial convolutional layer uses 5*5 convolutional kernels to extract low-level visual features. Starting from the second layer, multi-scale convolutional layers are used to obtain image context information. In each multi-scale convolutional layer, convolutional kernels of different sizes overlap at the same output position. Therefore, dilated convolutions with different dilation ratios are used to obtain an enhanced multi-scale receptive field.

[0069] For the same output position, a convolutional kernel with a larger dilation ratio has a larger receptive field, and only the center position overlaps with other convolutional kernels, which can reduce redundant information between different receptive fields. By embedding the feature map of the current convolutional layer through layer-by-layer convolution operations, different contextual information is integrated together. In this embodiment, the model selects three dilated convolutions with different dilation ratios of 1, 2, and 3 to replace the convolutional filters with kernel sizes of 3*3, 5*5, and 7*7 to obtain contextual information at different scales. The obtained feature maps are concatenated along the channel axis to form the final output of the current convolutional layer, effectively enhancing the visual contextual information.

[0070] (II) Construction of the Graph Convolutional Network Model: A human body analysis model is used to extract mask images of human body parts, obtaining the visual features of each body part. A lookup term embedding represents the pedestrian's attributes. Based on the obtained body part features and the filtered, dimensionality-reduced attribute features, a graph convolutional network is constructed. This graph contains key attribute nodes and body part nodes under the condition of the relevance matrix. Then, the graph convolutional network is used for inference, encoding more representative local features. Specifically, this includes:

[0071] (1) Method for representing body parts nodes

[0072] To obtain the visual representation of pedestrian body parts, the SCHP model is used to predict body part masks, and the body parts are divided into head, upper body, lower body, and arm regions. The parser segments each pedestrian image into N... P Each body part is masked. It means that H k It is a binary mask of the same size as the pedestrian image, where each mask H... k Scaling to the same size as the feature map F, and then applying L1 normalization, results in H′. k Feature map F∈R W×H×D It has W×H positions, and each position i is associated with a feature vector f. i ∈R D Related. The kth body part The calculation is shown in formula (1):

[0073]

[0074] in It is H′ k The scalar value at position i Projected onto a D-dimensional vector.

[0075] (2) Construct a graph network model using key attribute nodes and body part nodes.

[0076] The graph network structure model for feature extraction is constructed using attribute and body part information. This mainly includes the selection of graph nodes, graph construction, and inference of graph convolutional neural networks. First, attribute nodes with low correlation are filtered out based on the correlation between attributes. Then, attributes and body part nodes are constructed into a graph network structure. Finally, inference is performed on the graph network to encode more representative local features.

[0077] 1) Selection of attribute nodes

[0078] In the graph network model, key attributes and body parts are used as nodes. Redundant nodes will affect the network's computational efficiency. Based on the correlation between attributes and between attributes and pedestrians, attribute nodes with low reliability and little impact on the recognition results are filtered out to reduce the dimension of the correlation matrix.

[0079] To preserve key attributes and ensure recognition accuracy, one attribute is removed from each node at a time. The attributes with relatively minor impact are then identified and removed from the graph convolutional network. Based on experimental analysis on the Market-1501 dataset, the following attribute nodes were ultimately removed from the original 27: hat, age, upper body clothing color (including 9 colors), and lower body clothing color (including 8 colors). Eight key attribute nodes with significant impact on the recognition results were retained. Specifically, in this embodiment, these include gender, sleeve length, lower body clothing length, lower body clothing type, hair, backpack, handbag, and other bags.

[0080] 2) Graph Construction

[0081] Define the graph as G = (V, E), with nodes V = {V1, V2, ..., V...} N The total number of nodes is N. G =N A +N P , where N A N represents the number of attributes. P The number of body parts. Each node in the graph represents a pedestrian attribute or body part, using a D-dimensional feature vector x. v Initialize it. Attribute word embedding vector It means that N A D represents the number of attributes. W Let be the dimension of the word embedding vector.

[0082] The graph is derived from the correlation matrix. This means that it contains each edge (V) i V j The weights related to E are denoted as M. The correlation matrix M is specifically represented as follows:

[0083]

[0084] in, A matrix representing the correlation between attributes. This represents a matrix showing the correlation between body parts. This represents a correlation matrix between body parts and attributes. This represents the correlation matrix between attributes and body parts.

[0085] a) Correlation matrix between attributes

[0086] Construct a correlation matrix AA between attributes, where elements AA ij This represents the probability that attribute j will appear when attribute i appears, specifically as follows:

[0087]

[0088] Among them, K i L represents the number of times attribute i appears among pedestrians in the training set. ij This indicates the number of times attribute pairs i and j appear together.

[0089] b. Correlation matrix between body parts

[0090] Assuming that in the training set of pedestrians, the body parts of each pedestrian image can be identified, and all elements in the correlation matrix PP between body parts are set to 1, it can be inferred that if body part i is identified, the probability of identifying body part j is 1.

[0091] c. Correlation matrix between body parts and attributes

[0092] For the correlation matrix PA between body parts and attributes, where elements PA ij PA represents the probability of attribute j appearing when body part i is identified. Typically, some attributes are only related to specific body parts; therefore, if attribute j is not attached to body part i when identifying a body part i, PA... ij As shown in formula (4), otherwise as shown in formula (5):

[0093] PA ij =0 (4)

[0094] PA ij =K j (5)

[0095] Among them, K j It represents the percentage of attributes that appear among pedestrians in the training set.

[0096] d. Correlation matrix between attributes and body parts

[0097] For the correlation matrix AP between attributes and body parts, where the elements AP ij This represents the probability that body part i is identified when attribute j appears. Assuming all body parts of a pedestrian are identifiable in the training set, if attribute j is associated with body part i, then AP... ij As shown in formula (6), otherwise as shown in formula (7):

[0098] AP ij =1 (6)

[0099] APij =0 (7)

[0100] 3) Perform inference computation on graph convolutional networks

[0101] By constructing a graph using body parts and attributes as nodes, reasoning about it, and encoding it into more representative local features, and then connecting the obtained features with global features, the discrimination ability of the pedestrian re-identification network can be effectively improved.

[0102] Graph convolutional networks define a multi-layer propagation process on a graph G. Specifically, each layer in the graph is represented as a function f(X,M), which, under the influence of the correlation matrix M, propagates through the input nodes. Information is propagated between nodes to update their representations, using L (k) This represents the feature matrix after the input node X is passed to the k-th GCN layer.

[0103] Based on node features The corresponding correlation matrix M is taken as input and transformed through the GCN layer. Each GCN layer is represented as follows:

[0104]

[0105] Where, θ (k) For a specific layer, the trainable weight matrix, This is the normalized version of the correlation matrix M.

[0106] Represented as

[0107]

[0108] Where D is a diagonal matrix of M, It is an identity matrix.

[0109] Throughout the inference computation process, the input node features X are fed into the k-th stacked GCN layer, and then the node matrix is ​​output. Obtain subset node V a ∈V, where v a ∈V a For attribute nodes. V a The corresponding output node features are stacked together to form Used to parameterize multi-attribute classifiers for attribute prediction.

[0110] Step 2: Optimize image matching processing based on attribute prediction values: Based on the context-aware network and graph convolutional network constructed in Step 1, feature extraction and attribute prediction are performed on the images in the image library. Attribute predictions with high confidence are regarded as reliable predictions for both the query image and the image library images. Then, candidate images in the image library whose reliable attributes are different from those of the query image are deleted.

[0111] Specifically, a threshold φ is set; when the confidence score is higher than the threshold, the attribute is considered a reliable attribute. When an attribute is determined to be reliable for both the query image and images in the library, it is checked whether the two images have the same prediction for that attribute. If not, the image in the library is deleted. Attribute prediction can filter out most images from the library, effectively reducing the matching range.

[0112] During the image screening process in the image library, pedestrian images with unclear attribute representations may be mistakenly screened, affecting matching accuracy. Therefore, a balance needs to be struck between the values ​​of the threshold φ and other relevant parameters. If the threshold is set to a very small value, most candidate images will be filtered out from the image library, leaving only a few for matching. If the threshold is set to a large value, only some candidate images with reliable attribute predictions that differ from the query image will be deleted, affecting computational efficiency. This invention, through experimental analysis on a Market-1501, selected a threshold value of 0.7.

[0113] The verification process for the method of the present invention is as follows:

[0114] The system uses the PyTorch (1.4.0) deep learning framework, runs on Ubuntu 18.04, has a GeForce RTX 2080 Ti graphics card, an Intel(R) Xeon(R) Gold 6254 CPU @ 3.10GHz processor, and 16GB of RAM. The main parameter settings during network training and optimization are as follows: In training the multi-scale network, dilated convolutions with dilation ratios of 1, 2, and 3 are used, with a batch size of 64 and a learning rate decaying from 0.01 to 0.0001; when constructing the graph network model, the learning rate is 0.0003 and the batch size is 70; in training the person re-identification network, ResNet-50 is used as the baseline network, and the image size for both training and testing is set to 256*128. Data augmentation is performed using random horizontal flipping and random erasure, with a probability of 0.5 for both. The epoch is 110, the batch size is 128, the learning rate is initialized to 0.01, and the learning rate decays to 0.001 for the remaining 20 epochs, with dropout at 0.5; for each attribute, a fully connected layer after Pooling-5 layers is used as the classifier for attribute prediction.

[0115] To verify the rationality of the invented method, ablation experiments were conducted on two benchmark datasets, Market-1501 and DukeMTMC-reID. Three model methods were used for experimental analysis: Baseline, Baseline + Attribute, and Multi-Scale Convolution. The performance evaluation results are shown in Table 1 and Table 2, respectively.

[0116] Table 1. Performance evaluation results of different models on the Market1501 dataset.

[0117]

[0118] Table 2. Performance evaluation results of different models on the DukeMTMC-reID dataset.

[0119]

[0120] Tables 1 and 2 show that the new method improves the Rank-1 accuracy by 0.69% and mAP by 1.56% on the Market-1501 dataset compared to the baseline model, and improves the Rank-1 accuracy by 1.32% and mAP by 1.44% on the DukeMTMC-reID dataset. The results indicate that the baseline model, by introducing attribute information as additional local features, can fully utilize pedestrian identity semantic information to guide recognition. Simultaneously, the method of introducing multi-scale dilated convolution to obtain contextual information enhances fine-grained visual cues, captures information easily overlooked at single scales, ensures that the extracted features are more discriminative, and contributes to improving recognition performance.

[0121] To objectively evaluate the performance of the method of this invention, experiments were conducted on the Market-1501 and DukeMTMC-reID benchmark datasets, along with eight other typical methods, including MLFN, APR, AANet, APDR, AFFNet, PAAN, ACRN, and MultiScale. The results were evaluated based on Rank-1 and mAP metrics. The experimental results are shown in Tables 3 and 4. The superscript in bold in the tables represents the best, better, and second-best results among the compared methods.

[0122] Table 3 Comparison of Algorithm Recognition Performance on the Market1501 Dataset

[0123]

[0124] Table 4 Comparison of algorithm recognition performance on the DukeMTMC-reID dataset

[0125]

[0126] As shown in Tables 3 and 4, compared to the AANet method, which integrates global, local, and attribute features into a single network framework, the method presented in this invention achieves a 0.43% higher Rank-1 score and a 3.79% higher mAP score on the Market-1501 dataset, and a 0.72% higher Rank-1 score and a 1.76% higher mAP score on the DukeMTMC-reID dataset. Furthermore, the performance evaluation results significantly outperform the APR, AANet, and APDR methods that only utilize attribute information, and the MultiScale method that uses multi-scale convolution to obtain contextual information alone. Quantitative evaluation results demonstrate that multi-scale convolution processing can extract richer global features from pedestrian images, and that constructing a graph convolutional network using the correlation between attributes and body parts can obtain highly discriminative local features. This method, by fusing global and local features, learns more diverse pedestrian representations, effectively improving pedestrian re-identification performance.

Claims

1. A method for person re-identification using attribute features to construct a graph convolutional network, characterized in that, Includes the following steps: Step 1: Construct a multi-scale context-aware network to extract global features and a graph convolutional network to infer local features using pedestrian attributes and body parts: In the construction of the multi-scale context-aware network, dilated convolutions with different dilation ratios are used to obtain an enhanced multi-scale receptive field. In the construction of the graph convolutional network model: a human body analysis model is used to extract the mask map of human body parts to obtain the visual features of each body part. A search term is embedded to represent the pedestrian attributes. Based on the obtained body part features and the filtered and dimensionality-reduced attribute features, a graph convolutional network is constructed. This graph contains key attribute nodes and body part nodes under the condition of the correlation matrix. Then, the graph convolutional network is used to infer and encode more representative local features. Step 2: Optimize image matching processing based on attribute prediction values: Based on the context-aware network and graph convolutional network constructed in Step 1, perform feature extraction and attribute prediction on the image library images. Attribute predictions with high confidence are regarded as reliable predictions for the query image and the image library images. Then, delete the image library candidate images whose reliable attributes are different from those of the query image. The construction of the convolutional network model in step one includes the following specific methods: (1) Method for representing body parts nodes The body parts are divided into head, upper body, lower body, and arm regions. Each pedestrian image is segmented into N parts. P Each body part is masked. It means that H k It is a binary mask of the same size as the pedestrian image, where each mask H... k Scaling to the same size as the feature map F, and then applying L1 normalization, results in H. ′ k Feature map F∈R W×H×D It has W×H positions, and each position i is associated with a feature vector f. i ∈R D Related; k-th body part The calculation is shown in formula (1): in It is H ′ k The scalar value at position i Projected onto a D-dimensional vector; (2) Constructing a graph network model using key attribute nodes and body part nodes: including the optimal selection of key attribute nodes, graph construction, and inference of the graph convolutional neural network. ① Selection of attribute nodes Retain 8 key attribute nodes; ② Graph Construction Define the graph as G = (V, E), with nodes V = {V1, V2, ..., V...} N The total number of nodes is N. G =N A +N P , where N A N represents the number of attributes. P The number of body parts; each node in the graph represents a pedestrian attribute or body part, using a D-dimensional feature vector x. v Initialize it; attribute word embedding vector It means that N A D represents the number of attributes. W Let be the dimension of the word embedding vector; The graph is derived from the correlation matrix. This means that it contains each edge (V) i V j The weights related to E; the correlation matrix M is specifically represented as: in, A matrix representing the correlation between attributes. This represents a matrix showing the correlation between body parts. This represents a correlation matrix between body parts and attributes. A matrix representing the correlation between attributes and body parts; ③ Perform inference computation on graph convolutional networks A graph is constructed by using body parts and attributes as nodes. Reasoning is then performed on this graph, which is encoded into more representative local features. The obtained features are then connected with global features.

2. The person re-identification method based on attribute features to construct a graph convolutional network according to claim 1, characterized in that, In the multi-scale context-aware network of step one: the initial convolutional layer uses a 5*5 convolutional kernel to extract the low-level visual features, and from the second layer onwards, multi-scale convolutional layers with different dilation ratios are used to obtain image context information.

3. The person re-identification method based on attribute features to construct a graph convolutional network according to claim 2, characterized in that, The different dilation ratios include three types, which indicate that each convolutional layer has three convolutional kernels, namely convolutional kernels with dilation ratios of 1, 2 and 3.

4. The person re-identification method based on constructing a graph convolutional network using attribute features according to claim 3, characterized in that, The eight key attribute nodes are gender, sleeve length, lower body clothing length, lower body clothing type, hair, backpack, handbag and other bags.

5. The person re-identification method based on attribute features to construct a graph convolutional network according to claim 4, characterized in that, The construction of the graph includes the following specific matrix construction: a) Correlation matrix between attributes Construct a correlation matrix AA between attributes, where elements AA ij This represents the probability that attribute j will appear when attribute i appears, specifically as follows: Among them, K i L represents the number of times attribute i appears among pedestrians in the training set. ij This indicates the number of times attribute pairs i and j appear together; b. Correlation matrix between body parts Assuming that in the training set of pedestrians, the body parts of each pedestrian image can be identified, and all elements in the correlation matrix PP between body parts are set to 1, it can be inferred that if body part i is identified, the probability of identifying body part j is 1. c. Correlation matrix between body parts and attributes For the correlation matrix PA between body parts and attributes, where elements PA ij PA represents the probability of attribute j appearing when body part i is identified. Some attributes are only related to specific body parts; therefore, if attribute j is not attached to body part i when identifying a body part i, then PA... ij As shown in formula (4), otherwise as shown in formula (5): PA ij =0 (4) PA ij =K j (5) Among them, K j It represents the percentage of attributes that appear among pedestrians in the training set. d. Correlation matrix between attributes and body parts For the correlation matrix AP between attributes and body parts, where elements AP ij Let $\mathbf{j}$ represent the probability that body part $i$ is identified when attribute $j$ is present. Assuming all body parts of a pedestrian are identifiable in the training set, if attribute $j$ is associated with body part $i$, then $\mathbf{j}$ is the probability that $i$ is identified. ij As shown in formula (6), otherwise as shown in formula (7): AP ij =1 (6) AP ij =0 (7)。 6. The person re-identification method based on attribute features to construct a graph convolutional network according to claim 5, characterized in that, The specific process of inference computation for the graph convolutional network described in step ③ is as follows: Graph convolutional networks define a multi-layer propagation process on a graph G. Specifically, each layer in the graph is represented as a function f(X,M), which, under the influence of the correlation matrix M, propagates through the input nodes. Information is propagated between nodes to update node representations, using L (k) This represents the feature matrix after the input node X is passed to the k-th GCN layer; Based on node features The corresponding correlation matrix M is taken as input and transformed through the GCN layer. Each GCN layer is represented as follows: Where, θ (k) For a specific layer, the trainable weight matrix, This is the normalized version of the correlation matrix M; Represented as Where D is a diagonal matrix of M, It is the identity matrix; The input node features X are fed into the k-th stacked GCN layer, and then the output node matrix is ​​obtained. Obtain subset node V a ∈V, where v a ∈V a For attribute nodes, V a The corresponding output node features are stacked together to form 7. The person re-identification method according to claim 6, which utilizes attribute features to construct a graph convolutional network, is characterized in that... In step two, attributes with high confidence are determined by setting a threshold φ. When the confidence score is higher than the threshold, the attribute is considered a reliable attribute. When it is determined that an attribute is reliable for both the query image and the image in the library, it is checked whether the two images have the same prediction for the attribute. If not, the image in the library is deleted from the library.

8. The person re-identification method according to claim 7, which utilizes attribute features to construct a graph convolutional network, is characterized in that... In step two, the threshold is set to 0.7.

Images