Pedestrian feature recognition method based on multi-region attention correlation spatial relationship model
By using a spatial relationship model with multi-region attention association and leveraging the ResNet50 network and spatial adaptive graph convolution model, the problems of single local feature segmentation and insufficient spatial relationship in pedestrian re-identification are solved, thereby improving the accuracy and robustness of pedestrian feature recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN TEXTILE UNIV
- Filing Date
- 2022-04-28
- Publication Date
- 2026-05-12
AI Technical Summary
Existing pedestrian re-identification methods use a single standard to classify local features, resulting in a lack of diversity, confusion of pedestrian features with similar attributes, and insufficient construction of spatial relationships between local features, which affects the recognition effect.
A spatial relationship model based on multi-region attention association is adopted. Global features are extracted through ResNet50 network, a spatial attention model is constructed for local feature enhancement, and a spatial adaptive graph convolution model is used for feature fusion and association. The adjacency relationship matrix is calculated to suppress background interference, and a residual network is used to improve feature representation.
It effectively suppresses background information interference, improves pedestrian re-identification, enhances feature expression capabilities, and strengthens recognition accuracy.
Smart Images
Figure CN115188019B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of feature extraction technology, and in particular to a pedestrian feature recognition method based on a spatial relationship model of multi-region attention association. Background Technology
[0002] In recent years, feature extraction methods combining local and global approaches have become a popular trend and have received considerable attention in the field of person re-identification, achieving good results. However, it is not difficult to find that the local feature segmentation in this method often adopts a single standard. This single segmentation standard often leads to a lack of diversity in the grouping of local features, thus confusing different pedestrians with similar attributes. Furthermore, existing research has shown that constructing spatial relationships between local features can also enhance the representation of pedestrian features. Therefore, we propose a pedestrian feature recognition method based on a spatial relationship model of multi-region attention association. Summary of the Invention
[0003] The purpose of this invention is to address the shortcomings of existing technologies by proposing a pedestrian feature recognition method based on a spatial relationship model with multi-region attention association.
[0004] To achieve the above objectives, the present invention adopts the following technical solution:
[0005] A pedestrian feature recognition method based on a spatial relationship model of multi-region attention association is described below:
[0006] (1) Extract global features of pedestrian images: Import the collected pedestrian images into the ResNet50 network, and extract global features of each group of pedestrian images through the ResNet50 network;
[0007] (2) Obtaining local feature information and performing data augmentation: The global features are divided into multiple channels to obtain local features, a spatial attention model is constructed, and the extracted local features are augmented through the spatial attention model.
[0008] (3) Spatial association of local features in each group: Construct a spatial adaptive graph convolution model and import the local features in each group into the spatial adaptive graph convolution model. Then, perform feature fusion on the local features in each group and perform spatial association on the fused local features.
[0009] (4) Perform measurement calculations based on the feature information of each group: Collect the local features and global features that are the same as the original input, and perform measurement calculations between different pedestrians based on the acquired features.
[0010] As a further aspect of the present invention, the specific steps for local feature extraction in step (1) are as follows:
[0011] Step 1: The ResNet50 network performs coarse feature extraction on the pedestrian image to obtain a three-dimensional feature vector of W×H×C, where W and H represent the width and height of each local feature, respectively, and C represents the number of channels of each local feature.
[0012] Step 2: Divide the obtained feature map horizontally into multiple local feature regions according to different averaging criteria, and group the local feature regions obtained under the same criteria into the same group.
[0013] As a further aspect of the present invention, the specific steps of feature enhancement in step (2) are as follows:
[0014] Step 1: Collect the local features of each segmented group and import the collected local features into the spatial attention model;
[0015] The second step: The spatial attention model learns the attention mask by constructing an adjacency matrix, and uses an attention mechanism to extract pedestrian interest regions, and trains and enhances each group of local features based on the extracted information.
[0016] As a further aspect of the present invention, the specific calculation formula for the local feature enhancement process of the attention mechanism in the second step is as follows:
[0017] W l′ =W l ⊙M (1)
[0018] In the formula, W l For local features, M corresponds to the mask matrix for assigning attention weights, and W... l′ This represents the feature representation after attention enhancement.
[0019] As a further aspect of the present invention, the specific steps of local feature fusion in step (3) are as follows:
[0020] S1.1: The spatial adaptive graph convolutional model receives attention-weighted local features from each group and constructs the input feature set V. l in ∈R B×C×W×h Where B represents the number of local features in each group, and h represents the height of the spatial hierarchy of the local features;
[0021] S1.2: Transform each set of input features by global max pooling to obtain the most salient features in the original local information, and then input the transformed input feature set into the submodule OVSR in B steps;
[0022] S1.3: Enable each local feature to learn information from other local features. At the same time, divide the transformed input feature set into a baseline feature and other neighbor features. Perform global average pooling on the other neighbor features, and then perform convolution on the upper and lower branches and concatenate them.
[0023] S1.4: The idea of residual network is used to fuse the baseline local features with the spliced associated features to obtain local features enhanced by shallow relationships.
[0024] As a further aspect of the present invention, the specific steps of spatial association in step (3) are as follows:
[0025] S2.1: Dimensionality reduction and compression are performed on each group of local features enhanced by shallow relationships. At the same time, global features are obtained by processing the global pedestrian image through ResNet50 network and spatial attention model, and then imported into spatial adaptive graph convolution model.
[0026] S2.2: Perform global max pooling and dimensionality reduction compression on the global features, use the local feature information and global feature information of each group as the endpoints in the feature graph relationship, and perform amplification processing on each group of feature information to obtain multiple adjacency relationship matrices;
[0027] S2.3: Calculate the difference between each group's adjacency relation matrix and the global information feature map, eliminate irrelevant interference based on this difference, update the corresponding adjacency matrix, and then perform dimensionality reduction on the adjacency matrix by taking the absolute value, regularization, and full connection to obtain an adaptive adjacency matrix. A predefined relation matrix is then introduced to correct the adaptive adjacency matrix. The predefined form of the relation matrix is as follows:
[0028]
[0029] Among them, the elements corresponding to the relation matrix A are the relation information between different features;
[0030] S2.4: The weights of the relation matrix are updated through backpropagation. The adaptive adjacency matrix is multiplied by the predefined adjacency matrix to generate the final weight matrix. At the same time, the original local information and the weighted feature information are fused to obtain the corresponding two-dimensional feature matrix. The two-dimensional feature matrix is then augmented using the following formula:
[0031]
[0032]
[0033] Here, f1, f2, and f3 are three fully connected layers, respectively. These are the outputs of the final local and global features of the adaptive graph convolution module, respectively.
[0034] As a further aspect of the present invention, the specific formula for the measurement calculation in step (4) is as follows:
[0035]
[0036] In the formula, represents the cross-entropy loss and triplet loss of the i-th feature, respectively.
[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0038] Compared to previous methods, this pedestrian feature recognition method based on a spatial relationship model with multi-region attention association differs from previous methods. It imports collected local features into a spatial attention model, which uses an attention mechanism to extract pedestrian interest regions. The extracted information is then used to train and enhance each set of local features. Simultaneously, a spatial adaptive graph convolutional model is constructed to allow each local feature to initially learn information from other local features. Then, the idea of a residual network is used to fuse the baseline local features with the concatenated associated features to obtain local features enhanced by shallow relationships. The difference between each set of adjacency relationship matrices and the global information feature map is calculated, and irrelevant interference is eliminated based on this difference, updating the corresponding adjacency matrices. A predefined relationship matrix is introduced to correct the adaptive adjacency matrix, and the weights of the relationship matrix are updated through backpropagation. This method can extract deeper spatial dimension features of the human body, significantly suppressing background information interference and effectively ensuring better pedestrian re-identification results. Attached Figure Description
[0039] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.
[0040] Figure 1 This is a flowchart of the pedestrian feature recognition method based on a spatial relationship model with multi-region attention association proposed in this invention.
[0041] Figure 2 This is a flowchart of the multi-channel attention mechanism of the pedestrian feature recognition method based on a spatial relationship model of multi-region attention association proposed in this invention. Detailed Implementation
[0042] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0043] Reference Figure 1 A pedestrian feature recognition method based on a spatial relationship model of multi-region attention association is proposed. The specific steps of this recognition method are as follows:
[0044] Extracting global features from pedestrian images: The collected pedestrian images are imported into the ResNet50 network, and global features are extracted from each group of pedestrian images through the ResNet50 network.
[0045] Specifically, the ResNet50 network performs coarse feature extraction on pedestrian images to obtain a three-dimensional feature vector of W×H×C, where W and H represent the width and height of each local feature, respectively, and C represents the number of channels for each local feature. The obtained feature map is horizontally divided into multiple local feature regions according to different averaging criteria, and local feature regions obtained under the same criteria are grouped into the same group.
[0046] Acquire local feature information and perform data augmentation: Use a multi-channel equal distribution method to acquire local features from global features, construct a spatial attention model, and use this spatial attention model to augment the extracted local features.
[0047] Specifically, such as Figure 2 As shown, the computer collects the local features of each segmented group and imports them into the spatial attention model. The spatial attention model learns the attention mask by constructing an adjacency matrix and uses an attention mechanism to extract the pedestrian's region of interest. Based on the extracted information, it trains and enhances each group of local features.
[0048] It should be further explained that the specific calculation formula for the enhancement of local features by the attention mechanism is as follows:
[0049] W l′ =W l ⊙M (1)
[0050] In the formula, W l For local features, M corresponds to the mask matrix for assigning attention weights, and W... l′ This represents the feature representation after attention enhancement.
[0051] Spatial correlation of local features in each group: Construct a spatial adaptive graph convolution model, import the local features in each group into the spatial adaptive graph convolution model, then perform feature fusion on the local features in each group, and perform spatial correlation on the fused local features.
[0052] The spatial adaptive graph convolutional model receives attention-weighted local features from each group and constructs an input feature set V. lin ∈R B×C×W×h Here, B represents the number of local features in each group, and h represents the spatial hierarchy of the local features. Then, global max pooling is used to transform the input feature sets of each group to obtain the most salient features in the original local information. The transformed input feature sets are then input into the submodule OVSR B times, so that each local feature can learn the information of other local features. The transformed input feature sets are divided into baseline features and other neighbor features. Global average pooling is performed on the other neighbor features. Then, convolution is performed on the upper and lower branches and they are concatenated. Finally, the idea of residual network is used to fuse the baseline local features and the concatenated associated features to obtain local features enhanced by shallow relationships.
[0053] Specifically, the local features of each group enhanced by shallow relationships are reduced in dimension and compressed. Simultaneously, the global pedestrian image is processed by a ResNet50 network and a spatial attention model to obtain global features. Global max pooling and dimensionality reduction are then performed on the global features using a spatial adaptive graph convolution model. The local and global feature information are used as endpoints in the feature map relationship, and each group of feature information is augmented to obtain multiple adjacency relation matrices. The difference between each adjacency relation matrix and the global information feature map is calculated, and irrelevant interference is eliminated based on this difference, and the corresponding adjacency matrix is updated. Then, the adjacency matrix is reduced in dimension by taking absolute values, regularization, and full connectivity to obtain an adaptive adjacency matrix. A predefined relation matrix is introduced to correct the adaptive adjacency matrix, and the weights of the relation matrix are updated through backpropagation. The adaptive adjacency matrix is multiplied by the predefined adjacency matrix to generate the final weight matrix. Simultaneously, the original local information and the weighted feature information are fused to obtain the corresponding two-dimensional feature matrix, which is then augmented in dimension.
[0054] It should be further noted that its relation matrix is predefined in the following form:
[0055]
[0056] Among them, the elements corresponding to the relation matrix A are the relation information between different features;
[0057] The specific dimensionality enhancement formula is as follows:
[0058]
[0059]
[0060] Here, f1, f2, and f3 are three fully connected layers, respectively. These are the outputs of the final local and global features of the adaptive graph convolution module, respectively.
[0061] Measurement calculations are performed based on the feature information of each group: the number of local and global features obtained in the end is the same as the number of original inputs, and measurement calculations are performed between different pedestrians based on the obtained features.
[0062] Specifically, the formula for the measurement calculation is as follows:
[0063]
[0064] In the formula, represents the cross-entropy loss and triplet loss of the i-th feature, respectively.
[0065] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A pedestrian feature recognition method based on a spatial relationship model of multi-region attention association, characterized in that, The specific steps of this identification method are as follows: (1) Extract global features of pedestrian images: Import the collected pedestrian images into the ResNet50 network, and extract global features of each group of pedestrian images through the ResNet50 network; (2) Obtain local feature information and perform data augmentation: Use the multi-channel equal distribution method to obtain local features from global features, construct a spatial attention model, and use the spatial attention model to perform feature augmentation on each group of extracted local features; (3) Spatial association of local features in each group: Construct a spatial adaptive graph convolution model and import the local features in each group into the spatial adaptive graph convolution model. Then, perform feature fusion on the local features in each group and perform spatial association on the fused local features. (4) Perform measurement calculations based on the feature information of each group: collect the same number of local and global features as the original input, and perform measurement calculations between different pedestrians based on the acquired features; The specific steps for local feature extraction described in step (1) are as follows: Step 1: The ResNet50 network performs coarse feature extraction on the pedestrian image to obtain a three-dimensional feature vector of W×H×C, where W and H represent the width and height of each local feature, respectively, and C represents the number of channels of each local feature. Step 2: Divide the obtained feature map horizontally into multiple local feature regions according to different averaging criteria, and group the local feature regions obtained under the same criteria into the same group; The specific steps of feature enhancement described in step (2) are as follows: Step 1: Collect the local features of each segmented group and import the collected local features into the spatial attention model; The second step: The spatial attention model learns the attention mask by constructing an adjacency matrix, and uses an attention mechanism to extract pedestrian interest regions, and trains and enhances each group of local features based on the extracted information; The specific steps of local feature fusion in step (3) are as follows: S1.1: The spatial adaptive graph convolutional model receives attention-weighted local features from each group and constructs the input feature set. Where B represents the number of local features in each group, and h represents the height of the spatial hierarchy of the local features; S1.2: Transform each set of input features by global max pooling to obtain the most salient features in the original local information, and then input the transformed input feature set into the submodule OVSR in B steps; S1.3: Enable each local feature to learn information from other local features. At the same time, divide the transformed input feature set into a baseline feature and other neighbor features. Perform global average pooling on the other neighbor features, and then perform convolution on the upper and lower branches and concatenate them. S1.4: The idea of residual networks is used to fuse the baseline local features with the concatenated associated features to obtain local features enhanced by shallow relationships; The specific steps of spatial association described in step (3) are as follows: S2.1: Dimensionality reduction and compression are performed on each group of local features enhanced by shallow relationships. At the same time, global features are obtained by processing the global pedestrian image through ResNet50 network and spatial attention model, and then imported into spatial adaptive graph convolution model. S2.2: Perform global max pooling and dimensionality reduction compression on the global features, use the local feature information and global feature information of each group as the endpoints in the feature graph relationship, and perform amplification processing on each group of feature information to obtain multiple adjacency relationship matrices; S2.3: Calculate the difference between each group's adjacency relation matrix and the global information feature map, eliminate irrelevant interference based on this difference, update the corresponding adjacency matrix, and then perform dimensionality reduction on the adjacency matrix by taking the absolute value, regularization, and full connection to obtain an adaptive adjacency matrix. A predefined relation matrix is then introduced to correct the adaptive adjacency matrix. The predefined form of the relation matrix is as follows: （2） Among them, the relation matrix The corresponding elements represent the relationship information between different features; S2.4: The weights of the relation matrix are updated through backpropagation. The adaptive adjacency matrix is multiplied by the predefined adjacency matrix to generate the final weight matrix. At the same time, the original local information and the weighted feature information are fused to obtain the corresponding two-dimensional feature matrix. The two-dimensional feature matrix is then augmented using the following formula: （3） （4） in, , and Each of the three fully connected layers, , These are the outputs of the final local and global features of the adaptive graph convolution module, respectively.
2. The pedestrian feature recognition method based on a spatial relationship model with multi-region attention association as described in claim 1, characterized in that, The specific calculation formula for the local feature enhancement process of the attention mechanism described in the second step is as follows: （1） In the formula, where For local features, The mask matrix corresponding to the attention weights. This represents the feature representation after attention enhancement.
3. The pedestrian feature recognition method based on a spatial relationship model of multi-region attention association according to claim 1, characterized in that, The specific formula for the measurement calculation in step (4) is as follows: （5） In the formula, , represents the cross-entropy loss and triplet loss of the i-th feature, respectively.