A pedestrian re-identification method, a terminal device, and a storage medium
By combining DenseNet and graph convolutional networks with feature pyramid networks and multi-head attention mechanisms, the problems of limited global representation capabilities and semantic misalignment in person re-identification are solved, thereby improving recognition accuracy.
Patent Information
- Application Number
- CN202211308979.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-25
- Publication Date
- 2025-12-30
- Estimated Expiration
- 2042-10-25
AI Technical Summary
Existing deep learning methods lack attention to local differences in person re-identification tasks, and cannot effectively solve the misalignment problem caused by factors such as pose changes, viewpoint differences, background clutter and occlusion, resulting in limited global representation capabilities and semantic misalignment.
The DenseNet network is used to extract multi-channel features, and the basic convolutional neural network and graph convolutional network are combined to extract local features. High-level semantic features are extracted through feature pyramid network and multi-head attention mechanism. Clustering algorithm is used to group them into superpixels to build a pedestrian re-identification network model.
It improves global representation capabilities, reduces semantic misalignment errors caused by human body spatial misalignment, and improves the accuracy of pedestrian re-identification.
Smart Images
Figure CN115690840B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision, and more particularly to a pedestrian re-identification method, terminal device, and storage medium. Background Technology
[0002] The purpose of person re-identification is to use computer vision technology to retrieve the presence of a specific pedestrian in an image or video sequence. Person re-identification is a crucial component of intelligent surveillance systems. Early person re-identification tasks consisted of two parts: feature learning and metric learning. Feature learning was used to extract invariant features representing people of the same identity, while metric learning was used to measure the similarity between images.
[0003] Pedestrian re-identification aims to capture images of the same person at different physical locations. This task is challenging due to factors such as significant variations in pose and viewpoint, imperfect person detection, cluttered backgrounds, occlusion, and lighting variations. Some deep learning methods lack attention to local differences and lack explicit mechanisms to address misalignment, limiting their ability to represent global aspects. Furthermore, pose-based body part alignment fails to achieve satisfactory results. Due to spatial misalignment of the human body, the same spatial location does not correspond to the same semantic meaning. Summary of the Invention
[0004] To address the aforementioned problems, this invention proposes a pedestrian re-identification method, a terminal device, and a storage medium.
[0005] The specific plan is as follows:
[0006] A pedestrian re-identification method includes the following steps:
[0007] S1: Collect an image dataset consisting of pedestrian images corresponding to different people. At the same time, collect different images of the same person as the images in the image dataset as pedestrian images to be queried. Label the images of the same person in the image dataset that are the pedestrian images to be queried. Construct a training set based on the labeled pedestrian images to be queried and the image dataset.
[0008] S2: Construct a pedestrian re-identification network model, taking the image dataset and the image of the pedestrian to be queried as the input of the model, and the output of the model is the same person image in the image dataset as the pedestrian image to be queried; train the model using the training set;
[0009] The implementation process of the model includes:
[0010] S21: Extract multi-channel features from each image in the image dataset;
[0011] S22: Extract local features from each image in the image dataset;
[0012] S23: Merge multi-channel features and local features into global features, and construct a feature library for the image dataset from the global features of all images in the image dataset;
[0013] S24: Extract high-level semantic features from the pedestrian image to be queried;
[0014] S25: Calculate the similarity between the high-level semantic features of the pedestrian image to be queried and the global features of each image in the image dataset, and take the image in the image dataset with the highest similarity as the same person image of the pedestrian image to be queried;
[0015] S3: Recognize the pedestrian image to be queried using the trained pedestrian re-identification network model.
[0016] Furthermore, the multi-channel feature extraction process in step S21 includes the following steps:
[0017] S211: Input the image into the DenseNet network to obtain the image's feature map;
[0018] S212: Input the feature map into the max pooling layer to obtain the first feature vector;
[0019] S213: Input the feature map into the average pooling layer to obtain the second feature vector;
[0020] S214: Add the first feature vector to the second feature vector to obtain the multi-channel feature.
[0021] Furthermore, the local feature extraction process in step S22 includes the following steps:
[0022] S221: Crop each image evenly into multiple sub-images;
[0023] S222: Input the subgraph into the base convolutional neural network to obtain the feature map of each subgraph;
[0024] S223: Input the feature map of the subgraph into the max pooling layer to obtain the subgraph-level features, and input the feature map of the subgraph into the multilayer perceptron to obtain the correlation matrix of the subgraph;
[0025] S224: After inputting the subgraph-level features and the correlation matrix of the subgraphs into the graph convolutional neural network, the relationship between the subgraphs is obtained;
[0026] S225: Input the relationships between subgraphs into a multilayer perceptron to obtain local features of the image.
[0027] Furthermore, the extraction process of high-level semantic features in step S24 includes the following steps:
[0028] S241: Extract the 5-dimensional feature vector of the pedestrian image to be queried;
[0029] S242: Inputting the 5-dimensional feature vector into the feature pyramid network yields the semantic features of the pedestrian image to be queried;
[0030] S243: Based on the semantic features of the pedestrian images to be queried, a clustering algorithm is used to group the images into superpixels;
[0031] S244: Based on the superpixel grouping results, a multi-head attention mechanism is used to extract high-level semantic features of the pedestrian image to be queried.
[0032] Furthermore, the five dimensions in step S241 include two coordinates in the coordinate system and the three primary colors.
[0033] A pedestrian re-identification terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described above in the embodiments of the present invention.
[0034] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above in the embodiments of the present invention.
[0035] The present invention adopts the above technical solution, which improves the problem of the limitation of global representation ability in the current deep learning pedestrian re-identification network, and at the same time improves the problem of semantic misalignment caused by spatial misalignment of human body, thus causing errors. Attached Figure Description
[0036] Figure 1 The diagram shown is a flowchart of a method according to an embodiment of the present invention.
[0037] Figure 2 The diagram shows a flowchart of the model implementation process in Embodiment 1 of the present invention. Detailed Implementation
[0038] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention.
[0039] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.
[0040] Example 1:
[0041] This invention provides a pedestrian re-identification method, such as... Figure 1 As shown, the method includes the following steps:
[0042] S1: Collect an image dataset X = {I1, I2, ..., I...} consisting of pedestrian images corresponding to different individuals. N}, where N represents the number of images in the input image dataset X. At the same time, different images belonging to the same person as the images in the image dataset are collected as the pedestrian images to be queried. The images of the same person in the image dataset are labeled. A training set is constructed based on the labeled pedestrian images to be queried and the image dataset.
[0043] S2: Construct a pedestrian re-identification network model, taking the image dataset and the image of the pedestrian to be queried as input to the model, and the output of the model is the same person as the image of the pedestrian to be queried in the image dataset. Train the model using the training set.
[0044] like Figure 2 As shown, the implementation process of the model includes:
[0045] S21: Extract multi-channel features from each image in the image dataset.
[0046] The multi-channel feature extraction process in this embodiment includes the following steps:
[0047] S211: Input image I into the DenseNet network to obtain the feature map F of the image. M =DenseNet(Ι), H represents the height of the image, W represents the width of the image, and d represents the number of feature channels.
[0048] S212: Input the feature map into the max pooling layer to obtain the first feature vector F Max =Maxpooling(F M ).
[0049] S213: Input the feature map into the average pooling layer to obtain the second feature vector F. Avg =Avgpooling(F M ).
[0050] S214: Transfer the first eigenvector F Max With the second eigenvector F Avg Adding them together yields multi-channel features.
[0051] S22: Extract local features from each image in the image dataset.
[0052] The local feature extraction process in this embodiment includes the following steps:
[0053] S221: Uniformly crop each image I into multiple sub-images I = {s1, s2, ..., s...} k}, h represents the height of the subgraph, w represents the width of the subgraph, and k represents the number of subgraphs.
[0054] S222: Input the subgraph into the underlying convolutional neural network to obtain the feature map f of each subgraph. M =CNN(s k ).
[0055] S223: Input the feature map of the subgraph into the max pooling layer to obtain the subgraph-level feature f. k =Maxpooling(f M ), The feature map of the subgraph is input into a multilayer perceptron to obtain the correlation matrix of the subgraph.
[0056] S224: After inputting the subgraph-level features and the correlation matrix of the subgraphs into the graph convolutional neural network, the association relationship F between the subgraphs is obtained. R =GCN(f k M).
[0057] S225: Establish the association relationships between subgraphs F R The local features F of image I are obtained by inputting the multilayer perceptron. local =MLP(F R ).
[0058] S23: Multi-channel feature F merge and local features F local Merge into global feature F global =Concat(F merge ,F local This involves constructing a feature library for the image dataset from the global features of all images in the dataset.
[0059] S24: Extract high-level semantic features from the pedestrian image to be queried.
[0060] The extraction process of high-level semantic features in this embodiment includes the following steps:
[0061] S241: Extract the pedestrian image to be queried I q The 5-dimensional feature vector, in this embodiment, includes two coordinates (x, y) in the coordinate system and three primary colors (R, G, B).
[0062] S242: After inputting the 5-dimensional feature vector into the feature pyramid network, the pedestrian image to be queried is obtained. q semantic features F sem =FPN(F a ).
[0063] S243: Based on the pedestrian image to be queried I qsemantic features F sem The image is grouped into superpixel P using a clustering algorithm. super .
[0064] The K-means algorithm is used for clustering in this embodiment.
[0065] S244: Based on superpixel P super The grouping results are used to extract high-level semantic features F from the pedestrian images to be queried using a multi-head attention mechanism. q =MHAttention(P super ).
[0066] S25: Calculate the similarity between the high-level semantic features of the pedestrian image to be queried and the global features of each image in the image dataset, and take the image in the image dataset with the highest similarity as the same person image of the pedestrian image to be queried.
[0067] The model loss function in this embodiment Using triplet loss and cross-entropy loss Right now Where 0 < λ < 1 represents the weighting coefficient.
[0068] Triple loss The calculation formula is:
[0069]
[0070] in, This represents the distance metric. Representative and F q Positive samples belonging to the same person Representative and F q Negative samples belonging to different individuals, where m represents the marginal parameter.
[0071] The cross-entropy loss consists of two parts: the first cross-entropy loss and the second cross-entropy loss, wherein:
[0072] The first cross-entropy loss is used to calculate the accuracy of multi-channel feature extraction for each image in the image dataset. To calculate the first cross-entropy loss, the model also includes: processing the multi-channel features F... merge The classification probability of the image is obtained by inputting it into the fully connected layer and then using the Softmax function.
[0073] The second cross-entropy loss is used to calculate the accuracy of identifying the same person in the pedestrian image to be queried.
[0074] S3: Recognize the pedestrian image to be queried using the trained pedestrian re-identification network model.
[0075] This invention improves upon the limitation of global representation capabilities in current deep learning pedestrian re-identification networks by merging multi-channel features and local features to obtain global features, and by combining the global features of each image to form the feature library of the image dataset. Furthermore, this invention extracts semantic features of the query pedestrian image through a feature pyramid network, groups the image into superpixels using a clustering algorithm, and then utilizes multi-head attention to extract high-level semantic features. Therefore, this invention addresses the problem of errors caused by semantic misalignment due to spatial displacement of the human body.
[0076] Example 2:
[0077] The present invention also provides a pedestrian re-identification terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the method embodiment described above in Embodiment 1 of the present invention.
[0078] Furthermore, as an executable solution, the pedestrian re-identification terminal device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The pedestrian re-identification terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described composition of the pedestrian re-identification terminal device is merely an example and does not constitute a limitation on the pedestrian re-identification terminal device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the pedestrian re-identification terminal device may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.
[0079] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the pedestrian re-identification terminal device, connecting all parts of the entire pedestrian re-identification terminal device via various interfaces and lines.
[0080] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the pedestrian re-identification terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0081] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.
[0082] If the modules / units integrated in the pedestrian re-identification terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.
[0083] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.
Claims
1. A pedestrian re-identification method, characterized in that, The method comprises the following steps: S1: collecting an image dataset composed of pedestrian images corresponding to different persons, collecting different images of the same person as the image dataset as a to-be-queried pedestrian image, labeling the to-be-queried pedestrian image in the same person image of the image dataset, and constructing a training set based on the labeled to-be-queried pedestrian image and the image dataset; S2: constructing a pedestrian re-identification network model, taking the image dataset and the to-be-queried pedestrian image as inputs of the model, and taking the same person image of the to-be-queried pedestrian image in the image dataset as an output of the model; training the model through the training set; The implementation process of the model comprises: S21: extracting multi-channel features of each image in the image dataset; S22: extracting local features of each image in the image dataset; S23: merging the multi-channel features and the local features into global features, and constructing the global features of all images in the image dataset into a feature library of the image dataset; S24: extracting high-level semantic features of the to-be-queried pedestrian image; S25: calculating the similarity between the high-level semantic features of the to-be-queried pedestrian image and the global features of each image in the image dataset, and taking the image in the image dataset with the highest similarity as the same person image of the to-be-queried pedestrian image; S3: identifying the to-be-queried pedestrian image through the trained pedestrian re-identification network model; The extraction process of the multi-channel features in step S21 comprises the following steps: S211: inputting the image into a DenseNet network to obtain a feature map of the image; S212: inputting the feature map into a max-pooling layer to obtain a first feature vector; S213: inputting the feature map into an average-pooling layer to obtain a second feature vector; S214: adding the first feature vector and the second feature vector to obtain the multi-channel features; The extraction process of the local features in step S22 comprises the following steps: S221: uniformly cutting each image into a plurality of subgraphs; S222: inputting the subgraphs into a basic convolutional neural network to obtain a feature map of each subgraph; S223: inputting the feature map of the subgraph into a max-pooling layer to obtain a subgraph-level feature, and inputting the feature map of the subgraph into a multi-layer perceptron to obtain a correlation matrix of the subgraph; S224: inputting the subgraph-level feature and the correlation matrix of the subgraph into a graph convolutional neural network to obtain the association relationship between the subgraphs; S225: inputting the association relationship between the subgraphs into a multi-layer perceptron to obtain the local features of the image; The extraction process of the high-level semantic features in step S24 comprises the following steps: S241: extracting a 5-dimensional feature vector of the to-be-queried pedestrian image; S242: inputting the 5-dimensional feature vector into a feature pyramid network to obtain semantic features of the to-be-queried pedestrian image; S243: grouping the image into superpixels based on the semantic features of the to-be-queried pedestrian image by using a clustering algorithm; S244: extracting high-level semantic features of the to-be-queried pedestrian image based on the superpixel grouping result by using a multi-head attention mechanism. 2.The pedestrian re-identification method of claim 1, wherein: The 5-dimensional features in step S241 comprise two coordinates and three primary colors in a coordinate system. 3.The pedestrian re-identification method of claim 1, wherein: The loss function used in the model training process comprises a triplet loss and a cross-entropy loss. 4.A pedestrian re-identification terminal device, characterized in that: A computer program product comprising a computer readable medium having stored thereon instructions that, when executed by a processor, cause the processor to execute the steps of the method according to any one of claims 1 to 3.
5. A computer readable storage medium storing a computer program, characterized in that: The computer program is executed by a processor to implement the steps of the method according to any one of claims 1 to 3.