An efficient pedestrian image re-identification method
By using a classification neural network branch network to process pedestrian image features, the problems of high computational complexity and feature mismatch in existing technologies are solved, and efficient and accurate pedestrian re-identification is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU FABU TECH CO LTD
- Filing Date
- 2023-02-28
- Publication Date
- 2026-05-15
AI Technical Summary
Existing pedestrian re-identification methods have high computational complexity, lose global information, and suffer from feature mismatch issues.
A classification neural network branch network is used to process pedestrian image features. By combining global max pooling and global average pooling operations with local feature extraction from vertical segmentation, fused features are constructed and Euclidean distance is calculated for similarity judgment.
With moderate computational complexity, it can accurately extract global and local features, significantly improve re-identification accuracy, reduce dependence on detection models, and ensure the accuracy of feature matching in extreme cases.
Smart Images
Figure CN116778523B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an image re-identification method in the field of computer vision technology, and more specifically, to an efficient pedestrian image re-identification method. Background Technology
[0002] With the development of concepts such as intelligent driving, smart cities, and new retail, pedestrian re-identification has also developed rapidly, playing a decisive role in tasks such as pedestrian tracking, pedestrian retrieval, cross-camera pedestrian association, and pedestrian trajectory prediction.
[0003] Local feature models are currently the most mainstream method, but it has been found that existing related methods all suffer from high computational complexity, feature mismatch, or missing features.
[0004] PCB (Beyond Part Models: Person Retrieval with Refined Part Pooling, ECCV 2018). A single branch is vertically split into 6 blocks, and then average pooling is performed. This approach may result in feature mismatch (vertical splitting into 6 blocks may lead to feature mismatch) and loss of global information (all features are local features from vertical splitting, lacking global features).
[0005] MGN (Learning Discriminative Features with Multiple Granularities for Person Re-Identification, MM 2018) divides features into three branches, each vertically split into 1, 2, or 3 blocks, and then performs max pooling. This approach has high computational complexity (due to the large number of branches), may result in feature mismatches (vertical splitting into three blocks may lead to incorrect feature matching), and loses global information (max pooling only takes the feature at the position of the maximum response, losing other information).
[0006] PPA (Person re-identification with part prediction alignment, CVIU2021) vertically divides a single branch into 6 blocks and performs max pooling on them, while also performing max pooling on the undivided features. This approach may result in feature mismatch (vertical division into 6 blocks may lead to feature mismatch) and loss of global information (max pooling only takes the features at the position of the maximum response, losing other information).
[0007] PH-GCN (PH-GCN: Person Retrieval With Part-Based Hierarchical Graph Convolutional Network, TMM 2022) divides features into three branches, each of which is then vertically divided into 1, 3, or 6 blocks. Average pooling is then performed before connecting the segments to a graph neural network. This approach has high computational complexity (due to the numerous branches and the additional graph neural network) and may suffer from feature mismatches (vertical division into 3 or 6 blocks may lead to incorrect feature matching).
[0008] Therefore, existing re-identification methods suffer from problems such as high computational complexity, loss of global information, and potential feature mismatch. Summary of the Invention
[0009] To address the problems existing in the background technology, this invention proposes a simple and efficient pedestrian re-identification structure. This structure has moderate computational complexity, can extract global and local features simultaneously, and solves the feature mismatch problem in previous methods, resulting in a significant improvement in accuracy.
[0010] The technical solution adopted in this invention is:
[0011] 1) Establish a classification neural network for pedestrian image re-identification;
[0012] 2) Pedestrian images are input into a classification neural network, which outputs various features corresponding to the pedestrian images;
[0013] 3) Use the features corresponding to the pedestrian images obtained in step 2) to stitch together to obtain the fusion features of the pedestrian images;
[0014] 4) Process both pedestrian images according to steps 1) to 2) to obtain their respective fusion features, calculate the similarity based on the fusion features of the two images, and determine the re-identification result based on the similarity.
[0015] The classification neural network is pre-trained using a pedestrian image dataset with known pedestrian image re-identification results and its labels as input.
[0016] In the aforementioned classification neural network, after receiving a pedestrian image, it is processed in the following manner:
[0017] First, the pedestrian image is processed by a neural network pre-trained based on a classification task to extract initial features F_in;
[0018] The initial feature F_in is then fed into two branch networks N1 and N2 for processing:
[0019] In the first branch network N1, the initial feature F_in is subjected to global max pooling and global average pooling operations respectively to obtain the first local feature FL1 and the first global feature FG1 respectively;
[0020] In the second branch network N2, the initial feature F_in is subjected to global average pooling to obtain the second global feature FG2. At the same time, the initial feature F_in is vertically divided into upper and lower regions, and the upper and lower regions are subjected to global max pooling to obtain their respective second upper local feature FL2 and second lower local feature FL3.
[0021] Finally, the first local feature FL1, the first global feature FG1, the second global feature FG2, the second upper local feature FL2, and the second lower local feature FL3 are convolved to reduce the dimension to 256, which constitutes each feature.
[0022] In step 4), the Euclidean distance between the two pedestrian images is calculated as the similarity based on the fusion features of the two images.
[0023] In step 4), the two pedestrian images are derived from two consecutive frames in the pedestrian video.
[0024] The beneficial effects of this invention are:
[0025] This invention can significantly improve the performance of the model and further reduce the dependence on the pedestrian detection model. It can also achieve good results when the detected pedestrian images are inaccurate or incomplete.
[0026] This invention offers advantages such as accurate feature matching without losing global information, while maintaining low computational complexity. Attached Figure Description
[0027] Figure 1 This is a schematic diagram of a common pedestrian re-identification dataset;
[0028] Figure 2 This is a flowchart of the classification neural network of the present invention. Detailed Implementation
[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0030] The implementation of this invention is as follows:
[0031] like Figure 1 As shown, in a common pedestrian re-identification dataset, the first row contains normal images that can be aligned well, but the images in the second and third rows, whether divided into 3 or 6 blocks, still show feature matching errors.
[0032] For vertical segmentation, this invention simply cuts it into two pieces to ensure that, in extreme cases, some features can still be matched.
[0033] The implementation process is as follows:
[0034] 1) Establish a classification neural network for pedestrian image re-identification;
[0035] The classification neural network is pre-trained using a pedestrian image dataset with known pedestrian image re-identification results and its re-identification labels as input.
[0036] like Figure 2 As shown, in the classification neural network, after receiving a pedestrian image, it is processed in the following way:
[0037] First, the pedestrian image is processed by a neural network pre-trained based on a classification task to extract initial features F_in;
[0038] The initial feature F_in is then fed into two branch networks N1 and N2 for processing:
[0039] In the first branch network N1, the initial feature F_in is subjected to global max pooling and global average pooling operations respectively to obtain the first local feature FL1 and the first global feature FG1 respectively;
[0040] In the second branch network N2, the initial feature F_in is subjected to global average pooling to obtain the second global feature FG2. At the same time, the initial feature F_in is vertically divided into upper and lower regions, and the upper and lower regions are subjected to global max pooling to obtain their respective second upper local feature FL2 and second lower local feature FL3.
[0041] Finally, the first local feature FL1, the first global feature FG1, the second global feature FG2, the second upper local feature FL2, and the second lower local feature FL3 are convolved to reduce the dimension to 256, which constitutes each feature.
[0042] Figure 2 In this context, GMP represents global max pooling, GAP represents global average pooling, and the last five features are concatenated.
[0043] In practice, neural networks pre-trained for classification tasks are typically ResNet neural networks.
[0044] 2) Input each frame of the real-time pedestrian video into the classification neural network, and input the pedestrian images into the classification neural network to output the corresponding features of the pedestrian images;
[0045] 3) Use the features corresponding to the pedestrian images obtained in step 2) to stitch together to obtain the fusion feature F_Out of the pedestrian images;
[0046] 4) Process each pair of adjacent pedestrian images in the pedestrian video. Process both pedestrian images according to steps 1) to 2) to obtain their respective fusion features F_Out. Calculate the similarity based on the fusion features F_Out of the two images. Determine the re-identification result between each real-time pedestrian image and the previous pedestrian image based on the similarity.
[0047] Specifically, the similarity is calculated by measuring the Euclidean distance between two pedestrian images based on the fusion features of the two images.
[0048] In practice, the advantages and disadvantages of the implementation of the present invention were compared with those of other pedestrian re-identification methods based on local features in recent years. The results are shown in Table 3 below.
[0049] Table 1. Different Part Divisions and Different Information Collection Methods
[0050]
[0051] As shown in rows 1 to 6 of Table 1, losing global or local information will lead to a decrease in performance.
[0052] As shown in rows 7 to 10 of Table 1, the performance is optimal when the number of part partitions is equal to 2, which is better than no partitions at all. When the number of part partitions is greater than 2, the performance will continue to decline, indicating that feature mismatch will affect the performance.
[0053] In practice, the implementation of the present invention and other pedestrian re-identification methods based on local features in recent years were tested, and the results are shown in Table 2 below.
[0054] Table 2 shows the results of different feature extraction methods for the same backbone network.
[0055]
[0056] As shown in Table 3, the method of the present invention has a moderate computational load and its performance significantly exceeds that of other similar methods.
[0057] Finally, the present invention was compared with other existing network processing methods, and the results are shown in the table below.
[0058] Table 3 Comparison of characteristics of related methods
[0059]
[0060] As shown in the table above, the method of the present invention adopts a feature extraction process that simply divides the data into two parts. It can achieve the advantages of accurate feature matching without losing global information, and can accurately extract both global and local features without losing global information, while maintaining low computational complexity.
[0061] Therefore, this invention ensures that even in extreme cases, some features can still be matched, which is a significant advantage.
Claims
1. A highly efficient pedestrian image re-identification method, characterized in that: 1) Establish a classification neural network for pedestrian image re-identification; 2) Pedestrian images are input into a classification neural network, which outputs various features corresponding to the pedestrian images; In the aforementioned classification neural network, after receiving a pedestrian image, it is processed in the following manner: First, the pedestrian image is processed by a neural network pre-trained based on a classification task to extract initial features F_in; The initial feature F_in is then fed into two branch networks N1 and N2 for processing: In the first branch network N1, the initial feature F_in is subjected to global max pooling and global average pooling operations respectively to obtain the first local feature FL1 and the first global feature FG1 respectively; In the second branch network N2, the initial feature F_in is subjected to global average pooling to obtain the second global feature FG2. At the same time, the initial feature F_in is vertically divided into upper and lower regions, and the upper and lower regions are subjected to global max pooling to obtain their respective second upper local feature FL2 and second lower local feature FL3. Finally, the first local feature FL1, the first global feature FG1, the second global feature FG2, the second upper local feature FL2, and the second lower local feature FL3 are convolved to reduce the dimension to 256, which constitutes each feature. 3) Use the features corresponding to the pedestrian images obtained in step 2) to stitch together to obtain the fusion features of the pedestrian images; 4) Process both pedestrian images according to steps 1) to 2) to obtain their respective fusion features, calculate the similarity based on the fusion features of the two images, and determine the re-identification result based on the similarity.
2. The efficient pedestrian image re-identification method according to claim 1, characterized in that: The classification neural network is pre-trained using a pedestrian image dataset with known pedestrian image re-identification results and its labels as input.
3. The efficient pedestrian image re-identification method according to claim 1, characterized in that: In step 4), the Euclidean distance between the two pedestrian images is calculated as the similarity based on the fusion features of the two images.
4. The efficient pedestrian image re-identification method according to claim 1, characterized in that: In step 4), the two pedestrian images are derived from two consecutive frames in the pedestrian video.