A bias feature-based occluded pedestrian re-identification method

Abstract

The application relates to a kind of methods for identifying occluded pedestrians based on bias features, comprising constructing a Transformer-based occluded pedestrian re-identification backbone network;Constructing coarse-grained and fine-grained feature extraction modules;Construct a weight extraction module to extract fine-grained feature weights;Construct a feature weighting module to dynamically weight coarse-grained features using the weights extracted by fine-grained features to obtain bias features;Construct a loss function based on coarse-grained features and bias features, train the model using the training set, and select the optimal model from the test set. This method dynamically adjusts the weight of coarse-grained features by providing information from fine-grained features, reducing the interference caused by occlusion in coarse-grained features, and does not require the introduction of additional body cues to train the model, thereby achieving high-precision and high-efficiency occluded pedestrian re-identification.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of based on bias feature's occluded pedestrian re-identification method, belong to digital security, city management technical field. BACKGROUND

[0002] Person Re-identification, also known as pedestrian re-identification, is a task that aims to match the identity of pedestrians from images captured at different times, locations, or angles by comparing their appearance features, poses, and clothing.

[0003] Person Re-identification technology plays a key role in digital security, social safety, and urban management. In the field of digital security, Person Re-identification technology tracks pedestrians through images or videos captured by video surveillance. This technology effectively supports the matching and tracking of targets in surveillance videos, which are often affected by occlusion or partial information loss. It also helps overcome the challenges of matching and tracking targets in surveillance videos due to occlusion and missing information.

[0004] In the field of social safety, Person Re-identification technology helps identify and verify personnel in the field of public security. By combining Person Re-identification technology with access control systems, only authorized personnel can enter specific areas, improving security and reducing potential security threats. In the field of urban management, occluded pedestrian re-identification technology is used for traffic regulation and urban planning. By analyzing pedestrian mobility and behavior patterns, urban planners can better understand the movement of people in the city, providing strategies for optimizing urban road construction and public facility layout, improving urban operational efficiency, and enhancing the quality of life for residents.

[0005] Occlusion is a major challenge in this field, with various types of occlusions, including other pedestrians, vehicles, and environmental objects. This factor can make certain parts of pedestrians invisible or incomplete, increasing the difficulty of occluded pedestrian re-identification and reducing the accuracy of re-identification. Existing methods aim to improve accuracy by introducing pose estimation techniques to infer the location and pose of the occluded parts or using semantic segmentation techniques to segment pedestrians and occlusions to compensate for or infer the occluded areas. However, these two methods have disadvantages, as they require the introduction of additional network models and datasets, which can cause domain shift problems, i.e., the dataset used to train the additional network differs from the dataset used in actual application scenarios. Additionally, using additional networks requires more time and computational resources, increasing the cost of model training and inference. Therefore, unlike traditional methods that require additional body cues, the present application designs an occluded pedestrian re-identification method based on bias features, which adjusts the weight of features to achieve high-precision and efficient occluded pedestrian re-identification. SUMMARY

[0006] In order to overcome the shortcomings of the prior research, the application provides a bias feature-based occluded pedestrian re-identification method. The information provided by the fine-grained features is used to dynamically adjust the weight of the coarse-grained features, and the interference caused by the occlusion in the coarse-grained features is weakened, and at the same time, an additional body clue training model does not need to be introduced, so as to achieve high-precision and high-efficiency occluded pedestrian re-identification.

[0007] The application extracts coarse-grained and fine-grained features of pedestrian pictures through a Transformer network, extracts weights using fine-grained features, and obtains bias features by weighting coarse-grained features using the weights, and finally designs a loss function to train the model. The weighting method dynamically adjusts the weight of the coarse-grained features through the information provided by the fine-grained features, so that the features extracted from the pictures of the same pedestrian with different degrees of occlusion have higher similarity after weighting. The application can more effectively infer the identity of the pedestrian in the case of missing body parts of the pedestrian, thereby improving the accuracy of pedestrian re-identification in actual scenarios.

[0008] The specific steps of a bias feature-based occluded pedestrian re-identification method are as follows:

[0009] Step one: given different sizes of pedestrian pictures, uniformly adjust them to HxWxC, after data enhancement, use a sliding window to cut the pictures into N fixed size small blocks The step size of the sliding window is denoted as S, the size of each small block is denoted as PxPx C, and the calculation formula of N is as follows:

[0010]

[0011] Wherein is the bottom function, S is less than P, N H and N W respectively represent the number of divisions in height and width, and then each small block is flattened into a one-dimensional vector, denoted as

[0012] Step two: construct a Transformer-based occluded pedestrian re-identification backbone network;

[0013] An additional learnable mark is added to the input sequence, denoted as x cls . In order to learn the spatial information in the sequence, camera ID information is embedded in the input sequence Z'0 D is the embedding dimension, N C is the number of cameras. If the camera ID is r, the embedded information can be marked as S C [r], S C[r] is the same in all patches of one picture. The final input sequence Z0 is:

[0014]

[0015] After that, a k-layer pre-trained Transformer network is built to extract the features of the input sequence. The output features obtained are

[0016] Step three: build coarse-grained and fine-grained feature extraction modules. In the coarse-grained branch module, a layer of Transformer is used to extract coarse-grained features; in the fine-grained branch module, the coarse-grained features are segmented by the boundary padding strategy, and a layer of Transformer is used to extract four groups of fine-grained features;

[0017] The invention designs coarse-grained and fine-grained branch modules to capture coarse-grained and fine-grained features respectively. Coarse-grained features provide an overall appearance information, however, in real-world scenarios, the occlusion problem will affect the accuracy and integrity of coarse-grained features. At this time, the intervention of fine-grained features becomes crucial, they can make up for the part that coarse-grained features cannot cover, and provide more specific and detailed information. In the coarse-grained branch module, a layer of Transformer is used to extract coarse-grained features f g ; for the fine-grained branch module, in order to weaken the feature information loss problem caused by segmentation at the connection, the invention uses the boundary padding method to segment the coarse-grained features, and part of the overlap is reserved between the adjacent two groups of features, that is, Z k will be divided into four groups, where n is the offset, and the adjacent two groups have 2n overlapping units. After the 4 groups are connected in series through the shared , they are input into a layer of Transformer to extract 4 groups of fine-grained features, denoted as where is the jth group of features extracted by the fine-grained branch module.

[0018] Step three adopts the boundary padding strategy: part of the feature overlap is reserved between every two adjacent fine-grained features. The purpose of this is to ensure that the features will not be lost due to the processing of the boundary during the cutting process, so as to better preserve the relevance and continuity between the features.

[0019] Step four: build a weight extraction module to extract weights using the fine-grained features extracted in step three, which consists of two layers of Transformer and one linear layer;

[0020] After the coarse-grained and fine-grained features are extracted, a conventional method is directly used to calculate the loss, or a feature fusion method is used to fuse the features and then calculate the loss. This strategy ignores the influence of the missing appearance of pedestrians caused by occlusion, and there is noise information extracted by the occlusion in the extracted coarse-grained and fine-grained features, which is difficult to eliminate even by using the feature fusion strategy. On the other hand, due to the different positions of pedestrians in different pictures, when the fine-grained features are used to calculate the feature distance of two pictures, the feature misplacement problem will occur. To this end, in order to weaken the noise interference in the coarse-grained features and avoid the misplacement problem caused by the fine-grained features, the weight extraction module based on the fine-grained features is designed. Since the occlusion is distributed in blocks in the picture, the fine-grained features are used to extract the weight W The weight W l is extracted. The module is composed of two layers of Transformer layers and one layer of linear layers. The Transformer layer uses the self-attention mechanism to independently and in parallel calculate the weight of the fine-grained features, and the linear layer performs linear transformation to obtain the weight W l .

[0021] In step four, the weight extraction module is designed to extract the weight using the fine-grained features. The module is composed of two layers of Transformer layers and one layer of linear layers, and the advantage is that the self-attention mechanism of the Transformer layer can independently and in parallel calculate the weight of the fine-grained features, which can capture long-distance dependencies and has a more extensive global receptive field. The effective weight can be extracted with fewer layers and under the condition of ensuring the calculation amount.

[0022] Step five: constructing a feature weighting module, using the weight extracted by the fine-grained features to dynamically weight the coarse-grained features, and obtaining the bias feature;

[0023] The weight W l extracted in step four is used to weight the coarse-grained feature f g in step three, and the weighted bias feature f m is obtained. The weighting method is as follows:

[0024] f m = f g (W l ⊙ W) + bias

[0025] Where W is the default weight matrix of the linear layer; is the element-wise multiplication (W l is multiplied by each row of W); bias is the bias. Finally, two parts of features, the coarse-grained feature f g and the bias feature f mThe feature weighting strategy utilizes the weight extracted by the fine-grained feature to dynamically adjust the coarse-grained feature, and the purpose is to weaken the interference of noise information in the coarse-grained feature, so as to improve the similarity of the same pedestrian picture feature under different degrees of occlusion.

[0026] In step five, a weighting module is designed, and the weight extracted by the fine-grained feature is used to dynamically weight the coarse-grained feature to obtain a biased feature, and the purpose is to make the same pedestrian picture feature have higher similarity after dynamic weighting, so as to achieve high-precision and high-efficiency occluded pedestrian re-identification.

[0027] Step six: constructing a loss function based on coarse-grained features and biased features;

[0028] The application designs a loss function to train the model, and the ID loss and the triplet loss are jointly trained, The calculation formula is as follows:

[0029]

[0030] Among them , f g , f m corresponding ID loss, alpha is the hyperparameter of balancing the two, is the triplet loss, the ID loss and the triplet loss The calculation formula is as follows:

[0031]

[0032] Among them, M is the number of pedestrian IDs in the training set, y is the real ID of the picture, p i is the probability that the pedestrian belongs to the i-th pedestrian ID, d p and d q are the feature distances of the positive sample pair and the negative sample pair, beta is the triplet loss residue, [*] + is equivalent to max(*,0).

[0033] Step seven: training the model using the training set, and selecting the optimal model from the test set.

[0034] After the model is trained, the pedestrian picture to be searched in the given data set is extracted using the above network, and the biased feature f m is compared with the biased feature f m extracted from all pedestrian images in the warehouse. The comparison method is to calculate the Euclidean distance between the features, and the pedestrian picture corresponding to the feature vector with the smallest Euclidean distance is the re-identification result of the model.

[0035] Compared with the prior art, the application has the beneficial effects that:

[0036] The application can realize high-precision occluded pedestrian re-identification. First, the coarse-grained features of the picture are extracted by using the Transformer backbone network, then the fine-grained features are extracted by the boundary filling method, and the weight is extracted from the fine-grained features, finally the coarse-grained features are weighted and processed by using the weight to obtain the bias features. This processing method can make the picture features of the same pedestrian with different occlusion degrees have higher similarity after dynamic weighting, which helps to weaken the influence of noise caused by occlusion, thereby improving the recognition accuracy.

[0037] The application can realize high-efficiency occluded pedestrian re-identification. The application does not need to introduce additional body clues, that is, it does not need to use additional networks and datasets to train the model, and can improve the efficiency of occluded pedestrian re-identification. Compared with the conventional solution, the model reduces the demand for computing resources during training and reasoning, and the system can process the occlusion in a more economical way, so that the occluded pedestrian re-identification algorithm is easier to deploy in practical applications, and has higher practicability and feasibility. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, below the drawings needed to be used in the embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0039] Figure 1 The flowchart of the occluded pedestrian re-identification method based on bias features of the application;

[0040] Figure 2 The network model structure diagram constructed by the application;

[0041] Figure 3 The re-identification result comparison diagram of the TransREID method and the application;

[0042] Figure 4 The average precision mean (mAP) index calculation example diagram. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0044] The application is further explained using the Occluded-Duke dataset and in conjunction with the accompanying drawings:

[0045] The specific steps of the bias feature-based occluded pedestrian re-identification method are as follows:

[0046] Step one: unify the different sizes of pedestrian pictures in the training set to 258x128x3, and after random cropping, random scaling, and random erasing, use a sliding window to cut the pictures into small blocks, each picture is cut into 21 parts in height and 10 parts in width, a total of 210 small blocks The size of the sliding window is 16x16, and the step is 12. The size of each small block is 16x16x3. Then, each small block is flattened into a one-dimensional vector through a linear layer, and an output of 210x768 dimensions is obtained

[0047] Step two: embed an extra learnable mark of 1x768 dimensions in the input sequence Z'0, initialized to all zeros, denoted as x cls After the mark passes through the Transformer backbone network, a coarse-grained feature is output. In order to learn the spatial information in the sequence, camera ID information is embedded in the input sequence Z'0 The embedding dimension D is 1x768, N C is the number of cameras shooting pedestrian pictures in the Occluded-Duke dataset, which is 8. That is, an all-0 vector S of dimension 8x1x768 is generated C initialized using a truncated normal distribution, with a mean of 0.2, a variance of 1, and upper and lower limits of truncation of -2 and 2, respectively. If the camera ID is r, the embedded information can be marked as S C [r], S C [r] is the same in all small blocks of a picture. The final input sequence Z0 is:

[0048]

[0049] Then, an 11-layer pre-trained Transformer network is used to extract the features of the input sequence. The output result is denoted as with a dimension of 211x768.

[0050] Step three: as Figure 2 shown, the features Z 11 extracted by the Transformer backbone network are captured by the coarse-grained branch and the fine-grained branch to capture coarse-grained and fine-grained features. In the coarse-grained branch module, after passing through a layer of Transformer, Z is obtained, where The coarse-grained features learned by the model are denoted as f. g The dimension is 1×768; in the fine-grained branch module, in order to mitigate the loss of feature information at the connection points caused by segmentation, the boundary padding method is first used to segment Z. 11 It retains some feature overlap between two adjacent sets of fine-grained features, i.e., Z. 11 Will be divided into Four groups, with 20×768-dimensional feature overlap between adjacent groups of fine-grained features. These four groups are then processed through a shared... After concatenation, the input is fed into a single Transformer layer to obtain... Four sets of features, each set The fine-grained features learned by the model are denoted as... in The j-th feature extracted by the fine-grained branch module has a dimension of 1×768.

[0051] Step 4: After extracting coarse-grained and fine-grained features, in order to remove noise interference caused by occlusion in the coarse-grained features, this invention uses fine-grained features to extract the weight W. l .like Figure 2 As shown, four sets of fine-grained features The input is fed into the weight extraction module, and after passing through two Transformer layers and one linear layer, a weight W with a dimension of 1×768 can be extracted. l The advantage of this module lies in its ability to independently and in parallel compute weights for fine-grained features through the self-attention mechanism of the Transformer layer. Compared to the CNN module, it can capture long-distance dependencies and has a wider global receptive field, enabling the extraction of effective weights with fewer layers and guaranteed computational efficiency.

[0052] Step 5: Use the weights W extracted in Step 4 l For the coarse-grained feature f in step three g The bias feature f is obtained by dynamically weighting through a linear layer. m The weighting method is as follows:

[0053] f m =f g (W l ⊙W)+bias

[0054] Where W is the default weight matrix of the linear layer; ⊙ represents element-wise multiplication (using W). l(Multiplied by each row of W); bias is the bias, initialized to all zeros. The Occluded-Duke training set has 702 pedestrians, so the output dimension of the linear layer is 1×702, and the input feature f... g The dimension of W is 1×768, therefore, the dimension of the weights W of this linear layer is 702×768. l Multiplying by each row of W yields the final weights for the linear layer, for f g After weighting, we can obtain a weighted feature with dimension 1×702, f. m At the same time, the coarse-grained feature f g Adjusted to f m The same dimensions. Ultimately, two parts of features can be obtained: coarse-grained features f. g With bias feature f m This feature weighting strategy uses the weights extracted from fine-grained features to dynamically adjust coarse-grained features, with the aim of reducing the interference of noise information in coarse-grained features.

[0055] Step 6: Extract coarse-grained features f g With bias feature f m Then, through the loss function To train the model, we use a combination of ID loss and triplet loss for joint training. The calculation formula is as follows:

[0056]

[0057] in f g f m The corresponding ID loss, with α being a hyperparameter balancing the two, was found to be 0.8 after multiple experiments. The ID loss focuses on separating different pedestrians in the feature space through classification, while the triplet loss, through metric learning, aims to make the feature distances of pedestrians with the same identity closer and the feature distances of pedestrians with different identities farther apart. Jointly training the model with both methods can better optimize the feature space and improve the model's robustness. The hyperparameter α is used to balance f... g f m The corresponding ID loss allows the model to focus on coarse-grained features while adjusting network parameters using weighted features, thereby reducing the interference of noise in coarse-grained features.

[0058] Step 7: After the model is trained, given the pedestrian image to be retrieved, use the above network to extract the bias feature f. m The bias features f extracted from all pedestrian images in the warehouse mComparison. The comparison method is to calculate the Euclidean distance between the features, and the pedestrian picture corresponding to the feature vector with the smallest Euclidean distance is the re-identification result of the model. As shown in Figure 3 For a given pedestrian picture to be retrieved, a set of pedestrian pictures with the highest similarity is found in the warehouse, and the similarity gradually decreases from left to right. Figure 3 In the two examples in Table 2, the TransREID method has two picture query errors, while the present application correctly identifies. Compared with the TransREID method, the present application has better re-identification effect when dealing with the occlusion problem.

[0059] Experimental results

[0060] The present method is compared with other methods in experiments on Occluded-Duke and Occluded-REID data sets. These two pedestrian re-identification data sets contain a large number of occlusions, which may come from other pedestrians, objects, or shooting angles and other factors. These images reflect various challenges and complex situations that may be encountered in the real world. The construction of the data set aims to encourage researchers to develop more robust and generalizable pedestrian re-identification algorithms that can effectively cope with the challenges brought by occlusion. Rank-1 and mAP are used as evaluation indicators, and the experimental results and comparisons are shown in Table 1.

[0061] Table 1 Comparison of experimental results

[0062]

[0063] where * indicates the use of additional networks, such as human pose estimation networks, semantic segmentation networks, etc. According to the values in the table, the present method has higher re-identification accuracy compared to other methods; compared to methods that do not use additional networks, the re-identification accuracy of the present method improves by an average of 6.2% in Rank-1 and 6.8% in mAP; compared to models that use additional networks, the present method also has a large improvement: Rank-1 improves by an average of 7.8%, and mAP improves by an average of 10.4%.

[0064] The evaluation indicators are explained as follows:

[0065] (1) Rank-1: This index measures the probability that the model can correctly match the given retrieval image to the image with the same identity in the dataset. Specifically, when a retrieval image is input into the model, the model returns a series of images most similar to the retrieval image and ranks them. The Rank-1 index refers to whether the image ranked first (i.e., the most similar image) in the model's returned ranking list is the same identity as the retrieval image. If the highest similarity image has the same identity as the query image, the image is considered a successful match, otherwise, the match fails. This index is an important standard for measuring the performance of the pedestrian re-identification task, because it directly reflects the accuracy of the model in the most important matching.

[0066] (2) mAP: This index measures the retrieval effect of the model on a given dataset to be retrieved, reflecting the degree to which the retrieval target person is ranked in the top of the sorting list among all correct pictures in the warehouse, and can more comprehensively measure the performance of the ReID algorithm. As shown in Figure 4 : For the pictures P1, P2 to be retrieved, the feature distances of all pictures in the warehouse are sorted from small to large, the circles represent the correct answers, and the crosses represent the wrong answers. For the retrieval results, the proportion of correctly identified pictures to the total number of pictures at this serial number is the precision, and the average of the precision is the average precision (AP). Taking the average of the AP of all pictures in the query set can obtain the average precision (mAP) of the dataset.

[0067] The embodiments of the present application are described in detail above in combination with the drawings, but the present application is not limited to the described embodiments. For those skilled in the art, various changes, modifications, replacements and variations of these embodiments can be made without departing from the principles and spirits of the present application, and still fall within the protection scope of the present application.

Claims

1. A method for occluded pedestrian re-identification based on bias features, characterized in that: Comprising the following steps: Step one: Given different sizes of pedestrian pictures, adjust them to HxWxC uniformly, after data augmentation, use sliding window to cut the pictures into N small blocks of fixed size The step of sliding window is recorded as S, the size of each small block is recorded as PXPXC, and the calculation formula of N is as follows: where is the bottom function, S is smaller than P, N H and N W denote the number of splits in height and width respectively, after which each tile is flattened into a one-dimensional vector, denoted as Step two: build a Transformer-based occluded pedestrian re-identification backbone network; Step three: build coarse-grained and fine-grained feature extraction modules. In the coarse-grained branch module, coarse-grained features are extracted through a layer of Transformer. In the fine-grained branch module, the coarse-grained features are segmented by a boundary padding strategy, and four sets of fine-grained features are extracted through a layer of Transformer; Step four: build a weight extraction module to extract weights from the fine-grained features extracted in step three. This module consists of two layers of Transformer and one linear layer; Step five: build a feature weighting module to dynamically weight the coarse-grained features using the weights extracted from the fine-grained features, and obtain biased features; The step five specifically comprises: using the weight W extracted in step four l performing weighting on the coarse-grained feature f of step three g to obtain the weighted bias feature f m The weighting manner is as follows: f m = f g (W l ⊙W) + bias where W is a linear layer default weight matrix; is an element-wise multiplication; bias is a bias, which can obtain two parts of features, coarse-grained features f g and bias features f m ; Step six: build a loss function based on coarse-grained features and biased features; Step seven: train the model using the training set and select the optimal model using the test set.

2. The method of claim 1, wherein the method is a bias feature based occluded pedestrian re-identification method. The step three specifically comprises: designing coarse-grained and fine-grained branch modules for capturing coarse-grained and fine-grained features respectively, in the coarse-grained branch module, coarse-grained features f g are extracted through a layer of Transformer k For the fine-grained branch module, the coarse-grained features are segmented by the boundary padding method, and part of the overlap is reserved between the adjacent two groups of features, that is, Z four groups, where n is the offset, and there are 2n overlapping units between the adjacent two groups, and the four groups are input into a layer of Transformer in series through shared , and four groups of fine-grained features are extracted, denoted as , where is the jth group of features extracted by the fine-grained branch module.

3. The method of claim 1, wherein the method is a bias feature based occluded pedestrian re-identification method. The step four specifically comprises: utilizing the fine-grained features The weight W is extracted l The module is composed of two layers of Transformer layers and one layer of linear layers, the Transformer layers adopt a self-attention mechanism to independently and in parallel calculate the weight of the fine-grained features, and the linear layer is linearly transformed to obtain the weight W l .

4. The method of claim 1, wherein the method is a bias feature based occluded pedestrian re-identification method. The step six specifically comprises: designing a loss function to train the model, the ID loss and the triple loss are jointly trained, The calculation formula is as follows: wherein are f g , f m corresponding ID loss, and a is a hyperparameter balancing the two, is the triplet loss, and the ID loss is calculated as follows: Triplet loss The formula for calculating the triplet loss is as follows: where M is the number of pedestrian IDs in the training set, y is the true ID of the picture, p i is the probability that the pedestrian belongs to the i-th pedestrian ID, d p and d q are the feature distances of the positive sample pair and the negative sample pair, respectively, β is the triplet loss margin, and + is equivalent to max(*, 0).

5. The method of claim 1, wherein the method is a bias feature based occluded pedestrian re-identification method. The step seven specifically comprises: after the model training is completed, a pedestrian picture to be retrieved in a given data set is given, and the above network is used to extract the bias feature f m The bias feature f extracted from all pedestrian images in the warehouse is compared m The comparison method is to calculate the Euclidean distance between the features, and the pedestrian picture corresponding to the feature vector with the smallest Euclidean distance is the re-identification result of the model.

Images