An occluded pedestrian re-identification method based on occlusion simulation and feature fusion

Abstract

The application discloses a kind of based on occlusion simulation and feature fusion's occlusion pedestrian re-identification method, belong to computer vision technical field, need to construct a feature coupling network based on occlusion simulation and Token constraint, specific steps include: S1: integration pedestrian image dataset;S2: design based on the occlusion simulation strategy of block mixing;S3: establish backbone network;S4: design based on the occlusion simulation strategy of block mixing;S5: establish local-global feature coupling module;S6: design loss function;The method based on the application of Transformer can effectively handle the dependent relationship of long distance information, simultaneously without the aid of additional auxiliary model, can improve the robustness of model to deal with various occlusion scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a blocked pedestrian re-identification method based on occlusion simulation and feature fusion. BACKGROUND

[0002] In recent years, many researchers have been studying the blocked pedestrian re-identification method, and most of the existing methods solve these challenges by locating the visible body parts and aligning them. These methods can be roughly divided into two categories: 1) segmentation-based methods and 2) external cue-based methods. The segmentation-based method divides the pedestrian image or feature map into blocks and extracts local features from them for comparison, thereby achieving alignment between feature blocks. This alleviates the impact of occlusion to some extent, but causes the loss of semantic information of adjacent parts and destroys the order of the feature space. The external cue-based method usually uses auxiliary models of pose estimation and semantic segmentation to indicate the unoccluded body parts, both of which help to eliminate occluded features. However, the introduction of additional network structures increases the complexity of the model and requires more training time and computing resources. In addition, in complex occlusion scenarios, the effectiveness of the external cue-based method cannot be guaranteed due to the accuracy of the additional model. SUMMARY

[0003] The present application aims to provide a blocked pedestrian re-identification method based on occlusion simulation and feature fusion to solve the problems raised in the background.

[0004] To achieve the above purpose, the present application provides the following technical scheme:

[0005] A blocked pedestrian re-identification method based on occlusion simulation and feature fusion needs to construct a feature coupling network based on occlusion simulation and Token constraint, and the specific steps include:

[0006] S1: Integrate the pedestrian image dataset, divide the dataset into training set, validation set and test set for use in different stages, and crop the photo size of the dataset to 256*128;

[0007] S2: Design an occlusion simulation strategy based on block mixing, select two different training samples from a batch of original input images to fuse into a new training sample, simulate more diverse occlusion situations in the real world, and enhance the ability of the network to perceive the target person under various occlusion situations;

[0008] S3: Establish a backbone network, input the new training image into ViT, and use ViT-B-16 pre-training weight feature extraction backbone to extract global and local features;

[0009] S4: Design an occlusion simulation strategy based on block mixing, and obtain a distinguishable class token representation by passing the class token of the last Transformer layer through the Token orthogonal embedding module, so as to ensure that it maintains the representational difference with the patch token in the feature embedding space;

[0010] S5: Establish a local-global feature coupling module, which reshapes local classification features and local patch features and feeds them into the multi-scale Transformer feature integration module to obtain more fine-grained local enhancement features, which complement the global representation. This greatly reduces the adverse effects of occlusion or non-target regions on the network.

[0011] S6: Design a loss function to evaluate the model's prediction results.

[0012] Preferably, the occlusion simulation strategy based on block blending proposed in step S2 is specifically implemented by the following steps:

[0013] S21: Obtain a batch of raw images from the dataset. Where B represents the number of pedestrian categories in a batch. To better utilize occlusion enhancement to solve the occlusion problem, two different training samples (x1, y1) and (x2, y2) are selected and merged into a new training sample.

[0014] S22: Considering color differences in changing environments, the data undergoes local grayscale transformation to allow the model to adapt to significant changes caused by local color loss and reduce the resulting bias; a maximum number of trials is set to select a random aspect ratio and thus determine the random rectangular region R. g Height H r and width W r ;

[0015] S23: Unlike the traditional method of directly converting the selected area's RGB color space to grayscale, a random angle θ∈[0, 360°) is selected, with R... g geometric center (W) r / 2,H r / 2) is the axis of rotation for the rectangular region R. g The rotation transformation matrix M(θ) is defined as follows:

[0016]

[0017] in:

[0018] t x = (1-cos(θ))·W r / 2+sin(θ)·H r / 2

[0019] t y = (1 - cos(θ)) · H r / 2 - sin(θ) · W r / 2

[0020] The new image after rotation transformation Can be obtained by the following transformation:

[0021] R′ g (u, v) = R g (M -1 (θ) · (u, v, 1) T )

[0022] In the above formula, M -1 (θ) is the inverse matrix of M(θ), used to map back from the transformed space to the original space, (u, v) is the coordinate point in the transformed image, and then the choice of replacing the grayscale image on all color channels or only on a randomly selected color channel provides more variations for model training.

[0023] S24: Due to the vertical symmetry of human features, vertical segmentation may damage key body parts, so only horizontal occlusion simulation is considered, the image is divided into N equal size blocks along the horizontal axis, the 2D coordinates of each block are calculated and stored in the coordinate list ; Then assign an index to each block and generate an index list I l = [I1, I2,..., In N ]; At this time, the equally divided image can be represented as In the real world, occlusion is diverse and irregular, and randomly appears at any position in space; sample the segmented image blocks according to N / 2 to get the number of sample blocks n, randomly select n block indexes from the coordinate list C l , the formula is as follows:

[0024] I′ l = sample(range(len(C l )), n)

[0025] S25: Mix the image according to the index in equal proportion, so as to effectively cover different positions in space, the specific form is as follows:

[0026]

[0027] Since the generated occluded images contain labels from different identities, providing only one identity label at the network input stage can disturb the model's recognition ability; therefore, for labels y1 and y2, a combination ratio λ is set for their combination; here λ is expressed as n / N, and the formula is as follows:

[0028]

[0029] Thus, new training samples are obtained Because is obtained by fusing two different image segmentations, but each image needs to be optimized for IDLoss and TripletLoss, so the loss function is redefined as follows:

[0030]

[0031] Preferably, the step S3 proposes to construct a backbone network, and the specific steps include:

[0032] S31: Given an input image Where H, W and C represent the height, width and number of channels of the image, respectively;

[0033] S32: Divide the image X into N(h x w) patches using a sliding window Where N can be expressed as:

[0034]

[0035] In the above formula, S and P represent the step size of the sliding window and the size of the image patch, respectively; since vit needs sequences as input, each patch is embedded into a D-dimensional space by a linear projection function F(·);

[0036] S33: An additional learnable class token x cls Is attached to the input sequence to capture the semantic information of pedestrians and used as a global feature representation f gb of the encoder; before being fed into the transformer block, a learnable position embedding and a camera embedding are added to the patch embedding, which are used to preserve the position information of the image and the camera-specific information, respectively; the embedding process of the input sequence can be expressed as:

[0037]

[0038] In the above formula, λ is a hyperparameter used to balance the weight of the camera embedding;

[0039] S34: Input embedding The final output of the encoder is processed by L transformer layers Is divided into two parts (global features and local features): And In order to learn more distinctive pedestrian features, the local features are divided into K groups to form K groups of local patch features The size of each group is (N / / K) x D; a local class token is set in the local features The local class token and each group of local patch features are concatenated and fed into a shared transformer layer to learn K groups of local classification features

[0040] Preferably, the step S4 proposes a blocking mixed-based occlusion simulation strategy, and the specific steps include:

[0041] S41: Represent the class token of the last Transformer layer as ycls, where K is the number of class tokens; before this, in order to suppress overfitting while reducing the co-adaptation between features and stabilizing the gradient, Dropout and Batch Normalization are introduced to pre-process the input features; then L2 normalization is performed along the feature dimension to ensure that the feature embedding has a stable scale before entering the matrix operation, and the above process can be expressed as:

[0042]

[0043] In the above formula, ||·||F F represents the Frobenius norm, and ∈ is a number designed to prevent division by zero;

[0044] S42: Based on prior knowledge, inter-class separation can be achieved through orthogonal constraints while enhancing intra-class compactness, which helps the model improve the discrimination of similar classes; in order to maintain the difference between the class token and the patch token, the L2 normalized class token The orthogonal constraint is:

[0045]

[0046] In the above formula, I i，j ∈K×K is the identity matrix; is the Huber Loss, which is used to constrain the similarity between features to be close to orthogonal;

[0047] S43: In real scenes, occlusion has great uncertainty, which may cause the network to learn too complex and highly differentiated feature representations while imposing constraints on class tokens, so a regularization constraint is introduced to control the size of the model weights, helping the network to learn more sparse feature representations and avoid excessive dependence on certain features due to excessively large weights. The formula is as follows:

[0048]

[0049] In the above formula, represents the weight decay coefficient for controlling the strength of regularization; the final orthogonal constraint and regularization constraint work together, as follows:

[0050]

[0051] Preferably, the step S5 proposes to construct a local-global feature coupling module, and the specific steps include:

[0052] S51: Local classification features f l l 1 l 2 l K ] and local patch features are spliced respectively, and then reshaped to obtain feature shapes and

[0053] S52: The local classification feature branch is specifically analyzed, and the other branch is also applicable to this process; first, the multi-dimensional attention mechanism of ODconv is used to mine detailed features from multi-view and multi-scale images, which is different from directly acting on the feature map. A learnable dynamic weight w1 is initialized in the dynamic convolution branch, so as to enlarge the weight of the main line and let the dynamic convolution part make a mutually biased supplement. The specific process is as follows:

[0054]

[0055] S53: Use the MHSA mechanism and hierarchical feature extraction of the Transformer encoder to aggregate more subtle differences. Before this, the feature map based on one branch generates tokens of a specified size, as follows:

[0056]

[0057] In the above formula, the token represents the global information of the local patch feature;

[0058] ​​​S54: The outputs of the two branches after ODconv are represented as and Then, they are concatenated and fed into the Transformer encoder for self-attention calculation, and the specific calculation formula is described as follows:

[0059]

[0060] In the above formula, D represents the number of channels, represents matrix multiplication; q, k and v respectively represent the query matrix, key matrix and value matrix obtained by connecting and w q , w k and w v represent different weight matrices; then, the output A is fed into the FFN network to enhance the embedding representation capability; the final output of the local classification feature branch is represented as: Similarly, the output of the other branch is: Next, the outputs of the two branches are added to obtain the integrated features

[0061] S55: Further extract and enhance the integrated features, use a feature extractor composed of convolutional layers and pooling layers to obtain local enhanced features The formula is as follows:

[0062] f k = σ (Avg (G (F out )))

[0063] S55: Define a learnable weight matrix where C represents the number of categories, and the Hadamard product of the weight matrix and f k forces the model to give priority to the categories that contribute to the local features; the global feature f g is multiplied by the above weight to obtain a weight with higher classification accuracy The specific process is described as follows:

[0064]

[0065] ​​​S56: The multi-scale coupling of local features and global representation through the above operation enhances the classification accuracy, and Softmax loss is usually used for optimization, considering that there is a large optimization space for expanding the inter-class distance between different samples and reducing the intra-class distance between the same samples. An additional angle margin is added to the original Softmax loss for optimization, and the formula is as follows:

[0066]

[0067] Preferably, the step S6 proposes to construct a loss function, and the specific steps include:

[0068] S61: Re-ID is considered as an image classification task, and each ID is a class, so IDloss is used to train the model:

[0069]

[0070] In the above formula, B represents the number of images in batchsize, p(y i |x i ) represents the predicted probability that x i is identified as y i ;

[0071] S62: In order to reduce the intra-class distance and increase the inter-class distance, a triple loss is used for optimization:

[0072]

[0073] In the above formula, f a is the anchor sample; f p is the positive sample; f n is the negative sample, and ||·||2 represents L2-norm;

[0074] S63: Finally, the loss function can be expressed as follows:

[0075]

[0076] In the above formula, alpha and beta are hyperparameters for balancing and .

[0077] Compared with the prior art, the present application has the following beneficial effects:

[0078] The method based on the Transformer of the present application can effectively handle the dependency relationship of long-distance information, and does not need to rely on additional auxiliary models, thereby improving the robustness of the model in dealing with various occlusion scenes. BRIEF DESCRIPTION OF DRAWINGS

[0079] Figure 1 is a schematic diagram of an overall framework;

[0080] Figure 2 is a schematic diagram of a blocking simulation strategy based on block mixing;

[0081] Figure 3 is a schematic diagram of a multi-scale Transformer feature integration module;

[0082] Figure 4 is a flowchart of a blocking pedestrian re-identification method based on blocking simulation and feature fusion. DETAILED DESCRIPTION

[0083] The application will be further described in detail below in combination with the drawings and embodiments:

[0084] A blocking pedestrian re-identification method based on blocking simulation and feature fusion needs to construct a feature coupling network based on blocking simulation and Token constraint, as shown in Figure 1 The overall architecture of the proposed feature coupling network based on blocking simulation and Token constraint is composed of a blocking simulation strategy based on block mixing, a local-global feature coupling module and a Token orthogonal embedding module. The blocking samples generated from the blocking simulation strategy based on block mixing are fed into the pre-trained ViT as the backbone of feature extraction. The class labels are passed through the Token orthogonal embedding module in the last Transformer layer to obtain more distinctive feature representations. Then, the local-global feature coupling module couples the local features and global representations at multiple scales to facilitate comprehensive feature learning.

[0085] The specific implementation steps are as follows Figure 4 as shown):

[0086] Step 1: Integrate the pedestrian image dataset, divide the dataset into training set, validation set and test set for use at different stages, and at the same time, crop the photo size of the dataset to 256*128;

[0087] Step 2: Select two different training samples from a batch of original input images to fuse into a new training sample, simulate more diverse blocking situations in the real world, and enhance the network's ability to perceive the target person under various blocking situations;

[0088] Step 3: Input the new training image into the ViT (use ViT-B-16 pre-training weight) feature extraction backbone for extracting global and local features;

[0089] Step 4: The class token of the last Transformer layer is passed through the Token Orthogonal Embedding module to obtain a distinctive class token representation, ensuring that it maintains a representational difference from the patch tokens in the feature embedding space;

[0090] Step 5: The local classification features and local patch features are respectively reshaped and fed into the multi-scale Transformer feature integration module to obtain more fine-grained local enhanced features, which greatly reduce the adverse effects of occlusion or non-target regions on the network;

[0091] Step 6: Evaluate the model prediction results through the loss function.

[0092] Further, a block-based mixed occlusion simulation strategy is constructed;

[0093] The occlusion simulation proposed in step 2 is shown in Figure 2 The current occlusion enhancement strategy mostly uses random cropping and random erasing to form an occlusion scene. However, it can only cover some common occlusion scenes, while ignoring the influence of the external environment, thus cannot effectively solve the problem of occlusion diversity. In order to solve the above problems, the present application proposes an occlusion simulation strategy based on block mixing. First, a batch of original images is obtained from the data set, where B represents the number of human categories in a batch, in order to better utilize occlusion enhancement to solve the occlusion problem, two different training samples (x1, y1) and (x2, y2) are selected to be fused into a new training sample Before that, considering the color difference in the changing environment, the data is subjected to local grayscale transformation, so that the model can better adapt to the significant changes caused by the lack of local color and reduce the deviation caused thereby. By setting a maximum number of attempts, a random aspect ratio is selected to determine the random rectangular region R g , the height H r and the width W r of R g are determined. After the selection, unlike the traditional method of directly converting the selected region from the RGB color space to the grayscale color space, a random angle θ∈[0, 360°) is selected, and the rectangular region R r is rotated around the geometric center (W r / 2, H g / 2) of R g . The rotation transformation matrix M(θ) is defined as:

[0094]

[0095] where:

[0096] t x =(1-cos(θ))·Wr / 2+sin(θ)·H r / 2

[0097] t y = (1-cos(θ))·H r / 2-sin(θ)·W r / 2

[0098] The new image obtained after rotation transformation It can be obtained through the following transformation:

[0099] R′ g (u, v) = R g (M -1 (θ)·(u, v, 1) T )

[0100] In the above formula, M -1 (θ) is the inverse matrix of M(θ), used to map from the transformation space back to the original space, and (u, v) are the coordinates of the transformed image. Then, the choice is made to replace the grayscale image with all color channels or only with a randomly selected color channel, thus providing more variations for model training.

[0101] Furthermore, due to the vertical symmetry of human features, vertical segmentation may damage critical body parts (such as the head). Therefore, only horizontal occlusion simulation is considered. The image is divided into N equal-sized blocks along the horizontal axis, the 2D coordinates of each block are calculated, and stored in a coordinate list. Then, assign an index to each block and generate an index list I. l = [I1, I2, ..., I N At this point, the equally segmented image can be represented as: In the real world, occlusion is diverse and unpredictable, and can occur anywhere in space. The number of sampled blocks, n, is obtained by sampling the segmented image patches according to N / 2, and then using the coordinate list C... l The indices of n randomly selected blocks are given by the following formula:

[0102] I' l =sample(range(len(C) l )), n)

[0103] Furthermore, the images are blended proportionally according to the index, thereby effectively covering different locations in space.

[0104] The specific format is as follows:

[0105]

[0106] Since the generated occluded images contain labels from different identities, providing only one identity label at the network input stage can disturb the model's recognition ability. Therefore, for labels y1 and y2, a combination ratio λ is set to combine them. Here λ is expressed as n / N, and the formula is as follows:

[0107]

[0108] Thus, the new training sample is obtained Because is obtained by fusing two different image segmentations, but each image needs to be optimized for IDLoss and TripletLoss, so the loss function is redefined as follows:

[0109]

[0110] Further, the backbone network is constructed:

[0111] The ViT feature extraction backbone mentioned in step 3 is described as follows: given an input image where H, W, and C represent the height, width, and number of channels of the image, respectively; the image X is divided into N(h×w) using a sliding window where N can be expressed as:

[0112]

[0113] In the above formula, S and P represent the step size of the sliding window and the size of the image block, respectively. Since vit requires a sequence as input, each patch is embedded into a D-dimensional space through a linear projection function F(·). In addition, an additional learnable class token x cls is attached to the input sequence to capture the semantic information of pedestrians and serve as a global feature representation f gb of the encoder; a learnable position embedding and a camera embedding are added to the patch embedding before feeding into the transformer block to retain the position information of the image and the camera-specific information, respectively.

[0114] The embedding process of the input sequence can be expressed as:

[0115]

[0116] In the above formula, λ is a hyperparameter used to balance the weight of the camera embedding.

[0117] Further, the input embedding This will be processed by L transformer layers. The encoder's final output... It is divided into two parts (global features and local features): and To learn more distinctive pedestrian features, local features are divided into K groups to form K groups of local patch features. Each group has a size of (N / / K)×D. A local class token is set in the local features. The local class tokens and features from each group of local patches are then concatenated and fed into a shared transformer layer to learn K groups of local classification features f. l =[f l 1 f l 2 , ..., f l K ].

[0118] Furthermore, an occlusion simulation strategy based on block blending is constructed:

[0119] The Token orthogonal embedding module proposed in step 4 is as follows: Figure 1 As shown. Specifically, the class token of the last Transformer layer is represented as ycls, where K represents the number of class tokens. Prior to this, to suppress overfitting, reduce co-fitting between features, and stabilize gradients, Dropout and Batch Normalization are introduced to preprocess the input features. Then, L2 normalization is performed along the feature dimension to ensure that the feature embeddings have a stable scale before entering matrix operations. The above process can be described as follows:

[0120]

[0121] In the above formula, ||·|| F ∈ represents the Frobenius norm, where ∈ is a number designed to prevent division by zero.

[0122] Based on prior knowledge, orthogonal constraints can achieve inter-class separation while enhancing intra-class compactness, which helps the model improve its ability to distinguish similar classes. Therefore, to maintain the difference between class tokens and patch tokens, L2-normalized class tokens... Apply orthogonal constraints as follows:

[0123]

[0124] In the above formula, I i，j ∈K×K is the identity matrix; For HuberLoss, it is used to constrain the similarity between features to be close to orthogonal.

[0125] In real scenes, occlusion has great uncertainty, which may cause the network to learn too complex and highly differentiated feature representations while imposing constraints on class tokens, so a regularization constraint is introduced to control the size of model weights, helping the network to learn more sparse feature representations and avoid excessive dependence on certain features due to excessively large weights. The formula is as follows:

[0126]

[0127] In the above formula, represents the weight decay coefficient used to control the strength of regularization; the final orthogonal constraint and the regularization constraint work together, as follows:

[0128]

[0129] Further, a local-global feature coupling module is constructed:

[0130] In order to complement each other between local features and global representations, the application designs a local-global feature coupling module, as shown in Figure 1 . Among them, the multi-scale Transformer feature integration module mentioned in step 5 is as shown in Figure 3 . The local classification feature f l l 1 , f l 2 ,..., f l K and the local patch feature are spliced respectively, and then reshaped to obtain feature shapes and Specific analysis is made on the local classification feature branch, and the other branch is also applicable to this process. It first uses the multi-dimensional attention mechanism of ODconv to mine detailed features from multi-angle and multi-scale images, which is different from directly acting on the feature map. A learnable dynamic weight w1 is initialized in the dynamic convolution branch, so as to enlarge the weight of the main line and let the dynamic convolution part make mutual biased complement;

[0131] The specific process is as follows:

[0132]

[0133] Further, the MHSA mechanism and hierarchical feature extraction of the Transformer encoder are used to aggregate more subtle differences. Before that, tokens of a specified size are generated based on the feature map of one branch, as follows:​

[0134]

[0135] In the above formula, the token represents the global information of the local patch feature. The output of the two branches after ODconv is represented as and Then, it is concatenated with and fed into the Transformer encoder for self-attention calculation. The specific calculation formula is described as follows:

[0136]

[0137] In the above formula, D represents the number of channels, represents matrix multiplication. q, k and v represent the query matrix, key matrix and value matrix obtained by connecting and , that is, and w q , w k and w v represent different weight matrices. Subsequently, the output A is fed into the FFN network to enhance the embedding representation capability. The final output of the local classification feature branch is represented as: Similarly, the output of the other branch is: Next, the outputs of the two branches are added to obtain the integrated feature

[0138] In order to further extract and enhance the integrated feature, a feature extractor composed of convolutional layers and pooling layers is used to obtain the local enhanced feature The formula is as follows:

[0139] f k = σ (Avg (G (F out )))

[0140] Further, a learnable weight matrix is defined, where C represents the number of classes. The weight matrix and f k are Hadamard multiplied, forcing the model to give priority to the classes that contribute to the local feature. Then, the global feature f g is multiplied by the above weight by Matmul to obtain a weight with higher classification accuracy.

[0141] The specific process is described as follows:

[0142]

[0143] The multi-scale coupling of local features and global representation by the above operation enhances the classification accuracy, and Softmax loss is usually used for optimization. The present application considers that there is a great optimization space for expanding the inter-class distance between different samples and reducing the intra-class distance between the same samples. An additional angle margin is added to the original Softmax loss for optimization, and the formula is as follows:

[0144]

[0145] Further, the loss function is constructed:

[0146] Re-ID is considered as an image classification task, and each ID is a class, so IDloss is used to train the model:

[0147]

[0148] In the above formula, B represents the number of images in batchsize, p(y i |x i ) represents the predicted probability of x i being identified as y i . In order to reduce the intra-class distance and increase the inter-class distance, a triplet loss is used for optimization:

[0149]

[0150] In the above formula, f a is the anchor sample; f p is the positive sample; f n is the negative sample, and ||·||2 represents L2-norm.

[0151] Finally, the loss function of the present application can be represented by the following formula:

[0152]

[0153] In the above formula, α and β are hyperparameters for balancing and respectively.

[0154] The above is only an embodiment of the present application, and common technical solutions and / or characteristics in the scheme are not described in detail. It should be noted that for those skilled in the art, without departing from the technical solutions of the present application, a number of modifications and improvements can be made, which should also be considered as the protection scope of the present application, and these will not affect the effect and practicality of the present application. The protection scope of the present application should be subject to the content of its claims, and the specific implementation mode and the like in the specification can be used to explain the content of the claims.

Claims

1. A method for re-identifying occluded pedestrians based on occlusion simulation and feature fusion, characterized in that: The specific steps for constructing a feature-coupled network based on occlusion simulation and token constraints include: S1: Integrate the pedestrian image dataset, divide the dataset into training set, validation set and test set for use at different stages, and crop the dataset photos to 256*128; S2: Design an occlusion simulation strategy based on block fusion. Select two different training samples from a batch of original input images and fuse them into a new training sample to simulate more diverse occlusion situations in the real world and enhance the network's ability to perceive target people under various occlusion conditions. S3: Establish a backbone network by inputting new training images into the ViT feature extraction backbone network to extract global and local features; S4: Design an occlusion simulation strategy based on block mixing, and obtain a distinguishable class token representation by passing the class token of the last Transformer layer through the Token orthogonal embedding module, so as to ensure that it maintains the representational difference with the patch token in the feature embedding space; S5: Establish a local-global feature coupling module, which reshapes local classification features and local patch features and feeds them into the multi-scale Transformer feature integration module to obtain more fine-grained local enhancement features, which complement the global representation. This greatly reduces the adverse effects of occlusion or non-target regions on the network. S6: Design a loss function to evaluate the model's prediction results; Step S4 proposes an occlusion simulation strategy based on block-based blending, the specific steps of which include: S41: Represent the class token of the last Transformer layer as... ,in K represents the number of class tokens. Prior to this, to suppress overfitting, reduce co-fitting between features, and stabilize gradients, Dropout and Batch Normalization are introduced to preprocess the input features. Then, L2 normalization is performed along the feature dimension to ensure that the feature embeddings have a stable scale before entering matrix operations. The above process can be described as follows: ； In the above formula, Denotes the Frobenius norm. It is a number designed to prevent division by zero; S42: Based on prior knowledge, orthogonal constraints can achieve inter-class separation while enhancing intra-class compactness, which helps the model improve its ability to distinguish similar classes; in order to maintain the difference between class tokens and patch tokens, L2-normalized class tokens are... Apply orthogonal constraints as follows: ； In the above formula, It is the identity matrix; HuberLoss is used to constrain the similarity between features to make them nearly orthogonal. S43: In real-world scenarios, occlusion is highly uncertain. While imposing constraints on tokens, it may cause the network to learn overly complex and highly differentiated feature representations. Therefore, regularization constraints are introduced to control the magnitude of model weights, helping the network learn sparser feature representations and avoiding excessive reliance on certain features due to excessively large weights. The formula is as follows: ； In the above formula, The weight decay coefficient is used to control the strength of regularization; ultimately, the orthogonal constraint and the regularization constraint work together, as described below: 。 2. The occlusion-based pedestrian re-identification method according to claim 1, characterized in that: The occlusion simulation strategy based on block-based blending proposed in step S2 includes the following specific implementation steps: S21: Obtain a batch of raw images from the dataset. Where B represents the number of pedestrian categories in a batch, and to better utilize occlusion enhancement to solve the occlusion problem, two different training samples are selected. and To merge into a new training sample ; S22: Considering color differences in changing environments, the data undergoes local grayscale transformation to allow the model to adapt to significant changes caused by local color loss, reducing the resulting bias; a maximum number of trials is set to select a random aspect ratio and thus determine a random rectangular region. height and width ; S23: Unlike the traditional method of directly converting the selected area's RGB color space to grayscale, this method randomly selects an angle... ,by geometric center Using the axis of rotation, for a rectangular area Perform rotation, rotation transformation matrix Defined as: ； in: ； ； The new image obtained after rotation transformation It can be obtained through the following transformation: ； In the above formula, yes The inverse matrix is ​​used to map from the transformed space back to the original space. These are the coordinates of the transformed image. Then, the choice is made between replacing the grayscale image with all color channels or only with a randomly selected color channel, providing more variations for model training. S24: Due to the vertical symmetry of human body features, vertical segmentation may damage critical body parts. Therefore, only horizontal occlusion simulation is considered. The image is divided into N equal-sized blocks along the horizontal axis, the 2D coordinates of each block are calculated, and stored in a coordinate list. Then, assign an index to each block and generate a list of indexes. At this point, the equally segmented image can be represented as In the real world, occlusion is diverse and unpredictable, appearing randomly in any location in space; the number of sampled blocks n is obtained by sampling the segmented image blocks according to N / 2, and then using the coordinate list... The indices of n randomly selected blocks are given by the following formula: ； S25: The image is blended proportionally according to the index to effectively cover different locations in space, as shown in the following form: ； Since the generated occluded images contain labels from different identities, providing only one identity label during the network input stage may disrupt the model's recognition ability; therefore, for the label... and Set a combination ratio Combine them; here Represented as The formula is expressed as follows: ； Thus, new training samples are obtained. ;because It is obtained by fusing two segmented images, but each image needs to be optimized with IDRows and TripletLoss, so the loss function is redefined: ； 。 3. The occlusion-based pedestrian re-identification method according to claim 2, characterized in that: Step S3 proposes constructing a backbone network, which includes the following steps: S31: Given the input image Where H, W, and C represent the height, width, and number of channels of the image, respectively; S32: Divide the image X into N patches using a sliding window. N can be represented as: ； In the above formula, S and P represent the step size of the sliding window and the size of the image patch, respectively; since vit requires a sequence as input, each patch is processed through a linear projection function. Embedded into D-dimensional space; S33: Add an additional learnable class token Appended to the input sequence to capture the semantic information of pedestrians and used as a global feature representation for the encoder. A learnable position embedding is added to the patch embedding before it is fed into the transformer block. and camera embedded These are used to preserve the image's location information and camera-specific information, respectively. The embedding process of the input sequence can be described as follows: ； In the above formula, It is a hyperparameter used to balance the camera embedding weights; S34: Input Embedding The encoder's final output will be processed by L transformer layers. It is divided into two parts, including global features. and local features To learn more distinctive pedestrian features, local features are divided into K groups to form K groups of local patch features. The size of each group is Set a local class token in the local feature. The local class tokens and features of each group of local patches are concatenated and fed into a shared transformer layer to learn local classification features of K groups. .

4. The occlusion-based pedestrian re-identification method according to claim 3, characterized in that: Step S5 proposes constructing a local-global feature coupling module, the specific steps of which include: S51: Local Classification Features and local patch features They are spliced ​​together separately, and then reshaped to obtain the characteristic shape. and ; S52: A detailed analysis of the local classification feature branch is performed, and the same process applies to the other branch; firstly, the multidimensional attention mechanism of ODconv is used to mine detailed features from images at multiple views and scales. Unlike directly acting on the feature map, a learnable dynamic weight is initialized in the dynamic convolution branch. This increases the weight of the main line, allowing the dynamically convolutional parts to complement each other. The specific process is as follows: ； S53: Utilize the MHSA mechanism and hierarchical feature extraction of the Transformer encoder to aggregate more subtle differences; prior to this, generate a token of a specified size based on the feature map of one branch, as described below: ； In the above formula, the token Global information representing the characteristics of local patches; S54: The outputs of the two branches after ODconv are represented as follows and Then with The cascaded feeds are then sent to the Transformer encoder for self-attention calculation. The specific calculation formula is described below: ； In the above formula, D represents the number of channels. Represents matrix multiplication; , and They respectively represent the process of passing through. and The query matrix, key matrix, and value matrix obtained by joining are, i.e. , ,and ; , and These represent different weight matrices; subsequently, the output will be... Feeding into the FFN network to enhance embedded representation capabilities; the final output of the local classification feature branch is represented as: Similarly, the output of the other branch is: Next, the outputs of the two branches are added together to obtain the integrated features. ; S55: Further extract and enhance ensemble features by employing a feature extractor consisting of convolutional and pooling layers to obtain locally enhanced features. The formula is as follows: ； S56: Define a learnable weight matrix ,in To represent the number of categories, the weight matrix... and conduct The product forces the model to prioritize categories that contribute to local features; it introduces global features. Combined with the above weights The product yields weights with higher classification accuracy. The specific process is described as follows: ； S57: The above operations enhance classification accuracy by coupling local features and global representations at multiple scales. This is typically achieved using... To optimize this, we need to consider that there is significant room for improvement in increasing the inter-class distance between dissimilar samples and decreasing the intra-class distance between similar samples, in the original... Add an additional angle margin for optimization, as shown in the following formula: 。 5. The occlusion-based pedestrian re-identification method according to claim 4, characterized in that: Step S6 involves constructing a loss function, specifically including the following steps: S61: Re-ID is considered an image classification task, where each ID represents a class; therefore, IDloss is used to train the model. ； In the above formula, B represents Number of images in the middle express Identified as The predicted probability; S62: To reduce intra-class distance and increase inter-class distance, triplet loss is used for optimization. ； In the above formula, It is an anchor point sample; It is a positive sample; It is a negative sample. express ; S63: Finally, the loss function can be expressed by the following formula: ； In the above formula, and They are balance and Hyperparameters.

Images