A method for re-identifying pedestrians in different outfits based on context awareness and gait features

Abstract

This invention relates to the field of computer vision, and more particularly to a method for re-identifying pedestrians in different clothing based on context awareness and gait features. The adaptive multi-scale feature fusion algorithm introduced in this method improves the accuracy of person recognition under different clothing and environmental changes; edge-aware feature enhancement technology effectively enhances the model's ability to recognize person contours, especially against visually complex backgrounds; the comprehensive gait feature extraction framework enables the model to accurately capture subtle changes in gait, enhancing recognition accuracy; and the innovative design of the composite loss function improves the model's learning efficiency and ensures high recognition stability.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more particularly to a method for re-identifying pedestrians in different outfits based on context awareness and gait features. Background Technology

[0002] Re-identification of individuals (ReID) has always been a significant challenge in computer vision, especially in recognizing individuals after they have changed clothing. Existing ReID systems often suffer from decreased accuracy when dealing with appearance differences caused by clothing changes. To overcome this challenge, researchers are constantly exploring new datasets and algorithms to improve the accuracy and robustness of recognition in complex real-world scenarios. Current challenges include how to accurately extract and maintain consistent features of individuals under different clothing conditions, and how to effectively fuse multi-scale features to enhance recognition accuracy. Existing methods often rely on single-scale feature extraction, ignoring the impact of clothing changes on human contours and gait features, leading to a significant drop in recognition performance when clothing is changed. Furthermore, existing methods fail to fully utilize contextual information and detailed features, resulting in poor recognition performance in complex backgrounds or dynamic environments. Summary of the Invention

[0003] In view of the aforementioned shortcomings of existing technologies, this invention provides a method for pedestrian re-identification under different clothing and environmental changes based on context awareness and gait features, thereby improving the accuracy, efficiency, and stability of person recognition. The technical solution is as follows:

[0004] A method for re-identifying pedestrians in disguise based on context awareness and gait features includes:

[0005] Process (1) Obtain edge intensity map by integrating edge detectors, introduce edge information at the feature level, and use element-wise multiplication to enhance feature representation; use deep learning network to extract multi-scale features of human images, and dynamically adjust the fusion weight of each scale feature through adaptive algorithm;

[0006] Process (2) combines dilated convolution, self-attention mechanism, nonlocal operation and bidirectional LSTM network to form an integrated framework for comprehensive extraction and synthesis of gait features;

[0007] Process (3) Constructs a composite loss function to improve the learning efficiency of the model and ensure recognition stability;

[0008] In process (4), based on the composite loss function in process (3), gait features are matched in the database using a similarity algorithm, and the corresponding personnel information is returned.

[0009] Furthermore, process (1) includes:

[0010] S101: Input the original image of the person (I);

[0011] S102: Extracting multi-scale features using deep convolutional neural networks {F 1 ,F 2 ,…,F n}, where F i Represents the feature map at the i-th scale;

[0012] S103: Introduce an edge detector D and calculate the edge intensity map E = D(I);

[0013] S104: Define edge-aware features Where ⊙ represents element-wise multiplication to enhance edge information;

[0014] S105: Adaptive weights W are used i Fusion of edge-aware features at different scales Generate fusion feature F fusion :

[0015]

[0016] Furthermore, the deep convolutional neural network includes U-Net.

[0017] Furthermore, the edge detector employs the Sobel operator or the Canny edge detection algorithm.

[0018] Furthermore, process (2) includes:

[0019] S106: Construct a context module C using dilated convolution, from F fusion Extracting contextual feature information F context Thus, the character outline mask S is obtained;

[0020] S107: Input a human figure contour mask S. The encoder first performs a series of convolutional operations to extract low-level and high-level features, generating a feature map set Z, i.e.:

[0021] Z = {z1, z2, ... z} n}=Q(S;α Q )

[0022] Where, α Q These are encoder parameters, z i It is the feature map of the i-th layer;

[0023] S108: Design a contour space mapper C. For the position of each pixel, generate a two-dimensional Gaussian distribution to capture position information and enhance the central region of the contour, represented as:

[0024] c embed =C(position; τ)

[0025] Where τ is used to control the width of the Gaussian distribution;

[0026] S109: Feature fusionist F combines spatial mapping and feature map sets to generate enhanced feature maps:

[0027] z fusion =F(Z,c embed ;α F )

[0028] Where, α F These are the parameters of the feature fusion unit;

[0029] S110: Enhance the detail information of the feature map using the dynamic attention mechanism V.

[0030] z attention =V(z) fusion ;α V )

[0031] Where, α V These are parameters of the dynamic attention mechanism;

[0032] S111: Decoder W utilizes enhanced feature map z attention Predict the outline of the person, and output the following:

[0033] T = W(z) attention ;α W )

[0034] Where, α W These are the parameters of the decoder;

[0035] S113: The result T output from the input decoder is processed through multiple convolutional and pooling layers to extract the original gait features, represented as:

[0036] G raw =ConvPool(R)

[0037] Among them, G raw This represents the original gait feature vector extracted from the result R;

[0038] S114: A self-attention mechanism is used to weight the features at each location in order to capture long-distance dependencies, as shown below:

[0039] G selfatt =SelfAttention(G raw )

[0040] Among them, G selfattIt is a gait feature weighted by a self-attention mechanism;

[0041] S115: Regarding G selfatt Performing nonlocal operations to achieve global feature enhancement is represented as:

[0042] G nonlocal =NonLocal(G selfatt )

[0043] S116: Using bidirectional LSTM to process G nonlocal Capture the forward and backward timing information, represented as:

[0044] G temp =BiLSTM(G nonlocal )

[0045] Among them, G temp Gait features that include preceding and following temporal information;

[0046] S117: Fusing various features G through a fully connected layer raw G selfatt G nonlocal and G temp , represented as:

[0047] G fusion =FC(G raw G selfatt G nonlocal G temp )

[0048] Among them, G fusion It is a comprehensive gait feature that combines information from original features, self-attention weighted features, nonlocal features, and bidirectional temporal features.

[0049] The process (3) includes:

[0050] S112: Calculate the dual fusion loss function L1, expressed as:

[0051] L1=γ1||T-GT|| 2 +γ2L regular

[0052] Where GT is the actual annotation result; γ is an adjustment coefficient; L regular It is a regularization term;

[0053] S118: Define semantic consistency loss L2 and maximum mean difference loss L3;

[0054] Semantic consistency loss is used to measure the mean squared error between features, and is expressed as:

[0055]

[0056] Where N is the number of feature dimensions, F context (i) represents the i-th dimension of the features described below, G fusion (i) represents the i-th dimension of the comprehensive gait features;

[0057] Maximum mean difference loss is used to measure the difference between two feature distributions, and is expressed as:

[0058]

[0059] Where J is the kernel function, used to calculate the similarity between features;

[0060] S119: Calculate the semantic consistency loss L2;

[0061] S120: Calculate the maximum mean difference loss L3;

[0062] S121: Calculate the comprehensive loss function L, expressed as...

[0063] L = L1 + L2 + L3.

[0064] Furthermore, process (4) includes:

[0065] S122: Calculate the gait features G of the input image. fusion Gait features G of images in the database db The cosine similarity sim is represented as:

[0066]

[0067] Where · represents the dot product, and ||·||2 is the L2 norm.

[0068] S123: Select from the database the match with G fusion The gait feature with the highest cosine similarity is identified and its identifier is recorded; based on the identifier with the highest similarity, the corresponding personnel information is returned.

[0069] This invention achieves the following technical effects:

[0070] The adaptive multi-scale feature fusion algorithm introduced in this method improves the accuracy of person recognition under different clothing and environmental changes; the edge-aware feature enhancement technology effectively enhances the model's ability to recognize person contours, especially in visually complex backgrounds; the comprehensive gait feature extraction framework enables the model to accurately capture subtle changes in gait, enhancing recognition accuracy; and the innovative design of the composite loss function improves the model's learning efficiency and ensures high recognition stability. Attached Figure Description

[0071] Figure 1 This is a flowchart of the pedestrian re-identification method based on context awareness and gait features according to the present invention. Detailed Implementation

[0072] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention.

[0073] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0074] like Figure 1 As shown, this invention presents a method for re-identifying pedestrians in different outfits based on context awareness and gait features, the core of which is:

[0075] (1) By integrating edge detectors to obtain edge intensity maps, edge information is introduced at the feature level, and element-level multiplication is used to strengthen feature representation, which effectively improves the accuracy of human contour recognition, especially in environments with a lot of visual noise; deep learning networks are used to extract multi-scale features of human images, and an adaptive algorithm is innovatively used to dynamically adjust the fusion weight of features at each scale by relying on the learning method of small neural networks, which significantly improves the performance of human recognition under different clothing and environmental conditions.

[0076] (2) By combining dilated convolution, self-attention mechanism, non-local operation and bidirectional LSTM network, an integrated framework is formed to comprehensively extract and synthesize gait features. This framework can effectively capture the dynamic changes of gait and provide a new and efficient approach for gait recognition.

[0077] (3) A composite loss function design was adopted, including dual fusion loss function, semantic consistency loss and maximum mean difference loss. This design ensures the accuracy of feature extraction while strengthening the model's learning of human contour and gait features, thereby improving the overall robustness and accuracy of the recognition system.

[0078] Finally, based on the composite loss function in (3), gait features are matched in the database using a similarity algorithm, and the corresponding personnel information is returned.

[0079] Example 1:

[0080] like Figure 1 As shown in the figure, this embodiment presents a processing flow for a pedestrian re-identification method based on context awareness and gait features, including:

[0081] S101: Input the original image of the person.

[0082] S102: Extracting multi-scale features using deep convolutional neural networks (such as U-Net) {F 1 ,F 2 ,…,F n}, where F i This represents the feature map at the i-th scale.

[0083] S103: Introduce an edge detector D, such as the Sobel operator or the Canny edge detection algorithm, to calculate the edge intensity map E = D(I).

[0084] S104: Define edge-aware features Here, ⊙ represents element-wise multiplication to enhance edge information.

[0085] S105: Adaptive weights W are used i Fusion of edge-aware features at different scales Generate fusion feature F fusion :

[0086]

[0087] Wherein, weight W i The weights are learned through a small neural network N, which takes the features and edge maps at the current scale as input and outputs the corresponding weights.

[0088] W i =N(F i E)

[0089] From S101 to S105, edge intensity maps are obtained by integrating edge detectors, and edge information is introduced at the feature level. Element-wise multiplication is used to enhance feature representation, which effectively improves the accuracy of human contour recognition, especially in environments with a lot of visual noise. Deep learning networks are used to extract multi-scale features of human images. An innovative adaptive algorithm is used to dynamically adjust the fusion weights of features at each scale by relying on a small neural network learning method, which significantly improves the performance of human recognition under different clothing and environmental conditions.

[0090] S106: Construct a context module C using dilated convolution, from F fusion Extracting broad contextual feature information F context This allows for the acquisition of a more accurate character outline mask S.

[0091] S107: Input a human figure contour mask S. The encoder first performs a series of convolutional operations to extract low-level and high-level features, generating a feature map set Z, i.e.:

[0092] Z = {z1, z2, ... z} n}=Q(S;αQ )

[0093] Where, α Q These are encoder parameters, z i It is the feature map of the i-th layer.

[0094] S108: Design a contour space mapper C. For the position of each pixel, generate a two-dimensional Gaussian distribution to capture position information and enhance the central region of the contour, represented as:

[0095] c embed =C(position; τ)

[0096] Here, τ controls the width of the Gaussian distribution.

[0097] S109: Feature fusionist F combines spatial mapping and feature map sets to generate enhanced feature maps:

[0098] z fusion =F(Z,c embed ;α F )

[0099] Where, α F These are the parameters of the feature fusion processor.

[0100] S110: Using dynamic attention mechanism V to further enhance the detailed information of the feature maps:

[0101] z attention =V(z) fusion ;α V )

[0102] Where, α V These are parameters of the dynamic attention mechanism.

[0103] S111: Decoder W utilizes enhanced feature map z attention Predict the outline of the person, and output the following:

[0104] T = W(z) attention ;α W )

[0105] Where, α W These are the parameters of the decoder.

[0106] S112: Calculate the dual fusion loss function L1 to ensure the model can accurately reconstruct the input human figure outline, expressed as:

[0107] L1=γ1||T-GT|| 2 +γ2L regular

[0108] Here, GT represents the actual annotation result, and γ is an adjustment coefficient whose optimal value can be determined through cross-validation. regular It is a regularization term to prevent overfitting.

[0109] S113: The result T output from the input decoder is processed through multiple convolutional and pooling layers to extract the original gait features, represented as:

[0110] G raw =ConvPool(R)

[0111] Among them, G raw This represents the original gait feature vector extracted from the result R.

[0112] Preferably, the following steps are used to improve the discriminative power of gait features and form comprehensive gait features;

[0113] S114: A self-attention mechanism is used to weight the features at each location in order to capture long-distance dependencies, as shown below:

[0114] G selfatt =SelfAttention(G raw )

[0115] Among them, G selfatt It is a gait feature weighted by a self-attention mechanism.

[0116] S115: Regarding G selfatt Performing nonlocal operations allows each feature to interact with all global features, achieving global feature enhancement, as represented by:

[0117] G nonlocal =NonLocal(G selfatt )

[0118] S116: Using Bi-LSTM (Bi-LSTM) to process G nonlocal Capture the forward and backward timing information, represented as:

[0119] G temp =BiLSTM(G nonlocal )

[0120] Among them, G temp Gait features that include preceding and following temporal information.

[0121] S117: Fusing various features G through a fully connected layer raw G selfatt、 G nonlocal and G temp , is represented as:

[0122] G fusion =FC(G raw G selfattt G nonlocal G temp )

[0123] Among them, G fusion It is a comprehensive gait feature that combines information from original features, self-attention weighted features, nonlocal features, and bidirectional temporal features.

[0124] In steps S106 to S117, dilated convolution, self-attention mechanism, nonlocal operation and bidirectional LSTM network are combined to form an integrated framework, and gait features are comprehensively extracted and integrated through this framework.

[0125] S118: To ensure the extraction of contextual feature information F context With integrated gait features G fusion To assess semantic consistency, we define and compute two types of losses: semantic consistency loss L2 and maximum mean difference loss L3.

[0126] The semantic consistency loss measures the mean squared error between features, expressed as:

[0127]

[0128] Where N is the number of feature dimensions, F context (i) represents the i-th dimension of the features described below, G fusion (i) represents the i-th dimension of the comprehensive gait features.

[0129] The maximum mean difference loss measures the difference between two feature distributions, expressed as:

[0130]

[0131] Where K is the kernel function, used to calculate the similarity between features.

[0132] S119: Calculate the semantic consistency loss L2;

[0133] S120: Calculate the maximum mean difference loss L3;

[0134] S121: The comprehensive loss function is calculated as L = L1 + L2 + L3.

[0135] In this embodiment, a composite loss function design is adopted, which includes a dual fusion loss function (S112), semantic consistency loss, and maximum mean difference loss. This design ensures the accuracy of feature extraction while strengthening the model's learning of human contour and gait features, thereby improving the overall robustness and accuracy of the recognition system.

[0136] S122: Calculate the gait features G of the input image. fusion Gait features G of images in the database db The cosine similarity sim is represented as:

[0137]

[0138] Where · represents the dot product, and ||·||2 is the L2 norm (i.e., the Euclidean length of the vector).

[0139] In this step, the cosine similarity algorithm is the preferred similarity algorithm. Cosine similarity is independent of vector length and is suitable for measuring text similarity, user similarity, item similarity, etc.

[0140] S123: Select from the database the match with G fusion The gait feature with the highest cosine similarity is identified and its identifier is recorded. Based on the identifier with the highest similarity, the corresponding personnel information is returned.

[0141] The adaptive multi-scale feature fusion algorithm introduced in this invention improves the accuracy of person recognition under different clothing and environmental changes; the edge-aware feature enhancement technology effectively enhances the model's ability to recognize person outlines, especially in visually complex backgrounds; the comprehensive gait feature extraction framework enables the model to accurately capture subtle changes in gait, enhancing recognition accuracy; the innovative design of the composite loss function improves the model's learning efficiency and ensures high recognition stability.

[0142] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A method for re-identifying pedestrians in disguise based on context awareness and gait features, characterized in that, include: S101: Input the original image of the person ; S102: Extracting Multi-Scale Features Using Deep Convolutional Neural Networks ,in Indicates the first Feature maps at various scales; S103: Introduction of edge detector Calculate the edge intensity map ; S104: Define edge-aware features ,in Represents element-wise multiplication to enhance edge information; S105: Adaptive weighting is adopted Fusion of edge-aware features at different scales Generate fusion features : S106: Construct a context module using dilated convolutions Fusion features Extracting contextual feature information This allows us to obtain the character's outline mask. ; S107: Input a character outline mask The encoder Q first performs a series of convolutional operations to extract low-level and high-level features, generating a set of feature maps. ,Right now: in, These are encoder parameters. It is the first Feature map of the layer; S108: Design Contour Space Mapper For each pixel's position, a two-dimensional Gaussian distribution is generated to capture positional information, enhancing the central region of the contour, represented as: position in, Used to control the width of the Gaussian distribution; S109: Feature Fusioner Combine spatial mapping and feature map sets to generate enhanced feature maps: in, These are the parameters of the feature fusion unit; S110: Using dynamic attention mechanisms Enhanced feature map details: in, These are parameters of the dynamic attention mechanism; S111: Decoder Utilizing enhanced feature maps Predict the outline of the person, and output the following: in, These are the parameters of the decoder; S112: Calculate the dual fusion loss function , is represented as: in, These are the actual annotation results; It is a regularization term; S113: The result of input decoder output The original gait features are extracted through multiple convolutional and pooling layers, and are represented as follows: in, Representative from the result The extracted raw gait feature vector; S114: A self-attention mechanism is used to weight the features at each location in order to capture long-distance dependencies, represented as: in, It is a gait feature weighted by a self-attention mechanism; S115: Yes Performing nonlocal operations to achieve global feature enhancement is represented as: G nonlocal =NonLocal(G selfatt ) S116: Using bidirectional LSTM to process G nonlocal Capture the forward and backward timing information, represented as: G temp =BiLSTM(G nonlocal ) in, Gait features that include preceding and following temporal information; S117: Fusing various features G through a fully connected layer raw G selfatt G nonlocal and , is represented as: G fusion =FC(G raw ,G selfatt ,G nonlocal ,G temp ) in, It is a comprehensive gait feature that combines information from original features, self-attention weighted features, nonlocal features, and bidirectional temporal features; S118: Define semantic consistency loss and maximum mean difference loss ; Semantic consistency loss is used to measure the mean squared error between features, and is expressed as: in, The number of feature dimensions. The first feature of the following text dimension, The first comprehensive gait feature dimension; Maximum mean difference loss is used to measure the difference between two feature distributions, and is expressed as: in, This is a kernel function used to calculate the similarity between features; S119: Calculate semantic consistency loss ; S120: Calculate the maximum mean difference loss ; S121: Calculate the comprehensive loss function , represented as S122: Calculate the comprehensive gait features of the input image. Gait features of images in the database cosine similarity , is represented as: Where · represents the dot product, It is an L2 norm; S123: Select from the database and The gait feature with the highest cosine similarity is identified and its identifier is recorded; based on the identifier with the highest similarity, the corresponding personnel information is returned.

2. The method for re-identifying pedestrians in disguise based on context awareness and gait features as described in claim 1, characterized in that, The deep convolutional neural network includes U-Net.

3. The method for re-identifying pedestrians in disguise based on context awareness and gait features as described in claim 1, characterized in that, The edge detector uses either the Sobel operator or the Canny edge detection algorithm.

Images