Visible light-infrared pedestrian re-identification method based on hypergraph-driven soft semantic dynamic learning

Abstract

The invention discloses a visible light-infrared pedestrian re-identification method based on hypergraph-driven soft semantic dynamic learning, and the method comprises the following steps: dividing a pedestrian image into a training set and a test set, and carrying out the data enhancement; pedestrian hierarchical soft semantic features are extracted by using a self-attention mechanism; performing cross-stage feature fusion in channel and space directions by using a gating mechanism; the modal alignment loss is calculated, and the difference between visible light and infrared characteristics is reduced; dynamically exploring a relationship between the extracted soft semantics by using a hypergraph neural network; training the network by using the joint loss until convergence; and re-identifying pedestrians in the test set. By using the method, hierarchical soft semantic features are effectively captured and dynamically fused, modal-invariant features are extracted, the difference between visible light and infrared modals is bridged, the high-order relationship between semantics can be adaptively explored, modal-invariant semantics are effectively learned, the difference between modals is reduced, and the robustness of the method is improved. And the visible light-infrared pedestrian re-identification effect is obviously improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and pedestrian re-identification, and in particular relates to a visible light-infrared pedestrian re-identification method based on hypergraph driven soft semantic dynamic learning. Background Art

[0002] Person re-identification aims to match the image of a specific pedestrian with the gallery images from different cameras. Pedestrian re-identification technology plays a vital role in the fields of intelligent security, traffic monitoring and smart supermarkets. It can be applied in areas such as tracking the trajectory of criminal suspects, statistical analysis of human flow, and customer identity recognition in smart supermarkets.

[0003] In recent years, the visible light pedestrian re-identification task has made great progress, but the visible light camera cannot effectively capture pedestrian information in low light or nighttime environments, which limits its application in all-weather and all-scenario situations. To solve this problem, many of the latest surveillance systems use infrared cameras to capture pedestrian images under low light conditions. Therefore, the visible light-infrared pedestrian re-identification task has received increasing attention. It can directly retrieve images of one modality (visible or infrared) through images of another modality, and can be applied in all weather and all-scenario situations under various lighting conditions, which has great application value in the industrial field.

[0004] At present, the research methods for visible-infrared pedestrian re-identification can be roughly divided into two categories: (1) image-level methods, which use generative adversarial networks to convert infrared (or visible) images into corresponding visible (or infrared) images to reduce the modality gap. Liu et al. introduced an intermediate modality to enhance the effect of cross-modal retrieval (Liu H, Ma S, Xia D, et al. SFANet: A Spectrum-Aware Feature Augmentation Network for Visible-Infrared Person Reidentification [J] / / IEEE Transactions on Neural Networks and Learning Systems, 2023, 34 (4): 1958-1971). However, these methods require precisely aligned visible-infrared image pairs, as well as consistent perspective, time and resolution for image generation. Due to the lack of real precisely aligned visible-infrared image pairs, the generated cross-modal images usually contain noise. (2) Feature-level methods, which attempt to map visible and infrared features into a common embedding space to minimize the modality gap. Zhang et al. proposed a diversified embedding generation network to learn informative feature representations and reduce the modality difference between visible and infrared images (Zhang Y, Wang H. Diverse embedding expansion network and low-light cross-modality benchmark for visible-infrared person re-identification [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2023: 2153-2162). However, the huge modality gap makes modality alignment and cross-modality mapping difficult.

[0005] In recent years, popular VIR person re-ID methods mainly learn hard semantic features shared by modalities, such as features of specific body parts (such as arms, legs, feet, etc.), to narrow the modal differences between visible light and infrared. Although existing methods are effective, they ignore two important factors. (1) The sensitivity of hard semantic features to cross-modal changes. Hard semantic features have significant modality specificity and are extremely sensitive to cross-modal changes, which makes it difficult to effectively align modalities. In contrast, soft semantic features contain more modality-invariant information. These features can be dynamically learned through deep models, thereby more effectively aligning modalities and reducing the differences between modalities. (2) Learning high-order relationships between semantic features. Exploring high-order relationships between soft semantics (such as the association between local texture details and overall appearance) can effectively promote the learning of modality-invariant features, thereby achieving more accurate modality alignment. Therefore, understanding the potential connections between soft semantics is of great significance for the VIR person re-ID task. However, most current methods still focus on learning hard semantic relationships between features of different body parts to reduce modality differences, ignoring the potential high-level associations between soft semantics and failing to effectively bridge the modality gap. The present invention aims to solve the above two key visible light-infrared pedestrian re-identification problems.

[0006] In response to the above problems, the present invention proposes a visible light-infrared pedestrian re-identification method based on hypergraph-driven soft semantic dynamic learning, which aims to extract modality-invariant soft semantic features to enable the model to perform modal alignment more effectively. In addition, the present invention dynamically explores the potential associations between the extracted soft semantics and further mines high-order modality-invariant features, so that the model can better bridge the modality differences between visible light and infrared images and improve the performance of visible light-infrared pedestrian re-identification. Summary of the invention

[0007] In order to solve the above technical problems, the purpose of the present invention is to provide a visible light-infrared pedestrian re-identification method based on hypergraph driven soft semantics dynamic learning. This method can capture hierarchical soft semantic features in channel and spatial dimensions, and dynamically fuse them through a gating mechanism, thereby effectively aligning the modalities. In addition, this method uses a hypergraph neural network to explore the potential relationships between the captured soft semantics, and dynamically learns these semantics through a gating mechanism to further mine high-order modality-invariant features. Therefore, this method effectively reduces the modal difference between visible light and infrared images and improves the performance of visible light-infrared pedestrian re-identification.

[0008] The technical solution of the present invention:

[0009] A visible light-infrared pedestrian re-identification method based on hypergraph driven soft semantic dynamic learning, comprising the following steps:

[0010] Step 1: Divide pedestrian images into training sets and test sets, and perform data augmentation;

[0011] make represents the RGB image of identity i, Indicates infrared image. Randomly selected One channel (R, G, or B) is copied to the other channels to produce a grayscale image Will Each channel of the image is multiplied by a random factor between 0.02 and 1 to generate a pseudo-color image. The final enhanced sample is

[0012] Use the extractor E(·) to extract four sets of enhanced images The four groups of images are input into four independent extractors to obtain RGB features. Grayscale Features Infrared signature False color infrared signature

[0013]

[0014] In the testing phase, features of the same modality are fused:

[0015]

[0016] Where α is the fusion weight, which is set to 0.5.

[0017] No feature fusion is performed during the training phase to encourage the extractor to extract diverse modal features. The output of the feature extractor during the training phase is In the testing phase

[0018] Step 2: Build a soft semantic extraction module and use the self-attention mechanism to extract the hierarchical soft semantic features of pedestrians;

[0019] The soft semantic extraction module uses ResNet50 as the backbone network, where ResNet50 consists of five stages: stage-0, stage-1, stage-2, stage-3, and stage-4. Consider two soft semantic features before and after each stage: the low-level features before the stage and the high-level features after this stage Among them, C l , W l , H l and C h , W h , H hRepresents the number of channels, width, and height of low-level features and high-level features respectively.

[0020] First, the attention mechanism is used to perform cross-stage feature fusion in the channel direction. Three 1×1 convolutional layers Q1, V1 and K1 are used to transform f h and f l Transformed into three embeddings: Q1(f h ), V1(f l ) and K1(f l ). Then, the channel-wise similarity matrix M is calculated by matrix multiplication and softmax function. c :

[0021] M c =Softmax(Q1(f h )×K1(f l ) T )(4)

[0022] Step 3: Use the gating mechanism to perform cross-stage feature fusion in channel and spatial directions;

[0023] Next, by changing V1(f l ) and M c Matrix multiplication is performed to achieve cross-stage feature fusion in the channel direction and restore the channel dimension. After that, another 1×1 convolution layer ω c Adjust the feature map to be consistent with f h The same size, get the fused features

[0024]

[0025] In order to enhance the flexibility of feature fusion, a gating mechanism is introduced on the feature map. y Apply the sigmoid function σ to get a gate value gate between 0 and 1 c . Use gate c Dynamically and f h Perform weighted fusion to obtain cross-stage fusion features of the channel stage

[0026]

[0027] Then, use and f l Perform cross-stage feature fusion in the spatial direction, similar to the fusion in the channel direction, to obtain the output

[0028]

[0029] Where gate s is the gating value in the spatial direction, It is the cross-stage fusion feature of spatial direction;

[0030] Step 4: Calculate the modal alignment loss to reduce the difference between visible light and infrared features;

[0031] In order to reduce the modal differences between the four features extracted by the feature extractor, a modal alignment loss is designed to narrow the distance between the two modal features.

[0032] The features extracted by the model are i Includes four sets of features for visible and infrared images:

[0033]

[0034] Compute the eigencenters of the two modes:

[0035]

[0036] where c V represents the visible light feature center, c T represents the infrared feature center, i represents the identity, and N is the number of samples of identity i.

[0037] To encourage the features of one modality of the same person to be close to the center of his features of another modality, a modality alignment loss is calculated:

[0038]

[0039] Step 5: Construct a soft semantics dynamic learning module and use a hypergraph neural network to dynamically explore the relationship between the extracted soft semantics;

[0040] First, use K generators U d (d=1,2,…,K), each U d It is a 1×1 convolutional layer that generates multiple semantic type graphs S = [S1, S2, …, S K ], each generator focuses on a different soft semantic type graph S i :

[0041] S d =σ(U d (F))(d=1,2,…,K)(12)

[0042] Where σ is the Sigmoid activation function.

[0043] Next, K types of soft semantic features are extracted by multiplying the K soft semantic type graphs with the original feature F:

[0044] f d =S d ⊙F(d=1,2,…,K)(13)

[0045] where ⊙ represents element-wise multiplication.

[0046] The soft semantic feature f of the dth type d =R b×c×h×w The node representation f is obtained by flattening d =R b×c×hw , where b, c, h, and w represent the batch size, number of channels, height, and width of the feature, respectively. Build a hypergraph for each semantic type HG={v1,…,v N}, the hyperedge set ε={e1,…,e M}, W is the hyperedge weight matrix. Each 1×1×C grid is regarded as a node, and the node is represented as All nodes form Z = [z1; ...; z N ]∈R N×C .

[0047] Then, calculate the Euclidean distance matrix B between nodes:

[0048] B ij =|z i -z j |2,(i,j=1,…,N)(14)

[0049] Select k nearest neighbors for each node and construct a hyperedge e i . Construct the association matrix H as follows:

[0050]

[0051] Among them, H ij is the element in the i-th row and j-th column of the incidence matrix H, B ij is the Euclidean distance between nodes i and j, is the average distance from node i to other nodes.

[0052] Based on H, a hypergraph convolution operation is applied to aggregate high-order semantic information, and ded enhances the feature representation R i :

[0053]

[0054] Where W is the weight matrix, D v and D e They are the node degree matrix and the hyperedge degree matrix, H is the association matrix, and f dis the soft semantic feature of the dth type, and Θ is a learnable parameter. Then, a gating mechanism is used to dynamically learn high-order semantics. A linear layer and Sigmoid activation are applied to generate the gating value

[0055]

[0056] Where W g is the learnable weight matrix, b g is the bias term, and σ is the Sigmoid activation function.

[0057] Each type of gating With high-level semantic enhancement feature R d Element-wise multiplication:

[0058]

[0059] R d Perform global average pooling g(·) to obtain the pooling feature p d :

[0060] p d =g(R d )(19)

[0061] Then, each type of feature is concatenated with the global feature to obtain the output F of the soft semantic dynamic learning module. ′ :

[0062]

[0063] Finally, we introduce type separation loss to make the semantic type graph extract diversified soft semantics. h×w×K Reshape into S hw×K , and define the type separation loss L sep for:

[0064]

[0065] Among them, (S T S) ij is the similarity between the i-th and j-th type graphs;

[0066] Step 6: Train the network using the joint loss until convergence;

[0067] Joint loss L total The definition is as follows:

[0068] L total =L id +L sep +L ma (twenty two)

[0069] Among them, L id represents ID loss, L sep represents the type separation loss, L ma represents the modality alignment loss, and the ID loss is calculated as:

[0070]

[0071] Where M is the total number of samples, C is the number of identities, and t ij is the true label of sample i corresponding to category j, y ij is the predicted probability that sample i output by the model belongs to category j;

[0072] Step 7: Input the test set images into the soft semantic extraction module constructed in step 2 to extract the soft semantic features of pedestrians, and then input the extracted features into the soft semantic dynamic learning module constructed in step 5 to dynamically learn the potential relationship between the soft semantic features and obtain the output features after relationship enhancement;

[0073] Input the test set images into the module constructed in step 2 and step 5 to obtain the output features, and input them into the classifier to calculate the predicted value of each identity category. For the predicted value x of the i-th category i , first calculate its L2 norm norm2:

[0074]

[0075] Where C is the number of categories; then the predicted value is normalized to obtain the predicted probability prob of the ath category a :

[0076]

[0077] The predicted probability of each identity category is sorted in descending order, and the pedestrian identity with the highest probability is selected as the identity matching result of the input image.

[0078] The beneficial effects of the present invention are as follows: the present invention proposes a novel visible light-infrared pedestrian re-identification method based on hypergraph-driven soft semantics dynamic learning, which effectively bridges the modal gap by learning soft semantics. First, the present invention captures and dynamically fuses low-level and high-level semantic information through a self-attention network, extracts modality-invariant soft semantic features, and thus effectively bridges the modal gap. Through information fusion in channel and spatial dimensions, the model can better handle the differences between different modalities. Secondly, the present invention adaptively explores the high-order relationships between the extracted soft semantics through a hypergraph neural network, effectively learns the modality-invariant soft semantic features between the visible light and infrared modalities, effectively narrows the modal differences, and significantly enhances the accuracy of visible light-infrared pedestrian re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a flow chart of the visible light-infrared pedestrian re-identification method based on hypergraph driven soft semantic dynamic learning of the present invention;

[0080] Figure 2 It is an overall framework diagram of a specific embodiment of the present invention;

[0081] Figure 3 The figure is a flow chart of the test steps of a specific embodiment of the present invention. DETAILED DESCRIPTION

[0082] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0083] like Figure 1 As shown in the figure, a visible light-infrared pedestrian re-identification method based on hypergraph driven soft semantic dynamic learning is implemented as follows:

[0084] 1. Data Augmentation

[0085] For each identity i, the RGB image is represented as The infrared image is represented as Randomly select a color channel (red, green or blue) from the RGB image and copy the pixel values ​​of that channel to the other channels to obtain a grayscale image At the same time, the infrared image is enhanced by pseudo color, and the pixel value of each channel is multiplied by a random factor between 0.02 and 1 to obtain a pseudo color image. In this way, four sets of enhanced image data are obtained: RGB image Grayscale images Infrared image And false color infrared images Next, the feature extractor E(·) is used to extract the four sets of enhanced images respectively. Features, get RGB features Grayscale Features Infrared signature False color infrared signature

[0086] In the test phase, the features of the same modality are fused. For the features of RGB images, first the features of the RGB images are fused. and grayscale image features Perform weighted fusion according to the same weight to obtain the feature For the features of infrared images, similarly, the features of infrared images are and the characteristics of pseudo-color infrared images Perform weighted fusion to obtain features During the testing phase, no feature fusion is performed.

[0087] 2. Dynamic extraction of soft semantics

[0088] First, three 1×1 convolutional layers (Q1, V1, K1) are used to convolutionally transform the high-level features f before each of the five stages (stage-0, stage-1, stage-2, stage-3, stage-4) of the ResNet50 backbone network. h And the following low-level features f l Convert to Q1(f h )、V1(f l ) and K1(f l ). Then, calculate the channel-wise similarity matrix M c :

[0089] M c =Softmax(Q1(f h )×K1(f l ) T )(26)

[0090] Next, by adding M c With V1(f l ) performs matrix multiplication, resizes the feature map, and uses another 1×1 convolutional layer ω c Process it to obtain intermediate features

[0091]

[0092] right Apply the sigmoid function to get the gate value gate c , using this value and f h Perform weighted fusion to obtain cross-stage fusion features in the channel direction

[0093]

[0094] Finally, use and f l Perform cross-stage feature fusion in the spatial direction, similar to the fusion in the channel direction, to obtain the extracted soft semantic features:

[0095]

[0096] Among them, gate s is the gating value in the spatial direction, It is the fusion feature of spatial direction.

[0097] 3. Dynamically learn the relationship between soft semantics

[0098] Use K generators U d (each is a 1×1 convolutional layer), d=1,2,…,K, generates K semantic type graphs S=[S1,S2,…,S K ]:

[0099] S d =σ(U d (F))(d=1,2,…,K)(31)

[0100] Where σ is the Sigmoid function.

[0101] Multiply the semantic type map with the original feature F to obtain K types of semantic feature maps f d :

[0102] f d =S d ⊙F(d=1,2,…,K)(32)

[0103] Among them, ⊙ represents element-by-element multiplication;

[0104] Constructing a hyperedge i And the association matrix H. Aggregate high-order semantic information through hypergraph convolution to obtain the semantic relationship enhanced feature R d :

[0105]

[0106] Where W is the weight matrix, D v and D e They are the node degree matrix and the hyperedge degree matrix, H is the association matrix, and f d is the soft semantic feature of the dth type, and Θ is a learnable parameter. Then, a gating mechanism is used to dynamically learn high-order semantics, generating gating values ​​through linear layers and Sigmoid And multiply the gated value with the enhanced feature:

[0107]

[0108] Where W g is the learnable weight matrix, b g is the bias term.

[0109] Perform global average pooling g(·) on the enhanced features to obtain the pooled features p d . Connect each type of pooling features with the global features to get the final output F′:

[0110]

[0111] 4. Use joint loss to train the network

[0112] The network is trained for 140 epochs using the joint loss L total The definition is as follows:

[0113] L total =L id +L sep +L ma (37)

[0114] Among them, L id represents ID loss, L sep represents the type separation loss, L ma represents the modality alignment loss.

[0115] 5. Test set pedestrian retrieval

[0116] like Figure 3 As shown in Figure 1, the pedestrian images in the visible light or infrared mode in the test set are input into the best network model trained in step 5 for feature extraction. The extracted features are then input into the fully connected layer of the classifier to calculate the predicted value of each identity category. For the predicted value x of the i-th category i , first calculate its L2 norm norm2:

[0117]

[0118] Among them, C is the number of categories; then the predicted value is normalized to obtain the predicted probability prob of the ath category a :

[0119]

[0120] The predicted probabilities of all identity categories are sorted in descending order, and the pedestrian identity with the highest probability is selected as the identity matching result of the input image. Finally, the image of the visible light or infrared modality is used to retrieve the corresponding identity image of the other modality to obtain the visible light-infrared pedestrian re-identification result.

[0121] In summary, the present invention discloses a visible light-infrared pedestrian re-identification method based on hypergraph-driven soft semantics dynamic learning. The method effectively solves the modal difference problem between visible light and infrared images by introducing hypergraph structure and soft semantic feature learning, and improves the performance of visible light-infrared pedestrian re-identification. First, a soft semantic extraction module is adopted to capture and dynamically fuse hierarchical soft semantic features to extract modality-invariant features, thereby bridging the gap between visible light and infrared modalities. The module can effectively extract modality-invariant semantic information by capturing low-level and high-level semantics in the channel and spatial dimensions respectively, and adaptively fusing them through a gating mechanism. Secondly, a soft semantics dynamic learning module based on a hypergraph neural network is proposed to adaptively explore high-order relationships between extracted soft semantics. The module improves the expressiveness of features by constructing a hypergraph structure and using hypergraph convolution to mine potential associations between soft semantics, thereby more effectively learning modality-invariant semantics and reducing the differences between modalities. Finally, through extensive experimental verification on the SYSU-MM01 and RegDB datasets, the method of the present invention achieved significant performance improvement in the visible light-infrared pedestrian re-identification task, surpassing the existing state-of-the-art methods, demonstrating its broad application prospects in visible light-infrared pedestrian re-identification.

Claims

1. A visible light-infrared pedestrian re-identification method based on hypergraph driven soft semantic dynamic learning, characterized in that: The following steps are involved: Step 1: Divide pedestrian images into training sets and test sets, and perform data augmentation; make represents the RGB image of identity i, Represents infrared images; randomly selected One channel R, G or B is copied to the other channels to generate a grayscale image Will Each channel of the image is multiplied by a random factor between 0.02 and 1 to generate a pseudo-color image. The final enhanced sample is Use the extractor E(·) to extract four sets of enhanced images Characteristics of four groups The image is input into four independent extractors to obtain RGB features. Grayscale Features Infrared signature False color infrared signature Fusion of features from the same modality during testing: Among them, α is the fusion weight, which is set to 0.5; No feature fusion is performed during the training phase to encourage the extractor to extract diverse modal features; the output of the extractor during the training phase is In the testing phase Step 2: Build a soft semantic extraction module and use the self-attention mechanism to extract the hierarchical soft semantic features of pedestrians; The soft semantic extraction module uses ResNet50 as the backbone network, where ResNet50 consists of five stages: stage-0, stage-1, stage-2, stage-3, and stage-4. Two soft semantic features are considered before and after each stage: the low-level features before the stage and the high-level features after this stage Among them, C l , W l , H l and C h , W h , H h Represents the number, width, and height of channels for low-level and high-level features respectively; First, the attention mechanism is used to perform cross-stage feature fusion in the channel direction; three 1×1 convolutional layers Q1, V1 and K1 are used to transform f h and f l Transformed into three embeddings: Q1(f h ), V1(f l ) and K1(f l ); Then, the channel-wise similarity matrix M is calculated by matrix multiplication and softmax function. c : M c =Softmax(Q1(f h )×K1(f l ) T )(4) Step 3: Use the gating mechanism to perform cross-stage feature fusion in channel and spatial directions; Next, by changing V1(f l ) and M c Matrix multiplication is performed to achieve cross-stage feature fusion in the channel direction and restore the channel dimension; then, another 1×1 convolution layer ω is passed c Adjust the feature map to be consistent with f h The same size, get the fused features In order to enhance the flexibility of feature fusion, a gating mechanism is introduced on the feature map; Apply the sigmoid function σ to get a gate value gate between 0 and 1 c ;Use gate c Dynamically and f h Perform weighted fusion to obtain cross-stage fusion features of the channel stage Then, use and f l Perform cross-stage feature fusion in the spatial direction, similar to the fusion in the channel direction, to obtain the output Among them, gate s is the gating value in the spatial direction, It is the cross-stage fusion feature of spatial direction; Step 4: Calculate the modal alignment loss to reduce the difference between visible light and infrared features; In order to reduce the modal differences between the four features extracted by the extractor, a modal alignment loss is designed to reduce the distance between the two modal features; The features extracted by the model are i Includes four sets of features for visible and infrared images: Compute the eigencenters of the two modes: where c V represents the visible light feature center, c T represents the infrared feature center, i represents the identity, and N is the number of samples of identity i; To encourage the features of one modality of the same person to be close to the center of his features of another modality, a modality alignment loss is calculated: Step 5: Construct a soft semantics dynamic learning module and use a hypergraph neural network to dynamically explore the relationship between the extracted soft semantics; First, use K generators U d , d=1,2,…,K, each U d It is a 1×1 convolutional layer that generates multiple semantic type graphs S = [S1, S2, …, S K ], each generator focuses on a different soft semantic type graph S i : S d =σ(U d (F)) (d=1,2,…,K)(12) Where σ is the Sigmoid function; Next, K types of soft semantic features are extracted by multiplying the K soft semantic type graphs with the original feature F: f d =S d ⊙F (d=1,2,…,K)(13) Among them, ⊙ represents element-by-element multiplication; The soft semantic feature f of the dth type d =R b×c×h×w The node representation f is obtained by flattening d =R b×c×hw , where b, c, h, and w represent the batch size, number of channels, height, and width of the feature respectively; a hypergraph is constructed for each soft semantic type HG={v1,…,v N }, the hyperedge set ε={e1,…,e M }, W is the hyperedge weight matrix; each 1×1×C grid is regarded as a node, and the node is represented as All nodes form Z = [z1; ...; z N ]∈R N×C ; Then, calculate the Euclidean distance matrix B between nodes: B ij =|z i -z j |2,(i,j=1,…,N)(14) Select k nearest neighbors for each node and construct a hyperedge e i ; Construct the association matrix H as follows: Among them, H ij is the element in the i-th row and j-th column of the incidence matrix H, B ij is the Euclidean distance between nodes i and j, is the average distance from node i to other nodes; Based on H, a hypergraph convolution operation is applied to aggregate high-order semantic information to obtain an enhanced feature representation R d : Among them, D v and D e They are the node degree matrix and the hyperedge degree matrix, H is the association matrix, and f d is the d-th type of soft semantic feature, Θ is a learnable parameter; then, a gating mechanism is used to dynamically learn high-order semantics; a linear layer and Sigmoid activation are applied to generate the gating value Among them, W g is the learnable weight matrix, b g is the bias term; Each type of gating With high-level semantic enhancement feature R d Element-wise multiplication: R d Perform global average pooling g(·) to obtain the pooling feature p d : p i =g(R d )(19) Then, each type of feature is concatenated with the global feature to obtain the output F of the soft semantic dynamic learning module. ′ : Finally, the type separation loss is introduced to make the semantic type graph extract diversified soft semantics; the type graph S h×w×K Reshape into S hw×K , and define the type separation loss L sep for: Among them, (S T S) ij is the similarity between the i-th and j-th type graphs; Step 6: Train the network using the joint loss until convergence; Joint loss L total The definition is as follows: L total =L id +L sep +L ma (22) Among them, L id represents ID loss, L sep represents the type separation loss, L ma represents the modality alignment loss, and the ID loss is calculated as: Where M is the total number of samples, C is the number of identities, and t ij is the true label of sample i corresponding to category j, y ij is the predicted probability that sample i output by the model belongs to category j; Step 7: Input the test set images into the soft semantic extraction module constructed in step 2 to extract the soft semantic features of pedestrians, and then input the extracted features into the soft semantic dynamic learning module constructed in step 5 to dynamically learn the potential relationship between the soft semantic features and obtain the output features after relationship enhancement; Input the test set images into the module constructed in step 2 and step 5 to obtain the output features, and input them into the classifier to calculate the predicted value of each identity category. For the predicted value x of the i-th category i , first calculate its L2 norm norm2: Among them, C is the number of categories; then the predicted value is normalized to obtain the predicted probability prob of the ath category a : The predicted probability of each identity category is sorted in descending order, and the pedestrian identity with the highest probability is selected as the identity matching result of the input image.

Images