Covered pedestrian re-identification method of multi-scale hypergraph connection with KAN structure

Abstract

The invention belongs to the field of image recognition, and discloses a multi-scale hypergraph connected shielded pedestrian re-recognition method with a KAN structure, which comprises the following steps of: 1, fusing three-dimensional representation information and two-dimensional image information of a pedestrian image by using a feature fusion module to obtain a fused feature V; step 2, using a multi-scale hypergraph module with a KAN structure to learn and transmit semantic information of different areas, and promoting transmission of useful semantic feature information; 3, using a three-dimensional representation mask to participate in training of a loss function, and eliminating the influence of inactive shielding features; and step 4, using the trained model to carry out identification on the occlusion data set. According to the invention, a plurality of local features beneficial to recognition of sheltered pedestrians are extracted. A KAN network is used for combining hypergraph transfer learning of a multi-scale structure and transferring semantic information of different areas, so that transfer of meaningless and noise features is inhibited, and the effect of shielding pedestrian re-identification is effectively improved; and the function of useful features is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to an occluded pedestrian re-identification method with a multi-scale hypergraph connection with a KAN structure. Background Art

[0002] Pedestrian re-identification (ReID) refers to a method of retrieving other image information of a pedestrian from other camera devices based on the image information of the pedestrian appearing in a certain camera device. This method is widely used in the field of intelligent security.

[0003] Existing pedestrian re-identification methods are mainly designed for complete unobstructed pedestrian features. Considering the complexity and variability of real-life scenes, there is a high possibility that pedestrians are occluded when people and objects move randomly. Occlusion will seriously affect visual information, making the affected features unreliable and reducing the accuracy of feature matching. Most of the existing occluded pedestrian re-identification methods based on deep learning CNN networks simply segment the two-dimensional pedestrian images or combine them with traditional two-dimensional posture estimation methods to obtain local features of pedestrians, and then use graph convolution or very few single-scale hypergraph convolutions to connect the features to improve the occlusion situation. However, the features extracted by such methods are not discriminative enough, and the connectivity between features is not high enough. The re-identification effect is not good when pedestrians are occluded. Summary of the invention

[0004] The purpose of the present invention is to disclose an occluded pedestrian re-identification method with a multi-scale hypergraph connection with a KAN structure to solve the technical problems raised in the background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention provides an occluded person re-identification method with a multi-scale hypergraph connection with a KAN structure, comprising:

[0007] Step 1: Use a feature fusion module to fuse the three-dimensional representation information and the two-dimensional image information of the pedestrian image to obtain the fused feature V;

[0008] Step 2: Use a multi-scale hypergraph module with a KAN structure to learn and transfer semantic information of different regions to promote the transfer of useful semantic feature information;

[0009] Step 3: Use the 3D representation mask to participate in the training of the loss function to eliminate the influence of inactive occlusion features;

[0010] Step 4: Use the trained model to perform recognition on the occlusion dataset.

[0011] Preferably, step 1 comprises:

[0012] Step 1.1 Use the MotionBERT 3D representation extractor to extract the 3D representation of the input pedestrian image;

[0013] Step 1.2 uses a resnet50 network with simple self-attention to extract two-dimensional features from the input pedestrian image;

[0014] Step 1.3 multiplies the extracted three-dimensional representation and the extracted pedestrian two-dimensional features element by element on the corresponding channels and sums them up, thereby obtaining the mapping of the pedestrian two-dimensional image information in the three-dimensional space as the feature V after the fusion of the pedestrian two-dimensional image feature and the three-dimensional representation feature.

[0015] Preferably, step 2 comprises:

[0016] Step 2.1 Construct the scale-1 hypergraph foundation: Construct the human body connection matrix H1, which contains three hyperedges e1, e2, and e3, corresponding to the three body attributes of the human body: arms, legs, and torso. Construct the corresponding diagonal matrix W1 with positive weights, calculate the corresponding node degree D1, and calculate the corresponding hyperedge degree B1;

[0017] Step 2.2 Construct the scale-2 hypergraph foundation: Construct the human body connection matrix H2, which includes the five hyperedges e1, e2, e3, e4, and e5, corresponding to the five body attributes of the left arm, right arm, left leg, right leg, and torso, construct the corresponding positive weight W2 diagonal matrix, calculate the corresponding node degree D2, and calculate the corresponding hyperedge degree B2;

[0018] Step 2.3: Construct the scaled three-hypergraph foundation: construct the human body connection matrix H3, which includes the ten hyperedges from e1 to e10, the ten body attributes formed by connecting the three adjacent key point node features, construct the corresponding positive weight W3 diagonal matrix, calculate the corresponding node degree D3, and calculate the corresponding hyperedge degree B3;

[0019] Step 2.4 According to the hypergraph convolution formula: X l+1 =σ(D -1 HWB -1 H T X l P), and perform three-scale hypergraph convolution on the input respectively; where W is a diagonal matrix storing the positive weights of each hyperedge; H is the association matrix between hyperedges and nodes, and H is set to 1 for nodes on hyperedges, otherwise it is set to 0; D is the degree matrix of the vertex, which represents the sum of the weights of all hyperedges associated with the vertex; B is the degree matrix of the hyperedge, which represents the sum of the number of vertices associated with the corresponding hyperedge; X is the feature input of layer 1, X l+1 is the output of layer l and the input of layer l+1, P is the weight matrix between layer l and layer l+1; σ is the nonlinear activation function;

[0020] Step 2.5: Send the hypergraph convolution result to the KAN network structure layer; use the spline function to train the appropriate activation function and continuously perform nonlinear fitting on the input data;

[0021] In step 2.6, the convolution output is finally processed through a series of operations including concatenation and pooling to output the final result.

[0022] Preferably, the ten body attributes include the left forearm, left upper arm, right forearm, right upper arm, left calf, left thigh, right calf, right thigh, head and upper torso of the human body.

[0023] Preferably, step 3 comprises:

[0024] Step 3.1 uses the 3D representation extracted by the MotionBERT 3D representation extractor to calculate its feature pixel category score;

[0025] Step 3.2 extracts the index corresponding to the pixel feature with the highest pixel category score in the feature, encodes the index using the one-hot encoding method, and converts the encoding result into a Boolean mask;

[0026] Step 3.3: The output feature V of the feature fusion module and the mask obtained in step 3.2 are sent to the average local triplet loss function for training. The average local triplet loss function is trained by first calculating the pairwise distance matrix of local features according to the three-dimensional representation mask, and then calculating the pairwise distance of the samples. In this way, the distance between invalid local features is eliminated, the influence of inactive and inconspicuous features is eliminated, and the role of useful features is enhanced.

[0027] In step 3.4, the output of the multi-scale hypergraph module with the KAN structure is sent to the standard triplet loss function for training. After the output passes through the normalization layer, it passes through the classification linear layer and is sent to the cross entropy loss function for training.

[0028] Preferably, step 4 comprises:

[0029] Step 4.1 Save the trained model;

[0030] Step 4.2: Divide the Occluded-DukeMTMC pedestrian dataset into a query set of 2210 images and 519 pedestrian IDs and a gallery set of 17661 images and 1110 pedestrian IDs.

[0031] Step 4.3 loads the saved model, uses the saved model to infer all the pedestrian feature values ​​in the query set and gallery set in step 4.2, further calculates the cosine distance matrix between the two, and calculates the average accuracy of feature matching and the accuracy of the first successful match based on the cosine distance between all pedestrians.

[0032] Beneficial effects:

[0033] Advantage 1: Different from the traditional simple segmentation of two-dimensional pedestrian images or combining traditional two-dimensional posture estimation methods to obtain pedestrian local features, the features extracted by such methods are not discriminative enough. The method of the present invention integrates three-dimensional human body information for pedestrian feature extraction, contains more human body structure-related features, and removes the influence of background noise in pedestrian images, extracting more local features that are conducive to identifying occluded pedestrians.

[0034] Advantage 2: Different from the previous methods of using graph convolution or very few methods of using fixed single-scale hypergraph convolution to establish connections between features, the connections between features in such methods are not high enough. The method of the present invention uses the KAN network to combine the multi-scale structure of the hypergraph to transfer learning and transfer semantic information of different regions. The KAN structure makes the model fit the objective function better, and the multi-scale hypergraph convolution promotes the information transfer of useful semantic features, thereby suppressing the transfer of meaningless and noise features. The combination of the two effectively improves the effect of re-identification of occluded pedestrians;

[0035] Advantage 3: Using the 3D representation mask to participate in the training of the loss function assists the training of the entire model, eliminates the influence of inactive occluded features, and enhances the role of useful features. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0037] Figure 1 Schematic diagram of the relationship between steps 1-3 of the present invention.

[0038] Figure 2 Schematic diagram of the feature fusion module of the present invention.

[0039] Figure 3 Schematic diagram of a multi-scale hypergraph module with a KAN structure in the present invention.

[0040] Figure 4 It is a schematic diagram of the detailed process of step 3 of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The present invention provides an occluded person re-identification method with a multi-scale hypergraph connection with a KAN structure, comprising:

[0043] Step 1: Use a feature fusion module to fuse the three-dimensional representation information and the two-dimensional image information of the pedestrian image to obtain the fused feature V;

[0044] Feature fusion module such as Figure 2 As shown;

[0045] Step 2: Use a multi-scale hypergraph module with a KAN structure to learn and transfer semantic information of different regions to promote the transfer of useful semantic feature information;

[0046] The multi-scale hypergraph module with KAN structure is as follows Figure 3 As shown;

[0047] Step 3: Use the 3D representation mask to participate in the training of the loss function to eliminate the influence of inactive occlusion features;

[0048] Step 4: Use the trained model to perform recognition on the occlusion dataset.

[0049] The relationship between steps 1-3 of the present invention is as follows Figure 1 shown.

[0050] Preferably, step 1 comprises:

[0051] Step 1.1 Use MotionBERT, a new 3D representation extractor, to extract 3D representations of the input pedestrian image. The extracted 3D representation of the human body is helpful for identifying 17 local feature information of occluded pedestrians, including semantic information and structural association information of the human body in 3D space. In terms of meaning, MotionBERT removes background information and noise information in the image, making the model focus more on the human body.

[0052] Reshape the 3D representation of the human body extracted by MotionBERT to reshape the final eigenvalue dimension of the 3D representation of the human body into two dimensions: height and width.

[0053] Perform bilinear interpolation on the three-dimensional representation features of the human body with height and width to make its shape value consistent with the two-dimensional feature shape value of the pedestrian image, so as to facilitate the subsequent fusion operation of the two.

[0054] Normalize the output of bilinear interpolation. Normalization here means integrating the height and width of the three-dimensional representation of the human body into one dimension, performing a softmax operation on the three-dimensional representation in this dimension, calculating the importance distribution of the human body features in the three-dimensional space, and then reshaping the feature probability into the height and width dimensions before feature integration.

[0055] Step 1.2 uses a resnet50 network with simple self-attention to extract two-dimensional features of the input pedestrian image to extract rich and deep two-dimensional pedestrian image features;

[0056] Step 1.3 multiplies the extracted three-dimensional representation and the extracted pedestrian two-dimensional features element by element on the corresponding channels and sums them up, thereby obtaining the mapping of the pedestrian two-dimensional image information in the three-dimensional space as the feature V after the fusion of the pedestrian two-dimensional image feature and the three-dimensional representation feature.

[0057] Specifically, step 1.3 includes:

[0058] At this time, for the three-dimensional representation features, the second dimension represents the 17 local features corresponding to the 17 key points of the human body in the three-dimensional space. For these 17 local features, each local feature is replicated c times in the second dimension, where c represents the number of channels of the pedestrian two-dimensional image features;

[0059] Each element in the three-dimensional representation that has been copied c times is multiplied element-by-element with each element in the extracted pedestrian two-dimensional image features in the corresponding channel dimension. The multiplication results are summed in height and width, and only the eigenvalue of the corresponding dimension is retained. This operation is performed for each local feature. This process is repeated 17 times. Finally, all the features are spliced ​​together. The spliced ​​features can represent the mapping of the pedestrian two-dimensional image information in the three-dimensional space, and at the same time serve as the final feature V after the fusion of the pedestrian two-dimensional image features and the three-dimensional representation features.

[0060] Preferably, step 2 comprises:

[0061] Step 2.1 Construct the scale-1 hypergraph foundation: Construct the human body connection matrix H1, which contains three hyperedges e1, e2, and e3, corresponding to the three body attributes of the human body: arms, legs, and torso. Construct the corresponding diagonal matrix W1 with positive weights, calculate the corresponding node degree D1, and calculate the corresponding hyperedge degree B1;

[0062] Step 2.2 Construct the scale-2 hypergraph foundation: Construct the human body connection matrix H2, which includes the five hyperedges e1, e2, e3, e4, and e5, corresponding to the five body attributes of the left arm, right arm, left leg, right leg, and torso, construct the corresponding positive weight W2 diagonal matrix, calculate the corresponding node degree D2, and calculate the corresponding hyperedge degree B2;

[0063] Step 2.3: Construct the scaled three-hypergraph foundation: construct the human body connection matrix H3, which includes the ten hyperedges from e1 to e10, the ten body attributes formed by connecting the three adjacent key point node features, construct the corresponding positive weight W3 diagonal matrix, calculate the corresponding node degree D3, and calculate the corresponding hyperedge degree B3;

[0064] Step 2.4 According to the hypergraph convolution formula: X l+1 =σ(D -1 HWB -1 H T X l P), perform three-scale hypergraph convolution on the input to fully transfer the semantic information of different key points between features, promote the information transfer of semantic features, and thus suppress the transfer of meaningless and noisy features;

[0065] Where W is a diagonal matrix storing the positive weights of each hyperedge; H is the association matrix between hyperedges and nodes. H is set to 1 for nodes on hyperedges and 0 otherwise; D is the degree matrix of the vertex, which represents the sum of the weights of all hyperedges associated with the vertex; B is the degree matrix of the hyperedge, which represents the sum of the number of vertices associated with the corresponding hyperedge; X is the feature input of layer 1, X l+1 is the output of layer l and the input of layer l+1, P is the weight matrix between layer l and layer l+1; σ is the nonlinear activation function;

[0066] Step 2.4 will get the hypergraph convolution results X1, X2, and X3 of three scales. The hypergraph convolution results X1, X2, and X3 are added element by element to get the feature X. Then, X is average pooled in the third dimension. The result after average pooling is sent to the KAN network structure layer.

[0067] Step 2.5 sends the hypergraph convolution result to the KAN network structure layer. This model uses the KAN ([18, 16, 18]) structure, which directly uses the spline function to train the appropriate activation function and continuously performs nonlinear fitting on the input data. The KAN network structure has a smaller number of parameters and a better fitting effect on the objective function.

[0068] The input data sent in the previous step first passes through a KAN layer, then a BN layer and a Relu layer, and then is sent to three KAN layers to output three different results respectively. These three outputs also correspond to the outputs of three different scale hypergraph convolutions;

[0069] In step 2.6, the convolution output is finally processed through a series of operations including concatenation and pooling to output the final result.

[0070] Specifically, step 2.6 includes:

[0071] The output of the KAN network layer is fed into the Softmax function, and then multiplied with the convolution results of the three-scale hypergraph in step 2.4. The purpose is to assign different weights to the 17 key point area information at each scale in the hypergraph so that the network can pay attention to each area information at different levels.

[0072] The three multiplication results are concatenated into feature X in the second dimension, and then sent to the Softmax function. The output of the Softmax function is multiplied by feature X and used as the new feature X. The purpose is to assign different weights to the three-scale hypergraphs so that the network can pay attention to each hypergraph information at different levels.

[0073] The new feature X is concatenated with the initial input, and after pooling and reshaping, the final result output is output. The output is a two-dimensional feature quantity of N*c, where N represents the number of pedestrian samples, c represents the feature value corresponding to each pedestrian, and output represents the characteristics of the pedestrian.

[0074] Preferably, the ten body attributes include the left forearm, left upper arm, right forearm, right upper arm, left calf, left thigh, right calf, right thigh, head and upper torso of the human body.

[0075] Preferably, if Figure 4 As shown, step 3 includes:

[0076] Step 3.1 uses the 3D representation extracted by the MotionBERT 3D representation extractor to calculate the feature pixel category score. The higher the score, the higher the confidence that the pixel feature is a human body part. The lower the score, the lower the confidence that the pixel feature is a human body part.

[0077] Step 3.2: Extract the index corresponding to the pixel feature with the highest pixel category score in the feature, encode the index using the one-hot encoding method, and convert the encoding result into a Boolean mask. This mask is used as the basis for judging whether the pedestrian is blocked. The part with a value of True indicates that this part of the feature is highly active and is the part that is not blocked by the pedestrian. The part with a value of False indicates that this part of the feature is invalid and is the part that is blocked by the pedestrian.

[0078] Step 3.3: The output feature V of the feature fusion module and the mask obtained in step 3.2 are sent to the average local triplet loss function for training. The average local triplet loss function is trained by first calculating the pairwise distance matrix of local features according to the three-dimensional representation mask, and then calculating the pairwise distance of the samples. In this way, the distance between invalid local features is eliminated, the influence of inactive and inconspicuous features is eliminated, and the role of useful features is enhanced.

[0079] In step 3.4, the output of the multi-scale hypergraph module with the KAN structure is sent to the standard triplet loss function for training. After the output passes through the normalization layer, it passes through the classification linear layer and is sent to the cross entropy loss function for training.

[0080] Preferably, step 4 comprises:

[0081] Step 4.1 Save the trained model;

[0082] Step 4.2: The Occluded-DukeMTMC pedestrian dataset is divided into a query set of 2210 images and 519 pedestrian IDs and a gallery set of 17661 images and 1110 pedestrian IDs according to the public test requirements.

[0083] Step 4.3 loads the saved model and uses it to infer the feature values ​​of all pedestrians in the query set and gallery set in step 4.2. The cosine distance matrix between the two is further calculated. The average accuracy (MAP) of feature matching and the accuracy of the first successful match (Rank-1) are calculated based on the cosine distance between all pedestrians.

[0084] Compared with other CNN-based methods, the proposed method achieved better breakthroughs in both Map and Rank-1 indicators on the Occluded-DukeMTMC dataset, verifying the advanced nature of the proposed method.

[0085] Occluded-DukeMTMC is a public dataset of occluded pedestrians. The train set contains 15,618 different pictures and 702 pedestrian IDs; the query set contains 2,210 pictures and 519 pedestrian IDs; and the gallery set contains 17,661 pictures and 1,110 pedestrian IDs.

[0086] The evaluation index Map represents the average accuracy of feature matching, and Rank-1 represents the accuracy of the first successful matching.

[0087] Table 1 below is a comparison of the method of the present invention with other methods on the Occluded-DukeMTMC dataset:

[0088] Table 1 Test results of Occluded-DukeMTMC dataset

[0089] method years bac MAP rank-1 PGFA 2019 CNN 37.3％ 51.4％ HOReID 2020 CNN 43.8％ 55.15 OAMN 2021 CNN 46.1％ 62.6％ Pirt 2021 CNN 50.9％ 60.0％ SORN 2021 CNN 46.3％ 57.6％ QPM 2022 CNN 49.7％ 64.4％ RTGTA 2023 CNN 50.1％ 61.0％ Ours 2024 CNN 56.0％ 66.8％

[0090] Table 1 shows that the innovative points proposed in the present invention have greatly improved and solved the problems of background noise and occlusion in pedestrian re-identification. The test effect of the method of the present invention on the occluded pedestrian Occluded-DukeMTMC dataset has better performance than other CNN-based methods.

[0091] In addition to testing the pedestrian occlusion problem, the method of the present invention is also trained and tested on the complete pedestrian dataset Market1501 to verify the effectiveness of the model of the present invention when pedestrians are not occluded. The Market1501 dataset has 12,936 images in the train set and 751 pedestrian IDs; 3,368 images in the query set and 750 pedestrian IDs; and 19,732 images in the gallery set and 750 pedestrian IDs. Table 2 below is the comparison results of the method of the present invention with other methods on the Market1501 dataset:

[0092] Table 2 Test results of Market1501 dataset

[0093] method years bac MAP rank-1 PGFA 2019 CNN 76.8％ 91.2％ HOReID 2020 CNN 84.9％ 94.2％ OAMN 2021 CNN 79.8％ 93.2％ Pirt 2021 CNN 86.3％ 94.1％ SORN 2021 CNN 84.5％ 94.8％ QPM 2022 CNN RTGTA 2023 CNN 88.2％ 95.3％ Ours 2024 CNN 89.3％ 95.3％

[0094] Table 2 shows that the model of the present invention not only has good performance when pedestrians are occluded, but also has good results when pedestrians are not occluded, which is of great significance for the actual application of pedestrian re-identification.

[0095] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A multi-scale hypergraph connection method for occluded pedestrian re-identification with KAN structure, characterized in that: include: Step 1: Use a feature fusion module to fuse the three-dimensional representation information and the two-dimensional image information of the pedestrian image to obtain the fused feature V; Step 2: Use a multi-scale hypergraph module with a KAN structure to learn and transfer semantic information of different regions to promote the transfer of useful semantic feature information; Step 3: Use the 3D representation mask to participate in the training of the loss function to eliminate the influence of inactive occlusion features; Step 4: Use the trained model to perform recognition on the occlusion dataset.

2. According to claim 1, the method for occluded person re-identification with multi-scale hypergraph connection with KAN structure is characterized in that: Step 1 includes: Step 1.1 Use the MotionBERT 3D representation extractor to extract the 3D representation of the input pedestrian image; Step 1.2 uses a resnet50 network with simple self-attention to extract two-dimensional features from the input pedestrian image; Step 1.3 multiplies the extracted three-dimensional representation and the extracted pedestrian two-dimensional features element by element on the corresponding channels and sums them up, thereby obtaining the mapping of the pedestrian two-dimensional image information in the three-dimensional space as the feature V after the fusion of the pedestrian two-dimensional image feature and the three-dimensional representation feature.

3. According to claim 1, the method for occluded person re-identification with multi-scale hypergraph connection with KAN structure is characterized in that: Step 2 includes: Step 2.1 Construct the scale-1 hypergraph foundation: Construct the human body connection matrix H1, which contains three hyperedges e1, e2, and e3, corresponding to the three body attributes of the human body: arms, legs, and torso. Construct the corresponding diagonal matrix W1 with positive weights, calculate the corresponding node degree D1, and calculate the corresponding hyperedge degree B1; Step 2.2 Construct the scale-2 hypergraph foundation: Construct the human body connection matrix H2, which includes the five hyperedges e1, e2, e3, e4, and e5, corresponding to the five body attributes of the left arm, right arm, left leg, right leg, and torso, construct the corresponding positive weight W2 diagonal matrix, calculate the corresponding node degree D2, and calculate the corresponding hyperedge degree B2; Step 2.3: Construct the scaled three-hypergraph foundation: construct the human body connection matrix H3, which includes the ten hyperedges from e1 to e10, the ten body attributes formed by connecting the three adjacent key point node features, construct the corresponding positive weight W3 diagonal matrix, calculate the corresponding node degree D3, and calculate the corresponding hyperedge degree B3; Step 2.4 According to the hypergraph convolution formula: X l+1 =σ(D -1 HWB -1 H T X l P), and perform three-scale hypergraph convolution on the input respectively; where W is a diagonal matrix storing the positive weights of each hyperedge; H is the association matrix between hyperedges and nodes, and H is set to 1 for nodes on hyperedges, otherwise it is set to 0; D is the degree matrix of the vertex, which represents the sum of the weights of all hyperedges associated with the vertex; B is the degree matrix of the hyperedge, which represents the sum of the number of vertices associated with the corresponding hyperedge; X is the feature input of layer 1, X l+1 is the output of layer l and the input of layer l+1, P is the weight matrix between layer l and layer l+1; σ is the nonlinear activation function; Step 2.5: Send the hypergraph convolution result to the KAN network structure layer; use the spline function to train the appropriate activation function and continuously perform nonlinear fitting on the input data; In step 2.6, the convolution output is finally processed through a series of operations including concatenation and pooling to output the final result.

4. According to claim 3, the method for occluded person re-identification with multi-scale hypergraph connection with KAN structure is characterized in that: The ten body attributes include the human left forearm, left upper arm, right forearm, right upper arm, left calf, left thigh, right calf, right thigh, head and upper torso.

5. The occluded person re-identification method with multi-scale hypergraph connection with KAN structure according to claim 1 is characterized in that: Step 3 includes: Step 3.1 uses the 3D representation extracted by the MotionBERT 3D representation extractor to calculate its feature pixel category score; Step 3.2 extracts the index corresponding to the pixel feature with the highest pixel category score in the feature, encodes the index using the one-hot encoding method, and converts the encoding result into a Boolean mask; Step 3.3: The output feature V of the feature fusion module and the mask obtained in step 3.2 are sent to the average local triplet loss function for training. The average local triplet loss function is trained by first calculating the pairwise distance matrix of local features according to the three-dimensional representation mask, and then calculating the pairwise distance of the samples. In this way, the distance between invalid local features is eliminated, the influence of inactive and inconspicuous features is eliminated, and the role of useful features is enhanced. In step 3.4, the output of the multi-scale hypergraph module with the KAN structure is sent to the standard triplet loss function for training. After the output passes through the normalization layer, it passes through the classification linear layer and is sent to the cross entropy loss function for training.

6. The occluded person re-identification method with multi-scale hypergraph connection with KAN structure according to claim 1, characterized in that: Step 4 includes: Step 4.1 Save the trained model; Step 4.2: Divide the Occluded-DukeMTMC pedestrian dataset into a query set of 2210 images and 519 pedestrian IDs and a gallery set of 17661 images and 1110 pedestrian IDs. Step 4.3 loads the saved model, uses the saved model to infer all the pedestrian feature values ​​in the query set and gallery set in step 4.2, further calculates the cosine distance matrix between the two, and calculates the average accuracy of feature matching and the accuracy of the first successful match based on the cosine distance between all pedestrians.

Images