Pedestrian re-identification method based on parallel feature fusion network human action recognition

Abstract

A pedestrian re-identification method based on parallel feature fusion network human action recognition, comprising the following steps: 1) establishing a parallel feature fusion network: setting the backbone network of the parallel feature fusion network, which is used to extract multi-scale features from input images, and setting a branch network at each layer of the backbone network, which is used to improve the feature expression ability of the semantic information of the image at this layer and accumulate to the branch network of the next layer of the backbone network; the output features of the first layer of the backbone network are used as the input features of the first layer branch network, and from the second layer of the backbone network, the output features of the current layer of the backbone network are fused with the output features of the branch network of the previous layer to be used as the input features of the branch network of the current layer; 2) using the parallel feature fusion network to extract the semantic information of the to-be-retrieved pedestrian image, comparing the extracted semantic information with the semantic information of all images in the candidate pedestrian library one by one, and screening out the image with the highest similarity to the to-be-retrieved pedestrian image in the candidate pedestrian library.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a pedestrian re-identification method based on a parallel feature fusion network human action recognition. BACKGROUND

[0002] Computer vision is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further process images so that the computer processing becomes more suitable for human observation or transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and tries to establish an artificial intelligence system that can obtain 'information' from images or multidimensional data.

[0003] ReID, also known as Re-identification, is defined as a technology that enables computers to autonomously find a target to be searched in an image library using algorithms, so it is a sub-problem of image retrieval in computer vision.

[0004] Person Re-identification is one of the subfields of ReID, which aims to use computer vision technology to determine whether a specific pedestrian exists in a video library or image library captured by different cameras, and to establish a rank list according to the similarity of the retrieved pedestrian images. The basic process of the pedestrian re-identification task is as follows:

[0005] Given a pedestrian image to be searched (Query / Probe), the feature descriptor of the pedestrian is extracted through a specific technology, and the feature is compared with the image features in the candidate pedestrian library (Gallery). In the candidate pedestrian library, the images with high similarity to the pedestrian image to be searched are arranged at the front of the rank list, and the images with low similarity to the pedestrian image to be searched are arranged at the back of the rank list.

[0006] However, the current pedestrian re-identification technology is still affected by many reasons:

[0007] ① Low image resolution:

[0008] The resolution of the image captured by the camera is low, which directly affects the observation of the details of the pedestrian by the computer, making it difficult to identify a specific pedestrian using the facial features of the pedestrian;

[0009] ② Variable pedestrian posture:

[0010] Pedestrians have different postures at different times or places, which increases the difficulty of pedestrian re-identification;

[0011] ③ Environment diversification:

[0012] The environment of the same pedestrian is quite different in different time periods, and the diversification of the environment will inevitably reduce the recognition accuracy and efficiency of pedestrian re-identification;

[0013] ④ The target is blocked:

[0014] The body of the pedestrian may be blocked by buildings, cars, umbrellas and the like, which can easily cause the loss of semantic information of the pedestrian, thereby greatly reducing the recognition accuracy and efficiency of pedestrian re-identification.

[0015] How to overcome the above problems and greatly improve the recognition efficiency and accuracy of pedestrian re-identification has always been a problem to be solved in the field of computer vision. SUMMARY

[0016] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide a pedestrian re-identification method based on parallel feature fusion network human action recognition, which uses a grouping fusion attention layer in the parallel feature fusion network to continuously transmit single-level features from the backbone network to the branch network, so that the branch network continuously accumulates multi-level semantic information of the image, and the entire parallel feature fusion network has a much higher retrieval accuracy than the reference network, and better expresses the semantic information of the pedestrian in the image.

[0017] The purpose of the present application is achieved by the following scheme, a pedestrian re-identification method based on parallel feature fusion network human action recognition, comprising the following steps:

[0018] 1) Establish a parallel feature fusion network:

[0019] The backbone network of the parallel feature fusion network is set up to extract multi-scale features from the input image, and a branch network is set up at each layer of the backbone network, which is used to improve the feature expression ability of the image semantic information of the layer and accumulate to the branch network of the next layer of the backbone network;

[0020] The output features of the first layer of the backbone network are used as the input features of the first layer branch network, and from the second layer of the backbone network, the output features of the current layer of the backbone network are fused with the output features of the branch network of the previous layer to be used as the input features of the current layer branch network;

[0021] 2) Extract the semantic information of the pedestrian image to be searched using the parallel feature fusion network, and compare the extracted semantic information with the semantic information of all images in the candidate pedestrian library one by one, and select the image with the highest similarity to the pedestrian image to be searched from the candidate pedestrian library.

[0022] Preferably, the backbone network of the parallel feature fusion network is a deep residual neural network.

[0023] Preferably, the branch network comprises a region-guided Transformer layer for obtaining semantic information with global context dependency, and an output feature of the branch network is an output feature of the region-guided Transformer layer.

[0024] Preferably, the region-guided Transformer layer comprises a region-guided attention unit and a Transformer encoder, the region-guided attention unit comprises two fully connected layers and a sigmoid activation function module, and a channel weight of each local region feature is obtained as an input of the Transformer encoder, and all channel weights are constrained by a center bias loss function, the center bias loss function is shown in the following formula:

[0025]

[0026]

[0027]

[0028]

[0029] In the formula, L BM is a center bias loss function based on a weight distribution, δ is a weight interval, w L-σ is a weight lower than an average weight, w H-σ is a weight higher than the average weight, W σ is the average weight, that is, an average value of all channel weights;

[0030] N is a number of images in a batch, N1 is a number of images lower than the average weight, N2 is a number of images higher than the average weight, w i is a weight of an i-th image in all images lower than the average weight, w j is a weight of a j-th image in all images higher than the average weight, w k is a weight of a k-th image in all images in a batch.

[0031] Preferably, the branch network comprises a group fusion attention layer for capturing semantic information of different scales, and the group fusion attention layer is arranged between the region-guided Transformer layer and the backbone network.

[0032] Preferably, the group fusion attention layer comprises a channel grouping unit, a first channel, and a second channel, the first channel is composed of a global maximum pooling unit, a first full connection layer, and a second full connection layer, the second channel is composed of a global average pooling unit, a first full connection layer, and a second full connection layer, the output feature of the first full connection layer of the first channel is processed by a ReLU activation function and then used as the input feature of the second full connection layer of the first channel, the output feature of the first full connection layer of the second channel is processed by a ReLU activation function and then used as the input feature of the second full connection layer of the second channel, the output features of the second full connection layers of the first channel and the second channel are processed by a Sigmoid activation function, and then the channel splicing is performed and the output is obtained, and the mathematical expression of the group fusion attention layer is as follows:

[0033]

[0034]

[0035]

[0036]

[0037]

[0038] In the formula, Group (X i ) is defined as the input feature X i of the group fusion attention layer is divided into G s groups by the channel grouping unit in the channel dimension, G s is the total number of channel grouping, and

[0039] C s is the channel number of the semantic information of the layer after channel grouping, and s is the number of layers of the backbone network.

[0040] f k (h, w, c) is the semantic feature of the kth group of channels after channel grouping in terms of width, height, and channel number, h is the height of the semantic feature, w is the width of the semantic feature, and c is the channel number of the semantic information of the layer before channel grouping.

[0041] f o is the output feature.

[0042] represents the feature of the first channel after global maximum pooling;

[0043] represents the feature of the second channel after global average pooling;

[0044] ψ3 represents a 3x3 convolution layer.

[0045] Maxp(·) represents a global max pooling function;

[0046] Avgp(·) represents a global average pooling function;

[0047] Cat(·) is a function of concatenating feature blocks in the channel dimension, W S is a two-dimensional matrix of the first fully connected layer, GAP(F) is a global average pooling, W E is a two-dimensional matrix of the second fully connected layer, ReLU(·) is a ReLU activation function, BN(·) is a batch normalization function, and softmax(·) is a normalized exponential function.

[0048] Preferably, the branch network comprises a joint pooling layer for obtaining compressed semantic information of a unified size, which is arranged between the region-guided Transformer layer and the group fusion attention layer.

[0049] Preferably, the joint pooling layer comprises a max pooling layer and an average pooling layer, and the output features of the group fusion attention layer are subjected to adaptive max pooling and adaptive average pooling through the max pooling layer and the average pooling layer respectively, and then the output results of the max pooling layer and the average pooling layer are averaged, and finally the averaged result is output to the region-guided Transformer layer.

[0050] Preferably, the output features of each layer of the backbone network are subjected to feature constraint by using a loss function, and the output features of each branch network of the backbone network are fused with the input features of the branch network of the layer, and then subjected to feature constraint by using a loss function.

[0051] Preferably, the loss function comprises a cross-correlation penalty triplet loss function based on relative angles and a cross-entropy loss function, and the mathematical expression of the cross-correlation penalty triplet loss function based on relative angles is as follows:

[0052]

[0053]

[0054]

[0055] θ d = |max(θ(a, p))-min(θ(a, n))|

[0056] In the formula, L cr-Angleis a relative angle-based cross-correlation penalty triplet loss function, P is the number of randomly selected persons, K is the number of randomly selected images for each person, a is an anchor sample, p is a positive sample, n is a negative sample, theta (a, p) is the distance between the positive sample pair, theta (a, n) is the distance between the negative sample pair, is a first penalty factor of a hard sample, is a second penalty factor of a hard sample, T a is a set angle difference threshold, theta d is an angle difference, pi is a circular constant, m cr-angle is an angle threshold of a relative angle-based cross-correlation penalty triplet loss function;

[0057] The mathematical expression of the cross-entropy loss function is as follows:

[0058]

[0059] In the formula, T is the height of the pedestrian, q j is the true value label of whether the jth image belongs to the identity, p j is the predicted probability.

[0060] The advantages of the present application are as follows:

[0061] ① The branch network arranged at each layer of the backbone network can continuously accumulate multi-level semantic information of the image, so that the entire parallel feature fusion network has a retrieval accuracy much larger than that of the benchmark network, and better expresses the semantic information of the pedestrian in the image.

[0062] ② In view of the problem of insufficient channel semantic information of the input feature of the Transformer encoder, a region-guided attention unit is added before the traditional Transformer encoder to obtain the channel weight of each local region feature as the input feature of the traditional Transformer encoder, and a center deviation loss function is used to assist the Transformer to better capture the context information.

[0063] ③ In view of the problem that the traditional triplet loss function can only rely on fixed gradient to guide the network training, the traditional hard sample mining triplet loss function is improved, the concept of "penalty" is used to strengthen the constraint of the positive and negative sample pairs, and the distribution information between the samples is fused into the penalty factor, and the training of the parallel feature fusion network is guided by dynamic gradient. BRIEF DESCRIPTION OF DRAWINGS

[0064] Fig. 1 is a schematic diagram of the parallel feature fusion network according to the present application;

[0065] Fig. 2A schematic diagram of the grouping fusion attention layer described in the present application. DETAILED DESCRIPTION

[0066] As shown in the figure, a pedestrian re-identification method based on a parallel feature fusion network human action recognition includes the following steps: Figs. 1-2

[0067] 1) Establish a parallel feature fusion network:

[0068] Set the backbone network of the parallel feature fusion network, which is used to extract multi-scale features from the input image, and set a branch network at each layer of the backbone network, which is used to improve the feature expression ability of the semantic information of the image at this layer and accumulate to the branch network of the next layer of the backbone network.

[0069] The output feature of the first layer of the backbone network is used as the input feature of the first layer branch network, and starting from the second layer of the backbone network, the output feature of the current layer of the backbone network is fused with the output feature of the branch network of the previous layer to serve as the input feature of the branch network of the current layer.

[0070] In this embodiment, the fusion of the output feature of the current layer of the backbone network and the output feature of the branch network of the previous layer is a weighted average of the output feature of the current layer of the backbone network and the output feature of the branch network of the previous layer, and the specific formula is as follows:

[0071]

[0072] In the formula, X f is the fused semantic feature, is the output feature of the current layer of the backbone network, and is the output feature of the branch network of the previous layer of the backbone network, and is the weight of the output feature of the current layer of the backbone network, is the weight of the output feature of the branch network of the previous layer of the backbone network, and are empirical values.

[0073] It is worth noting that in this embodiment, the two features mentioned above need to be resized before being fused.

[0074] 2) Extract the semantic information of the to-be-searched pedestrian image using the parallel feature fusion network, compare the extracted semantic information with the semantic information of all images in the candidate pedestrian library one by one, and select the image in the candidate pedestrian library that is most similar to the to-be-searched pedestrian image.

[0075] ​The backbone network of the parallel feature fusion network is a deep residual neural network, which is a residual network Resnet50. ResNet34, ResNet101, ResNet152, etc. can also be used as the backbone network of the parallel feature fusion network. The more layers of the residual neural network, the stronger the expression ability of the semantics. The residual neural network can effectively solve the problem of gradient disappearance when the number of network layers is large through identity mapping. At the same time, the residual neural network also reduces the time overhead of network training.

[0076] Of course, generally speaking, the low-level semantic information of the backbone network, such as color, texture, trajectory, etc., is not helpful for pedestrian re-identification. It is not cost-effective to set a branch network at the first layer. The performance of the entire parallel feature fusion network not only does not greatly improve, but also leads to an increase in the computational complexity of the entire parallel feature fusion network. Therefore, generally, the first layer of the backbone network is not set with a branch network. Starting from the second layer of the backbone network, a branch network is set each time, that is, the output feature of the first layer of the backbone network is taken as the input feature of the first layer branch network, the output feature of the second layer of the backbone network is taken as the input feature of the second layer branch network, and starting from the third layer of the backbone network, the output feature of the current layer of the backbone network is taken as the input feature of the current layer branch network after being fused with the output feature of the previous layer branch network.

[0077] In this embodiment, the branch network includes a region-guided Transformer layer for obtaining semantic information with global context dependency, and the output feature of the branch network is the output feature of the region-guided Transformer layer. The region-guided Transformer layer includes a region-guided attention unit and a Transformer encoder. The region-guided attention unit includes two fully connected layers and a sigmoid activation function module, which is used to obtain the channel weight of each local region feature as the input of the Transformer encoder. The local region features of the image are compressed in channel number through the two fully connected layers, the parameters are reduced, and then expanded to the original channel number. Finally, the channel weight of each local region feature is obtained through the sigmoid activation function, which provides more robust local region features for the Transformer, so that the Transformer can more easily explore the correlation between these region features.

[0078] While too uniform weight distribution will cause the network to degrade in discriminability, and weight distribution with greater difference will make the output features of the grouping attention module more delicate, therefore the weights of all channels are constrained by a center deviation loss function, which can alleviate the semantic ambiguity problem as much as possible, guide the regional guided attention unit to learn more robust features, and provide effective local region information for exploring the context relationship of features. The center deviation loss function based on weight distribution is shown in the following formula:

[0079]

[0080]

[0081]

[0082]

[0083] In the formula, L BM is the center deviation loss function based on weight distribution, δ is the weight interval, w L-σ is the weight lower than the average weight, w H-σ is the weight higher than the average weight, W σ is the average weight, that is, the average value of the weights of all channels;

[0084] N is the number of images in a batch, N1 is the number of images lower than the average weight, N2 is the number of images higher than the average weight, w i is the weight of the i-th image lower than the average weight, w j is the weight of the j-th image higher than the average weight, w k is the weight of the k-th image in all images in a batch;

[0085] In the embodiment, the branch network includes a grouping fusion attention layer for capturing semantic information of different scales, which is arranged between the regional guided Transformer layer and the backbone network.

[0086] In order to obtain compressed semantic information of a uniform size, the branch network comprises a joint pooling layer configured to obtain compressed semantic information of a uniform size, which is arranged between the region-guided Transformer layer and the group fusion attention layer. The joint pooling layer comprises a max pooling layer and an average pooling layer, and the output features of the group fusion attention layer are respectively subjected to adaptive max pooling and adaptive average pooling through the max pooling layer and the average pooling layer, and then the output results of the max pooling layer and the average pooling layer are averaged, and finally the averaged result is output to the region-guided Transformer layer. In this embodiment, the output features of the joint pooling layer are flattened and then output to the region-guided Transformer layer.

[0087] The output features of the group fusion attention layer are sent to the region-guided Transformer layer after being subjected to the joint pooling layer, so as to extract multi-scale semantic features, enhance the semantic expression of relevant channels, and suppress irrelevant channel semantic information by using the structure of the group fusion attention layer.

[0088] In the parallel feature fusion network of this embodiment, the output features of the group fusion attention layer are sent to the joint pooling layer, and the output features of the joint pooling layer are sent to the region-guided attention unit after being unfolded, so as to obtain important subtle clues of each local region feature. Then, the enhanced local region features are sent to the Transformer encoder, so that the Transformer encoder explores the context information of the features, thereby forming output features with more closely global information, and the output features are recursively fused with high-level semantic features of the next layer of the backbone network, so as to realize semantic accumulation effect and enrich the feature expression of each layer of the backbone network.

[0089] Since the number of channels of the semantic features of the low layer of the backbone network is relatively small, and the number of channels of the semantic features of the high layer of the backbone network is relatively large, that is, the dependence of the low-level semantic features on the global channel is relatively strong, and the dependence of the high-level semantic features on the local region channel is relatively strong. The connection between adjacent channels in the high-level semantic features is closer. The channel grouping unit can not only strengthen the interaction of the features in the group, but also reduce the parameter amount of the network. Therefore, different channel grouping methods are adopted in the group fusion attention layers of each stage of the ResNet50, appropriate group channel weights are effectively learned, and different scale channel information is used to capture the semantic information of the image, so as to not only reduce the training time of the network, but also provide good channel semantic features for the downstream network.

[0090] Since the second layer of the ResNet50 tends to capture low-level semantic information (such as color and texture), these low-level semantic features have more compact global channel connections, so the features of the second layer of the backbone network are fully grouped.

[0091] With the deepening of different stages of ResNet50, high-level semantic information gradually attracts attention, and the grouping method of the network should follow certain principles. Since the channel number of the output features of each layer of ResNet50 is a power of 2, the simplest grouping method can be:

[0092]

[0093] In the formula, a = 2, b = 0, C s is the channel number of the semantic information of this layer after channel grouping, G s is the total number of channel grouping.

[0094] Therefore, in the embodiment, the grouping fusion attention layer includes a channel grouping unit, a first channel, and a second channel. The first channel is composed of a global maximum pooling unit, a first full connection layer, and a second full connection layer. The second channel is composed of a global average pooling unit, a first full connection layer, and a second full connection layer. The input features of the global maximum pooling unit and the global average pooling unit are first processed by a 3x3 convolution layer. The output features of the first full connection layer of the first channel are processed by a ReLU activation function and then used as the input features of the second full connection layer of the first channel. The output features of the first full connection layer of the second channel are processed by a ReLU activation function and then used as the input features of the second full connection layer of the second channel. The output features of the second full connection layer of the first channel and the second full connection layer of the second channel are processed by a Sigmoid activation function, concatenated, and then output. The mathematical expression of the grouping fusion attention layer is as follows:

[0095]

[0096]

[0097]

[0098]

[0099]

[0100] In the formula, Group(X i ) is defined as the input features X i of the grouping fusion attention layer. Through the channel grouping unit, the input features are divided into G s groups according to the channel dimension, i.e., G s is the total number of channel grouping, and

[0101] C s is the channel number of the semantic information of this layer after channel grouping, and s is the number of layers of the backbone network.

[0102] f k (h, w, c) is the semantic feature of the kth group of channels after channel grouping, h is the height of the semantic feature, w is the width of the semantic feature, and c is the number of channels of the semantic information of the layer before channel grouping;

[0103] f o is the output feature;

[0104] represents the feature of the first channel after global maximum pooling;

[0105] represents the feature of the second channel after global average pooling;

[0106] ψ3 represents a 3x3 convolution layer;

[0107] Maxp(·) represents a global maximum pooling function;

[0108] Avgp(·) represents a global average pooling function;

[0109] Cat(·) is a function of concatenating feature blocks in the channel dimension, W S is a two-dimensional matrix of the first fully connected layer, GAP(F) is global average pooling, W E is a two-dimensional matrix of the second fully connected layer, ReLU(·) is a ReLU activation function, BN(·) is a batch normalization function, and softmax(·) is a normalized exponential function.

[0110] In the embodiment, the output features of each layer of the backbone network are constrained by a loss function, and the output features of each branch network of the backbone network are constrained by a loss function after being fused with the input features of the branch network. The loss function includes a cross-correlation penalty triplet loss function based on a relative angle and a cross-entropy loss function, that is, the classification loss of the application adopts a softmax loss, that is, an ID loss. Given a pedestrian image, a fixed-dimensional output vector is extracted from the fully connected layer, and the dimension is equal to the number of body parts. Then, the softmax loss is calculated according to the true value label and the prediction probability.

[0111] The mathematical expression of the cross-correlation penalty triplet loss function based on the relative angle is as follows:

[0112]

[0113]

[0114]

[0115] θ d= |max(θ(a, p)) - min(θ(a, n))|

[0116] wherein L cr-Angle is a relative angle-based cross-correlation penalty triplet loss function, P is the number of persons randomly selected, K is the number of images randomly selected for each person, a is an anchor sample, p is a positive sample, n is a negative sample, θ(a, p) is the distance between a pair of positive samples, θ(a, n) is the distance between a pair of negative samples, is a first penalty factor of a hard sample, is a second penalty factor of a hard sample, T a is a set angle difference threshold, θ d is an angle difference, π is a constant, m cr-angle is an angle threshold of the relative angle-based cross-correlation penalty triplet loss function, which is a calibrated value;

[0117] The mathematical expression of the cross-entropy loss function is as follows:

[0118]

[0119] wherein T is the height of the pedestrian, q j is a true value label of whether the jth image belongs to the identity, 1 if yes, and 0 if no, p j is a predicted probability.

[0120] As described in the above embodiment, the entire parallel feature fusion network takes ResNet50 as the backbone network, and sends the semantic features of each layer of ResNet50 into the grouping fusion attention layer in the corresponding branch network of each layer to enhance the channel semantic features of each layer of the backbone network. The enhanced features are sent into the region-guided Transformer layer after joint pooling, to obtain features with closer context contact. Then, the strong semantic image features are transmitted to the next layer of the network of ResNet50, and are fused with the output features of the next layer of the network. Finally, the original semantic information is fused with the output of the region-guided Transformer layer, and is constrained by a loss function.

[0121] After the parallel feature fusion network of the embodiment is trained by using Market1501, DukeMTMC, MSMT17, CUHK03, Occluded-Duke and Partial-iLIDS as a training set, the semantic information of the pedestrian image to be searched is extracted by using the parallel feature fusion network, so that the high-level network can pay attention to the features of the current layer and also perceive the accumulated semantic features of the low-level network, thereby capturing more hierarchical features. Compared with other classic pedestrian re-identification networks, the parallel feature fusion network has higher mAP and Rank-1 indexes on the Market1501, DukeMTMC, MSMT17 and CUHK03 data sets, and has better performance, that is, the application has excellent learning performance and generalization performance.

[0122] Table 1

[0123]

[0124] In Table 1, mAP is mean average precision, and rank-1 is the probability that the most forward image in the search result is the correct result.

[0125] The above description is only the preferred embodiment of the application and is not used to limit the application. The modifications made by those skilled in the art without departing from the spirit of the application fall within the protection scope of the application.

Claims

1. A pedestrian re-identification method based on parallel feature fusion network for human action recognition, characterized in that, Includes the following steps: 1) Establish a parallel feature fusion network: A parallel feature fusion network is set up as the backbone network to extract multi-scale features from the input image. A branch network is set up in each layer of the backbone network. The branch network is used to improve the feature representation ability of the semantic information of the image in this layer and accumulates to the branch network of the next layer of the backbone network. The output features of the first layer of the backbone network are used as the input features of the first layer of the branch network. Starting from the second layer of the backbone network, the output features of the current layer of the backbone network are fused with the output features of the previous layer of the branch network and used as the input features of the current layer of the branch network. The branch network includes a region-guided Transformer layer for obtaining semantic information with global context dependencies, and the output features of the branch network are the output features of the region-guided Transformer layer. The branch network includes a grouped fusion attention layer for capturing semantic information at different scales. This grouped fusion attention layer is set between the region-guided Transformer layer and the backbone network. The region-guided Transformer layer includes a region-guided attention unit and a Transformer encoder. The region-guided attention unit includes two fully connected layers and a sigmoid activation function module, which is used to obtain the channel weights of each local region feature as the input of the Transformer encoder. The weights of all channels are constrained by a center bias loss function. 2) Use a parallel feature fusion network to extract semantic information from the pedestrian image to be retrieved, compare the extracted semantic information with the semantic information of all images in the candidate pedestrian database, and select the image with the highest similarity to the pedestrian image to be retrieved from the candidate pedestrian database.

2. The pedestrian re-identification method based on parallel feature fusion network human action recognition according to claim 1, characterized in that, The backbone network of the parallel feature fusion network is a deep residual neural network.

3. The pedestrian re-identification method based on parallel feature fusion network human action recognition according to claim 1, characterized in that, The center deviation loss function is shown in the following formula: In the formula, L BM Let w be the center deviation loss function based on the weight distribution, where δ is the weight interval and w is the weight distribution. L-σ For weights below the average weight, w H-σ W is a weight higher than the average weight. σ This is the average weight, which is the average weight of all channels; N is the number of images in a batch, N1 is the number of images with weights below the average, N2 is the number of images with weights above the average, and w i w is the weight of the i-th image among all images with weights below the average weight. j w is the weight of the j-th image among all images with weights higher than the average weight. k This represents the weight of the k-th image out of all images in a batch.

4. The pedestrian re-identification method based on parallel feature fusion network human action recognition according to claim 1, characterized in that, The grouped fusion attention layer includes a channel grouping unit, a first channel, and a second channel. The first channel consists of a global max pooling unit, a first fully connected layer, and a second fully connected layer. The second channel consists of a global average pooling unit, a first fully connected layer, and a second fully connected layer. The output features of the first fully connected layer of the first channel are processed by the ReLU activation function and then used as the input features of the second fully connected layer of the first channel. The output features of the first fully connected layer of the second channel are also processed by the ReLU activation function and then used as the input features of the second fully connected layer of the second channel. The output features of the second fully connected layers of the first and second channels are processed by the Sigmoid activation function, then concatenated before output. The mathematical expression of the grouped fusion attention layer is as follows: In the formula, Group(X) i The definition of ) is to group and fuse the input features X of the attention layer. i Divided into G by channel dimension through channel grouping unit s Group, i.e., G s The total number of channel groups, and C s The number of channels for semantic information in this layer after channel grouping, where s is the number of layers in the backbone network; f k (h,w,c) represents the semantic features of the k-th channel after channel grouping, with respect to width, height, and number of channels. h is the height of the semantic feature, w is the width of the semantic feature, and c is the number of channels of the semantic information in this layer before channel grouping. f o For output features; This represents the feature of the first channel after global max pooling; This represents the feature of the second channel after global average pooling; ψ3 represents a 3×3 convolutional layer; Maxp(·) represents the global max pooling function; Avgp(·) represents the global average pooling function; Cat(·) is a function that concatenates feature blocks along the channel dimension, W S It is a two-dimensional matrix of the first fully connected layer, GAP(·) represents global average pooling, and W E Let be the two-dimensional matrix of the second fully connected layer, ReLU(·) be the ReLU activation function, BN(·) be the batch normalization function, and softmax(·) be the normalization exponential function.

5. The pedestrian re-identification method based on a parallel feature fusion network according to claim 1, characterized in that, The branch network includes a joint pooling layer for acquiring compressed semantic information of uniform size, which is set between the region-guided Transformer layer and the group fusion attention layer.

6. The pedestrian re-identification method based on parallel feature fusion network human action recognition according to claim 5, characterized in that, The joint pooling layer includes a max pooling layer and an average pooling layer. The output features of the grouped fusion attention layer are subjected to adaptive max pooling and adaptive average pooling through the max pooling layer and the average pooling layer, respectively. The output results of the max pooling layer and the average pooling layer are then averaged, and finally the averaged result is output to the region-guided Transformer layer.

7. The pedestrian re-identification method based on a parallel feature fusion network according to claim 1, characterized in that, The output features of each layer of the backbone network are constrained by a loss function. The output features of each branch network of the backbone network are fused with the input features of that branch network and then constrained by a loss function.

8. The pedestrian re-identification method based on parallel feature fusion network human action recognition according to claim 7, characterized in that, The loss function includes a cross-correlation penalty triplet loss function based on relative angle and a cross-entropy loss function. The mathematical expression of the cross-correlation penalty triplet loss function based on relative angle is as follows: i d =|max(θ(a,p))-min(θ(a,n))| In the formula, L cr-Angle Let θ(a,p) be the cross-correlation penalty triplet loss function based on relative angle, where P is the number of randomly selected people, K is the number of images randomly selected for each person, a is the anchor sample, p is the positive sample, n is the negative sample, θ(a,p) is the distance between positive sample pairs, and θ(a,n) is the distance between negative sample pairs. The first penalty factor for making the sample difficult. The second penalty factor for difficult samples, T a θ is the set angular difference threshold. d It's a difference in angle, π is the mathematical constant pi, and m cr-angl The angle threshold for the cross-correlation penalty triplet loss function based on relative angle; The mathematical expression for the cross-entropy loss function is as follows: In the formula, T represents the number of pedestrian identities, and q j p is the truth label for whether the j-th image belongs to that identity. j To predict probabilities.

Images