Pedestrian re-identification method, system and related equipment based on polarized self-attention

Abstract

The present invention is applicable to the field of computer vision and provides a person re-identification method, system, and related equipment based on polarized self-attention. The method comprises: obtaining a captured dataset containing pedestrian images and preprocessing the captured dataset to obtain a dataset to be divided; dividing the dataset to be divided into a training set and a test set, with each pedestrian image having a true label; constructing a person re-identification model comprising a dual-branch structure and a polarized self-attention mechanism; using the training set as input and the true labels corresponding to the training set as a reference for parameter debugging, training the person re-identification model using the Adam optimization algorithm to obtain training parameter weights; using the training parameter weights as test weights and the test set as input to the person re-identification model, obtaining a person re-identification result for the test set. The present invention reduces information loss during feature extraction and improves the accuracy of person re-identification by combining global and local feature extraction.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a pedestrian re-identification method, system and related equipment based on polarized self-attention. Background Art

[0002] In recent years, with the increasing demand for intelligent security and video surveillance, person re-identification (PRE) research has received increasing attention and research. In video surveillance, factors such as low camera resolution, insufficient lighting, poor camera angles, and obstructions make it difficult to capture clear facial information of pedestrians, making it difficult to identify pedestrians based on their facial information. PRE can be viewed as an image retrieval task, leveraging computer vision techniques to determine whether a specific pedestrian exists in a given image or video sequence. This technology can overcome the difficulties inherent in identifying pedestrians based on facial information due to factors such as low camera resolution, insufficient lighting, poor camera angles, and obstructions.

[0003] Person re-identification is typically achieved using deep learning methods. This involves using convolutional neural networks to identify pedestrian features and then identify pedestrians with the same features across different images. Existing person re-identification methods primarily consider coarse-grained features of the entire image, specifically the overall characteristics of the pedestrian, while lacking attention to fine-grained features such as hairstyle, clothing color, and the presence of a backpack. This results in inadequate feature extraction and low accuracy. Summary of the Invention

[0004] The embodiments of the present invention provide a pedestrian re-identification method, system and related equipment based on polarized self-attention, aiming to solve the problem that traditional pedestrian re-identification methods do not fully extract different coarse-grained features of pedestrians, resulting in low accuracy.

[0005] In a first aspect, an embodiment of the present invention provides a person re-identification method based on polarized self-attention, the method comprising:

[0006] Acquire a photographic dataset containing pedestrian images, and preprocess the photographic dataset to obtain a dataset to be divided;

[0007] Dividing the data set to be divided into a training set and a test set, wherein each of the pedestrian images in the training set and the data set has a true label;

[0008] Construct a person re-identification model with a dual-branch structure and a polarized self-attention mechanism;

[0009] The person re-identification model is trained using the training set as input of the person re-identification model and the true labels corresponding to the training set as a reference for parameter debugging, using the Adam optimization algorithm to train the person re-identification model to obtain training parameter weights;

[0010] The training parameter weights are used as test weights of the person re-identification model, and the test set is used as input of the person re-identification model to obtain a person re-identification result for the test set.

[0011] Furthermore, in the step of obtaining a photographic dataset having pedestrian images and preprocessing the photographic dataset to obtain a dataset to be divided, the preprocessing specifically includes:

[0012] The size of each picture in the shooting dataset is normalized, and each picture in the shooting dataset is flipped, randomly cropped, and data enhancement is performed.

[0013] Furthermore, the pedestrian re-identification model uses a convolutional neural network as a feature extraction network, and the convolutional neural network includes an input layer, a convolution layer, a feature extraction layer, and an output layer, wherein the convolution layer includes multiple layers, the dual-branch structure includes a global branch and a local branch, and the dual-branch structure is located between the convolution layer and the output layer, and the polarized self-attention mechanism structure includes a channel self-attention branch and a spatial self-attention branch, and the polarized self-attention mechanism structure is located after each of the convolution layers.

[0014] Furthermore, for the polarized self-attention mechanism structure, the feature matrix of the pedestrian image output by the convolutional layer is defined as X, and the weight of the channel self-attention branch is A sp (X), then the weight A of the channel self-attention branch sp (X) satisfies the relationship (1):

[0015] A sp (X) = F SG [σ3(F SM (σ1(F GP (W q (X))))×σ2(W v (X)))] (1)

[0016] Define the weight of the spatial self-attention branch as A ch (X), then the weight A of the spatial self-attention branch ch (X) satisfies the relationship (2):

[0017]

[0018] In the above equations (1) and (2), W q 、W v 、 All are 1*1 convolution operations, σ1, σ2, σ3 are tensor deformation operations, F SM is the softmax operation, F GP is the global average pooling operation, F SG is the Sigmoid function, and the × operator is the matrix dot multiplication;

[0019] Define the weight A of the channel self-attention branch sp (X) and the weight A of the spatial self-attention branch ch The parallel fusion result of (X) is PSA p (X), then the parallel fusion result PSA p (X) satisfies the relationship (3):

[0020] PSA p (X) = Z ch +Z sp =A ch (X)⊙ ch X+A sp (X)⊙ sp X (3)

[0021] In the above equation (3), PSA p (X) is the output of the polarized self-attention mechanism structure, ⊙ ch For channel-wise multiplication operation, sp For spatial multiplication operation, Z sp is the output of the spatial self-attention branch, Z ch is the output of the channel self-attention branch.

[0022] Furthermore, for the dual-branch structure, the global branch has the same structure as the feature extraction layer of the convolutional neural network, and is used to take the parallel fusion result output by the polarized self-attention mechanism structure as input and extract the global features corresponding to the pedestrian image. The local branch is at a parallel level with the global branch, and is used to take the parallel fusion result output by the polarized self-attention mechanism structure as input and extract the local features corresponding to the pedestrian image.

[0023] Furthermore, the output layer of the pedestrian re-identification model uses the fusion result of the global features and the local features as the output result.

[0024] Furthermore, the loss function used in the training process of the pedestrian re-identification model includes at least one of triple loss, classification loss, and center loss.

[0025] In a second aspect, an embodiment of the present invention further provides a person re-identification system based on polarized self-attention, comprising:

[0026] A preprocessing module is used to obtain a shooting data set containing pedestrian images and preprocess the shooting data set to obtain a data set to be divided;

[0027] A data partitioning module, configured to divide the data set to be partitioned into a training set and a test set, wherein each of the pedestrian images in the training set and the data set has a true label;

[0028] A model building module is used to build a person re-identification model with a dual-branch structure and a polarized self-attention mechanism.

[0029] a model training module, configured to use the training set as input to the person re-identification model, and the true labels corresponding to the training set as a reference for parameter debugging, to train the person re-identification model using the Adam optimization algorithm to obtain training parameter weights;

[0030] The data recognition module is used to use the training parameter weights as the test weights of the pedestrian re-identification model and the test set as the input of the pedestrian re-identification model to obtain the pedestrian re-identification result for the test set.

[0031] In a third aspect, an embodiment of the present invention further provides a computer device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the person re-identification method based on polarized self-attention as described in any one of the above embodiments are implemented.

[0032] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the pedestrian re-identification method based on polarization self-attention as described in any one of the above embodiments are implemented.

[0033] The beneficial effects achieved by the present invention are that, due to the adoption of a two-dimensional polarized self-attention mechanism structure, information in different feature spaces can be adaptively learned, reducing information loss caused by dimensionality reduction. At the same time, a two-branch structure network is used to extract global features and local features respectively, further improving the recognition accuracy of the pedestrian re-identification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flowchart of the steps of a person re-identification method based on polarized self-attention provided by an embodiment of the present invention;

[0035] Figure 2 is a schematic structural diagram of a person re-identification model provided by an embodiment of the present invention;

[0036] Figure 3 Schematic diagram of the structure of the polarized self-attention mechanism provided by an embodiment of the present invention;

[0037] Figure 4 2 is a schematic structural diagram of a person re-identification system 200 based on polarized self-attention provided by an embodiment of the present invention;

[0038] Figure 5 It is a structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] Please refer to Figure 1 , Figure 1 This is a flowchart of a person re-identification method based on polarized self-attention provided by an embodiment of the present invention, which specifically includes the following steps:

[0041] S101: Acquire a captured data set containing pedestrian images, and preprocess the captured data set to obtain a data set to be divided.

[0042] In the step of obtaining a photographic dataset containing pedestrian images and preprocessing the photographic dataset to obtain a dataset to be divided, the preprocessing method is specifically as follows:

[0043] The size of each picture in the shooting dataset is normalized, and each picture in the shooting dataset is flipped, randomly cropped, and data enhancement is performed.

[0044] Exemplarily, the shooting dataset selected in the embodiment of the present invention may be the public Market1501 dataset, which contains 1,501 pedestrian targets captured by a camera, and each pedestrian target also includes multiple pedestrian images in the dataset. In step S101, the pedestrian images in the Market1501 dataset are preprocessed by normalization, flipping, random cropping, random erasing, etc. to obtain the dataset to be divided.

[0045] S102: Divide the data set to be divided into a training set and a test set, wherein each of the pedestrian images in the training set and the data set has a true label.

[0046] Specifically, the data to be divided is divided into the training set and the test set. In the Market1501 dataset used in the embodiment of the present invention, the training set and the test set containing different pedestrian targets are obtained according to their existing content. The training set contains 751 pedestrian targets and 12,936 pedestrian images, and the test set contains 750 pedestrian targets and 19,732 pedestrian images. Each of the pedestrian images has a true label, and the true label is used to mark a certain feature shown in the pedestrian image.

[0047] Preferably, the training set is divided into N batches, each batch contains P different pedestrian targets, and each pedestrian target corresponds to K pedestrian pictures, that is, each batch contains B=P*K pedestrian pictures as training samples.

[0048] S103. Construct a person re-identification model that includes a dual-branch structure and a polarized self-attention mechanism.

[0049] For details, please refer to Figure 2 , Figure 2 This is a structural diagram of a person re-identification model provided by an embodiment of the present invention. The person re-identification model uses a convolutional neural network (CNN) as a feature extraction network. The convolutional neural network includes an input layer, a convolution layer, a feature extraction layer, and an output layer. The convolution layer includes multiple layers. On the basis of the convolutional neural network, the dual-branch structure is arranged at the position of the feature extraction layer and includes a global branch and a local branch. The dual-branch structure is located between the convolution layer and the output layer of the overall neural network structure.

[0050] In an embodiment of the present invention, the structure of the global branch is the same as the feature extraction layer of the original convolutional neural network, that is, the original overall feature extraction of the pedestrian image is realized, mainly by using global maximum pooling after two unified sets of convolution operations to obtain output data of the same size as the original input data; the local branch can be regarded as an additional structure added between the convolution layer and the output layer. Unlike the global branch, the local branch does not include the structure for performing the global maximum pooling operation contained in the global branch. Instead, after two sets of convolution operations, the convolved image data is divided into smaller local images, and then feature extraction is performed on the local images, thereby extracting fine-grained features that are not easy to extract by hierarchical features. Preferably, the structure for dividing the local image into local images and performing local feature extraction in the local branch can also include multiple structures. When the size of the pedestrian image is large enough or the number of pedestrian targets in the image is large and the environment is complex, the multiple local features obtained by feature extraction by multiple local branches are spliced ​​to obtain output data of the same size as the original input data. By using the same loss function as the output result of the global branch to calculate the loss, the final feature extraction effect will also be better. The dual-branch structure in the embodiment of the present invention is used for illustration only, and it should be considered that the multi-branch neural network structure made on the basis of the embodiment of the present invention also falls within the scope of protection of the present invention.

[0051] In an embodiment of the present invention, the polarized self-attention mechanism structure includes a channel self-attention branch and a spatial self-attention branch, and the polarized self-attention mechanism structure is located after each of the convolutional layers of the convolutional neural network.

[0052] For details, please refer to Figure 3 , Figure 3 is a schematic diagram of the structure of the polarized self-attention mechanism provided by an embodiment of the present invention, Figure 3 Where C, H, and W correspond to the number of channels, height, and width of the pedestrian image, respectively. For the polarized self-attention mechanism structure, the feature matrix of the pedestrian image output by the convolutional layer is defined as X, and the weight of the channel self-attention branch is A. sp (X), then the weight A of the channel self-attention branch sp (X) satisfies the relationship (1):

[0053] A sp (X) = F SG [σ3(F SM (σ1(F GP (W q (X))))×σ2(W v (X)))] (1)

[0054] Define the weight of the spatial self-attention branch as A ch (X), then the weight A of the spatial self-attention branch ch (X) satisfies the relationship (2):

[0055]

[0056] In the above equations (1) and (2), W q 、W v 、 All are 1*1 convolution operations, σ1, σ2, σ3 are tensor deformation operations, F SM is the softmax operation, F GP is the global average pooling operation, F SG is the Sigmoid function;

[0057] Define the weight A of the channel self-attention branch sp (X) and the weight A of the spatial self-attention branch ch The parallel fusion result of (X) is PSA p (X), then the parallel fusion result PSA p (X) satisfies the relationship (3):

[0058] PSA p (X) = Z ch +Z sp =A ch (X)⊙ ch X+A sp (X)⊙ sp X (3)

[0059] In the above equation (3), PSA p (X) is the output of the polarized self-attention mechanism structure, ⊙ ch For channel-wise multiplication operation, sp For spatial multiplication operation, Z sp is the output of the spatial self-attention branch, Z ch is the output of the channel self-attention branch.

[0060] For the dual-branch structure, the global branch has the same structure as the feature extraction layer of the convolutional neural network, and is used to take the parallel fusion result output by the polarized self-attention mechanism structure as input and extract the global features corresponding to the pedestrian image. The local branch is at a parallel level with the global branch, and is used to take the parallel fusion result output by the polarized self-attention mechanism structure as input and extract the local features corresponding to the pedestrian image.

[0061] Specifically, in the pedestrian re-identification model constructed in an embodiment of the present invention, corresponding to the structure of the original convolutional neural network, the polarized self-attention mechanism structure acts on the convolution layer of the original convolutional neural network, and the dual-branch structure acts on the feature extraction layer of the original convolutional neural network. The dual-branch structure as a whole takes the output of the convolution layer after the polarized self-attention mechanism structure acts as input, and in the dual-branch structure, there is no structure for directly transmitting information between the global branch and the local branch. At the data input level, the global branch and the local branch both use the same input content, and use different feature extraction methods to extract features from the input content.

[0062] The output layer of the pedestrian re-identification model uses the fusion result of the global features and the local features as the output result.

[0063] It should be noted that the convolutional neural network refers to a neural network that includes a convolution structure. Some commonly used feature extraction networks, such as ResNet (Deep residual network), OSNet (Omni-Scale Network, multi-scale network), etc., are used as feature extraction networks. In their convolution structures, the polarized self-attention mechanism structure described in the embodiment of the present invention can be used, and at the end of their convolution structure, the dual-branch structure described in the embodiment of the present invention can be used. Moreover, both the polarized self-attention mechanism structure and the dual-branch structure can achieve the same technical effect as the embodiment of the present invention by participating in convolution calculations and realizing feature fusion of different branches through the output layer. Therefore, the embodiment of the present invention does not limit the basic type of the convolutional neural network. It should be considered that any neural network model constructed based on the objectives of tasks such as feature extraction, target classification, and scene segmentation can use the structure in the embodiment of the present invention and is within the scope of protection of the present invention.

[0064] S104 , using the training set as the input of the person re-identification model, and using the true label corresponding to the training set as a reference for parameter debugging, the person re-identification model is trained using the Adam optimization algorithm to obtain training parameter weights.

[0065] Specifically, using the real label corresponding to the training set as a reference for parameter debugging refers to the evaluation parameter specified for evaluating the pedestrian re-identification effect of the current model during the training process of the pedestrian re-identification model. In the embodiment of the present invention, the real label refers to the characteristics of the pedestrian target in the pedestrian picture. The re-identification degree of the real label can reflect the degree to which the overall characteristics of the pedestrian target are re-identified.

[0066] Preferably, the loss function used in the training process of the pedestrian re-identification model includes at least one of triple loss, classification loss, and center loss. In an embodiment of the present invention, the loss function used by the pedestrian re-identification model includes all three of the above, and the object of loss calculation using the loss function is the data output after the fusion of the global branch and the local branch features. The training parameter weight refers to the weight A of the channel self-attention branch in the above embodiment. sp (X) and the weight A of the spatial self-attention branch ch (X), and the parallel fusion result PSA p (X).

[0067] The number of person re-identification training iterations is related to the number of batches into which the training set is divided. In this embodiment of the present invention, if the training set is divided into N batches, the person re-identification model undergoes at least N iterations of training to fully train using the portions of the training set corresponding to all batches. The person re-identification model uses the Adam optimizer for parameter optimization during training.

[0068] S105 , using the training parameter weights as test weights of the person re-identification model, using the test set as input of the person re-identification model, and obtaining a person re-identification result for the test set.

[0069] In an embodiment of the present invention, the training parameter weight refers to the various weight parameters in the polarized self-attention mechanism structure. During the training process, due to continuous iterative optimization, the various weight parameters in the polarized self-attention mechanism structure gradually approach the value of the optimal re-identification effect. The test weight uses the parameter value obtained when the training is completed as the actual value used. The test set is used as the input of the pedestrian re-identification model after the training is completed in step S104, and the pedestrian re-identification result corresponding to the pedestrian target in the test set is obtained.

[0070] For example, please refer to Table 1, which shows the comparison results between the pedestrian re-identification method provided by the embodiment of the present invention and other public algorithms.

[0071] Table 1 Comparison results of person re-identification methods with other public algorithms

[0072]

[0073]

[0074] Both mAP and Rank-1 are commonly used evaluation indicators in the field of pedestrian re-identification, where mAP represents the recognition accuracy and Rank-1 represents the first-place hit rate. The data in Table 1 show that compared with the existing pedestrian re-identification algorithm, the pedestrian re-identification model based on the polarized self-attention mechanism structure and the dual-branch structure provided by the embodiment of the present invention can achieve significantly improved recognition accuracy and first-place hit rate.

[0075] The beneficial effects achieved by the present invention are that, due to the adoption of a two-dimensional polarized self-attention mechanism structure, information in different feature spaces can be adaptively learned, reducing information loss caused by dimensionality reduction. At the same time, a two-branch structure network is used to extract global features and local features respectively, further improving the recognition accuracy of the pedestrian re-identification model.

[0076] The embodiment of the present invention also provides a pedestrian re-identification system based on polarized self-attention, please refer to Figure 4 , Figure 4 is a schematic diagram of the structure of a person re-identification system 200 based on polarized self-attention provided by an embodiment of the present invention. The person re-identification system 200 includes:

[0077] The preprocessing module 201 is used to obtain a captured data set containing pedestrian images and preprocess the captured data set to obtain a data set to be divided;

[0078] A data partitioning module 202 is configured to partition the data set into a training set and a test set, wherein each of the pedestrian images in the training set and the data set has a true label;

[0079] A model building module 203 is used to build a person re-identification model including a dual-branch structure and a polarized self-attention mechanism structure;

[0080] A model training module 204 is configured to use the training set as input to the person re-identification model and the true labels corresponding to the training set as a reference for parameter debugging, and to train the person re-identification model using the Adam optimization algorithm to obtain training parameter weights;

[0081] The data identification module 205 is configured to use the training parameter weights as test weights of the person re-identification model and the test set as input of the person re-identification model to obtain a person re-identification result for the test set.

[0082] The pedestrian re-identification system 200 based on polarization self-attention can implement the steps in the pedestrian re-identification method based on polarization self-attention in the above embodiment, and can achieve the same technical effects. Please refer to the description in the above embodiment and will not repeat it here.

[0083] The embodiment of the present invention also provides a computer device, please refer to Figure 5 , Figure 5 3 is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. The computer device 300 includes: a memory 302, a processor 301, and a computer program stored in the memory 302 and executable on the processor 301.

[0084] The processor 301 calls the computer program stored in the memory 302 to execute the steps of the pedestrian re-identification method provided by the embodiment of the present invention. Figure 1 , specifically including:

[0085] S101: Acquire a captured data set containing pedestrian images, and preprocess the captured data set to obtain a data set to be divided.

[0086] Furthermore, in the step of obtaining a photographic dataset having pedestrian images and preprocessing the photographic dataset to obtain a dataset to be divided, the preprocessing specifically includes:

[0087] The size of each picture in the shooting dataset is normalized, and each picture in the shooting dataset is flipped, randomly cropped, and data enhancement is performed.

[0088] S102: Divide the data set to be divided into a training set and a test set, wherein each of the pedestrian images in the training set and the data set has a true label.

[0089] S103. Construct a person re-identification model that includes a dual-branch structure and a polarized self-attention mechanism.

[0090] Furthermore, the pedestrian re-identification model uses a convolutional neural network as a feature extraction network, and the convolutional neural network includes an input layer, a convolution layer, a feature extraction layer, and an output layer, wherein the convolution layer includes multiple layers, the dual-branch structure includes a global branch and a local branch, and the dual-branch structure is located between the convolution layer and the output layer, and the polarized self-attention mechanism structure includes a channel self-attention branch and a spatial self-attention branch, and the polarized self-attention mechanism structure is located after each of the convolution layers.

[0091] Furthermore, for the polarized self-attention mechanism structure, the feature matrix of the pedestrian image output by the convolutional layer is defined as X, and the weight of the channel self-attention branch is A sp (X), then the weight A of the channel self-attention branch sp (X) satisfies the relationship (1):

[0092] A sp(X) = F SG [σ3(F SM (σ1(F GP (W q (X))))×σ2(W v (X)))] (1)

[0093] Define the weight of the spatial self-attention branch as A ch (X), then the weight A of the spatial self-attention branch ch (X) satisfies the relationship (2):

[0094]

[0095] In the above equations (1) and (2), W q 、W v 、 All are 1*1 convolution operations, σ1, σ2, σ3 are tensor deformation operations, F SM is the softmax operation, F GP is the global average pooling operation, F SG is the Sigmoid function;

[0096] Define the weight A of the channel self-attention branch sp (X) and the weight A of the spatial self-attention branch ch The parallel fusion result of (X) is PSA p (X), then the parallel fusion result PSA p (X) satisfies the relationship (3):

[0097] PSA p (X) = Z ch +Z sp =A ch (X)⊙ ch X+A sp (X)⊙ sp X (3)

[0098] In the above equation (3), PSA p (X) is the output of the polarized self-attention mechanism structure, ⊙ ch For channel-wise multiplication operation, sp For spatial multiplication operation, Z sp is the output of the spatial self-attention branch, Z ch is the output of the channel self-attention branch.

[0099] Furthermore, for the dual-branch structure, the global branch has the same structure as the feature extraction layer of the convolutional neural network, and is used to take the parallel fusion result output by the polarized self-attention mechanism structure as input and extract the global features corresponding to the pedestrian image. The local branch is at a parallel level with the global branch, and is used to take the parallel fusion result output by the polarized self-attention mechanism structure as input and extract the local features corresponding to the pedestrian image.

[0100] Furthermore, the output layer of the pedestrian re-identification model uses the fusion result of the global features and the local features as the output result.

[0101] S104 , using the training set as the input of the person re-identification model, and using the true label corresponding to the training set as a reference for parameter debugging, the person re-identification model is trained using the Adam optimization algorithm to obtain training parameter weights.

[0102] Furthermore, the loss function used in the training process of the pedestrian re-identification model includes at least one of triple loss, classification loss, and center loss.

[0103] S105 , using the training parameter weights as test weights of the person re-identification model, using the test set as input of the person re-identification model, and obtaining a person re-identification result for the test set.

[0104] For example, when the computer device 300 used in the embodiment of the present invention executes the campus management pedestrian re-identification method provided in the embodiment of the present invention and obtains the data in Table 1 of the above embodiment, the computer hardware environment used is based on Intel's central processing unit Intel Xeon E5-2630 V4, the memory size used is 128GB, and the graphics processor used is NVIDIA GeForce CTX1080Ti; the software environment where the computer program is located is based on Ubuntu 20.04, the programming language is Python 3.6, the deep learning framework is Pytorch 1.8, and the CUDA version is 11.4. In addition, the computer device 300 provided in the embodiment of the present invention can also be implemented on hardware based on platforms such as Jetson Nano and HiSilicon 3559.

[0105] The computer device 300 provided in an embodiment of the present invention can implement the steps in the pedestrian re-identification method based on polarized self-attention in the above embodiment, and can achieve the same technical effects. Please refer to the description in the above embodiment and will not be repeated here.

[0106] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes and steps in the pedestrian re-identification method based on polarized self-attention provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0107] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0108] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0109] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0110] The embodiments of the present invention are described above in conjunction with the accompanying drawings. What is disclosed is only a preferred embodiment of the present invention. However, the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms and equivalent changes without departing from the scope of protection of the purpose of the present invention and the claims, which are all within the protection of the present invention.

Claims

1. A person re-identification method based on polarized self-attention, characterized in that: The method comprises: Acquire a photographic dataset containing pedestrian images, and preprocess the photographic dataset to obtain a dataset to be divided; Dividing the data set to be divided into a training set and a test set, wherein each of the pedestrian images in the training set and the data set has a true label; Construct a person re-identification model with a dual-branch structure and a polarized self-attention mechanism; The person re-identification model is trained using the training set as input of the person re-identification model and the true labels corresponding to the training set as a reference for parameter debugging, using the Adam optimization algorithm to train the person re-identification model to obtain training parameter weights; Using the training parameter weights as test weights of the person re-identification model and the test set as input of the person re-identification model, to obtain a person re-identification result for the test set; The person re-identification model uses a convolutional neural network as a feature extraction network. The convolutional neural network includes an input layer, a convolutional layer, a feature extraction layer, and an output layer, wherein the convolutional layer includes multiple layers. The dual-branch structure includes a global branch and a local branch, and the dual-branch structure is located between the convolutional layer and the output layer. The polarized self-attention mechanism structure includes a channel self-attention branch and a spatial self-attention branch, and the polarized self-attention mechanism structure is located after each convolutional layer. For the polarized self-attention mechanism structure, the feature matrix of the pedestrian image output by the convolutional layer is defined as X, and the weight of the channel self-attention branch is A sp (X), then the weight A of the channel self-attention branch sp (X) satisfies the relationship (1): A sp (X)=F SG [σ3(F SM (σ1(F GP (W q (X))))×σ2(W v (X)))] (1) Define the weight of the spatial self-attention branch as A ch (X), then the weight A of the spatial self-attention branch ch (X) satisfies the relationship (2): In the above equations (1) and (2), W q 、W v 、 All are 1*1 convolution operations, σ1, σ2, σ3 are tensor deformation operations, F SM is the softmax operation, F GP is the global average pooling operation, F SG is the Sigmoid function; Define the weight A of the channel self-attention branch sp (X) and the weight A of the spatial self-attention branch ch The parallel fusion result of (X) is PSA p (X), then the parallel fusion result PSA p (X) satisfies the relationship (3): PSA p (X)=Z ch +Z sp =A ch (X)⊙ ch X+A sp (X)⊙ sp X (3) In the above equation (3), PSA p (X) is the output of the polarized self-attention mechanism structure, ⊙ ch For channel-wise multiplication operation, sp For spatial multiplication operation, Z sp is the output of the spatial self-attention branch, Z ch is the output of the channel self-attention branch.

2. The person re-identification method based on polarized self-attention according to claim 1, characterized in that: In the step of obtaining a photographic dataset containing pedestrian images and preprocessing the photographic dataset to obtain a dataset to be divided, the preprocessing specifically includes: The size of each picture in the shooting dataset is normalized, and each picture in the shooting dataset is flipped, randomly cropped, and data enhancement is performed.

3. The person re-identification method based on polarized self-attention according to claim 1, wherein: For the dual-branch structure: the global branch has the same structure as the feature extraction layer of the convolutional neural network, and is used to take the parallel fusion result output by the polarized self-attention mechanism structure as input and extract the global features corresponding to the pedestrian image; the local branch is at a parallel level with the global branch, and is used to take the parallel fusion result output by the polarized self-attention mechanism structure as input and extract the local features corresponding to the pedestrian image.

4. The person re-identification method based on polarized self-attention according to claim 3, wherein: The output layer of the pedestrian re-identification model uses the fusion result of the global features and the local features as the output result.

5. The person re-identification method based on polarized self-attention according to claim 1, wherein: The loss function used in the pedestrian re-identification model during the training process includes at least one of triple loss, classification loss, and center loss.

6. A person re-identification system based on polarized self-attention, characterized in that: include: A preprocessing module is used to obtain a shooting data set containing pedestrian images and preprocess the shooting data set to obtain a data set to be divided; A data partitioning module, configured to divide the data set to be partitioned into a training set and a test set, wherein each of the pedestrian images in the training set and the data set has a true label; A model building module is used to build a person re-identification model with a dual-branch structure and a polarized self-attention mechanism. a model training module, configured to use the training set as input to the person re-identification model, and the true labels corresponding to the training set as a reference for parameter debugging, to train the person re-identification model using the Adam optimization algorithm to obtain training parameter weights; a data recognition module, configured to use the training parameter weights as test weights of the person re-identification model, use the test set as input of the person re-identification model, and obtain a person re-identification result for the test set; The person re-identification model uses a convolutional neural network as a feature extraction network. The convolutional neural network includes an input layer, a convolutional layer, a feature extraction layer, and an output layer, wherein the convolutional layer includes multiple layers. The dual-branch structure includes a global branch and a local branch, and the dual-branch structure is located between the convolutional layer and the output layer. The polarized self-attention mechanism structure includes a channel self-attention branch and a spatial self-attention branch, and the polarized self-attention mechanism structure is located after each convolutional layer. For the polarized self-attention mechanism structure, the feature matrix of the pedestrian image output by the convolutional layer is defined as X, and the weight of the channel self-attention branch is A sp (X), then the weight A of the channel self-attention branch sp (X) satisfies the relationship (1): A sp (X)=F SG [σ3(F SM (σ1(F GP (W q (X))))×σ2(W v (X)))] (1) Define the weight of the spatial self-attention branch as A ch (X), then the weight A of the spatial self-attention branch ch (X) satisfies the relationship (2): In the above equations (1) and (2), W q 、W v 、 All are 1*1 convolution operations, σ1, σ2, σ3 are tensor deformation operations, F SM is the softmax operation, F GP is the global average pooling operation, F SG is the Sigmoid function; Define the weight A of the channel self-attention branch sp (X) and the weight A of the spatial self-attention branch ch The parallel fusion result of (X) is PSA p (X), then the parallel fusion result PSA p (X) satisfies the relationship (3): PSA p (X)=Z ch +Z sp =A ch (X)⊙ ch X+A sp (X)⊙ sp X (3) In the above equation (3), PSA p (X) is the output of the polarized self-attention mechanism structure, ⊙ ch For channel-wise multiplication operation, sp For spatial multiplication operation, Z sp is the output of the spatial self-attention branch, Z ch is the output of the channel self-attention branch.

7. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the person re-identification method based on polarized self-attention are implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the pedestrian re-identification method based on polarization self-attention as described in any one of claims 1 to 5.

Images