Pedestrian re-identification method, device and computer equipment based on Transformer network model

Abstract

The present invention belongs to the field of computer vision, and relates to a method, device and computer equipment for pedestrian re-identification based on a Transformer network model; the method comprises obtaining a target pedestrian image and pre-processing it to generate a standard pedestrian image; dividing the standard pedestrian image into a plurality of square sub-images with overlapping parts by using a sliding window; performing horizontal linear projection and vertical linear projection on each square sub-image to obtain a horizontal feature matrix and a vertical feature matrix; inputting the sub-images into a pre-trained improved Transformer network model to predict the recognition result of the target pedestrian image. The present invention divides the input image into a plurality of square blocks with overlapping parts by using a sliding window, and highlights the features of pedestrians at the junction of occluders and pedestrians; using an improved Transformer network structure, the association of pedestrian features in all directions is strengthened, and the accuracy of pedestrian re-identification is improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and relates to a pedestrian re-identification method, device and computer equipment based on a Transformer network model. Background Art

[0002] Person Re-IDentification (Re-ID) can be considered as a sub-problem in the field of image retrieval that focuses on pedestrian images. In this task, a pedestrian image to be detected is first given, and then the pedestrian is detected again from multiple scenes containing pedestrian images taken by multiple cameras.

[0003] In real scenes, pedestrian images come from different cameras in different scenes. Pedestrian images are inevitably affected by shooting angles and unpredictable object occlusions, which makes the images of the same pedestrian very different. Therefore, it is very necessary to design a pedestrian re-identification method that can effectively extract image features under complex scene conditions.

[0004] At present, the pedestrian re-identification methods with relatively good results are all based on feature fusion methods. Among them, the method of extracting local features is crucial, such as PCB and Pyramid. PCB is the earliest method to extract local features by geometrically dividing pedestrian features in the horizontal direction. This method divides the input image into multiple horizontal strips and then trains the convolutional neural network on them to obtain local features, and then trains them through multiple non-shared weight classifiers. Pyramid considers different division granularities on the basis of PCB to better integrate global features with local features.

[0005] However, the above method still has the following problems:

[0006] The occlusion of objects causes the failure of local feature extraction of some pedestrian images, resulting in a large difference in features between the complete image and the occluded image of the same pedestrian, leading to detection failure. Summary of the invention

[0007] Based on the problems existing in the prior art, the present invention proposes a pedestrian re-identification method, device and computer equipment based on the Transformer network model, which can solve the problem of pedestrian re-identification under object occlusion.

[0008] In a first aspect of the present invention, the present invention provides a pedestrian re-identification method based on a Transformer network model, the method comprising:

[0009] Acquire a target pedestrian image, and pre-process the target pedestrian image to generate a standard pedestrian image; the target pedestrian image is a pedestrian image to be subjected to pedestrian re-identification;

[0010] Dividing the standard pedestrian image into a plurality of square sub-images with overlapping parts using a sliding window;

[0011] Each square sub-image is subjected to horizontal linear projection and vertical linear projection to obtain a horizontal feature matrix and a vertical feature matrix;

[0012] The vertical feature matrix and the horizontal feature matrix are input into a pre-trained Transformer network model to predict the recognition result of the target pedestrian image.

[0013] In a second aspect of the present invention, the present invention further proposes a pedestrian re-identification device based on a Transformer network model, the device comprising:

[0014] An image acquisition module is used to acquire a target pedestrian image; the target pedestrian image is a pedestrian image to be re-identified;

[0015] An image preprocessing module, used for preprocessing the target pedestrian image to generate a standard pedestrian image;

[0016] An image segmentation module, used for dividing the standard pedestrian image into a plurality of square sub-images with overlapping parts by using a sliding window;

[0017] An image mapping module is used to perform horizontal linear projection and vertical linear projection on each square sub-image to obtain a horizontal feature matrix and a vertical feature matrix;

[0018] The image recognition module is used to input the horizontal feature matrix and the vertical feature matrix into a pre-trained improved Transformer network model to predict the recognition result of the target pedestrian image.

[0019] In the third aspect of the present invention, the present invention also provides a computer device, comprising a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is used to call the program instructions to execute a pedestrian re-identification method based on a Transformer network model as described in the first aspect of the present invention.

[0020] The beneficial effects of the present invention are:

[0021] In response to the occlusion problem, the present invention proposes a pedestrian re-identification method, device and computer equipment based on the Transformer network model. The present invention divides the pedestrian image into multiple grouped sub-image inputs, reducing the impact of occlusions on pedestrian features. And uses a sliding window to divide the input image into multiple square sub-images with overlapping parts, highlighting the features of pedestrians at the intersection of occlusions and pedestrians. The present invention also constructs an improved Transformer network, and in the first layer of the network, a dual multi-scale Transformer structure based on a multi-branch multi-head attention module is constructed to obtain the pedestrian features of the image in both horizontal and vertical directions, enriching the feature representation of pedestrians and improving the accuracy of pedestrian re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0023] Figure 1 A schematic diagram of a pedestrian re-identification method based on a Transformer network model of the present invention;

[0024] Figure 2 A schematic diagram of the network structure of a pedestrian re-identification method based on a Transformer network model of the present invention;

[0025] Figure 3 This is a schematic diagram of a multi-branch multi-head attention module in a pedestrian re-identification method based on a Transformer network model in the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0028] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0029] The present application provides a pedestrian re-identification method based on the Transformer network model, which can perform image feature extraction, image matching and related data acquisition and processing based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0030] Figure 1 FIG. 1 is a flow chart of a pedestrian re-identification method based on a Transformer network model of the present invention; Figure 1 As shown, the method comprises the following steps:

[0031] 101. Acquire a target pedestrian image, and pre-process the target pedestrian image to generate a standard pedestrian image; the target pedestrian image is a pedestrian image to be subjected to pedestrian re-identification;

[0032] In an embodiment of the present invention, the pedestrian image to be re-identified can be a training set image in an existing pedestrian image dataset, a validation set image in an existing pedestrian image dataset, or a test set image in an existing pedestrian image dataset; it can also be a video image collected from video surveillance, wherein the video image contains a pedestrian image. In an embodiment of the present invention, the standard pedestrian image is a pedestrian image with the same preset size. Taking Market-1501 as an example of a dataset, after preprocessing the pedestrian images in the dataset, a standard pedestrian image with a size of 256×128 can be obtained.

[0033] In some embodiments, after the pedestrian images in the data set are standardized, the 33,217 images of 1,501 pedestrians can be divided into a training set and a test set in a ratio of 6:4, where the training set includes 19,926 pedestrian images and the test set includes 13,291 pedestrian images.

[0034] Figure 2 Schematic diagram of the network structure of a pedestrian re-identification method based on the Transformer network model of the present invention, as shown in Figure 2 As shown, after obtaining the standard pedestrian image, it is necessary to use a sliding window to take blocks of the standard pedestrian image, and input them into the improved Transformer network model using horizontal linear projection and vertical linear projection respectively, and finally obtain the corresponding recognition results. The network structure will be described in detail below in conjunction with steps 102 to 104.

[0035] 102. Divide the standard pedestrian image into a plurality of square sub-images with overlapping parts using a sliding window;

[0036] In some embodiments of the present invention, the standard pedestrian image may be segmented into a plurality of non-overlapping grouped sub-images; and each of the grouped sub-images may be segmented into a plurality of square sub-images with overlapping portions using a sliding window.

[0037] Each of the standard pedestrian images is divided into N square sub-images with overlapping parts, where the calculation formula of N is as follows:

[0038]

[0039] Among them, H is the height of the first standard pedestrian image, W is the width of the standard pedestrian image, S is the step size, P is the height and width of the square sub-image, K represents the total number of grouped sub-images, K=K1×K2; K1 represents the number of grouped sub-images divided vertically, and K2 represents the number of grouped sub-images divided horizontally.

[0040] For the convenience of description, this embodiment divides the standard pedestrian image into two upper and lower sub-images, and uses a sliding window to divide the upper and lower sub-images into N square sub-images with overlapping parts. At this time, the calculation formula of N is as follows:

[0041]

[0042] Among them, in the above formula, K=2, K1=2, K2=1.

[0043] In the embodiment of the present invention, a standard pedestrian image of 256×128 is taken as an example. Figure 2 As shown in FIG. 1 , the standard pedestrian image can be divided into two upper and lower group sub-images a and b, each with a size of 128×128. Each group sub-image is divided into N square sub-images with overlapping parts using a sliding window. In this example, the size of the square image block is 16×16, and the step size is 14, so the pedestrian image is divided into 162 16×16 square image blocks.

[0044] It can be understood that the present invention divides the pedestrian image into multiple sub-image inputs, and the occluders and pedestrians will most likely be divided into different sub-image blocks. When the network model extracts features from the sub-images separately, it can extract the features of the pedestrians and the features of the occluders separately, thereby reducing the impact of the occluders on the pedestrian features.

[0045] The input image is divided into a plurality of overlapping square blocks using a sliding window. The overlapping portion of the image contains the edge portion of the pedestrian that is blocked. The improved model of the present invention performs additional feature extraction on the pedestrian portion of these edges, thereby highlighting the features of the pedestrian at the junction of the blocker and the pedestrian. 103. Each square sub-image is subjected to horizontal linear projection and vertical linear projection to obtain a horizontal feature matrix and a vertical feature matrix;

[0046] In the embodiment of the present invention, each square sub-image obtained in step 102 is linearly projected in a horizontal expansion manner and then spliced ​​in the vertical direction to obtain a horizontal feature matrix; each square image is linearly projected in a vertical expansion manner and then spliced ​​in the horizontal direction to obtain a vertical feature matrix; that is, the vertical feature matrix Z h , the horizontal feature matrix Z w , where Z w The dimension is N×3P 2 , Z h Dimension is 3P 2 ×N.

[0047] Z h =[F h (x 1 );F h (x 2 );…F h (x N )

[0048] Z w =[F w (x 1 );F w (x 2 );…F w (x N )

[0049] Among them, {x i |i=1,2,…,N} represents N different square sub-images, F w Indicates horizontal expansion, F h Indicates vertical expansion.

[0050] Taking the standard pedestrian image of 256×128 in step 102 as an example, the input image is divided into 162 16×16 square image blocks, which are spliced ​​in the vertical direction by the linear projection of the horizontal expansion method and in the horizontal direction by the linear projection of the vertical expansion method. Thus, the horizontal feature matrix Z w The size is 162×768, the vertical feature matrix Z h The size is 768×162.

[0051] 104. Input the vertical feature matrix and the horizontal feature matrix into a pre-trained Transformer network model to predict the recognition result of the target pedestrian image.

[0052] In the embodiment of the present invention, the Transformer network model is an improved Transformer network model, such as Figure 2 As shown, the improved Transformer network model includes a dual multi-scale Transformer structure of the first layer, an L-1 layer Origin Transformer structure and a fully connected layer of the last layer; the dual multi-scale Transformer structure is used to calculate two sets of global feature matrices and local feature matrices of different scales, and fuse the global feature matrices and the local feature matrices; the L-1 layer Origin Transformer structure is used to process the fused features and extract pedestrian features; the fully connected layer is used to process the pedestrian features and predict the recognition results of the target pedestrian image. In this example, L=10, and there are 10 layers of Transformer structure. Of course, the specific number of layers of L can be set according to the actual situation, at least to ensure that L≥2.

[0053] The dual multi-scale Transformer structure includes two multi-scale Transformer structures, which are named horizontal multi-scale Transformer structure and vertical multi-scale Transformer structure respectively. The horizontal multi-scale Transformer structure calculates the horizontal feature matrix, and the vertical multi-scale Transformer structure calculates the vertical feature matrix. Each multi-scale Transformer structure includes a multi-branch multi-head attention module (MBMHA) based on top-level branches, middle-level branches and bottom-level branches, wherein the top-level branch does not perform any processing on the input feature matrix and directly extracts the global features of the pedestrian, while the middle-level branch divides the input feature matrix into two equal parts to extract the local features of the pedestrian, and the bottom-level branch divides the input feature matrix into three equal parts to extract the local features of the pedestrian. The global features and local features of the horizontal scale Transformer structure are fused and spliced ​​to output a horizontal fusion feature matrix; the global features and local features of the vertical multi-scale Transformer structure are fused and spliced ​​to output a vertical fusion feature matrix; the horizontal fusion feature matrix is ​​projected using the horizontal weight matrix to obtain the feature matrix of the horizontal multi-scale Transformer structure; the vertical fusion feature is projected using the vertical weight matrix to obtain the feature matrix of the vertical multi-scale Transformer structure. The horizontal feature matrix and the vertical feature matrix are then fused by weight matrix projection to obtain the feature matrix of the first-layer dual multi-scale Transformer structure.

[0054] It can be understood that in the embodiment of the present invention, it is necessary to add corresponding position information and grouping information G to the horizontal feature matrix and the vertical feature matrix respectively, and the dimension of G is N×2 to form two horizontal embedding feature matrices (E w ) and the vertical embedding feature matrix (E h ), E h The dimension is (3P 2 +2)×N,E w The dimension is N×(3P 2 +2), E h ,E w It can be expressed as:

[0055] E h =Concat(Z h ,G T )

[0056] E w =Concat(Z w ,G)

[0057] Taking the standard pedestrian image of 256×128 in step 102 as an example, the horizontal feature matrix Z w The size is 162×768, the vertical feature matrix Z h The size is 768×162; since the dimension of the position and grouping information G is N×2, and N=162 here, it is equivalent to adding two dimensions of feature matrices, so the horizontal embedding feature matrix dimension is 162×770, and the vertical embedding feature matrix dimension is 770×162.

[0058] The horizontal feature matrix E w and the vertical feature matrix E h They are input into the dual multi-scale Transformer structure of the first layer respectively, and then Figure 3 The three different branches shown are Up-Layer, Mid-Layer, and Down-Layer. The three branches can extract pedestrian features at multiple scales.

[0059] For Up-Layer, E h ,E w The feature matrix is ​​segmented to extract global features, while Mid-Layer and Down-Layer respectively horizontally divide the embedding feature matrix into two equal parts and three equal parts to extract local features.

[0060] For Mid-Layer, E w Divided into two dimensions: The characteristic matrix and E h Divided into two dimensions: The characteristic matrix and

[0061] For Down-Layer, E w Divided into three dimensions: The characteristic matrix as well as E h Divided into three dimensions: The characteristic matrix as well as After that, the three branches perform multi-head attention mechanism operations respectively.

[0062] When calculating multi-branch multi-head self-attention, for each feature matrix, first use the weight matrix W Q ,WK and W V Project it into Q, K, V, calculate the attention matrix head of a single head, concatenate the attention matrices of M heads, and pass through the weight matrix W O The transformation obtains the horizontal and vertical output feature matrices on one branch.

[0063] The horizontal and vertical multi-branch multi-head self-attention values ​​can be expressed as and Where v is equal to 1, 2, and 3 respectively, corresponding to three branches, and there are M heads in each branch.

[0064] For Up-Layer, and The calculation formula is as follows:

[0065]

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] in Both represent weight matrices with dimension N×N, The dimension is (3P 2 +2)×(3P 2 +2) weight matrix. The dimension is (3P 2 +2)×N, The dimension is N×(3P 2 +2), The dimension is (3P 2 +2)×(M(3P 2 +2)) weight matrix, The dimension is (M(3P 2 +2))×(3P 2 +2) weight matrix.

[0073] For Mid-Layer, and The calculation formula is as follows:

[0074]

[0075]

[0076]

[0077]

[0078]

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] in, E h The matrix after the feature matrix is ​​horizontally divided into two equal parts has the dimension E w The matrix after the feature matrix is ​​horizontally divided into two equal parts has the dimension represents a weight matrix of dimension N×N, The dimension is (3P 2 +2)×(3P 2 +2) weight matrix. and The dimensions are consistent with those in the Up-Layer.

[0085] For Down-Layer, and The calculation formula is as follows:

[0086]

[0087]

[0088]

[0089]

[0090]

[0091]

[0092]

[0093]

[0094]

[0095]

[0096] in, E h The matrix after the feature matrix is ​​divided into three equal parts horizontally has the dimension E w The matrix after the feature matrix is ​​divided into three equal parts horizontally has the dimension represents a weight matrix of dimension N×N, The dimension is (3P 2 +2)×(3P 2 +2) weight matrix. and The dimensions are consistent with those in the Up-Layer.

[0097] In summary, and are the two sets of feature matrices output at the end. It contains three dimensions of N×(3P 2 +2), There are three dimensions (3P 2 +2)×N feature matrix.

[0098] In this example, for the Up-Layer, the horizontal embedding feature matrix dimension input to each head is 162×770, and the vertical embedding feature matrix dimension is 770×162. Map the horizontal embedding feature matrix dimension of 162×770 to Q up-w ,K up-w ,V up-w , mapping the vertical embedding feature matrix of dimension 770×162 to Q up-h ,K up-h ,V up-h .in The dimensions are 770×770. The dimensions are all 162 × 162. The formula is as follows:

[0099]

[0100]

[0101]

[0102] Calculated Q up-w ,K up-w ,V up-w The dimensions are 162×770, Q up-h ,K up-h ,V up-h The dimension is 770 × 162. Then calculate the self-attention matrix Attention(Q,K,V) of a single head, the formula is as follows:

[0103]

[0104] Wherein d is a constant, which is 64 in this example.

[0105] Horizontal Attention(Q,K,V) up-w The dimension is 162×770, vertical Attention (Q, K, V) up-h The dimension is 770×162, and the dimensions are concatenated separately. The horizontal feature matrix is ​​concatenated to obtain a 162×9240 feature matrix, which is consistent with the matrix with a dimension of 9240×770. After dot multiplication, we get a 162×770 matrix. The vertical feature matrix is ​​concatenated to get a 9240×162 feature matrix with a dimension of 770×9240. After dot multiplication, we get a matrix of 770×162. In this way, we get the horizontal feature matrix and vertical feature matrix output by Up-Layer.

[0106] According to the above process, the feature vectors of the Mid-Layer and Down-Layer branches in the dual structure are extracted respectively, so that two sets of feature matrices are obtained. One set of horizontal matrices has three feature matrices with dimensions of 162×770, and the other set of vertical matrices has three feature matrices with dimensions of 770×162. Each set has one global feature and two local features.

[0107] It can be understood that the dual Transformer network structure of the present invention extracts features from the pedestrian image from the vertical direction and the horizontal direction respectively. This method strengthens the connection between each pixel in the pedestrian image in the horizontal and vertical directions respectively, and improves the accuracy of pedestrian re-identification. The multi-branch multi-head attention module extracts the features of pedestrians at different scales, and the fusion thereof greatly optimizes the performance of the model. In an embodiment of the present invention, two sets of global feature matrices and local feature matrices of different scales can be obtained through the above calculations, and the features are fused to obtain two fused feature matrices respectively, that is, the global feature matrix and the local feature matrix of the vertical feature matrix are spliced ​​and fused to output a vertical fused feature matrix; the global feature vectors and the local feature matrix of the horizontal feature matrix are spliced ​​and fused; the horizontal fused feature matrix is ​​output; the horizontal fused feature matrix is ​​projected using the horizontal weight matrix to obtain the output feature matrix of the horizontal multi-scale Transformer structure; the vertical fusion matrix is ​​projected using the vertical weight matrix to obtain the output feature matrix of the vertical multi-scale Transformer structure.

[0108] Taking the matrix dimensions corresponding to the above embodiment as an example, in this example, a horizontal feature matrix of 162×2310 and a vertical matrix of 2310×162 are obtained after splicing. After the dot multiplication, the final output matrix of the horizontal multi-scale Transformer structure is obtained, with a dimension of 162×770. A matrix with a dimension of 770×2310 The final output matrix of the vertical multi-scale Transformer structure is obtained by dot multiplication with the vertical feature matrix of 2310×162, with a dimension of 770×162. Finally, the final output matrix of the two Transformer structures is fused with the feature matrix to obtain the output matrix of the dual multi-scale Transformer layer. In this example, the horizontal feature matrix of 162×770 is dot multiplied with the vertical matrix of 770×162 to obtain a feature matrix of 162×162, and the matrix W of 162×770 is dot multiplied to obtain the final output matrix of 162×770.

[0109] In an embodiment of the present invention, after the dual multi-scale Transformer structure is processed, the pedestrian features are extracted through the L-1 layer Origin Transformer structure, and then the pedestrian features are predicted by the fully connected layer to obtain the recognition result of the target pedestrian image; the L-1 layer Origin Transformer structure and the fully connected layer are similar to the structure of the traditional Transformer network model, and the present invention will not repeat them.

[0110] In an embodiment of the present invention, the Transformer network model can be pre-trained using training set images. When the Transformer network model is trained, it can be used for recognition prediction of target pedestrian images.

[0111] In the embodiment of the present invention, the training of the Transformer network model can use a loss function to train the entire process and model to obtain trained model parameters. Finally, the pedestrian image to be recognized is input into the network model to obtain a recognition result.

[0112] In an embodiment of the present invention, the pre-training process of the improved Transformer network model includes:

[0113] Acquire a training pedestrian image, and preprocess the training pedestrian image to generate a standard pedestrian image; the training pedestrian image is a pedestrian image used to train the improved Transformer network model;

[0114] Dividing the standard pedestrian image corresponding to the training pedestrian image into a plurality of square sub-images with overlapping parts using a sliding window;

[0115] Performing horizontal linear projection and vertical linear projection on each square sub-image corresponding to the training pedestrian image to obtain a vertical feature matrix and a horizontal feature matrix;

[0116] Inputting the horizontal feature matrix and the vertical feature matrix corresponding to the training pedestrian image into the improved Transformer network model, and using the loss function to train the parameters of the network model;

[0117] When the improved Transformer network model reaches a preset number of iterations or converges, the training of the improved Transformer network model is completed.

[0118] Among them, this embodiment can also compare and train the pedestrian features obtained after the training set images go through steps 101 to 104 with the real pedestrian features of the training set images, that is, use the loss function to perform multiple rounds of training and learning on the entire process and model. In this example, the loss function includes identity information loss and triplet loss.

[0119] Identity loss (L id )The formula is as follows:

[0120]

[0121] Among them, y represents the label of the image and c represents the number of identities.

[0122] Triplet loss (L t )The formula is as follows:

[0123] L t =d(x a ,x p )-d(x a ,x n )+m] +

[0124] Among them, x a is the benchmark sample, x p is a positive sample, x n is a negative sample, d is the distance representation function, m is the distance between triplet losses, [·] + , which is equivalent to max(.,0).

[0125] Combining global features with local features, the overall loss function (L) formula is as follows:

[0126]

[0127] Train for multiple rounds until the loss value basically does not change. In this example, 64 rounds of training are performed, where the initial learning rate is set to 0.000125, and the optimization strategy is optimizing_method:sgd (stochastic gradient descent). The learning rate warm start round epochs is 4, and the decay rate weight_deca is 1e -7 In the first 16 training rounds, the learning rate was hot-started. After 16 rounds, the learning rate reached a stable state. In the next 48 rounds, the model was trained relatively smoothly. After 60 rounds, the loss value remained basically unchanged, the model converged, and finally the parameters of the network model were obtained.

[0128] In this example, the relevant parameters of the Transformer network model trained on the training set images can be stored offline, and the network model can be used to recognize the target pedestrian image. The test results of the network model in the present invention are shown in Table 1.

[0129] Table 1 Recognition test results of Transformer network model

[0130]

[0131]

[0132] When querying pedestrian images, you only need to put the query image into the network to extract features, and then you can quickly calculate the distance between it and all images in the test set, and sort the distances in ascending order, and use the sorting results as the output of the pedestrian re-identification model.

[0133] The embodiment of the present invention further provides a pedestrian re-identification device based on a Transformer network model, the device comprising:

[0134] An image acquisition module is used to acquire a target pedestrian image; the target pedestrian image is a pedestrian image to be re-identified;

[0135] An image preprocessing module, used for preprocessing the target pedestrian image to generate a standard pedestrian image;

[0136] An image segmentation module, used for dividing the standard pedestrian image into a plurality of square sub-images with overlapping parts by using a sliding window;

[0137] An image mapping module is used to perform horizontal linear projection and vertical linear projection on each square sub-image to obtain a horizontal feature matrix and a vertical feature matrix;

[0138] The image recognition module is used to input the horizontal feature matrix and the vertical feature matrix into a pre-trained improved Transformer network model to predict the recognition result of the target pedestrian image.

[0139] An embodiment of the present invention also provides a computer device, comprising a processor and a memory, wherein the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is used to call the program instructions to execute a pedestrian re-identification method based on a Transformer network model as described in the present invention.

[0140] An embodiment of the present invention provides a processing device, including a processor and a storage device; the processor is suitable for executing various programs; the storage device is suitable for storing multiple programs; the program is suitable for being loaded and executed by the processor to implement the above-mentioned pedestrian re-identification method based on the Transformer network model.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

[0142] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, disk or CD, etc.

[0143] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A pedestrian re-identification method based on Transformer network model, characterized in that: The method comprises: Acquire a target pedestrian image, and pre-process the target pedestrian image to generate a standard pedestrian image; the target pedestrian image is a pedestrian image to be re-identified; Dividing the standard pedestrian image into a plurality of square sub-images with overlapping parts using a sliding window; Each square sub-image is subjected to horizontal linear projection and vertical linear projection to obtain a vertical feature matrix and a horizontal feature matrix; Inputting the vertical feature matrix and the horizontal feature matrix into a pre-trained improved Transformer network model to predict the recognition result of the target pedestrian image; The improved Transformer network model includes a dual multi-scale Transformer structure, an L-1 layer Origin Transformer structure and a fully connected layer; the dual multi-scale Transformer structure is used to calculate global feature matrices and local feature matrices of different scales obtained based on horizontal feature matrices and vertical feature matrices, and to perform feature fusion on the global feature matrices and the local feature matrices; the L-1 layer Origin Transformer structure is used to process the fused feature matrix to extract pedestrian features; the fully connected layer is used to process the pedestrian features and predict the recognition result of the target pedestrian image; The dual multi-scale Transformer structure includes two multi-scale Transformer structures, namely a horizontal multi-scale Transformer structure and a vertical multi-scale Transformer structure; wherein the horizontal multi-scale Transformer structure calculates the horizontal feature matrix, and the vertical multi-scale Transformer structure calculates the vertical feature matrix; each multi-scale Transformer structure includes a multi-branch multi-head attention module based on a top-level branch, a middle-level branch and a bottom-level branch, wherein the top-level branch does not perform any processing on the input feature matrix and directly extracts the global features of the pedestrian, while the middle-level branch divides the input feature matrix into two equal parts to extract the local features of the pedestrian, and the bottom-level branch divides the input feature matrix into three equal parts to extract the pedestrian. local features of people; the global features of the horizontal multi-scale Transformer structure and the local features of the two corresponding branches are fused and spliced ​​to output a horizontal fused feature matrix; the global features of the vertical multi-scale Transformer structure and the local features of the two corresponding branches are fused and spliced ​​to output a vertical fused feature matrix; the horizontal fused feature matrix is ​​projected using the horizontal weight matrix to obtain the feature matrix of the horizontal multi-scale Transformer structure; the vertical fused features are projected using the vertical weight matrix to obtain the feature matrix of the vertical multi-scale Transformer structure; the horizontal feature matrix and the vertical feature matrix are then fused by means of weight matrix projection to obtain the feature matrix of the first-layer dual multi-scale Transformer structure.

2. According to the method for person re-identification based on the Transformer network model of claim 1, It is characterized in that The sub-image division process of the standard pedestrian image includes: dividing the standard pedestrian image into a plurality of non-overlapping group sub-images; and dividing each group sub-image into a plurality of square sub-images with overlapping parts by using a sliding window.

3. According to claim 2, a pedestrian re-identification method based on a Transformer network model is characterized in that: Each of the standard pedestrian images is divided into N square sub-images with overlapping parts, where N is calculated as follows: Among them, H is the height of the standard pedestrian image, W is the width of the first standard pedestrian image, S is the step size, P is the height and width of the square sub-image, K represents the total number of grouped sub-images, K=K1×K2; K1 represents the number of grouped sub-images divided vertically, and K2 represents the number of grouped sub-images divided horizontally.

4. According to the method for person re-identification based on Transformer network model in claim 1, it is characterized in that: The method of performing horizontal linear projection and vertical linear projection on each square sub-image to obtain a horizontal feature matrix and a vertical feature matrix comprises: linearly projecting each square sub-image in a horizontal expansion manner to obtain a horizontal expansion vector of each sub-image, and then splicing N horizontal expansion vectors in a vertical direction to obtain N×P 2 The horizontal feature matrix of P is obtained by linearly projecting each square sub-image in a vertical expansion mode, and then splicing the N vertical expansion vectors in the horizontal direction to obtain P 2 ×N vertical feature matrix, where P is the height and width of the square sub-image.

5. According to the method for person re-identification based on Transformer network model in claim 1, it is characterized in that: Before calculating the horizontal feature matrix, the method further includes adding position and grouping information to the horizontal feature matrix; before calculating the vertical feature matrix, the method further includes adding position and grouping information to the vertical feature matrix.

6. According to the method for person re-identification based on Transformer network model in claim 1, it is characterized in that: The pre-training process of the improved Transformer network model includes: Acquire a training pedestrian image, and preprocess the training pedestrian image to generate a standard pedestrian image; the training pedestrian image is a pedestrian image used to train the improved Transformer network model; Dividing the standard pedestrian image corresponding to the training pedestrian image into a plurality of square sub-images with overlapping parts using a sliding window; Performing horizontal linear projection and vertical linear projection on each square sub-image corresponding to the training pedestrian image to obtain a vertical feature matrix and a horizontal feature matrix; Inputting the horizontal feature matrix and the vertical feature matrix corresponding to the training pedestrian image into the improved Transformer network model, and using the loss function to train the parameters of the network model; When the improved Transformer network model reaches a preset number of iterations or converges, the training of the improved Transformer network model is completed.

7. A pedestrian re-identification device based on a Transformer network model, characterized in that: The device comprises: An image acquisition module is used to acquire a target pedestrian image; the target pedestrian image is a pedestrian image to be re-identified; An image preprocessing module, used for preprocessing the target pedestrian image to generate a standard pedestrian image; An image segmentation module, used for dividing the standard pedestrian image into a plurality of square sub-images with overlapping parts by using a sliding window; An image mapping module is used to perform horizontal linear projection and vertical linear projection on each square sub-image to obtain a horizontal feature matrix and a vertical feature matrix; An image recognition module, used for inputting the horizontal feature matrix and the vertical feature matrix into a pre-trained improved Transformer network model to predict the recognition result of the target pedestrian image; The improved Transformer network model includes a dual multi-scale Transformer structure, an L-1 layer Origin Transformer structure and a fully connected layer; the dual multi-scale Transformer structure is used to calculate global feature matrices and local feature matrices of different scales obtained based on horizontal feature matrices and vertical feature matrices, and to perform feature fusion on the global feature matrices and the local feature matrices; the L-1 layer Origin Transformer structure is used to process the fused feature matrix to extract pedestrian features; the fully connected layer is used to process the pedestrian features and predict the recognition result of the target pedestrian image; The dual multi-scale Transformer structure includes two multi-scale Transformer structures, namely a horizontal multi-scale Transformer structure and a vertical multi-scale Transformer structure; wherein the horizontal multi-scale Transformer structure calculates the horizontal feature matrix, and the vertical multi-scale Transformer structure calculates the vertical feature matrix; each multi-scale Transformer structure includes a multi-branch multi-head attention module based on a top-level branch, a middle-level branch and a bottom-level branch, wherein the top-level branch does not perform any processing on the input feature matrix and directly extracts the global features of the pedestrian, while the middle-level branch divides the input feature matrix into two equal parts to extract the local features of the pedestrian, and the bottom-level branch divides the input feature matrix into three equal parts to extract the pedestrian. local features of people; the global features of the horizontal multi-scale Transformer structure and the local features of the two corresponding branches are fused and spliced ​​to output a horizontal fused feature matrix; the global features of the vertical multi-scale Transformer structure and the local features of the two corresponding branches are fused and spliced ​​to output a vertical fused feature matrix; the horizontal fused feature matrix is ​​projected using the horizontal weight matrix to obtain the feature matrix of the horizontal multi-scale Transformer structure; the vertical fused features are projected using the vertical weight matrix to obtain the feature matrix of the vertical multi-scale Transformer structure; the horizontal feature matrix and the vertical feature matrix are then fused by means of weight matrix projection to obtain the feature matrix of the first-layer dual multi-scale Transformer structure.

8. A computer device, characterized in that: The computer device includes a processor and a memory, the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to call the program instructions to execute a pedestrian re-identification method based on a Transformer network model as described in any one of claims 1-6.

Images