A pedestrian re-identification feature extraction method and device, electronic equipment and medium

Abstract

The application provides a pedestrian re-identification feature extraction method and device, electronic equipment and medium. The method comprises: inputting a to-be-extracted image into a feature extraction model of a backbone network adopting a ResNet-50 model to obtain complementary attention features output by a third layer convolution block, a fourth layer convolution block and a fifth layer convolution block of the feature extraction model respectively; and extracting a pedestrian re-identification feature of the to-be-extracted image according to the complementary attention features output by the third layer convolution block, the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block. The application extracts the pedestrian re-identification feature of the to-be-extracted image by combining the complementary attention features of the multi-scale spatial attention and the multi-scale channel attention, can extract the pedestrian re-identification feature of the to-be-extracted image, and improves the efficiency and accuracy of extracting the pedestrian re-identification feature.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of pedestrian re-identification, in particular to a pedestrian re-identification feature extraction method and device, an electronic device and a medium. BACKGROUND

[0002] Pedestrian re-identification technology monitors pedestrians in a video monitoring scene, extracts pedestrian re-identification features in the monitored image to be extracted, and then finds a specific pedestrian. However, in real-world scenarios, pedestrian re-identification tasks still face challenges such as pedestrian occlusion, similar appearance, and background interference. Currently, technical personnel manually extract pedestrian re-identification features in the image to be extracted through various operators to find a specific pedestrian.

[0003] However, when the number of images to be extracted is large, the efficiency of manually extracting pedestrian re-identification features in the image to be extracted by technical personnel through various operators is low, and the extraction accuracy of pedestrian re-identification features is low. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a pedestrian re-identification feature extraction method and device, an electronic device and a medium, which can extract pedestrian re-identification features from an image to be extracted, improve the efficiency and accuracy of extracting pedestrian re-identification features.

[0005] In a first aspect, the present application provides a pedestrian re-identification feature extraction method, which comprises:

[0006] Obtaining an image to be extracted;

[0007] Inputting the image to be extracted into a feature extraction model of a backbone network using a ResNet-50 model to obtain complementary attention features output by a third convolutional block, a fourth convolutional block and a fifth convolutional block in the feature extraction model;

[0008] Wherein, the complementary attention features are obtained by combining multi-scale spatial attention and multi-scale channel attention;

[0009] According to the complementary attention features output by the third convolutional block, the complementary attention features output by the fourth convolutional block and the complementary attention features output by the fifth convolutional block, the pedestrian re-identification features of the image to be extracted are extracted.

[0010] In a possible implementation, according to the complementary attention features output by the third convolutional block, the complementary attention features output by the fourth convolutional block and the complementary attention features output by the fifth convolutional block, the pedestrian re-identification features of the image to be extracted are extracted, comprising:

[0011] The complementary attention features output by the third layer convolution block, the complementary attention features output by the fourth layer convolution block, and the complementary attention features output by the fifth layer convolution block are propagated and fused to obtain a first pedestrian re-identification feature;

[0012] The complementary attention features output by the fifth layer convolution block are subjected to a global average pooling operation to obtain a second pedestrian re-identification feature;

[0013] The first pedestrian re-identification feature and the second pedestrian re-identification feature are subjected to a connection operation to obtain a pedestrian re-identification feature of the image to be extracted.

[0014] In a possible implementation, the complementary attention features output by the third layer convolution block, the complementary attention features output by the fourth layer convolution block, and the complementary attention features output by the fifth layer convolution block are propagated and fused to obtain a first pedestrian re-identification feature, including:

[0015] The first intermediate feature is obtained according to the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block;

[0016] The second intermediate feature is obtained according to the complementary attention features output by the third layer convolution block and the first intermediate feature;

[0017] The third intermediate feature is obtained according to the first intermediate feature and the second intermediate feature;

[0018] The fourth intermediate feature is obtained according to the third intermediate feature and the complementary attention features output by the fifth layer convolution block;

[0019] The second intermediate feature, the third intermediate feature, and the fourth intermediate feature are sequentially added and fused to obtain the first pedestrian re-identification feature.

[0020] In a possible implementation, the first intermediate feature is obtained according to the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block, including:

[0021] The size of the complementary attention features output by the fifth layer convolution block is adjusted to the size of the complementary attention features output by the fourth layer convolution block, and then the complementary attention features output by the fourth layer convolution block are connected to obtain connected features;

[0022] The connected features are input into the first attention module to obtain first spatial maximum pooling features, first spatial average pooling features, first channel maximum pooling features, and first channel average pooling features;

[0023] The first intermediate feature is obtained according to the connected features, the first spatial maximum pooling features, the first spatial average pooling features, the first channel maximum pooling features, and the first channel average pooling features.

[0024] In a possible implementation, the first intermediate feature is obtained according to the connected feature, the first spatial maximum pooling feature, the first spatial average pooling feature, the first channel maximum pooling feature and the first channel average pooling feature, and the obtaining includes:

[0025] The first weighted multi-scale spatial attention feature is obtained according to the connected feature, the first spatial maximum pooling feature and the first spatial average pooling feature.

[0026] The first weighted multi-scale channel attention feature is obtained according to the connected feature, the first channel maximum pooling feature and the first channel average pooling feature.

[0027] The first intermediate feature is obtained by adding the connected feature, the first weighted multi-scale spatial attention feature and the first weighted multi-scale channel attention feature.

[0028] In a possible implementation, the first weighted multi-scale spatial attention feature is obtained according to the connected feature, the first spatial maximum pooling feature and the first spatial average pooling feature, and the obtaining includes:

[0029] The first spatial maximum pooling feature and the first spatial average pooling feature are respectively subjected to two-dimensional convolution.

[0030] The first spatial maximum pooling feature after convolution and the first spatial average pooling feature after convolution are connected to obtain initial spatial attention weights.

[0031] The initial spatial attention weights are sequentially subjected to two-dimensional convolution and Sigmoid operation to obtain target spatial attention weights.

[0032] The weighted multi-scale spatial attention feature is obtained by performing bit-by-bit multiplication on the connected feature and the target spatial attention weights.

[0033] In a possible implementation, each convolution block in the feature extraction model includes a second attention module; the image to be extracted is input into the feature extraction model of the backbone network adopting the ResNet-50 model to obtain a complementary attention feature output by a third convolution block in the feature extraction model, and the obtaining includes:

[0034] The image to be extracted is input into the feature extraction model of the backbone network adopting the ResNet-50 model to obtain a complementary attention feature output by a second convolution block, and the second spatial maximum pooling feature, the second spatial average pooling feature, the second channel maximum pooling feature and the second channel average pooling feature output by the second attention module in the third convolution block.

[0035] According to the complementary attention features output by the second layer convolutional block, the second spatial maximum pooling features, and the second spatial average pooling features, second weighted multi-scale spatial attention features are obtained.

[0036] According to the complementary attention features output by the second layer convolutional block, the second channel maximum pooling features, and the second channel average pooling features, second weighted multi-scale channel attention features are obtained.

[0037] The complementary attention features output by the second layer convolutional block, the second weighted multi-scale spatial attention features, and the second weighted multi-scale channel attention features are added to obtain complementary attention features output by a third layer convolutional block.

[0038] In a second aspect, the embodiments of the present application further provide a pedestrian re-identification feature extraction device, which comprises:

[0039] An acquisition module is configured to acquire a to-be-extracted image.

[0040] An input module is configured to input the to-be-extracted image into a feature extraction model of a backbone network adopting a ResNet-50 model to obtain complementary attention features output by a third layer convolutional block, a fourth layer convolutional block, and a fifth layer convolutional block of the feature extraction model, respectively.

[0041] The complementary attention features are obtained by combining multi-scale spatial attention and multi-scale channel attention.

[0042] An extraction module is configured to extract a pedestrian re-identification feature of the to-be-extracted image according to the complementary attention features output by the third layer convolutional block, the complementary attention features output by the fourth layer convolutional block, and the complementary attention features output by the fifth layer convolutional block.

[0043] In a third aspect, the embodiments of the present application further provide an electronic device, which comprises a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus. The processor executes the machine-readable instructions to perform the steps of the pedestrian re-identification feature extraction method according to any one of the first aspect.

[0044] In a fourth aspect, the embodiments of the present application further provide a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, the steps of the pedestrian re-identification feature extraction method according to any one of the first aspect are executed.

[0045] This application provides a method, apparatus, electronic device, and medium for extracting pedestrian re-identification features. The method includes: acquiring an image to be extracted; inputting the image to be extracted into a feature extraction model using a ResNet-50 backbone network to obtain complementary attention features output by the third, fourth, and fifth convolutional blocks of the feature extraction model; wherein the complementary attention features are obtained by combining multi-scale spatial attention and multi-scale channel attention; and extracting pedestrian re-identification features from the image to be extracted based on the complementary attention features output by the third, fourth, and fifth convolutional blocks. This application extracts pedestrian re-identification features from the image to be extracted by combining complementary attention features of multi-scale spatial attention and multi-scale channel attention, thereby improving the efficiency and accuracy of pedestrian re-identification feature extraction. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of a method for extracting pedestrian re-identification features according to an embodiment of this application is shown;

[0048] Figure 2 A flowchart of another method for extracting pedestrian re-identification features provided in an embodiment of this application is shown;

[0049] Figure 3 A flowchart of another method for extracting pedestrian re-identification features provided in an embodiment of this application is shown;

[0050] Figure 4 A schematic diagram of the structure of a pedestrian re-identification feature extraction device provided in an embodiment of this application is shown;

[0051] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0052] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of description and illustration, and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowchart used in the present application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowchart or one or more operations can be removed from the flowchart under the guidance of the content of the present application.

[0053] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0054] In order to enable those skilled in the art to use the content of the present application, the following implementation is given in combination with a specific application scenario "pedestrian re-identification field". For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of the present application. Although the present application is mainly described around the "pedestrian re-identification field", it should be understood that this is only an exemplary embodiment.

[0055] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0056] The following will describe in detail a pedestrian re-identification feature extraction method provided by the embodiments of the present application.

[0057] Referring to Figure 1 As shown in the flowchart of the pedestrian re-identification feature extraction method provided by the embodiments of the present application, the specific execution process of the pedestrian re-identification feature extraction method is as follows:

[0058] S101, acquiring an image to be extracted.

[0059] S102, input the image to be extracted into a feature extraction model of a backbone network adopting a ResNet-50 model, and obtain complementary attention features output by a third layer convolution block, a fourth layer convolution block and a fifth layer convolution block in the feature extraction model respectively.

[0060] S103, extract a pedestrian re-identification feature of the image to be extracted according to the complementary attention features output by the third layer convolution block, the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block.

[0061] The embodiment of the present application provides a pedestrian re-identification feature extraction method, device, electronic equipment and medium, which comprises the following steps: acquiring an image to be extracted; inputting the image to be extracted into a feature extraction model of a backbone network adopting a ResNet-50 model, and obtaining complementary attention features output by a third layer convolution block, a fourth layer convolution block and a fifth layer convolution block in the feature extraction model respectively; wherein the complementary attention features are obtained by combining multi-scale spatial attention and multi-scale channel attention; and extracting a pedestrian re-identification feature of the image to be extracted according to the complementary attention features output by the third layer convolution block, the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block. The pedestrian re-identification feature of the image to be extracted is extracted by the complementary attention features combining the multi-scale spatial attention and the multi-scale channel attention, so that the efficiency and accuracy of extracting the pedestrian re-identification feature are improved.

[0062] The steps of the embodiment of the present application are described below:

[0063] S101, acquiring an image to be extracted.

[0064] In the embodiment of the present application, a monitoring image monitored in a video monitoring scene is acquired and taken as the image to be extracted; the image to be extracted refers to an image whose pedestrian re-identification feature needs to be extracted.

[0065] S102, input the image to be extracted into a feature extraction model of a backbone network adopting a ResNet-50 model, and obtain complementary attention features output by a third layer convolution block, a fourth layer convolution block and a fifth layer convolution block in the feature extraction model respectively.

[0066] The complementary attention features are obtained by combining multi-scale spatial attention and multi-scale channel attention.

[0067] Here, the ResNet-50 model is divided into five convolutional blocks, so the feature extraction model using the backbone network of the ResNet-50 model is also divided into five convolutional blocks, and a second attention module is introduced in each convolutional block in the feature extraction model; the second attention module includes a second spatial attention module introducing a second multi-scale spatial attention, and a second channel attention module introducing a second multi-scale channel attention. The feature extraction model is obtained by training the sample image and the corresponding complementary attention feature; the loss value of the feature extraction model is determined by the cross-entropy loss, the triplet loss and the center loss of the feature extraction model to optimize the optimization parameters of the feature extraction model:

[0068] The loss value of the feature extraction model is determined as follows: L total =L ID +L Tri +βL C ;

[0069] Wherein, L total is the loss value of the feature extraction model, L ID is the cross-entropy loss of the feature extraction model, L Tri is the triplet loss of the feature extraction model, β is the weight balance value of the center loss of the feature extraction model, L C is the center loss of the feature extraction model.

[0070] Specifically, the complementary attention feature output by the third convolutional block in the feature extraction model is obtained by the following steps:

[0071] I, input the image to be extracted into the feature extraction model using the backbone network of the ResNet-50 model to obtain the complementary attention feature output by the second convolutional block, and the second spatial maximum pooling feature, the second spatial average pooling feature, the second channel maximum pooling feature and the second channel average pooling feature output by the second attention module in the third convolutional block.

[0072] In the embodiment of the application, the image to be extracted is input into the feature extraction model using the backbone network of the ResNet-50 model, to obtain the complementary attention feature output by the second convolutional block, the second spatial maximum pooling feature and the second spatial average pooling feature output by the second spatial attention module of the second attention module in the third convolutional block, and the second channel maximum pooling feature and the second channel average pooling feature output by the second channel attention module of the second attention module in the third convolutional block.

[0073] II, according to the complementary attention feature output by the second convolutional block, the second spatial maximum pooling feature and the second spatial average pooling feature, the second weighted multi-scale spatial attention feature is obtained.

[0074] In the embodiments of the present application, the second spatial maximum pooling features and the second spatial average pooling features output by the third layer convolutional block are respectively subjected to two-dimensional convolution to obtain spatial maximum pooling weights and spatial average pooling weights; the spatial maximum pooling weights and the spatial average pooling weights are connected to obtain spatial attention weights of the third layer convolutional block; the value after the spatial attention weights of the third layer convolutional block are subjected to two-dimensional convolution and Sigmoid operation in sequence is multiplied by the complementary attention features output by the second layer convolutional block bit by bit to obtain the second weighted multi-scale spatial attention features.

[0075] III. The second weighted multi-scale channel attention features are obtained according to the complementary attention features output by the second layer convolutional block, the second channel maximum pooling features and the second channel average pooling features.

[0076] In the embodiments of the present application, the second channel maximum pooling features and the second channel average pooling features output by the third layer convolutional block are respectively subjected to one-dimensional convolution to obtain channel maximum pooling weights and channel average pooling weights; the channel maximum pooling weights and the channel average pooling weights are connected to obtain channel attention weights of the third layer convolutional block; the value after the channel attention weights of the third layer convolutional block are subjected to one-dimensional convolution and RELU operation in sequence is multiplied by the complementary attention features output by the second layer convolutional block bit by bit to obtain the second weighted multi-scale channel attention features.

[0077] IV. The complementary attention features output by the second layer convolutional block, the second weighted multi-scale spatial attention features and the second weighted multi-scale channel attention features are added to obtain the complementary attention features output by the third layer convolutional block.

[0078] Specifically, the complementary attention features output by the fourth layer convolutional block in the feature extraction model are obtained by the following steps.

[0079] In the embodiment of the present application, the image to be extracted is input into the feature extraction model of the backbone network adopting the ResNet-50 model to obtain the complementary attention features output by the third layer convolution block, and the second spatial maximum pooling features, the second spatial average pooling features, the second channel maximum pooling features and the second channel average pooling features output by the second attention module in the fourth layer convolution block; the second weighted multi-scale spatial attention features of the fourth layer convolution block are obtained according to the complementary attention features output by the third layer convolution block, the second spatial maximum pooling features output by the fourth layer convolution block and the second spatial average pooling features output by the fourth layer convolution block; the second weighted multi-scale channel attention features of the fourth layer convolution block are obtained according to the complementary attention features output by the third layer convolution block, the second channel maximum pooling features of the fourth layer convolution block and the second channel average pooling features of the fourth layer convolution block; and the complementary attention features output by the fourth layer convolution block are obtained by adding the complementary attention features output by the third layer convolution block, the second weighted multi-scale spatial attention features of the fourth layer convolution block and the second weighted multi-scale channel attention features of the fourth layer convolution block.

[0080] Here, the specific implementation is the same as that of obtaining the complementary attention features output by the third layer convolution block in the feature extraction model, and will not be repeated here.

[0081] Specifically, the complementary attention features output by the fifth layer convolution block in the feature extraction model are obtained by the following steps.

[0082] In the embodiment of the present application, the image to be extracted is input into the feature extraction model of the backbone network adopting the ResNet-50 model to obtain the complementary attention features output by the fourth layer convolution block, and the second spatial maximum pooling features, the second spatial average pooling features, the second channel maximum pooling features and the second channel average pooling features output by the second attention module in the fifth layer convolution block; the second weighted multi-scale spatial attention features of the fifth layer convolution block are obtained according to the complementary attention features output by the fourth layer convolution block, the second spatial maximum pooling features output by the fifth layer convolution block and the second spatial average pooling features output by the fifth layer convolution block; the second weighted multi-scale channel attention features of the fifth layer convolution block are obtained according to the complementary attention features output by the fourth layer convolution block, the second channel maximum pooling features of the fifth layer convolution block and the second channel average pooling features of the fifth layer convolution block; and the complementary attention features output by the fifth layer convolution block are obtained by adding the complementary attention features output by the fourth layer convolution block, the second weighted multi-scale spatial attention features of the fifth layer convolution block and the second weighted multi-scale channel attention features of the fifth layer convolution block.

[0083] Here, the specific implementation is the same as that of obtaining the complementary attention features output by the third layer convolution block in the feature extraction model, and will not be repeated here.

[0084] S103, extracting the pedestrian re-identification feature of the image to be extracted according to the complementary attention features output by the third layer convolution block, the complementary attention features output by the fourth layer convolution block, and the complementary attention features output by the fifth layer convolution block.

[0085] Further, the target pedestrian identity corresponding to the pedestrian re-identification feature of the image to be extracted is determined by using the similarity matrix.

[0086] Referring to Figure 2 Fig. 4 is a flowchart of another method for extracting a pedestrian re-identification feature provided by an embodiment of the present application. The specific execution process of the method for extracting a pedestrian re-identification feature is as follows:

[0087] S201, propagating and fusing the complementary attention features output by the third layer convolution block, the complementary attention features output by the fourth layer convolution block, and the complementary attention features output by the fifth layer convolution block to obtain a first pedestrian re-identification feature.

[0088] Specifically, a first intermediate feature is obtained according to the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block; a second intermediate feature is obtained according to the complementary attention features output by the third layer convolution block and the first intermediate feature; a third intermediate feature is obtained according to the first intermediate feature and the second intermediate feature; a fourth intermediate feature is obtained according to the third intermediate feature and the complementary attention features output by the fifth layer convolution block; and the second intermediate feature, the third intermediate feature, and the fourth intermediate feature are continuously added and fused to obtain the first pedestrian re-identification feature.

[0089] Here, the complementary attention features of multiple scales are propagated in turn by the bottom-up and top-down manner, and then the second intermediate feature, the third intermediate feature, and the fourth intermediate feature obtained after the propagation are continuously added and fused to obtain the first pedestrian re-identification feature. The first pedestrian re-identification feature obtained by the method of the present application contains fine-grained visual clues and coarse-grained visual clues.

[0090] S202, performing a global average pooling operation on the complementary attention features output by the fifth layer convolution block to obtain a second pedestrian re-identification feature.

[0091] S203, performing a connection operation on the first pedestrian re-identification feature and the second pedestrian re-identification feature to obtain the pedestrian re-identification feature of the image to be extracted.

[0092] Referring to Figure 3 Fig. 4 is a flowchart of another method for extracting a pedestrian re-identification feature provided by an embodiment of the present application. The specific execution process of the method for extracting a pedestrian re-identification feature is as follows:

[0093] S301、According to the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block, a first intermediate feature is obtained.

[0094] Here, according to the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block, the first intermediate feature is obtained by the following steps:

[0095] I. After adjusting the size of the complementary attention features output by the fifth layer convolution block to the size of the complementary attention features output by the fourth layer convolution block, the complementary attention features output by the fourth layer convolution block are connected to obtain connected features.

[0096] II. The connected features are input into the first attention module to obtain the first spatial maximum pooling feature, the first spatial average pooling feature, the first channel maximum pooling feature and the first channel average pooling feature.

[0097] In the embodiments of the present application, the first attention module includes a first spatial attention module and a first channel attention module. The connected features are input into the first attention module to obtain the first spatial maximum pooling feature and the first spatial average pooling feature output by the first spatial attention module, and the first channel maximum pooling feature and the first channel average pooling feature output by the first channel attention module.

[0098] III. According to the connected features, the first spatial maximum pooling feature, the first spatial average pooling feature, the first channel maximum pooling feature and the first channel average pooling feature, a first intermediate feature is obtained.

[0099] Here, according to the connected features, the first spatial maximum pooling feature, the first spatial average pooling feature, the first channel maximum pooling feature and the first channel average pooling feature, the first intermediate feature is obtained by the following steps:

[0100] Step one, according to the connected features, the first spatial maximum pooling feature and the first spatial average pooling feature, a first weighted multi-scale spatial attention feature is obtained.

[0101] Specifically, the first spatial maximum pooling feature and the first spatial average pooling feature are respectively subjected to two-dimensional convolution.

[0102] Specifically, the first spatial maximum pooling feature after convolution and the first spatial average pooling feature after convolution are connected to obtain an initial spatial attention weight.

[0103] Specifically, the initial spatial attention weight is sequentially subjected to two-dimensional convolution and Sigmoid operation to obtain a target spatial attention weight.

[0104] Specifically, the connected feature is multiplied by the target spatial attention weight bit by bit to obtain a weighted multi-scale spatial attention feature.

[0105] Step two, a first weighted multi-scale channel attention feature is obtained according to the connected feature, the first channel maximum pooling feature and the first channel average pooling feature.

[0106] Specifically, the first channel maximum pooling feature and the first channel average pooling feature are respectively subjected to one-dimensional convolution.

[0107] Specifically, the first channel maximum pooling feature after convolution and the first channel average pooling feature after convolution are connected to obtain an initial channel attention weight.

[0108] Specifically, the initial channel attention weight is sequentially subjected to one-dimensional convolution and RELU operation to obtain a target channel attention weight.

[0109] Specifically, the connected feature is multiplied by the target channel attention weight bit by bit to obtain a weighted multi-scale channel attention feature.

[0110] Step three, the connected feature, the first weighted multi-scale spatial attention feature and the first weighted multi-scale channel attention feature are added to obtain a first intermediate feature.

[0111] S302, a second intermediate feature is obtained according to the complementary attention feature output by the third layer convolution block and the first intermediate feature.

[0112] Specifically, the size of the first intermediate feature is adjusted to the size of the complementary attention feature output by the third layer convolution block, and then the first intermediate feature is connected with the complementary attention feature output by the third layer convolution block to obtain a connected feature; the connected feature is input into the first attention module to obtain a first spatial maximum pooling feature, a first spatial average pooling feature, a first channel maximum pooling feature and a first channel average pooling feature; and the second intermediate feature is obtained according to the connected feature, the first spatial maximum pooling feature, the first spatial average pooling feature, the first channel maximum pooling feature and the first channel average pooling feature.

[0113] Here, the specific implementation process is the same as that of obtaining the first intermediate feature, which will not be repeated here.

[0114] S303, a third intermediate feature is obtained according to the first intermediate feature and the second intermediate feature.

[0115] Specifically, the value of the second intermediate feature after two-dimensional convolution is connected with the first intermediate feature to obtain a third intermediate feature.

[0116] S304, obtaining a fourth intermediate feature according to the complementary attention feature output by the third intermediate feature and the fifth layer convolution block.

[0117] Specifically, the value of the third intermediate feature after two-dimensional convolution is connected with the complementary attention feature output by the fifth layer convolution block to obtain a spliced feature, and the connected feature is input into the first attention module to obtain the fourth intermediate feature.

[0118] S305, sequentially adding and fusing the second intermediate feature, the third intermediate feature and the fourth intermediate feature to obtain a first pedestrian re-identification feature.

[0119] Based on the same inventive concept, the embodiments of the present application also provide a pedestrian re-identification feature extraction device corresponding to the pedestrian re-identification feature extraction method. Since the principle of the device in the embodiments of the present application solves the problem, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0120] Referring to Figure 4 Fig. 1 shows a schematic diagram of a pedestrian re-identification feature extraction device provided by the embodiments of the present application. The pedestrian re-identification feature extraction device comprises:

[0121] The acquisition module 401 is configured to acquire a to-be-extracted image.

[0122] The input module 402 is configured to input the to-be-extracted image into a feature extraction model of a backbone network adopting a ResNet-50 model to obtain complementary attention features output by a third layer convolution block, a fourth layer convolution block and a fifth layer convolution block of the feature extraction model respectively.

[0123] The complementary attention features are obtained by combining multi-scale spatial attention and multi-scale channel attention.

[0124] The extraction module 403 is configured to extract a pedestrian re-identification feature of the to-be-extracted image according to the complementary attention features output by the third layer convolution block, the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block.

[0125] In a possible implementation, the extraction module 403 is specifically configured to propagate and fuse the complementary attention features output by the third layer convolution block, the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block to obtain a first pedestrian re-identification feature; perform a global average pooling operation on the complementary attention features output by the fifth layer convolution block to obtain a second pedestrian re-identification feature; and perform a connection operation on the first pedestrian re-identification feature and the second pedestrian re-identification feature to obtain the pedestrian re-identification feature of the to-be-extracted image.

[0126] In a possible implementation, the extraction module 403 is specifically configured to obtain a first intermediate feature according to the complementary attention features output by the fourth layer convolution block and the complementary attention features output by the fifth layer convolution block; obtain a second intermediate feature according to the complementary attention features output by the third layer convolution block and the first intermediate feature; obtain a third intermediate feature according to the first intermediate feature and the second intermediate feature; obtain a fourth intermediate feature according to the third intermediate feature and the complementary attention features output by the fifth layer convolution block; and perform continuous addition fusion on the second intermediate feature, the third intermediate feature and the fourth intermediate feature to obtain the first pedestrian re-identification feature.

[0127] In a possible implementation, the extraction module 403 is specifically configured to adjust the size of the complementary attention features output by the fifth layer convolution block to the size of the complementary attention features output by the fourth layer convolution block, and perform a connection operation on the complementary attention features output by the fourth layer convolution block to obtain connected features; input the connected features into the first attention module to obtain a first spatial maximum pooling feature, a first spatial average pooling feature, a first channel maximum pooling feature and a first channel average pooling feature; and obtain a first intermediate feature according to the connected features, the first spatial maximum pooling feature, the first spatial average pooling feature, the first channel maximum pooling feature and the first channel average pooling feature.

[0128] In a possible implementation, the extraction module 403 is specifically configured to obtain a first weighted multi-scale spatial attention feature according to the connected features, the first spatial maximum pooling feature and the first spatial average pooling feature; obtain a first weighted multi-scale channel attention feature according to the connected features, the first channel maximum pooling feature and the first channel average pooling feature; and perform addition on the connected features, the first weighted multi-scale spatial attention feature and the first weighted multi-scale channel attention feature to obtain the first intermediate feature.

[0129] In a possible implementation, the extraction module 403 is specifically configured to perform two-dimensional convolution on the first spatial maximum pooling feature and the first spatial average pooling feature respectively; connect the convolved first spatial maximum pooling feature and the convolved first spatial average pooling feature to obtain an initial spatial attention weight; and perform two-dimensional convolution and Sigmoid operation on the initial spatial attention weight in sequence to obtain a target spatial attention weight; and perform bitwise multiplication on the connected features and the target spatial attention weight to obtain a weighted multi-scale spatial attention feature.

[0130] In a possible implementation, each of the convolutional blocks in the feature extraction model comprises a second attention module; the input module 402 is specifically configured to input the image to be extracted into the feature extraction model adopting the backbone network of the ResNet-50 model to obtain the complementary attention features output by the second convolutional block, and the second spatial maximum pooling features, the second spatial average pooling features, the second channel maximum pooling features and the second channel average pooling features output by the second attention module in the third convolutional block; the second weighted multi-scale spatial attention features are obtained according to the complementary attention features output by the second convolutional block and the second spatial maximum pooling features and the second spatial average pooling features; the second weighted multi-scale channel attention features are obtained according to the complementary attention features output by the second convolutional block, the second channel maximum pooling features and the second channel average pooling features; and the complementary attention features output by the second convolutional block, the second weighted multi-scale spatial attention features and the second weighted multi-scale channel attention features are added to obtain the complementary attention features output by the third convolutional block.

[0131] The embodiment of the present application provides a pedestrian re-identification feature extraction device, which can extract the pedestrian re-identification features of the image to be extracted, and improves the efficiency and accuracy of extracting the pedestrian re-identification features.

[0132] As shown in Figure 5 The embodiment of the present application provides an electronic device 500, which comprises a processor 501, a memory 502 and a bus. The memory 502 stores machine readable instructions executable by the processor 501. When the electronic device is running, the processor 501 and the memory 502 communicate through the bus. The processor 501 executes the machine readable instructions to perform the steps of the pedestrian re-identification feature extraction method.

[0133] Specifically, the memory 502 and the processor 501 can be general memory and processor, which are not specifically limited here. When the processor 501 runs the computer program stored in the memory 502, the pedestrian re-identification feature extraction method can be executed.

[0134] Corresponding to the pedestrian re-identification feature extraction method, the embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run by the processor, the steps of the pedestrian re-identification feature extraction method are executed.

[0135] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system and the device described above can refer to the corresponding process in the method embodiment, and will not be repeated in the present application. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other means. The above-described device embodiments are only schematic, for example, the division of the modules is only a logical function division, and the actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual elements can be indirect coupling or communication connection through some communication interface, device or module, which can be electrical, mechanical or other forms.

[0136] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical units, i.e. can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0137] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0138] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the information processing method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk and various program code storage media.

[0139] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for extracting pedestrian re-identification features, characterized in that, The method for extracting pedestrian re-identification features includes: Obtain the image to be extracted; The image to be extracted is input into the feature extraction model of the backbone network using the ResNet-50 model to obtain the complementary attention features output by the third, fourth and fifth convolutional blocks in the feature extraction model. The complementary attention feature is obtained by combining multi-scale spatial attention and multi-scale channel attention. Based on the complementary attention features output by the third, fourth, and fifth convolutional blocks, the pedestrian re-identification features of the image to be extracted are extracted. The step of extracting pedestrian re-identification features from the image to be extracted based on the complementary attention features output by the third, fourth, and fifth convolutional blocks includes: propagating and fusing the complementary attention features output by the third, fourth, and fifth convolutional blocks to obtain a first pedestrian re-identification feature; performing global average pooling on the complementary attention features output by the fifth convolutional block to obtain a second pedestrian re-identification feature; and concatenating the first and second pedestrian re-identification features to obtain the pedestrian re-identification features of the image to be extracted. Each convolutional block in the feature extraction model includes a second attention module. The image to be extracted is input into the feature extraction model of the backbone network using the ResNet-50 model to obtain complementary attention features output by the third convolutional block in the feature extraction model. This includes: inputting the image to be extracted into the feature extraction model of the backbone network using the ResNet-50 model to obtain complementary attention features output by the second convolutional block, and second spatial max-pooling features, second spatial average pooling features, second channel max-pooling features, and second channel average pooling features output by the second attention module in the third convolutional block; obtaining second weighted multi-scale spatial attention features based on the complementary attention features, second spatial max-pooling features, and second spatial average pooling features output by the second convolutional block; obtaining second weighted multi-scale channel attention features based on the complementary attention features, second channel max-pooling features, and second channel average pooling features output by the second convolutional block; and adding the complementary attention features, second weighted multi-scale spatial attention features, and second weighted multi-scale channel attention features output by the second convolutional block to obtain the complementary attention features output by the third convolutional block.

2. The method for extracting pedestrian re-identification features according to claim 1, characterized in that, The complementary attention features output from the third, fourth, and fifth convolutional blocks are propagated and fused to obtain the first pedestrian re-identification feature, including: The first intermediate feature is obtained based on the complementary attention features output by the fourth convolutional block and the complementary attention features output by the fifth convolutional block; The second intermediate feature is obtained based on the complementary attention features output by the third convolutional block and the first intermediate feature; Based on the first intermediate feature and the second intermediate feature, a third intermediate feature is obtained; The fourth intermediate feature is obtained based on the third intermediate feature and the complementary attention feature output by the fifth convolutional block; The second intermediate feature, the third intermediate feature, and the fourth intermediate feature are continuously added together and fused to obtain the first pedestrian re-identification feature.

3. The method for extracting pedestrian re-identification features according to claim 2, characterized in that, The step of obtaining the first intermediate feature based on the complementary attention features output by the fourth convolutional block and the complementary attention features output by the fifth convolutional block includes: After adjusting the size of the complementary attention feature output by the fifth convolutional block to the size of the complementary attention feature output by the fourth convolutional block, a concatenation operation is performed with the complementary attention feature output by the fourth convolutional block to obtain the concatenated feature. The concatenated features are input into the first attention module to obtain the first spatial max pooling feature, the first spatial average pooling feature, the first channel max pooling feature, and the first channel average pooling feature. The first intermediate feature is obtained based on the connected features, the first spatial max pooling feature, the first spatial average pooling feature, the first channel max pooling feature, and the first channel average pooling feature.

4. The method for extracting pedestrian re-identification features according to claim 3, characterized in that, The process of obtaining the first intermediate feature based on the concatenated features, the first spatial max pooling feature, the first spatial average pooling feature, the first channel max pooling feature, and the first channel average pooling feature includes: Based on the concatenated features, the first spatial max pooling feature, and the first spatial average pooling feature, the first weighted multi-scale spatial attention feature is obtained. Based on the connected features, the first channel max pooling feature, and the first channel average pooling feature, the first weighted multi-scale channel attention feature is obtained. The first intermediate feature is obtained by adding the concatenated feature, the first weighted multi-scale spatial attention feature, and the first weighted multi-scale channel attention feature.

5. The method for extracting pedestrian re-identification features according to claim 4, characterized in that, The step of obtaining the first weighted multi-scale spatial attention feature based on the concatenated features, the first spatial max-pooling feature, and the first spatial average-pooling feature includes: Two-dimensional convolutions are performed on the first spatial max pooling feature and the first spatial average pooling feature, respectively; The first spatial max pooling feature after convolution and the first spatial average pooling feature after convolution are concatenated to obtain the initial spatial attention weights. The initial spatial attention weights are sequentially subjected to two-dimensional convolution and Sigmoid operations to obtain the target spatial attention weights; The weighted multi-scale spatial attention features are obtained by multiplying the connected features with the target spatial attention weights bitwise.

6. A device for extracting pedestrian re-identification features, characterized in that, The pedestrian re-identification feature extraction device includes: The acquisition module is used to acquire the image to be extracted; The input module is used to input the image to be extracted into the feature extraction model of the backbone network using the ResNet-50 model, and obtain the complementary attention features output by the third, fourth and fifth convolutional blocks in the feature extraction model respectively; The complementary attention feature is obtained by combining multi-scale spatial attention and multi-scale channel attention. The extraction module is used to extract pedestrian re-identification features from the image to be extracted based on the complementary attention features output by the third convolutional block, the fourth convolutional block, and the fifth convolutional block. The extraction module is specifically used to propagate and fuse the complementary attention features output by the third, fourth, and fifth convolutional blocks to obtain a first pedestrian re-identification feature; to perform global average pooling on the complementary attention features output by the fifth convolutional block to obtain a second pedestrian re-identification feature; and to concatenate the first and second pedestrian re-identification features to obtain the pedestrian re-identification feature of the image to be extracted. Each convolutional block in the feature extraction model includes a second attention module; an input module, specifically used to input the image to be extracted into the feature extraction model of the backbone network using the ResNet-50 model to obtain the complementary attention features output by the second convolutional block, and the second spatial max pooling features, second spatial average pooling features, second channel max pooling features, and second channel average pooling features output by the second attention module in the third convolutional block; based on the complementary attention features, second spatial max pooling features, and second spatial average pooling features output by the second convolutional block, a second weighted multi-scale spatial attention feature is obtained; based on the complementary attention features, second channel max pooling features, and second channel average pooling features output by the second convolutional block, a second weighted multi-scale channel attention feature is obtained; the complementary attention features, second weighted multi-scale spatial attention features, and second weighted multi-scale channel attention features output by the second convolutional block are added together to obtain the complementary attention features output by the third convolutional block.

7. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the pedestrian re-identification feature extraction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the pedestrian re-identification feature extraction method as described in any one of claims 1 to 5.

Images