Pedestrian re-identification method and system

Abstract

The application provides a pedestrian re-identification method and system, which comprises the following steps: determining a to-be-identified pedestrian image; inputting the to-be-identified pedestrian image into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model; splicing and fusing the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the to-be-identified pedestrian image; and matching the to-be-identified pedestrian image with pedestrian images in a preset image library based on the complete feature expression to determine a pedestrian re-identification result of the to-be-identified pedestrian image; wherein the local feature vector comprises a plurality of local feature vectors of a feature map determined by the feature extraction model in the vertical direction and the depth direction. The method extracts multi-dimensional features such as the global feature vector and the local feature vector, and uses macro global features and micro local features of the to-be-identified pedestrian image to perform pedestrian re-identification, controls the whole and the details, and improves the accuracy of pedestrian re-identification.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a pedestrian re-identification method and system. Background Art

[0002] As urban population density and travel frequency increase, surveillance systems are becoming increasingly common. However, with the proliferation of cameras and the growth of data volumes, there is an urgent need for surveillance systems to operate intelligently, enabling automated pedestrian detection, identification, and tracking. Based on these requirements, person re-identification (ReID) technology has been rapidly developed.

[0003] Person re-identification, also known as pedestrian re-ID, utilizes computer vision techniques to determine whether a specific pedestrian exists in an image or video sequence. Person re-ID is widely considered a subproblem of image retrieval: given a monitored pedestrian image, it is necessary to retrieve images of that pedestrian across multiple devices. Person re-ID, as an important complement to facial recognition technology, can fully utilize cameras in cities to perform identity matching. Based on this, pedestrian movement trajectories can be derived. This technology is of great significance in security inspections, missing persons, and smart business applications, and is widely applicable to intelligent video surveillance, smart security, and other fields.

[0004] At present, due to the differences between different camera devices, and the fact that pedestrians have both rigid and flexible characteristics, their appearance is easily affected by lighting, clothing, scale, occlusion, posture and viewing angle, the re-identification accuracy of existing pedestrian re-identification models is not high. Summary of the Invention

[0005] The present invention provides a pedestrian re-identification method and system, which are used to solve the defect of low re-identification accuracy of pedestrian re-identification methods in the prior art and realize pedestrian re-identification with a higher accuracy.

[0006] The present invention provides a pedestrian re-identification method, comprising:

[0007] Determine a pedestrian image to be identified;

[0008] Inputting the pedestrian image to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model;

[0009] Splicing and fusing the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the image of the pedestrian to be identified;

[0010] Based on the complete feature expression, the pedestrian image to be identified is matched with pedestrian images in a preset image library to determine a pedestrian re-identification result of the pedestrian image to be identified;

[0011] Among them, the local feature vector includes multiple local feature vectors in the vertical and depth directions of the feature map determined by the feature extraction model; the feature extraction model includes a repositioning layer, which is used to perform attention operations on the feature map at the spatial and depth levels; the feature extraction model is trained in two stages based on pedestrian images carrying pedestrian labels; the pedestrian images include pedestrian images collected by different devices and containing the same pedestrian.

[0012] According to a pedestrian re-identification method provided by the present invention, the pedestrian image to be identified is input into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model, specifically comprising:

[0013] Inputting the pedestrian image to be identified into the backbone network of the feature extraction model to obtain a feature map extracted by the backbone network;

[0014] Inputting the feature map into a global residual convolution layer and three local residual convolution layers of the feature extraction model to obtain a global feature map and three separable feature maps respectively;

[0015] Inputting the global feature map and each separable feature map into the relocation layer of the feature extraction model to obtain a global relocation feature map corresponding to the global feature map and a separable relocation feature map corresponding to each separable feature map;

[0016] The global relocation feature map and each separable relocation feature map are input into four types of pooling convolution layers of different segmentation granularities of the feature extraction model to obtain global feature vectors and local feature vectors output by each type of pooling convolution layer.

[0017] According to a person re-identification method provided by the present invention, the local residual convolution layer includes a first local residual convolution layer, a second local residual convolution layer, and a third local residual convolution layer; accordingly, the feature map is input into a global residual convolution layer and three local residual convolution layers of the feature extraction model to obtain a global feature map and three separable feature maps, respectively, specifically including:

[0018] Inputting the feature map into the global residual convolution layer to obtain a global feature map output by the global residual convolution layer;

[0019] Inputting the feature map into the first local residual convolution layer, and performing feature extraction on the feature map by the first local residual convolution layer to obtain a first separable feature map;

[0020] Inputting the feature map into the second local residual convolution layer, and performing feature extraction on the feature map by the second local residual convolution layer to obtain a second separable feature map;

[0021] The feature map is input into the third local residual convolution layer, and the third local residual convolution layer performs feature extraction on the feature map to obtain a third separable feature map.

[0022] According to a person re-identification method provided by the present invention, the global feature map and each segmentable feature map are input into the relocation layer of the feature extraction model to obtain a global relocation feature map corresponding to the global feature map and a segmentable relocation feature map corresponding to each segmentable feature map, respectively, which specifically includes:

[0023] Perform convolution operations on the global feature map, the first separable feature map, and the second separable feature map to generate corresponding feature matrices, and normalize the feature matrices to obtain spatial attention matrices corresponding to the feature matrices;

[0024] Performing maximum pooling on the third separable feature map to generate a pooling vector, performing a linear transformation on the pooling vector, and normalizing the linearly transformed pooling vector to obtain a deep attention vector corresponding to the pooling vector;

[0025] Multiplying the spatial attention matrix by the feature matrices corresponding to each dimension of the corresponding global feature map, the first separable feature map, and the second separable feature map respectively to obtain the global relocation feature map, the first separable relocation feature map, and the second separable relocation feature map;

[0026] The depth attention vector is multiplied by the feature vector corresponding to each pixel in the third separable feature map to obtain a third separable relocation feature map.

[0027] According to a person re-identification method provided by the present invention, the global relocation feature map and each separable relocation feature map are input into four types of pooling convolution layers of the feature extraction model to obtain global feature vectors and local feature vectors output by each type of pooling convolution layer, specifically including:

[0028] Inputting the global relocation feature map and each separable relocation feature map into the pooling layer of the first type of pooling convolution layer to obtain four global feature vectors;

[0029] Divide the first separable relocatable feature map into two parts along the vertical direction of the image, input the two parts into the pooling layer of the second type of pooling convolution layer, perform pooling operation, and obtain two local feature vectors;

[0030] Divide the second separable relocatable feature map into three parts along the vertical direction of the image, input the three parts into the pooling layer of the third type of pooling convolution layer respectively, perform pooling operation, and obtain three local feature vectors;

[0031] Dividing the third separable relocatable feature map into four parts along the depth direction of the image, inputting the four parts into the pooling layer of the fourth type of pooling convolution layer respectively, performing a pooling operation, and obtaining four local feature vectors;

[0032] Each global feature vector and each local feature vector are respectively input into the convolution layer of each pooling convolution layer, and the dimension is transformed to obtain the global feature vector and local feature vector of the same dimension.

[0033] According to a pedestrian re-identification method provided by the present invention, the feature extraction model is trained based on the following method:

[0034] Inputting the pedestrian image into a feature extraction model to be trained, and training the basic structure of the feature extraction model to be trained except for the relocalization layer based on the value of a preset loss function and an adaptive moment estimation algorithm; the basic structure includes a backbone network, a global residual convolution layer, a local residual convolution layer, a pooling convolution layer, and a fully connected layer;

[0035] Based on the trained basic structure, the relocation layer in the feature extraction model to be trained is trained to obtain the feature extraction model.

[0036] According to a pedestrian re-identification method provided by the present invention, the preset loss function includes a triple loss function and a cross entropy loss function;

[0037] Accordingly, the pedestrian image is input into the feature extraction model to be trained, and the basic structure of the feature extraction model to be trained except the relocalization layer is trained based on the value of the preset loss function and the adaptive moment estimation algorithm, specifically including:

[0038] Inputting the pedestrian image into the backbone network, the global residual convolution layer, the local residual convolution layer, and the pooling convolution layer in sequence to obtain four global feature vectors and nine local feature vectors, and inputting each global feature vector and each local feature vector into the fully connected layer, and normalizing the output of the fully connected layer;

[0039] Calculating the value of the triplet loss function based on each global eigenvector, and calculating the value of the cross entropy loss function based on the result of the normalization process;

[0040] The value of the triplet loss function and the value of the cross entropy loss function are summed to obtain the value of the preset loss function, and the basic structure is trained based on the value of the preset loss function and the adaptive moment estimation algorithm.

[0041] The present invention also provides a pedestrian re-identification system, comprising:

[0042] A pedestrian image determination module, used to determine the pedestrian image to be identified;

[0043] A feature vector determination module is used to input the pedestrian image to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model;

[0044] A feature splicing and fusion module is used to splice and fuse the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the image of the pedestrian to be identified;

[0045] a pedestrian re-identification module, configured to match the image of the pedestrian to be identified with pedestrian images in a preset image library based on the complete feature expression, and determine a pedestrian re-identification result of the image of the pedestrian to be identified;

[0046] Among them, the local feature vector includes multiple local feature vectors in the vertical and depth directions of the feature map determined by the feature extraction model; the feature extraction model includes a repositioning layer, which is used to perform attention operations on the feature map at the spatial and depth levels; the feature extraction model is trained in two stages based on pedestrian images carrying pedestrian labels; the pedestrian images include pedestrian images collected by different devices and containing the same pedestrian.

[0047] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-described pedestrian re-identification methods are implemented.

[0048] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described pedestrian re-identification methods.

[0049] The pedestrian re-identification method and system provided by the present invention obtains the global feature vector and local feature vector of the pedestrian image to be identified through a feature extraction model, and then splices and fuses the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the pedestrian image to be identified; and based on the complete feature expression, matches the pedestrian image to be identified with pedestrian images in a preset image library to determine the pedestrian re-identification result of the pedestrian image to be identified. Among them, the local feature vector includes multiple local feature vectors in the vertical direction and depth direction of the feature map determined by the feature extraction model. This method extracts multi-dimensional features such as global feature vectors and local feature vectors, and uses the macro-global features and micro-local features of the pedestrian image to be identified to perform pedestrian re-identification, controlling the overall and details, and improving the accuracy of pedestrian re-identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 1 is a flow chart of a pedestrian re-identification method provided by an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of the structure of the feature extraction model provided by the embodiment of the present invention;

[0053] Figure 3 Schematic diagram of the training process of the basic structure of the feature extraction model in an embodiment of the present invention;

[0054] Figure 4 1 is a schematic diagram of the training process of the relocation layer of the feature extraction model in an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the structure of the pedestrian re-identification system provided by an embodiment of the present invention.

[0056] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0057] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0058] Due to the differences between different camera devices, and the fact that pedestrians have both rigidity and flexibility, their appearance is easily affected by lighting, clothing, scale, occlusion, posture, and viewing angle, resulting in low recognition accuracy of existing pedestrian re-identification models. Therefore, the present invention provides a pedestrian re-identification method.

[0059] Figure 1 FIG. 1 is a flow chart of a pedestrian re-identification method according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0060] S1, determine the pedestrian image to be identified;

[0061] S2, inputting the pedestrian image to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model;

[0062] S3, concatenating and fusing the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the image of the pedestrian to be identified;

[0063] S4, matching the image of the pedestrian to be identified with pedestrian images in a preset image library based on the complete feature expression, and determining a pedestrian re-identification result of the image of the pedestrian to be identified;

[0064] Among them, the local feature vector includes multiple local feature vectors in the vertical and depth directions of the feature map determined by the feature extraction model; the feature extraction model includes a repositioning layer, which is used to perform attention operations on the feature map at the spatial and depth levels; the feature extraction model is trained in two stages based on pedestrian images carrying pedestrian labels; the pedestrian images include pedestrian images collected by different devices and containing the same pedestrian.

[0065] Specifically, the pedestrian re-identification method provided in the embodiment of the present invention is executed by a server, which can be a local server or a cloud server. The local server can specifically be a computer, a tablet computer, a smart phone, etc., and this is not specifically limited in the embodiment of the present invention.

[0066] First, execute step S1. Determine a pedestrian image to be identified. The pedestrian image to be identified can be any pedestrian image captured by any camera, or a pedestrian image from an open-source dataset. The open-source dataset may be the Market 1501 dataset, the DukeMTMC dataset, or the MSMT17 dataset. This embodiment of the present invention does not specifically limit the source or resolution of the pedestrian image to be identified.

[0067] Then, step S2 is executed. The image of the pedestrian to be identified is input into the feature extraction model, and the global feature vector and local feature vector of the image of the pedestrian to be identified are obtained as output by the feature extraction model. The function of the feature extraction model is to extract features from the image of the pedestrian to be identified. Feature extraction refers to the process of extracting characteristic information from the image of the pedestrian to be identified. The feature extraction model can be a convolutional neural network (CNN) model or other feature extraction model capable of extracting global feature vectors and local feature vectors from the image of the pedestrian to be identified.

[0068] The feature extraction model can include a relocalization layer, which allows for spatial and depth-level attention operations on the feature maps determined by the feature extraction model. The spatial level refers to the two-dimensional space of width (w) and height (h), while the depth level refers to the depth-wise level.

[0069] The feature extraction model is trained in two stages using labeled pedestrian images. Specifically, the feature extraction model is trained as follows: First, a large number of pedestrian images are collected and annotated, effectively labeling them with pedestrian labels. Next, the infrastructure of the initial model, excluding the relocalization layer, is trained using labeled pedestrian images. The relocalization layer is then trained again, resulting in the feature extraction model.

[0070] Since pedestrian re-identification is to retrieve the pedestrian images across devices given a monitored pedestrian image, the pedestrian images need to include multiple different pedestrian images collected by different devices and containing the same pedestrian.

[0071] The global feature vector is used to represent the global features of the pedestrian image to be identified. Global features refer to the overall attributes of the pedestrian image to be identified. Common global features include color features, texture features, and shape features, such as intensity histograms.

[0072] Local feature vectors are used to characterize the local features of the image of the pedestrian to be identified. Local features refer to the local attributes of the image of the pedestrian to be identified, that is, some of the most representative and stable feature points are used to represent the object. Therefore, when there is local occlusion or affine transformation, the overall information of the image of the pedestrian to be identified can still be well restored.

[0073] The local feature vector may include multiple local feature vectors in the vertical direction and the depth direction of the feature map determined by the feature extraction model. The vertical direction refers to the longitudinal direction of the feature map, and the depth direction refers to the channel direction of the feature map. In this embodiment of the present invention, the number of local feature vectors may be two or more.

[0074] Then, step S3 is executed. After obtaining the global feature vector and the local feature vector, the global feature vector and the local feature vector can be concatenated to achieve splicing and fusion, and a complete feature expression corresponding to the image of the pedestrian to be identified is obtained.

[0075] Finally, step S4 is executed to match the pedestrian image to be identified with pedestrian images in the preset image library based on the complete feature expression corresponding to the pedestrian image to be identified obtained in step S3, and finally determine the pedestrian re-identification result of the pedestrian image to be identified.

[0076] Among them, the pedestrian image to be identified and the pedestrian images in the preset image library can be matched based on the re-ranking method. The specific matching method can be a k-neighbor encoding method, based on the output feature vector of the pedestrian image to be identified and the output feature vector of the pedestrian images in the preset image library, the Mahalanobis distance and the Jaccard distance are calculated, and then the two distances are weighted and summed to obtain the final distance. The closer the distance, the higher the similarity. The pedestrian re-identification results of the output pedestrian image to be identified are sorted according to the similarity, that is, the image with the highest similarity to the pedestrian image to be identified is ranked first, and then the matching results are sorted in descending order of similarity, and finally the pedestrian re-identification result of the pedestrian image to be identified is determined.

[0077] The preset image library can be a pre-established pedestrian image library that includes images captured by different devices and containing the same pedestrian, or it can be the above-mentioned open source data set, which includes multiple different pedestrian images captured by different cameras and containing the same pedestrian. The data set can be selected according to actual needs, and the embodiments of the present invention do not make specific limitations on this.

[0078] The pedestrian re-identification method in the embodiment of the present invention obtains the global feature vector and local feature vector of the pedestrian image to be identified through a feature extraction model, and then splices and fuses the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the pedestrian image to be identified; and based on the complete feature expression, the pedestrian image to be identified is matched with pedestrian images in a preset image library to determine the pedestrian re-identification result of the pedestrian image to be identified. Among them, the local feature vector includes multiple local feature vectors in the vertical direction and depth direction of the feature map determined by the feature extraction model. This method extracts multi-dimensional features such as global feature vectors and local feature vectors, and uses the macro-global features and micro-local features of the pedestrian image to be identified to perform pedestrian re-identification, controlling the overall and details, and improving the accuracy of pedestrian re-identification.

[0079] Based on the above embodiment, the pedestrian re-identification method provided by the embodiment of the present invention, wherein the image of the pedestrian to be identified is input into the feature extraction model to obtain the global feature vector and the local feature vector output by the feature extraction model, specifically includes:

[0080] Inputting the pedestrian image to be identified into the backbone network of the feature extraction model to obtain a feature map extracted by the backbone network;

[0081] Inputting the feature map into a global residual convolution layer and three local residual convolution layers of the feature extraction model to obtain a global feature map and three separable feature maps respectively;

[0082] Inputting the global feature map and each separable feature map into the relocation layer of the feature extraction model to obtain a global relocation feature map corresponding to the global feature map and a separable relocation feature map corresponding to each separable feature map;

[0083] The global relocation feature map and each separable relocation feature map are input into four types of pooling convolution layers of different segmentation granularities of the feature extraction model to obtain global feature vectors and local feature vectors output by each type of pooling convolution layer.

[0084] Specifically, if Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of the feature extraction model provided by the embodiment of the present invention. Figure 2As can be seen in the figure, the feature extraction model includes a backbone network, a global residual convolution layer, a local residual convolution layer, a relocalization layer, and a pooling convolution layer. The backbone network can be a convolutional neural network (CNN). The embodiment of the present invention does not specifically limit the specific form of the backbone network, and it can be selected according to actual needs, as long as it can extract features from the input pedestrian image to be identified. After extraction by the backbone network, a feature map F of the pedestrian image to be identified can be obtained.

[0085] The global residual convolution layer and the local residual convolution layer can be parameter-independent residual networks. The residual network can be an 18-layer residual network or a 50-layer residual network, which is not specifically limited in this embodiment of the present invention. There can be one global residual convolution layer and three local residual convolution layers. After the feature map passes through one global residual convolution layer and three local residual convolution layers, a global feature map and three separable feature maps are obtained respectively.

[0086] The global feature map and each separable feature map are input into the relocalization layer of the feature extraction network to obtain the relocalization feature map output by the relocalization layer. The relocalization layer can perform convolution processing on the global feature map and the separable feature map, and perform maximum pooling processing on multiple separable feature maps in the depth direction, ultimately obtaining the global relocalization feature map and the separable relocalization feature map.

[0087] The global relocalized feature map and each segmentable relocalized feature map are then input into four types of pooling convolution layers of different segmentation granularities in the feature extraction model. Pooling is first performed, and then a convolution operation is performed on the pooled data to obtain a global feature vector and a local feature vector. The pooling can be maximum pooling, and the convolution operation can be a convolution operation with a convolution kernel of 1.

[0088] The pedestrian re-identification method in the embodiment of the present invention obtains a feature map through a backbone network, obtains a global feature map and a separable feature map through a residual convolution layer, then obtains a global relocalization feature map and a separable relocalization feature map through a relocalization layer, and finally obtains a global feature vector and a local feature vector through a pooling convolution layer. The model structure of the feature extraction model is clear and easy to operate.

[0089] On the basis of the above embodiment, the pedestrian re-identification method provided by the embodiment of the present invention, the local residual convolution layer includes a first local residual convolution layer, a second local residual convolution layer and a third local residual convolution layer; accordingly, the feature map is input into a global residual convolution layer and three local residual convolution layers of the feature extraction model to obtain a global feature map and three separable feature maps, respectively, specifically including:

[0090] Inputting the feature map into the global residual convolution layer to obtain a global feature map output by the global residual convolution layer;

[0091] Inputting the feature map into the first local residual convolution layer, and performing feature extraction on the feature map by the first local residual convolution layer to obtain a first separable feature map;

[0092] Inputting the feature map into the second local residual convolution layer, and performing feature extraction on the feature map by the second local residual convolution layer to obtain a second separable feature map;

[0093] The feature map is input into the third local residual convolution layer, and the third local residual convolution layer performs feature extraction on the feature map to obtain a third separable feature map.

[0094] Specifically, in an embodiment of the present invention, the local residual convolution layer includes a first local residual convolution layer, a second local residual convolution layer and a third local residual convolution layer. That is to say, inputting the feature map F into the global residual convolution layer and the local residual convolution layer of the feature extraction model is to input the feature map F into four residual networks with independent parameters to obtain the feature maps output by these four residual networks.

[0095] The feature map F is input to the global residual convolution layer to obtain the global feature map output by the global residual convolution layer. The global feature map can be represented by T1, and the size of T1 is 12×4×2048 (height×width×dimension).

[0096] The feature map is input to the first local residual convolution layer, which extracts features from the feature map to obtain a first separable feature map T2. The size of the first separable feature map T2 can be 24×8×2048.

[0097] The feature map is input to the second local residual convolution layer, which extracts features from the feature map to obtain a second separable feature map T3. The size of the second separable feature map T3 can also be 24×8×2048.

[0098] The feature map is input to the third local residual convolution layer, which performs feature extraction on the feature map to obtain a third separable feature map T4. The size of the third separable feature map T4 can also be 24×8×2048.

[0099] The pedestrian re-identification method in the embodiment of the present invention can obtain the global feature map and the first separable feature map, the second separable feature map, and the third separable feature map corresponding to the feature map through the residual convolution layer. Since the feature maps are extracted by the same backbone network, parameter sharing is achieved to the greatest extent, and the macro and micro features of the image of the pedestrian to be identified are utilized for pedestrian re-identification, grasping the overall and details, and improving the accuracy of pedestrian re-identification.

[0100] On the basis of the above embodiment, the pedestrian re-identification method provided by the embodiment of the present invention, wherein the global feature map and each segmentable feature map are input into the relocation layer of the feature extraction model to obtain a global relocation feature map corresponding to the global feature map and a segmentable relocation feature map corresponding to each segmentable feature map, specifically includes:

[0101] Perform convolution operations on the global feature map, the first separable feature map, and the second separable feature map to generate corresponding feature matrices, and normalize the feature matrices to obtain spatial attention matrices corresponding to the feature matrices;

[0102] Performing maximum pooling on the third separable feature map to generate a pooling vector, performing a linear transformation on the pooling vector, and normalizing the linearly transformed pooling vector to obtain a deep attention vector corresponding to the pooling vector;

[0103] Multiplying the spatial attention matrix by the feature matrices corresponding to each dimension of the corresponding global feature map, the first separable feature map, and the second separable feature map respectively to obtain the global relocation feature map, the first separable relocation feature map, and the second separable relocation feature map;

[0104] The depth attention vector is multiplied by the feature vector corresponding to each pixel in the third separable feature map to obtain a third separable relocation feature map.

[0105] Specifically, in an embodiment of the present invention, a convolution operation can be performed on the global feature map T1, the first separable feature map T2, and the second separable feature map T3 to generate corresponding feature matrices M1, M2, and M3. The convolution operation can be a convolution operation performed on T1, T2, and T3 with a convolution kernel of 1. After the convolution operation, the dimension of the feature matrices M1, M2, and M3 is 1, and the length and width are consistent with the corresponding feature maps, that is, the size of M1 is 12×4×1, and the sizes of M2 and M3 are 24×8×1.

[0106] The feature matrix is ​​then normalized to obtain the spatial attention matrix corresponding to the feature matrix. Normalization refers to normalizing the values ​​in M1, M2, and M3 to within a preset range. Normalization can be performed using a Sigmoid function, which has a preset range of (0, 1).

[0107] The main purpose of the Sigmoid function is to normalize the values ​​in M1, M2, and M3 to between (0, 1), reflecting the numerical size of the feature matrix in a relatively smooth way. Its meaning is the importance of each pixel in the feature map. The feature matrix is ​​a secondary optimization of the previous division operation, making the division of the local feature map more accurate. The formula of the Sigmoid function is as follows:

[0108]

[0109] After normalization, the spatial attention matrices M1′, M2′, and M3′ corresponding to the global feature map, the first separable feature map, and the second separable feature map can be obtained.

[0110] The third separable feature map T4 is subjected to maximum pooling processing to generate a pooling vector M4. The dimension of M4 is the same as that of T4, that is, M4 is a 2048-dimensional feature vector. M4 is subjected to a dimension-invariant linear transformation operation. After the linear transformation, the dimension of the pooling vector M4 is 2048.

[0111] The linearly transformed M4 is normalized by the above-mentioned Sigmoid function to obtain the deep attention vector M4′ corresponding to the pooling vector M4.

[0112] The above-mentioned spatial attention matrices M1′, M2′ and M3′ are multiplied by the corresponding feature matrices corresponding to each dimension in the corresponding global feature map T1, the first separable feature map T2 and the second separable feature map T3 to obtain the global relocation feature map M1″, the first separable relocation feature map M2″ and the second separable relocation feature map M3″ respectively.

[0113] The depth attention vector M4′ is multiplied by the feature vector corresponding to each pixel in the third separable feature map T4 to obtain the third separable relocation feature map M4″.

[0114] The pedestrian re-identification method provided by the embodiment of the present invention processes the global feature map and the separable feature map by setting a relocalization layer, performs fine-grained optimization on the local features divided in the separable feature map, and improves the accuracy of the division.

[0115] Based on the above embodiment, the pedestrian re-identification method provided by the embodiment of the present invention, wherein the global relocation feature map and each separable relocation feature map are input into the four types of pooling convolution layers of the feature extraction model, and the global feature vectors and local feature vectors output by each type of pooling convolution layer are obtained respectively, specifically includes:

[0116] Inputting the global relocation feature map and each separable relocation feature map into the pooling layer of the first type of pooling convolution layer to obtain four global feature vectors;

[0117] Divide the first separable relocatable feature map into two parts along the vertical direction of the image, input the two parts into the pooling layer of the second type of pooling convolution layer, perform pooling operation, and obtain two local feature vectors;

[0118] Divide the second separable relocatable feature map into three parts along the vertical direction of the image, input the three parts into the pooling layer of the third type of pooling convolution layer respectively, perform pooling operation, and obtain three local feature vectors;

[0119] Dividing the third separable relocatable feature map into four parts along the image depth direction, inputting the four parts into the pooling layer of the fourth type of pooling convolution layer respectively, performing a pooling operation, and obtaining four local feature vectors;

[0120] Each global feature vector and each local feature vector are respectively input into the convolution layer of each pooling convolution layer, and the dimension is transformed to obtain the global feature vector and local feature vector of the same dimension.

[0121] Specifically, in an embodiment of the present invention, the four types of pooling convolution layers may include a first type of pooling convolution layer, a second type of pooling convolution layer, a third type of pooling convolution layer, and a fourth type of pooling convolution layer. The function of the pooling layer in the four types of pooling convolution layers is to obtain feature vectors, and the function of the convolution layer is to make the feature vectors have the same dimension.

[0122] First, the global relocation feature map M1″ and each separable relocation feature map M2″, M3″, and M4″ can be input into the pooling layer of the first type of pooling convolution layer, and the maximum pooling process is performed through the pooling layer of the first type of pooling convolution layer, that is, the maximum value in the image area is selected as the value of the area after pooling. When used, maximum pooling is divided into overlapping pooling and non-overlapping pooling. The embodiment of the present invention does not specifically limit overlapping pooling and non-overlapping pooling. Four global feature vectors can be obtained through the first type of pooling convolution layer, which are represented as g1′, g2′, g3′, and g4′ respectively.

[0123] Then, the first separable relocatable feature map M2″ is divided into two parts along the vertical direction of the image, and input into the pooling layer of the second type of pooling convolution layer respectively, and the maximum pooling is performed through the pooling layer of the second type of pooling convolution layer to obtain two local feature vectors. The size of the first separable relocatable feature map M2″ is 24×8×2048, and it is divided into two tensors of 12×8×2048 in the vertical direction. Therefore, when performing maximum pooling, both tensors need to be maximum pooled to obtain two local feature vectors, which are represented as p21′ and p22′ respectively.

[0124] Then, the second separable relocatable feature map M3″ is divided into three parts along the vertical direction of the image, and respectively input into the pooling layer of the third type of pooling convolution layer for pooling operation to obtain three local feature vectors. The size of the second separable relocatable feature map M3″ is also 24×8×2048, and it is divided into three 8×8×2048 tensors in the vertical direction. Therefore, when performing maximum pooling, it is necessary to perform maximum pooling on all three tensors to obtain three local feature vectors, which are represented as p31′, p32′, and p33′ respectively.

[0125] Then, the third separable relocatable feature map M4″ is divided into four parts along the depth direction of the image, and respectively input into the pooling layer of the fourth type of pooling convolution layer for pooling operation to obtain four local feature vectors. The size of the third separable relocatable feature map M4″ is 24×8×2048, and it is divided into four 24×8×512 tensors in the vertical direction. Therefore, when performing maximum pooling, it is necessary to perform maximum pooling on these four tensors to obtain four local feature vectors, which are represented as p41′, p42′, p43′, and p44′ respectively. Therefore, there are 9 local feature vectors in total.

[0126] Finally, after obtaining the global and local feature vectors, they are fed into the convolutional layers within each pooling convolutional layer for dimensionality transformation, resulting in global and local feature vectors of the same dimension. This dimensionality transformation by the convolutional layers within each pooling convolutional layer yields global feature vectors g1, g2, g3, and g4, and local feature vectors p21, p22, p31, p32, p33, p41, p42, p43, and p44. Both the global and local feature vectors have 256 dimensions.

[0127] The pedestrian re-identification method provided by the embodiment of the present invention processes the re-localization feature map through a pooling convolution layer to obtain a global feature vector and a local feature vector, making full use of the macroscopic and microscopic features of the pedestrian image to be identified, and can improve the recognition accuracy.

[0128] On the basis of the above embodiments, in the pedestrian re-identification method provided by the embodiment of the present invention, the feature extraction model is trained based on the following method:

[0129] Inputting the pedestrian image into a feature extraction model to be trained, and training the basic structure of the feature extraction model to be trained except for the relocalization layer based on the value of a preset loss function and an adaptive moment estimation algorithm; the basic structure includes a backbone network, a global residual convolution layer, a local residual convolution layer, a pooling convolution layer, and a fully connected layer;

[0130] Based on the trained basic structure, the relocation layer in the feature extraction model to be trained is trained to obtain the feature extraction model.

[0131] Specifically, in an embodiment of the present invention, when training a feature extraction model, a pedestrian image can be input into the feature extraction model to be trained, and based on the value of a preset loss function and an adaptive moment estimation algorithm, the basic structure except the relocalization layer in the feature extraction model to be trained is trained, wherein the basic structure includes a backbone network, a global residual convolution layer, a local residual convolution layer, a pooling convolution layer, and a fully connected layer.

[0132] When selecting pedestrian images, training data can be read in a manner of n pedestrians in each batch size (batch), with k images of each pedestrian. Wherein, n and k can be set according to actual needs and are not specifically limited in the embodiment of the present invention.

[0133] Pedestrian images can also be selected from the Market-1501 dataset. For example, 751 people and 12,936 images can be selected to form a training set for training, with an average of 17.2 training data images per person.

[0134] In order to enrich the training data, pedestrian images can also be randomly flipped and occluded to simulate real scenes and introduce noise.

[0135] Before inputting the pedestrian image into the feature extraction model, the data of each color channel of the pedestrian image can also be normalized to standardize the data range and improve the subsequent training speed.

[0136] In the basic architecture of the feature extraction model, fully connected layers follow the pooled convolutional layers. These layers perform feature classification, which is achieved through convolution operations. The four global feature vectors and nine local feature vectors output by the pooled convolutional layers are fed into the fully connected layers. In other words, each feature vector corresponds to a fully connected layer. The global and local feature vectors are then fed into 13 fully connected layers, yielding the classification results.

[0137] The loss function is used to evaluate the degree to which the predicted value of the feature extraction model differs from the true value. The value of the preset loss function can be set according to actual needs and is not specifically limited in this embodiment of the present invention. The loss function can be a triple loss function or a cross entropy loss function, etc.

[0138] The adaptive moment estimation algorithm designs independent adaptive learning rates for different parameters by calculating the first-order moment estimation and the second-order moment estimation of the gradient.

[0139] The parameters of the feature extraction model are determined by presetting the loss function and the adaptive moment estimation algorithm, thereby achieving the purpose of model training. After the basic structure training is completed, the parameters of the basic structure are fixed. Based on the trained basic structure, the relocation layer of the feature extraction model is trained to obtain the feature extraction model.

[0140] It should be noted that after the network in the feature extraction model converges, the previously fixed parameters can be released and the entire feature extraction model can be fine-tuned to obtain the feature extraction model.

[0141] The pedestrian re-identification method provided by an embodiment of the present invention first trains the basic structure except the relocalization layer in the feature extraction model based on the value of a preset loss function and an adaptive moment estimation algorithm, and then trains the relocalization layer separately based on the trained basic structure. The partial training method effectively reduces the amount of calculation and has a faster recognition speed.

[0142] On the basis of the above embodiment, in the pedestrian re-identification method provided by the embodiment of the present invention, the preset loss function includes a triple loss function and a cross entropy loss function;

[0143] Accordingly, the pedestrian image is input into the feature extraction model to be trained, and the basic structure of the feature extraction model to be trained except the relocalization layer is trained based on the value of the preset loss function and the adaptive moment estimation algorithm, specifically including:

[0144] Inputting the pedestrian image into the backbone network, the global residual convolution layer, the local residual convolution layer, and the pooling convolution layer in sequence to obtain four global feature vectors and nine local feature vectors, and inputting each global feature vector and each local feature vector into the fully connected layer, and normalizing the output of the fully connected layer;

[0145] Calculating the value of the triplet loss function based on each global eigenvector, and calculating the value of the cross entropy loss function based on the result of the normalization process;

[0146] The value of the triplet loss function and the value of the cross entropy loss function are summed to obtain the value of the preset loss function, and the basic structure is trained based on the value of the preset loss function and the adaptive moment estimation algorithm.

[0147] Specifically, in an embodiment of the present invention, the preset loss function includes a triple loss function and a cross entropy loss function.

[0148] When training the basic structure except the relocalization layer in the feature extraction model to be trained, the pedestrian image can be input into the backbone network, the global residual convolution layer, the local residual convolution layer and the pooling convolution layer in sequence to obtain the global feature vector and the local feature vector, and the global feature vector and the local feature vector are input into the fully connected layer, and the output of the fully connected layer is normalized.

[0149] After obtaining the global eigenvector and the local eigenvector, the value of the triplet loss function can be calculated based on the global eigenvector; and the value of the cross entropy loss function can be calculated based on the normalized result of the output of the fully connected layer.

[0150] The three elements in the triplet loss function are anchor, positive, and negative. An anchor is a randomly selected image from a set of pedestrian images. Positive images belong to the same class as the anchor, while negative images belong to different classes. "Same class" means that the positive image and the anchor belong to the same pedestrian.

[0151] In each batch of the training dataset, the training data is read as n pedestrians and k images of each pedestrian. Then for each anchor, there are k-1 positive and (n-1)*k negative images. Therefore, k*(k-1) APN pairs can be formed in a batch, and the triplet loss is formed on this basis.

[0152] The objective function of the triplet loss function is as follows:

[0153]

[0154] Among them, i is the serial number of the selected image, N is the total number, It represents the Euclidean distance from the anchor to the positive. Similarly, It represents the Euclidean distance from the anchor to the negative, and f represents the output feature of the feature sequence; + means that when the value in the brackets is greater than zero, the value is taken as the loss, and when it is less than zero, the loss is zero; α indicates that there is a minimum interval between the distance between the anchor and the negative and the distance between the anchor and the positive. α can be set according to actual needs, for example, it can be set to 1.2 or other values, and the embodiment of the present invention does not make specific limitations on this.

[0155] It can be seen from the above embodiment that there are four global eigenvectors. Therefore, the above triplet loss function calculation needs to be performed for all four global eigenvectors.

[0156] During training, the triplet loss is reduced as much as possible during the training iteration. That is, when selecting APN pairs, the positive with the farthest distance (worst similarity) and the negative with the closest distance (best similarity) are selected to reduce the loss, optimize the parameters, shorten the distance between positive pairs, and increase the distance between negative pairs, so as to achieve the purpose of training using the value of the triplet loss function.

[0157] The global feature vector and the local feature vector are input to the fully connected layer. From the above embodiment, it can be seen that there are 4 global feature vectors and 9 local feature vectors, and the dimensions of the global feature vector and the local feature vector are both 256 dimensions.

[0158] The 13 feature vectors are input into 13 fully connected layers respectively. The dimensions of the global feature vector and the local feature vector are changed from 256 dimensions to C dimensions, where C is the number of training set categories. The training set categories are the different pedestrian categories in the training set. For example, if there are 10 different pedestrians in the training set, then C is 10.

[0159] The output vector of the fully connected layer is in the form of [A0, A1, A2, ... A C ], normalize the C-dimensional vector output by the fully connected layer to obtain the different probabilities that the image belongs to the training set category, and calculate the value of the cross entropy loss function based on the normalized output of the fully connected layer.

[0160] Among them, normalization can be performed through the Softmax layer. After normalization, the formula for obtaining the different probabilities of the image belonging to the training set category can be:

[0161]

[0162] Among them, C is the training set category, i is the i-th item in the vector group, A k is the kth item in the vector group, p(C k ) represents the probability that the input pedestrian image belongs to a certain category in the training set.

[0163] Based on the above, the value of the cross entropy loss function is calculated. The cross entropy loss function can be expressed as:

[0164]

[0165] Among them, C is the number of categories in the training set mentioned above, y k It is an indicator variable, which is 1 when the true category of the input pedestrian image is the same, and 0 when it is different. kis the probability obtained in the above formula, that is, the predicted probability that the input pedestrian image belongs to a certain category.

[0166] After determining the values ​​of the triplet loss function and the cross-entropy loss function, the values ​​of the triplet loss function and the cross-entropy loss function can be accumulated to obtain the value of the preset loss function. Based on the value of the preset loss function and combined with the adaptive moment estimation algorithm, the basic structure of the feature extraction model except for the relocalization layer can be trained.

[0167] like Figure 3 As shown, Figure 3 It is a schematic diagram of the training process of the basic structure of the feature extraction model in an embodiment of the present invention.

[0168] exist Figure 3 As can be seen from the figure, the pedestrian image is input into the backbone network of the feature extraction model to obtain the feature map of the pedestrian image, the feature map is input into the global residual convolution layer to obtain the global feature map 301, the feature map is input into the local residual convolution layer to obtain the first segmentable feature map 302, the second segmentable feature map 303 and the third segmentable feature map 304; and by Figure 3 As can be seen from the above embodiments, the first separable feature map can be divided into two parts in the vertical direction; the second separable feature map can be divided into three parts in the vertical direction; and the third separable feature map can be divided into four parts in the image depth direction.

[0169] Then input the above global feature map and any separable feature map into the pooling layer in the pooling convolution layer to obtain the global feature vector and local feature vector output by the pooling layer. And according to Figure 3 As can be seen from the above embodiment, the pooling layer performs maximum pooling on the global feature map and any separable feature map to obtain 4 global feature vectors, namely Figure 3 311, 312, 313 and 314 in the pooling layer; the pooling layer also performs maximum pooling on the divided parts in the first separable feature map, the second separable feature map and the third separable feature map to obtain 9 local feature vectors, namely Figure 3 321 represents the two local feature vectors obtained by performing maximum pooling on the two parts divided in the first separable feature map, 322 represents the three local feature vectors obtained by performing maximum pooling on the three parts divided in the second separable feature map, and 323 represents the four local feature vectors obtained by performing maximum pooling on the four parts divided in the third separable feature map.

[0170] Then the global feature vector and local feature vector are input into the convolution layer in the pooling convolution layer to perform convolution operation to obtain the corresponding feature vector after convolution processing. Figure 3331, 332, 333 and 334 represent the global feature vector after the global feature vector is convolved; 341, 342 and 343 represent the local feature vector after the corresponding local feature vector is convolved.

[0171] The global feature vector and local feature vector are input to the fully connected layer respectively, and the output of the fully connected layer is normalized using softmax to obtain the classification result. Figure 3 352 is used to represent the classification results obtained after normalization using softmax.

[0172] During training, the basic structure of the feature extraction model is trained based on the values ​​of the triplet loss function and the cross entropy loss function combined with the adaptive moment estimation algorithm. During training, the classification results are also obtained. Figure 3 It is represented by 352.

[0173] Among them, the output classification result is the probability of the input pedestrian image in each category of the training set.

[0174] In one embodiment of the present invention, the Market 1501 dataset may be used to test the performance of the feature extraction model.

[0175] The Market-1501 dataset was collected on a campus in the summer and was constructed and made public in 2015. It includes 1,501 pedestrians captured by six cameras (five high-definition cameras and one low-definition camera) and 32,668 detected pedestrian rectangles. Each pedestrian is captured by at least two cameras, and a single camera may have multiple images.

[0176] Data from the Market-1501 dataset was selected to construct the query set and test gallery set. The test set includes a preset number of pedestrians and their images. The query set consists of one randomly selected image from each camera for each pedestrian in the test set. Therefore, a maximum of six queries can be generated for a single person.

[0177] For example, the test set may include 750 people, containing 19,732 images, with an average of 26.3 test data per person. The query set is formed by randomly selecting one image from each camera for each of the 750 people in the test set.

[0178] It should be noted that since pedestrian re-identification is cross-camera recognition, the images of the same pedestrian in the test set taken with the same camera as the query set are not included in the result calculation.

[0179] The pedestrian images to be identified in the query set are input into the basic structure of the feature extraction model, and the classification results of the pedestrian images to be identified are obtained through the process in the above embodiment.

[0180] In embodiments of the present invention, the performance of person re-identification methods can be evaluated using Rank1 accuracy and mAP accuracy. Rank1 accuracy refers to the proportion of images in the query set ranking results where the first-ranked person in the test set matches the person corresponding to the query set image. mAP accuracy refers to the average accuracy of all test results, specifically the degree to which all images in the test set that match the query set are ranked first.

[0181] In this embodiment of the present invention, the basic structure of the feature extraction model achieved a mAP of 0.9460, rank 1 of 0.9650, rank 3 of 0.9762, rank 5 of 0.9795, and rank 10 of 0.9846 on the Market1501 dataset. This method achieved higher accuracy on the Market1501 dataset than other currently published person re-ID methods.

[0182] like Figure 4 As shown, Figure 4 2 is a schematic diagram of the training process of the relocation layer of the feature extraction model in an embodiment of the present invention.

[0183] Since the embodiment of the present invention adopts a partial training method, that is, after the basic structure training of the feature extraction model is completed, the relocation layer is trained. Therefore, Figure 4 is Figure 3 Improvements based on Figure 4 exist Figure 3 A relocalization layer is added before the pooling layer in Figure 3 Stay consistent.

[0184] like Figure 4 As shown, the global feature map and the local feature map need to pass through the relocation layer to obtain the global relocation feature map and the separable relocation feature map, and then the global relocation feature map and the separable relocation feature map are input into the pooling layer, and the subsequent steps are performed according to the process in the above embodiment, and finally the classification result is obtained, so as to achieve the training purpose of the relocation layer and finally obtain the feature extraction model.

[0185] The pedestrian re-identification method in the embodiment of the present invention trains the feature extraction model through the triple loss function and the cross entropy loss function, thereby improving the recognition accuracy.

[0186] Based on the above embodiment, the present invention also provides a pedestrian re-identification system. Figure 5 As shown, the system includes:

[0187] A pedestrian image determination module 501 is used to determine a pedestrian image to be identified;

[0188] A feature vector determination module 502 is configured to input the image of the pedestrian to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model;

[0189] A feature splicing and fusion module 503 is used to splice and fuse the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the image of the pedestrian to be identified;

[0190] A pedestrian re-identification module 504 is configured to match the image of the pedestrian to be identified with pedestrian images in a preset image library based on the complete feature expression, and determine a pedestrian re-identification result of the image of the pedestrian to be identified;

[0191] Among them, the local feature vector includes multiple local feature vectors in the vertical and depth directions of the feature map determined by the feature extraction model; the feature extraction model includes a repositioning layer, which is used to perform attention operations on the feature map at the spatial and depth levels; the feature extraction model is trained in two stages based on pedestrian images carrying pedestrian labels; the pedestrian images include pedestrian images collected by different devices and containing the same pedestrian.

[0192] Based on the above embodiment, an embodiment of the present invention provides a person re-identification system, wherein the feature vector determination module specifically includes:

[0193] A backbone network submodule, configured to input the pedestrian image to be identified into the backbone network of the feature extraction model to obtain a feature map extracted by the backbone network;

[0194] A residual convolution submodule, configured to input the feature map into a global residual convolution layer and three local residual convolution layers of the feature extraction model to obtain a global feature map and three separable feature maps respectively;

[0195] A relocation submodule, configured to input the global feature map and each separable feature map into a relocation layer of the feature extraction model, and obtain a global relocation feature map corresponding to the global feature map and a separable relocation feature map corresponding to each separable feature map;

[0196] The pooling convolution submodule is used to input the global relocation feature map and each separable relocation feature map into four types of pooling convolution layers with different segmentation granularities of the feature extraction model, and obtain the global feature vector and local feature vector output by each type of pooling convolution layer.

[0197] On the basis of the above embodiment, an embodiment of the present invention provides a person re-identification system, wherein the local residual convolution layer includes a first local residual convolution layer, a second local residual convolution layer and a third local residual convolution layer; accordingly,

[0198] The residual convolution submodule specifically includes:

[0199] A global residual convolution subunit, configured to input the feature map into the global residual convolution layer to obtain a global feature map output by the global residual convolution layer;

[0200] a first local residual convolution subunit, configured to input the feature map into the first local residual convolution layer, and perform feature extraction on the feature map by the first local residual convolution layer to obtain a first separable feature map;

[0201] a second local residual convolution subunit, configured to input the feature map into the second local residual convolution layer, and perform feature extraction on the feature map by the second local residual convolution layer to obtain a second separable feature map;

[0202] The third local residual convolution subunit is used to input the feature map into the third local residual convolution layer, and the third local residual convolution layer performs feature extraction on the feature map to obtain a third separable feature map.

[0203] Based on the above embodiment, an embodiment of the present invention provides a person re-identification system, wherein the relocation submodule specifically includes:

[0204] a spatial attention matrix generating subunit, configured to perform convolution operations on the global feature map, the first separable feature map, and the second separable feature map, respectively, to generate corresponding feature matrices, and to normalize the feature matrices to obtain spatial attention matrices corresponding to the feature matrices;

[0205] a deep attention vector generating subunit, configured to perform maximum pooling on the third separable feature map to generate a pooling vector, perform a linear transformation operation on the pooling vector, and normalize the linearly transformed pooling vector to obtain a deep attention vector corresponding to the pooling vector;

[0206] A corresponding bit multiplication subunit is used to perform corresponding bit multiplication on the spatial attention matrix and the feature matrix corresponding to each dimension of the corresponding global feature map, the first separable feature map and the second separable feature map, respectively, to obtain the global relocation feature map, the first separable relocation feature map and the second separable relocation feature map;

[0207] The corresponding bit multiplication subunit is further used to perform corresponding bit multiplication on the depth attention vector and the feature vector corresponding to each pixel point in the third separable feature map to obtain a third separable repositionable feature map.

[0208] Based on the above embodiment, an embodiment of the present invention provides a person re-identification system, wherein the pooling convolution submodule specifically includes:

[0209] A first pooling subunit is configured to input the global relocation feature map and each separable relocation feature map into a pooling layer of a first type of pooling convolutional layer to obtain four global feature vectors;

[0210] A second pooling subunit is used to divide the first separable relocatable feature map into two parts along the vertical direction of the image, input the two parts into the pooling layer of the second type of pooling convolution layer, perform a pooling operation, and obtain two local feature vectors;

[0211] A third pooling subunit is configured to divide the second separable relocatable feature map into three parts along the vertical direction of the image, and input the three parts into the pooling layer of the third type of pooling convolution layer respectively to perform a pooling operation to obtain three local feature vectors;

[0212] a fourth pooling subunit, configured to divide the third separable relocatable feature map into four parts along the image depth direction, input the four parts respectively into the pooling layer of the fourth type of pooling convolution layer, perform a pooling operation, and obtain four local feature vectors;

[0213] The convolution subunit is used to input each global feature vector and each local feature vector into the convolution layer of each pooling convolution layer respectively, perform dimensionality transformation, and obtain global feature vectors and local feature vectors of the same dimension.

[0214] Based on the above embodiment, an embodiment of the present invention provides a person re-identification system, wherein the feature extraction model is trained based on the following method:

[0215] Inputting the pedestrian image into a feature extraction model to be trained, and training the basic structure of the feature extraction model to be trained except for the relocalization layer based on the value of a preset loss function and an adaptive moment estimation algorithm; the basic structure includes a backbone network, a global residual convolution layer, a local residual convolution layer, a pooling convolution layer, and a fully connected layer;

[0216] Based on the trained basic structure, the relocation layer in the feature extraction model to be trained is trained to obtain the feature extraction model.

[0217] On the basis of the above embodiment, an embodiment of the present invention provides a person re-identification system, wherein the preset loss function includes a triplet loss function and a cross entropy loss function;

[0218] Accordingly, the pedestrian image is input into the feature extraction model to be trained, and the basic structure of the feature extraction model to be trained except the relocalization layer is trained based on the value of the preset loss function and the adaptive moment estimation algorithm, specifically including:

[0219] Inputting the pedestrian image into the backbone network, the global residual convolution layer, the local residual convolution layer, and the pooling convolution layer in sequence to obtain four global feature vectors and nine local feature vectors, and inputting each global feature vector and each local feature vector into the fully connected layer, and normalizing the output of the fully connected layer;

[0220] Calculating the value of the triplet loss function based on each global eigenvector, and calculating the value of the cross entropy loss function based on the result of the normalization process;

[0221] The value of the triplet loss function and the value of the cross entropy loss function are summed to obtain the value of the preset loss function, and the basic structure is trained based on the value of the preset loss function and the adaptive moment estimation algorithm.

[0222] Specifically, the functions of each module in the pedestrian re-identification system provided in the embodiment of the present invention correspond one-to-one to the operating procedures of each step in the above-mentioned method embodiment, and the effects achieved are also consistent. Please refer to the above-mentioned embodiment for details, and no further details will be given in the embodiment of the present invention.

[0223] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6As shown, the electronic device may include: a processor (processor) 610, a communication interface (Communications Interface) 620, a memory (memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logic instructions in the memory 630 to execute the pedestrian re-identification method provided in the above-mentioned embodiments, which includes: determining a pedestrian image to be identified; inputting the pedestrian image to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model; splicing and fusing the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the pedestrian image to be identified; based on the complete feature expression, matching the pedestrian image to be identified with pedestrian images in a preset image library to determine the pedestrian re-identification result of the pedestrian image to be identified; wherein the local feature vector includes multiple local feature vectors in the vertical direction and the depth direction of the feature map determined by the feature extraction model; the feature extraction model includes a relocation layer, which is used to perform spatial and depth-level attention operations on the feature map; the feature extraction model is trained in two stages based on pedestrian images carrying pedestrian labels; the pedestrian images include pedestrian images collected by different devices and containing the same pedestrian.

[0224] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0225] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the pedestrian re-identification method provided in the above embodiments, the method comprising: determining a pedestrian image to be identified; inputting the pedestrian image to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model; splicing and fusing the global feature vector and the local feature vector to obtain a complete feature table corresponding to the pedestrian image to be identified. Based on the complete feature expression, the pedestrian image to be identified is matched with the pedestrian images in the preset image library to determine the pedestrian re-identification result of the pedestrian image to be identified; wherein, the local feature vector includes multiple local feature vectors in the vertical direction and the depth direction of the feature map determined by the feature extraction model; the feature extraction model includes a repositioning layer, and the repositioning layer is used to perform spatial and depth attention operations on the feature map; the feature extraction model is trained in two stages based on pedestrian images carrying pedestrian labels; the pedestrian images include pedestrian images collected by different devices and containing the same pedestrian.

[0226] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the pedestrian re-identification method provided in the above-mentioned embodiments, the method comprising: determining a pedestrian image to be identified; inputting the pedestrian image to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model; splicing and fusing the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the pedestrian image to be identified; based on the complete feature expression, matching the pedestrian image to be identified with pedestrian images in a preset image library to determine a pedestrian re-identification result of the pedestrian image to be identified; wherein the local feature vector includes multiple local feature vectors in the vertical direction and the depth direction of the feature map determined by the feature extraction model; the feature extraction model includes a relocalization layer, which is used to perform spatial and depth-level attention operations on the feature map; the feature extraction model is trained in two stages based on pedestrian images carrying pedestrian labels; the pedestrian images include pedestrian images collected by different devices and containing the same pedestrian.

[0227] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0228] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0229] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A pedestrian re-identification method, characterized in that: include: Determine a pedestrian image to be identified; Inputting the pedestrian image to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model; Splicing and fusing the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the image of the pedestrian to be identified; Based on the complete feature expression, the pedestrian image to be identified is matched with pedestrian images in a preset image library to determine a pedestrian re-identification result of the pedestrian image to be identified; The local feature vectors include multiple local feature vectors in the vertical and depth directions of the feature map determined by the feature extraction model; the feature extraction model includes a relocalization layer, which is used to perform spatial and depth-level attention operations on the feature map; the feature extraction model is trained in two stages based on pedestrian images carrying pedestrian labels; the pedestrian images include pedestrian images collected by different devices and containing the same pedestrian; The feature extraction model is trained based on the following method: Inputting the pedestrian image into a feature extraction model to be trained, and training the basic structure of the feature extraction model to be trained except the relocalization layer based on the value of a preset loss function and an adaptive moment estimation algorithm; the basic structure includes a backbone network, a global residual convolution layer, a local residual convolution layer, a pooling convolution layer, and a fully connected layer; Based on the trained basic structure, the relocation layer in the feature extraction model to be trained is trained to obtain the feature extraction model; The feature extraction model includes a third local residual convolution layer, and the third local residual convolution layer is used to extract features from the feature map to obtain a third separable feature map; The relocation layer is specifically used for: Performing maximum pooling on the third separable feature map to generate a pooling vector, performing a linear transformation on the pooling vector, and normalizing the linearly transformed pooling vector to obtain a deep attention vector corresponding to the pooling vector; Multiplying the depth attention vector by the feature vector corresponding to each pixel in the third separable feature map to obtain a third separable relocalization feature map; The local feature vector of the feature map in the depth direction is obtained by processing the third separable relocatable feature map by a pooling convolution layer.

2. The pedestrian re-identification method according to claim 1, characterized in that: Inputting the pedestrian image to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model specifically includes: Inputting the pedestrian image to be identified into the backbone network of the feature extraction model to obtain a feature map extracted by the backbone network; Inputting the feature map into a global residual convolution layer and three local residual convolution layers of the feature extraction model to obtain a global feature map and three separable feature maps respectively; Inputting the global feature map and each separable feature map into the relocation layer of the feature extraction model to obtain a global relocation feature map corresponding to the global feature map and a separable relocation feature map corresponding to each separable feature map; The global relocation feature map and each separable relocation feature map are input into four types of pooling convolution layers of different segmentation granularities of the feature extraction model to obtain global feature vectors and local feature vectors output by each type of pooling convolution layer.

3. The pedestrian re-identification method according to claim 2, characterized in that: The local residual convolution layer includes a first local residual convolution layer, a second local residual convolution layer, and a third local residual convolution layer; accordingly, the feature map is input into a global residual convolution layer and three local residual convolution layers of the feature extraction model to obtain a global feature map and three separable feature maps, respectively, specifically including: Inputting the feature map into the global residual convolution layer to obtain a global feature map output by the global residual convolution layer; Inputting the feature map into the first local residual convolution layer, and performing feature extraction on the feature map by the first local residual convolution layer to obtain a first separable feature map; Inputting the feature map into the second local residual convolution layer, and performing feature extraction on the feature map by the second local residual convolution layer to obtain a second separable feature map; The feature map is input into the third local residual convolution layer, and the third local residual convolution layer performs feature extraction on the feature map to obtain a third separable feature map.

4. The pedestrian re-identification method according to claim 3, characterized in that: The step of inputting the global feature map and each separable feature map into the relocation layer of the feature extraction model to obtain a global relocation feature map corresponding to the global feature map and a separable relocation feature map corresponding to each separable feature map, specifically includes: Perform convolution operations on the global feature map, the first separable feature map, and the second separable feature map to generate corresponding feature matrices, and normalize the feature matrices to obtain spatial attention matrices corresponding to the feature matrices; The spatial attention matrix is ​​multiplied by the corresponding feature matrices corresponding to each dimension in the corresponding global feature map, the first separable feature map and the second separable feature map respectively to obtain the global relocation feature map, the first separable relocation feature map and the second separable relocation feature map.

5. The pedestrian re-identification method according to claim 4, characterized in that: described The global relocation feature map and each separable relocation feature map are input into the four types of pooling convolution layers of the feature extraction model to obtain the global feature vector and local feature vector output by each type of pooling convolution layer, specifically including: Inputting the global relocation feature map and each separable relocation feature map into the pooling layer of the first type of pooling convolution layer to obtain four global feature vectors; Divide the first separable relocatable feature map into two parts along the vertical direction of the image, input the two parts into the pooling layer of the second type of pooling convolution layer, perform pooling operation, and obtain two local feature vectors; Divide the second separable relocatable feature map into three parts along the vertical direction of the image, input the three parts into the pooling layer of the third type of pooling convolution layer respectively, perform pooling operation, and obtain three local feature vectors; Dividing the third separable relocatable feature map into four parts along the image depth direction, inputting the four parts into the pooling layer of the fourth type of pooling convolution layer respectively, performing a pooling operation, and obtaining four local feature vectors; Each global feature vector and each local feature vector are respectively input into the convolution layer of each pooling convolution layer, and the dimension is transformed to obtain the global feature vector and local feature vector of the same dimension.

6. The pedestrian re-identification method according to claim 1, characterized in that: The preset loss function includes a triple loss function and a cross entropy loss function; Accordingly, the pedestrian image is input into the feature extraction model to be trained, and the basic structure of the feature extraction model to be trained except the relocalization layer is trained based on the value of the preset loss function and the adaptive moment estimation algorithm, specifically including: Inputting the pedestrian image into the backbone network, the global residual convolution layer, the local residual convolution layer, and the pooling convolution layer in sequence to obtain four global feature vectors and nine local feature vectors, and inputting each global feature vector and each local feature vector into the fully connected layer, and normalizing the output of the fully connected layer; Calculating the value of the triplet loss function based on each global eigenvector, and calculating the value of the cross entropy loss function based on the result of the normalization process; The value of the triplet loss function and the value of the cross entropy loss function are summed to obtain the value of the preset loss function, and the basic structure is trained based on the value of the preset loss function and the adaptive moment estimation algorithm.

7. A pedestrian re-identification system, characterized in that: include: A pedestrian image determination module, used to determine the pedestrian image to be identified; A feature vector determination module is used to input the pedestrian image to be identified into a feature extraction model to obtain a global feature vector and a local feature vector output by the feature extraction model; A feature splicing and fusion module is used to splice and fuse the global feature vector and the local feature vector to obtain a complete feature expression corresponding to the image of the pedestrian to be identified; a pedestrian re-identification module, configured to match the image of the pedestrian to be identified with pedestrian images in a preset image library based on the complete feature expression, and determine a pedestrian re-identification result of the image of the pedestrian to be identified; The local feature vectors include multiple local feature vectors in the vertical and depth directions of the feature map determined by the feature extraction model; the feature extraction model includes a relocalization layer, which is used to perform spatial and depth-level attention operations on the feature map; the feature extraction model is trained in two stages based on pedestrian images carrying pedestrian labels; the pedestrian images include pedestrian images collected by different devices and containing the same pedestrian; The feature extraction model is trained based on the following method: Inputting the pedestrian image into a feature extraction model to be trained, and training the basic structure of the feature extraction model to be trained except the relocalization layer based on the value of a preset loss function and an adaptive moment estimation algorithm; the basic structure includes a backbone network, a global residual convolution layer, a local residual convolution layer, a pooling convolution layer, and a fully connected layer; Based on the trained basic structure, the relocation layer in the feature extraction model to be trained is trained to obtain the feature extraction model; The feature extraction model includes a third local residual convolution layer, and the third local residual convolution layer is used to extract features from the feature map to obtain a third separable feature map; The relocation layer is specifically used for: Performing maximum pooling on the third separable feature map to generate a pooling vector, performing a linear transformation on the pooling vector, and normalizing the linearly transformed pooling vector to obtain a deep attention vector corresponding to the pooling vector; Multiplying the depth attention vector by the feature vector corresponding to each pixel in the third separable feature map to obtain a third separable relocalization feature map; The local feature vector of the feature map in the depth direction is obtained by processing the third separable relocatable feature map by a pooling convolution layer.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the pedestrian re-identification method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the pedestrian re-identification method according to any one of claims 1 to 6 are implemented.

Images