Pedestrian re-identification method and device based on mask-guided two-stream network

Abstract

The present invention discloses a pedestrian re-identification method and device based on a mask-guided dual-stream network, which relates to the field of image recognition. A pedestrian re-identification model, a first teacher model and a second teacher model are constructed. The pedestrian re-identification model includes a contour unit and an exposure unit. A contour space attention module is used as a first student model and is distilled and learned with the first teacher model to construct an attention loss guided by a mask in a contour flow; the exposure space attention module is used as a second student model and is distilled and learned with the second teacher model to construct an attention loss guided by a mask in an exposure flow; facial information loss and REID loss are constructed; a total loss function is constructed according to the attention loss guided by a mask in the contour flow, the attention loss guided by a mask in the exposure flow, the facial information loss and the REID loss, and the pedestrian re-identification model is trained to obtain a trained pedestrian re-identification model for pedestrian re-identification, thereby solving the interference problem of background and clothing factors.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a pedestrian re-identification method and device based on a mask-guided dual-stream network. Background Art

[0002] Person re-identification is a technology that uses computer vision technology to determine whether there are pedestrians with known identities in images or videos. It has been widely used in the fields of intelligent video surveillance, intelligent security, etc. However, existing pedestrian re-identification technologies have the following defects:

[0003] 1. Not considering the interference of background factors;

[0004] Some researchers did not consider the interference of background and clothing on the re-identification (ReID) problem, and the model's attention was scattered throughout the image, which was equivalent to Figure 1 In Mask1, Herzog et al. proposed a lightweight multi-branch network to capture overall features, which resulted in the model not learning identity-sensitive features in a targeted manner, and was interfered by both background factors and clothing factors.

[0005] 2. The interference of clothing factors is not considered;

[0006] Another part of the methods, as shown in Mask2 in the same figure, focuses on the entire pedestrian including clothes. For example, Chen et al. proposed a spatial attention network to extract individual spatial features. Although this method eliminates the negative impact of background factors on the Clothes-Changing ReID (CC-ReID) problem, the significant impact of clothes on the CC-ReID problem still exists.

[0007] 3. Remove the interference of background and clothing factors while erasing sensitive identity information;

[0008] Another group of researchers have adopted the same method as Mask 3 in the figure, which improves robustness by reducing the interference of clothing on pedestrian re-identification. For example, Yang et al. proposed a causal-based automatic intervention model to reduce clothing interference in CC-ReID; Wu et al. introduced a clothing-independent spatial attention framework to eliminate the interference of clothing appearance by acquiring knowledge of human parsing modules. Although these methods reduce the interference of clothing on CC-ReID problems, they do not take into account the identity-sensitive information of the exposed area of ​​pedestrians. Summary of the invention

[0009] The purpose of this application is to propose a pedestrian re-identification method and device based on a mask-guided dual-stream network to address the above-mentioned technical problems.

[0010] In a first aspect, the present invention provides a method for pedestrian re-identification based on a mask-guided two-stream network, comprising the following steps:

[0011] Obtaining a pedestrian image to be queried containing a target pedestrian and an image set containing at least one pedestrian image to be searched;

[0012] A pedestrian re-identification model, a first teacher model and a second teacher model are constructed. The pedestrian re-identification model includes a contour unit and an exposure unit. The contour unit includes a first feature extraction module, a contour space attention module and a first MBN network. The pedestrian image is input into the first feature extraction module to obtain a first intermediate feature map. The first intermediate feature map is input into the contour space attention module to obtain a first attention map. The first attention map is applied to the first intermediate feature map to obtain a first fine feature map. The first fine feature map is input into the first MBN network to obtain a first contour embedding feature and a second contour embedding feature. The exposure unit includes a second feature extraction module, an exposure space attention module and a second MBN network. The pedestrian image is input into the second feature extraction module to obtain a second intermediate feature map. The second intermediate feature map is input into the exposure space attention module to obtain a second attention map. The second attention map is applied to the second intermediate feature map to obtain a second fine feature map. The second fine feature map is input into the second MBN network to obtain a first exposure embedding feature and a second exposure embedding feature. ; In the training process of the pedestrian re-identification model, the contour space attention module is used as the first student model and is distilled and learned with the first teacher model to construct the mask-guided attention loss in the contour flow; the exposure space attention module is used as the second student model and is distilled and learned with the second teacher model to construct the mask-guided attention loss in the exposure flow; facial information loss is constructed based on the first student model and the first teacher module and the second student model and the second teacher model; the first cross entropy loss and the second cross entropy loss are constructed based on the first contour embedding feature and the first exposure embedding feature, respectively, and the batch hard triplet loss is constructed based on the second contour embedding feature and the second exposure embedding feature, and the REID loss is constructed according to the first cross entropy loss, the second cross entropy loss and the batch hard triplet loss; the total loss function is constructed according to the mask-guided attention loss in the contour flow, the mask-guided attention loss in the exposure flow, the facial information loss and the REID loss, and the pedestrian re-identification model is trained based on the total loss function to obtain the trained pedestrian re-identification model;

[0013] The trained pedestrian re-identification model is used to combine the pedestrian image to be queried and the pedestrian image to be searched in the image set to perform pedestrian re-identification and obtain the pedestrian re-identification result.

[0014] Preferably, the pedestrian image is input into the first feature extraction module to obtain a first intermediate feature map, the first intermediate feature map is input into the contour space attention module to obtain a first attention map, the first attention map is applied to the first intermediate feature map to obtain a first fine feature map, the first fine feature map is input into the first MBN network to obtain a first contour embedding feature and a second contour embedding feature, specifically including:

[0015] The pedestrian image passes through the first feature extraction module, where the first feature extraction module uses OSNet to obtain the first intermediate feature map, as shown in the following formula:

[0016] ;

[0017] in, Represents the function corresponding to OSNet, represents a pedestrian image, Represents the first intermediate feature map, whose size is , , and Respectively represent the height, width and number of channels of the first intermediate feature map;

[0018] The first intermediate feature map passes through the contour space attention module to obtain the first attention map, as shown in the following formula:

[0019] ;

[0020] in, represents the convolution operator, represents a convolution kernel of size 1×1, represents the sigmoid activation function, represents the first attention map;

[0021] The first attention map is applied to the first intermediate feature map using the following formula to obtain the first refined feature map:

[0022] ;

[0023] in, represents the Hadamard matrix product, represents the first fine feature map;

[0024] First fine feature map After the first MBN network, the first contour embedding feature is obtained and the second contour embedding feature .

[0025] Preferably, the contour space attention module is used as the first student model and distilled with the first teacher model to construct the mask-guided attention loss in the contour flow, specifically including:

[0026] The pedestrian image is input into the first teacher model. The first teacher model includes a pre-trained HED network, an opening operator, a closing operator and a Gaussian filter connected in sequence. The pedestrian image passes through the pre-trained HED network to obtain the third intermediate feature map, as shown in the following formula:

[0027] ;

[0028] in, represents a pedestrian image, represents the third intermediate feature map, Represents the function corresponding to the pre-trained HED network;

[0029] The third intermediate feature map After passing through the opening operator, closing operator and Gaussian filter in sequence, the first mask feature map is obtained. The calculation formula is as follows:

[0030] ;

[0031] ;

[0032] ;

[0033] ;

[0034] in," " is the open operator corresponding to the open operator, " is the closing operator corresponding to the closed operator, B represents the 5×5 differential operator, O represents the output feature of the open operator, C represents the output feature of the closed operator, G(x1, x2,σ) represents the two-dimensional Gaussian function, x1 represents the horizontal coordinate, x2 represents the vertical coordinate, σ represents the standard deviation, represents the convolution operator, represents the first mask feature map;

[0035] Mask-guided Attention Loss in Contour Flow Defined as:

[0036] ;

[0037] Among them, i and j represent the horizontal and vertical coordinates of the pixel points in the first attention map or the first mask feature map, respectively. , and Respectively represent the height, width and number of channels of the first intermediate feature map, represents the first attention map.

[0038] Preferably, the pedestrian image is input into the second feature extraction module to obtain a second intermediate feature map, the second intermediate feature map is input into the exposure space attention module to obtain a second attention map, the second attention map is applied to the second intermediate feature map to obtain a second fine feature map, the second fine feature map is input into the second MBN network to obtain a first exposure embedding feature and a second exposure embedding feature, specifically including:

[0039] The pedestrian image passes through the second feature extraction module, where the second feature extraction module uses CNN to obtain the second intermediate feature map, as shown in the following formula:

[0040] ;

[0041] in, Represents the function corresponding to CNN, represents a pedestrian image, Represents the second intermediate feature map, whose size is , , and Respectively represent the height, width and number of channels of the second intermediate feature map;

[0042] The second intermediate feature map is passed through the exposed spatial attention module to obtain the second attention map, as shown in the following formula:

[0043] ;

[0044] in, represents the convolution operator, x represents the convolution kernel of size 1×1, represents the sigmoid activation function, represents the second attention map;

[0045] The second attention map is applied to the second intermediate feature map using the following formula to obtain the second refined feature map:

[0046] ;

[0047] in, represents the Hadamard matrix product, represents the second fine feature map;

[0048] The second fine feature map After the second MBN network, the second contour embedding feature is obtained and the second contour embedding feature .

[0049] Preferably, the exposure space attention module is used as the second student model and distilled with the second teacher model to construct the mask-guided attention loss in the exposure stream, specifically including:

[0050] The pedestrian image is input into the second teacher model. The second teacher model includes a pre-trained SAM encoder and a pre-trained SAM decoder connected in sequence. The pedestrian image is passed through the pre-trained SAM encoder to obtain the fourth intermediate feature map, as shown in the following formula:

[0051] ;

[0052] in, represents a pedestrian image, represents the fourth intermediate feature map, Represents the function corresponding to the pre-trained SAM encoder;

[0053] Fourth intermediate feature map After the pre-trained SAM decoder, the second mask feature map is obtained, and the calculation formula is as follows:

[0054] ;

[0055] in, Represents the function corresponding to the pre-trained SAM decoder, represents the second mask feature map;

[0056] Mask-guided attention loss in exposure stream Defined as:

[0057] ;

[0058] Among them, m and n represent the horizontal and vertical coordinates of the pixel points in the second attention map or the second mask feature map, respectively. , and Respectively represent the height, width and number of channels of the second intermediate feature map, represents the second attention map.

[0059] As a preference, facial information loss for:

[0060] ;

[0061] Among them, KL represents Kullback-Leibler divergence, S represents Softmax function, τ represents parameter, represents the first mask feature map output by the first teacher model or the second mask feature map output by the second teacher model, represents the first attention map output by the first student model or the second attention map output by the second student model, k represents the kth category, then , N represents the total number of categories, Represents the difference between two probability distributions;

[0062] First cross entropy loss for:

[0063] ;

[0064] Among them, CE represents the cross entropy loss function, y represents the true identity label of the pedestrian image, represents the predicted identity label of the pedestrian image corresponding to the first contour embedding feature;

[0065] Second cross entropy loss for:

[0066] ;

[0067] in, The predicted identity label of the pedestrian image corresponding to the first exposed embedded feature;

[0068] Batch Hard Triplet Loss for:

[0069] ;

[0070] Wherein, Trip represents the batch hard triplet loss function, f1 represents the predicted identity label of the pedestrian image corresponding to the second contour embedding feature, and f2 represents the predicted identity label of the pedestrian image corresponding to the second exposure embedding feature;

[0071] REID loss for:

[0072] ;

[0073] Total loss function for:

[0074] ;

[0075] Among them, λ1, λ2, λ3 and λ4 represent the first weight, the second weight, the third weight and the fourth weight respectively.

[0076] Preferably, the trained person re-identification model is used to perform person re-identification in combination with the pedestrian image to be queried and the pedestrian image to be searched in the image set to obtain the pedestrian re-identification result, which specifically includes:

[0077] Inputting the queried pedestrian image and the pedestrian image to be searched in the image set into the trained pedestrian re-identification model respectively to obtain the query feature and the search feature;

[0078] In response to determining that a pedestrian in the exposed area is detected in the queried pedestrian image and / or the pedestrian image to be searched, a first embedding vector obtained before the last fully connected layer in the first MBN network of the contour unit and a second embedding vector obtained before the last fully connected layer in the second MBN network of the exposure unit are added to obtain a query feature and / or a search feature;

[0079] In response to determining that no pedestrian in the exposed area is detected in the queried pedestrian image and / or the pedestrian image to be searched, a first embedding vector obtained before the last fully connected layer in the first MBN network of the contour unit is used as a query feature and / or a search feature;

[0080] The similarity between the query feature and each search feature is calculated, and the identity label corresponding to the search feature with the greatest similarity to the query feature is used as the identity information of the target pedestrian in the pedestrian image to be queried.

[0081] In a second aspect, the present invention provides a pedestrian re-identification device based on a mask-guided dual-stream network, comprising:

[0082] A data acquisition module is configured to acquire a pedestrian image to be queried containing a target pedestrian and an image set containing at least one pedestrian image to be searched;

[0083] The model building module is configured to build a pedestrian re-identification model, a first teacher model and a second teacher model. The pedestrian re-identification model includes a contour unit and an exposure unit. The contour unit includes a first feature extraction module, a contour space attention module and a first MBN network. The pedestrian image is input into the first feature extraction module to obtain a first intermediate feature map. The first intermediate feature map is input into the contour space attention module to obtain a first attention map. The first attention map is applied to the first intermediate feature map to obtain a first fine feature map. The first fine feature map is input into the first MBN network to obtain a first contour embedding feature and a second contour embedding feature; the exposure unit includes a second feature extraction module, an exposure space attention module and a second MBN network. The pedestrian image is input into the second feature extraction module to obtain a second intermediate feature map. The second intermediate feature map is input into the exposure space attention module to obtain a second attention map. The second attention map is applied to the second intermediate feature map to obtain a second fine feature map. The second fine feature map is input into the second MBN network to obtain the first exposure embedding feature and the second exposure embedding feature; in the training process of the pedestrian re-identification model, the contour space attention module is used as the first student model and is distilled and learned with the first teacher model to construct the mask-guided attention loss in the contour flow; the exposure space attention module is used as the second student model and is distilled and learned with the second teacher model to construct the mask-guided attention loss in the exposure flow; facial information loss is constructed based on the first student model and the first teacher module and the second student model and the second teacher model; a first cross entropy loss and a second cross entropy loss are constructed based on the first contour embedding feature and the first exposure embedding feature, respectively, and a batch hard triplet loss is constructed based on the second contour embedding feature and the second exposure embedding feature, and a REID loss is constructed according to the first cross entropy loss, the second cross entropy loss and the batch hard triplet loss; a total loss function is constructed according to the mask-guided attention loss in the contour flow, the mask-guided attention loss in the exposure flow, the facial information loss and the REID loss, and the pedestrian re-identification model is trained based on the total loss function to obtain a trained pedestrian re-identification model;

[0084] The re-identification module is configured to use the trained pedestrian re-identification model in combination with the pedestrian image to be queried and the pedestrian image to be searched in the image set to perform pedestrian re-identification to obtain a pedestrian re-identification result.

[0085] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0086] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0087] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0088] Compared with the prior art, the present invention has the following beneficial effects:

[0089] (1) The mask-guided dual-stream network-based pedestrian re-identification method proposed in this paper proposes a contour stream to extract the pedestrian's contour, which effectively suppresses the influence of background factors on identity information recognition while retaining the pedestrian's identity sensitive information.

[0090] (2) The contour flow of the pedestrian re-identification method based on mask-guided two-stream network proposed in the present invention not only suppresses the influence of background factors through the mask-guided attention mechanism, but also effectively suppresses the influence of clothing factors on identity information recognition.

[0091] (3) The mask-guided dual-stream network-based pedestrian re-identification method proposed in this paper proposes an exposure stream to extract the exposed area of ​​the pedestrian, which effectively increases the pedestrian's identity sensitive information and further enhances the model's sensitivity to identity information recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0093] Figure 1 A comparison diagram of masks extracted by a pedestrian re-identification method based on a mask-guided dual-stream network in the prior art and an embodiment of the present application;

[0094] Figure 2 A schematic diagram of a process of a pedestrian re-identification method based on a mask-guided dual-stream network according to an embodiment of the present application;

[0095] Figure 3 A schematic diagram of the training process of a pedestrian re-identification method based on a mask-guided dual-stream network according to an embodiment of the present application;

[0096] Figure 4A schematic diagram of a process of performing knowledge distillation between a contour unit and a first teacher model in a pedestrian re-identification method based on a mask-guided two-stream network according to an embodiment of the present application;

[0097] Figure 5 A schematic diagram of a process for performing knowledge distillation between an exposure unit and a second teacher model in a person re-identification method based on a mask-guided two-stream network according to an embodiment of the present application;

[0098] Figure 6 A schematic diagram of a pedestrian re-identification device based on a mask-guided dual-stream network according to an embodiment of the present application;

[0099] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0100] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0101] Figure 2 A method for pedestrian re-identification based on a mask-guided dual-stream network provided in an embodiment of the present application is shown, comprising the following steps:

[0102] S1, obtaining a pedestrian image to be queried containing a target pedestrian and an image set containing at least one pedestrian image to be searched.

[0103] Specifically, the target pedestrian is a pedestrian of unknown identity, and the pedestrians in the pedestrian images to be searched in the image set are pedestrians of known identities. The goal of pedestrian re-identification in the embodiment of the present application is to find the pedestrian in the pedestrian images to be searched who has the same identity as the target pedestrian.

[0104] S2, construct a pedestrian re-identification model, a first teacher model and a second teacher model. The pedestrian re-identification model includes a contour unit and an exposure unit. The contour unit includes a first feature extraction module, a contour space attention module and a first MBN network. The pedestrian image is input into the first feature extraction module to obtain a first intermediate feature map. The first intermediate feature map is input into the contour space attention module to obtain a first attention map. The first attention map is applied to the first intermediate feature map to obtain a first fine feature map. The first fine feature map is input into the first MBN network to obtain a first contour embedding feature and a second contour embedding feature; the exposure unit includes a second feature extraction module, an exposure space attention module and a second MBN network. The pedestrian image is input into the second feature extraction module to obtain a second intermediate feature map. The second intermediate feature map is input into the exposure space attention module to obtain a second attention map. The second attention map is applied to the second intermediate feature map to obtain a second fine feature map. The second fine feature map is input into the second MBN network to obtain a first exposure embedding feature and a second exposure embedding feature. Features; in the training process of the pedestrian re-identification model, the contour space attention module is used as the first student model and is distilled and learned with the first teacher model to construct the mask-guided attention loss in the contour flow; the exposure space attention module is used as the second student model and is distilled and learned with the second teacher model to construct the mask-guided attention loss in the exposure flow; facial information loss is constructed based on the first student model and the first teacher module and the second student model and the second teacher model; the first cross entropy loss and the second cross entropy loss are constructed based on the first contour embedding feature and the first exposure embedding feature, respectively, and the batch hard triplet loss is constructed based on the second contour embedding feature and the second exposure embedding feature, and the REID loss is constructed according to the first cross entropy loss, the second cross entropy loss and the batch hard triplet loss; the total loss function is constructed according to the mask-guided attention loss in the contour flow, the mask-guided attention loss in the exposure flow, the facial information loss and the REID loss, and the pedestrian re-identification model is trained based on the total loss function to obtain the trained pedestrian re-identification model.

[0105] In a specific embodiment, a pedestrian image is input into a first feature extraction module to obtain a first intermediate feature map, the first intermediate feature map is input into a contour space attention module to obtain a first attention map, the first attention map is applied to the first intermediate feature map to obtain a first fine feature map, the first fine feature map is input into a first MBN network to obtain a first contour embedding feature and a second contour embedding feature, specifically including:

[0106] The pedestrian image passes through the first feature extraction module, where the first feature extraction module uses OSNet to obtain the first intermediate feature map, as shown in the following formula:

[0107] ;

[0108] in, Represents the function corresponding to OSNet, represents a pedestrian image, Represents the first intermediate feature map, whose size is , , and Respectively represent the height, width and number of channels of the first intermediate feature map;

[0109] The first intermediate feature map passes through the contour space attention module to obtain the first attention map, as shown in the following formula:

[0110] ;

[0111] in, represents the convolution operator, represents a convolution kernel of size 1×1, represents the sigmoid activation function, represents the first attention map;

[0112] The first attention map is applied to the first intermediate feature map using the following formula to obtain the first refined feature map:

[0113] ;

[0114] in, represents the Hadamard matrix product, represents the first fine feature map;

[0115] First fine feature map After the first MBN network, the first contour embedding feature is obtained and the second contour embedding feature .

[0116] In a specific embodiment, the contour space attention module is used as the first student model and distilled with the first teacher model to construct the mask-guided attention loss in the contour flow, specifically including:

[0117] The pedestrian image is input into the first teacher model. The first teacher model includes a pre-trained HED network, an opening operator, a closing operator and a Gaussian filter connected in sequence. The pedestrian image passes through the pre-trained HED network to obtain the third intermediate feature map, as shown in the following formula:

[0118] ;

[0119] in, represents a pedestrian image, represents the third intermediate feature map, Represents the function corresponding to the pre-trained HED network;

[0120] The third intermediate feature map After passing through the opening operator, closing operator and Gaussian filter in sequence, the first mask feature map is obtained. The calculation formula is as follows:

[0121] ;

[0122] ;

[0123] ;

[0124] ;

[0125] in," " is the open operator corresponding to the open operator, " is the closing operator corresponding to the closed operator, B represents the 5×5 differential operator, O represents the output feature of the open operator, C represents the output feature of the closed operator, G(x1, x2,σ) represents the two-dimensional Gaussian function, x1 represents the horizontal coordinate, x2 represents the vertical coordinate, σ represents the standard deviation, represents the convolution operator, represents the first mask feature map;

[0126] Mask-guided Attention Loss in Contour Flow Defined as:

[0127] ;

[0128] Among them, i and j represent the horizontal and vertical coordinates of the pixel points in the first attention map or the first mask feature map, respectively. , and Respectively represent the height, width and number of channels of the first intermediate feature map, represents the first attention map.

[0129] Specifically, refer to Figure 3 , the embodiment of the present application constructs a contour flow and an exposure flow. For the contour flow, in the scenario of clothing change, clothing is an interference term in the CC-ReID problem. A simple method is to use the spatial attention mechanism to explicitly suppress clothing features. However, without auxiliary supervision, it is difficult to learn effective attention weights. Based on this, the embodiment of the present application designs a new clothing-independent contour spatial attention (CSAO) module to learn an effective attention map. Figure 4 As shown, given a pedestrian image , let it pass through OSNet to get the first intermediate feature map . Then, As the input of the CSAO module, the CSAO module includes a convolution layer with a convolution kernel of 1×1 and a sigmoid activation function. The CSAO module then generates a first attention map that is independent of clothing, and then converts the first attention map into Applied to the first intermediate feature map The first fine feature map is input into the first MBN network, which is an existing multi-branch network (MBN) that combines a multi-branch architecture based on global, part-based and channel features. According to existing research, the generated embeddings are divided into two groups: the first contour embedding feature and the second contour embedding feature .

[0130] First attention map It is expected that there will be lower scores in the clothing area. However, without additional auxiliary supervision, it is difficult to ensure that the CSAO module can produce attention scores that are independent of clothing. To this end, the embodiments of the present application introduce a mask-guided attention loss in the contour flow to enable the CSAO module to learn an effective attention map. Figure 4 As shown, in the first teacher model, a pedestrian image is first given Through the pre-trained HED network, the third intermediate feature map is obtained The pre-trained HED network belongs to the existing network, and its network structure is not described here. As input, it is sequentially passed through an opening operator, a closing operator and a Gaussian filter, and is opened, closed and Gaussian filtered respectively, so that the image can reduce noise while maintaining important features and structural information of the image as much as possible, so as to obtain a first mask feature map that is independent of clothing. Based on the first attention map and the first mask feature map, the mask-guided attention loss in the contour flow is constructed. The first student model and the first teacher model are subjected to knowledge distillation through the mask-guided attention loss in the contour flow, so that the first student model and the first teacher model after knowledge distillation also have the ability to obtain the first attention map that is independent of clothing.

[0131] In a specific embodiment, the pedestrian image is input into the second feature extraction module to obtain a second intermediate feature map, the second intermediate feature map is input into the exposure space attention module to obtain a second attention map, the second attention map is applied to the second intermediate feature map to obtain a second fine feature map, the second fine feature map is input into the second MBN network to obtain a first exposure embedding feature and a second exposure embedding feature, specifically including:

[0132] The pedestrian image passes through the second feature extraction module, where the second feature extraction module uses CNN to obtain the second intermediate feature map, as shown in the following formula:

[0133] ;

[0134] in, Represents the function corresponding to CNN, represents a pedestrian image, Represents the second intermediate feature map, whose size is , , and Respectively represent the height, width and number of channels of the second intermediate feature map;

[0135] The second intermediate feature map is passed through the exposed spatial attention module to obtain the second attention map, as shown in the following formula:

[0136] ;

[0137] in, represents the convolution operator, x represents the convolution kernel of size 1×1, represents the sigmoid activation function, represents the second attention map;

[0138] The second attention map is applied to the second intermediate feature map using the following formula to obtain the second refined feature map:

[0139] ;

[0140] in, represents the Hadamard matrix product, represents the second fine feature map;

[0141] The second fine feature map After the second MBN network, the second contour embedding feature is obtained and the second contour embedding feature .

[0142] In a specific embodiment, the exposure space attention module is used as the second student model and distilled with the second teacher model to construct the mask-guided attention loss in the exposure stream, specifically including:

[0143] The pedestrian image is input into the second teacher model. The second teacher model includes a pre-trained SAM encoder and a pre-trained SAM decoder connected in sequence. The pedestrian image is passed through the pre-trained SAM encoder to obtain the fourth intermediate feature map, as shown in the following formula:

[0144] ;

[0145] in, represents a pedestrian image, represents the fourth intermediate feature map, Represents the function corresponding to the pre-trained SAM encoder;

[0146] Fourth intermediate feature map After the pre-trained SAM decoder, the second mask feature map is obtained, and the calculation formula is as follows:

[0147] ;

[0148] in, Represents the function corresponding to the pre-trained SAM decoder, represents the second mask feature map;

[0149] Mask-guided attention loss in exposure stream Defined as:

[0150] ;

[0151] Among them, m and n represent the horizontal and vertical coordinates of the pixel points in the second attention map or the second mask feature map, respectively. , and Respectively represent the height, width and number of channels of the second intermediate feature map, represents the second attention map.

[0152] Specifically, for the exposure stream, similar to the contour stream, the embodiment of the present application also designs a new clothing-independent exposure spatial attention (CSAE) module for the exposure stream. Unlike the contour stream, the exposure stream focuses on the exposure area, and it will learn an effective attention map under the guidance of the SAM model, which includes a SAM encoder and a SAM decoder. Figure 5 As shown, given a pedestrian image , let it pass through the CNN network to get the second intermediate feature map .Will As the input of the CSAE module, the CSAE module then generates a second attention map that is independent of clothing. , the second attention map Applied to the second intermediate feature map The second fine feature map is then input into the second MBN network, which combines a multi-branch architecture based on global, part-based and channel features. According to existing research, the generated embeddings are divided into two groups: the second contour embedding feature and the second contour embedding feature .

[0153] Similar to the contour flow, the embodiment of the present application also introduces a mask-guided attention loss for the exposure flow to enable the CSAE module to learn an effective attention map. Figure 5 As shown, first a pedestrian image is given Through the pre-trained SAM encoder, the fourth intermediate feature map is obtained . The fourth intermediate feature map As input, a second mask feature map that is independent of clothing is obtained through the pre-trained SAM decoder. The mask-guided attention loss in the exposure stream is constructed based on the second attention map and the second mask feature map. The second student model and the second teacher model are subjected to knowledge distillation through the mask-guided attention loss in the exposure stream, so that the second student model and the second teacher model after knowledge distillation also have the ability to obtain the second attention map that is independent of clothing.

[0154] In a specific embodiment, the facial information loss for:

[0155] ;

[0156] Among them, KL represents Kullback-Leibler divergence, S represents Softmax function, τ represents parameter, represents the first mask feature map output by the first teacher model or the second mask feature map output by the second teacher model, represents the first attention map output by the first student model or the second attention map output by the second student model, k represents the kth category, then , N represents the total number of categories, Represents the difference between two probability distributions;

[0157] First cross entropy loss for:

[0158] ;

[0159] Among them, CE represents the cross entropy loss, y represents the true identity label of the pedestrian image, represents the predicted identity label of the pedestrian image corresponding to the first contour embedding feature;

[0160] Second cross entropy loss for:

[0161] ;

[0162] in, The predicted identity label of the pedestrian image corresponding to the first exposed embedded feature;

[0163] Batch Hard Triplet Loss for:

[0164] ;

[0165] Wherein, Trip represents the batch hard triplet loss function, f1 represents the predicted identity label of the pedestrian image corresponding to the second contour embedding feature, and f2 represents the predicted identity label of the pedestrian image corresponding to the second exposure embedding feature;

[0166] REID loss for:

[0167] ;

[0168] Total loss function for:

[0169] ;

[0170] Among them, λ1, λ2, λ3 and λ4 represent the first weight, the second weight, the third weight and the fourth weight respectively.

[0171] Specifically, facial information plays an important role in the pedestrian ReID problem. In order to obtain more accurate facial information, an embodiment of the present application uses a facial restoration method provided by Deskpro to restore facial details, and constructs facial information loss based on the first mask feature map output by the first teacher model or the second mask feature map output by the second teacher model and the first attention map output by the first student model or the second attention map output by the second student model.

[0172] In addition to the above losses, the embodiments of the present application also utilize All features in optimize the batch hard triplet loss function, and use All features in the image are optimized for cross entropy loss function to achieve basic discriminant learning, so as to construct the loss and total loss function of traditional ReID. The whole framework optimizes the total loss function The first weight, the second weight, the third weight, and the fourth weight are used to balance different losses.

[0173] S3, using the trained pedestrian re-identification model in combination with the pedestrian image to be queried and the pedestrian image to be searched in the image set to perform pedestrian re-identification, and obtain a pedestrian re-identification result.

[0174] In a specific embodiment, step S3 specifically includes:

[0175] Inputting the queried pedestrian image and the pedestrian image to be searched in the image set into the trained pedestrian re-identification model respectively to obtain the query feature and the search feature;

[0176] In response to determining that a pedestrian in the exposed area is detected in the queried pedestrian image and / or the pedestrian image to be searched, a first embedding vector obtained before the last fully connected layer in the first MBN network of the contour unit and a second embedding vector obtained before the last fully connected layer in the second MBN network of the exposure unit are added to obtain a query feature and / or a search feature;

[0177] In response to determining that no pedestrian in the exposed area is detected in the queried pedestrian image and / or the pedestrian image to be searched, a first embedding vector obtained before the last fully connected layer in the first MBN network of the contour unit is used as a query feature and / or a search feature;

[0178] The similarity between the query feature and each search feature is calculated, and the identity label corresponding to the search feature with the greatest similarity to the query feature is used as the identity information of the target pedestrian in the pedestrian image to be queried.

[0179] Specifically, in the inference stage, the first teacher model in the contour stream and the second teacher model in the exposure stream are removed. This process can be regarded as a retrieval process based on cosine similarity. In order to further enhance the sensitivity of the model to identity information recognition, the embodiment of the present application proposes an exposure stream to extract the exposed area of ​​the pedestrian, which effectively increases the pedestrian identity sensitive information. Combining the two streams to obtain a mask, such as Figure 1 As shown in Mask No. 4 in the figure, the mask-guided attention mechanism not only eliminates the interference of clothing on the pedestrian ReID problem, but also retains the identity-sensitive information of the exposed area of ​​the pedestrian.

[0180] In one embodiment, the embodiment of the present application may use the SAM model to detect whether there is an exposed area in the pedestrian image to be queried. If the score output by the SAM model is greater than or equal to the threshold, it is determined that the exposed area is detected, and the first embedding vector and the second embedding vector obtained before the last fully connected layer in the first MBN network and the second MBN network are connected to perform similarity calculation. In addition, if the score output by the SAM model is less than the threshold, it is determined that the exposed area is not detected. For pedestrian images without detected exposed areas, only the contour flow is enabled to calculate the feature vector, that is, the first embedding vector corresponding to the pedestrian image to be queried and the first embedding vector corresponding to the pedestrian image to be searched are used to perform similarity calculation, as shown in the following formula:

[0181] ;

[0182] in, represents the function corresponding to the pedestrian re-identification model, and X represents the pedestrian image to be queried.

[0183] In other embodiments, other models may be used to detect whether there is an exposed area. Detecting whether there is an exposed area is not the focus of this application, and the model is not limited.

[0184] Further references Figure 6 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a pedestrian re-identification device based on a mask-guided dual-stream network. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0185] The embodiment of the present application provides a pedestrian re-identification device based on a mask-guided dual-stream network, comprising:

[0186] A data acquisition module 1 is configured to acquire a pedestrian image to be queried that includes a target pedestrian and an image set that includes at least one pedestrian image to be searched;

[0187] The model building module 2 is configured to build a pedestrian re-identification model, a first teacher model and a second teacher model. The pedestrian re-identification model includes a contour unit and an exposure unit. The contour unit includes a first feature extraction module, a contour space attention module and a first MBN network. The pedestrian image is input into the first feature extraction module to obtain a first intermediate feature map. The first intermediate feature map is input into the contour space attention module to obtain a first attention map. The first attention map is applied to the first intermediate feature map to obtain a first fine feature map. The first fine feature map is input into the first MBN network to obtain a first contour embedding feature and a second contour embedding feature; the exposure unit includes a second feature extraction module, an exposure space attention module and a second MBN network. The pedestrian image is input into the second feature extraction module to obtain a second intermediate feature map. The second intermediate feature map is input into the exposure space attention module to obtain a second attention map. The second attention map is applied to the second intermediate feature map to obtain a second fine feature map. The second fine feature map is input into the second MBN network to obtain the first exposure embedding feature and the second exposure embedding feature; in the training process of the pedestrian re-identification model, the contour space attention module is used as the first student model and is distilled and learned with the first teacher model to construct the mask-guided attention loss in the contour flow; the exposure space attention module is used as the second student model and is distilled and learned with the second teacher model to construct the mask-guided attention loss in the exposure flow; facial information loss is constructed based on the first student model and the first teacher module and the second student model and the second teacher model; a first cross entropy loss and a second cross entropy loss are constructed based on the first contour embedding feature and the first exposure embedding feature, respectively, and a batch hard triplet loss is constructed based on the second contour embedding feature and the second exposure embedding feature, and a REID loss is constructed according to the first cross entropy loss, the second cross entropy loss and the batch hard triplet loss; a total loss function is constructed according to the mask-guided attention loss in the contour flow, the mask-guided attention loss in the exposure flow, the facial information loss and the REID loss, and the pedestrian re-identification model is trained based on the total loss function to obtain a trained pedestrian re-identification model;

[0188] The re-identification module 3 is configured to perform pedestrian re-identification by using the trained pedestrian re-identification model in combination with the pedestrian image to be queried and the pedestrian image to be searched in the image set to obtain a pedestrian re-identification result.

[0189] Figure 7 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Figure 7As shown, the electronic device of this embodiment includes: a processor 701 and a memory 702; wherein the memory 702 is used to store computer-executable instructions; the processor 701 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0190] Optionally, the memory 702 may be independent or integrated with the processor 701 .

[0191] When the memory 702 is independently provided, the electronic device further includes a bus 703 for connecting the memory 702 and the processor 701 .

[0192] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 701 executes the computer execution instructions, the above method is implemented.

[0193] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 701, the above method is implemented.

[0194] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0195] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0196] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned module-composed unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0197] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 701 to perform some steps of the methods of various embodiments of the present application.

[0198] It should be understood that the processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 701 may be any conventional processor 701, etc. The steps of the method disclosed in the invention may be directly embodied in the execution of the hardware processor 701, or may be executed by a combination of hardware and software modules in the processor 701.

[0199] The memory 702 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0200] The bus 703 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 703 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 703 in the drawings of the present application is not limited to only one bus 703 or one type of bus 703.

[0201] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0202] An exemplary storage medium is coupled to the processor 701, so that the processor 701 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 701. The processor 701 and the storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as ASIC). Of course, the processor 701 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0203] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A person re-identification method based on mask-guided two-stream network, characterized in that: The following steps are involved: Obtaining a pedestrian image to be queried containing a target pedestrian and an image set containing at least one pedestrian image to be searched; A pedestrian re-identification model, a first teacher model and a second teacher model are constructed, wherein the pedestrian re-identification model includes a contour unit and an exposure unit, wherein the contour unit includes a first feature extraction module, a contour space attention module and a first MBN network, wherein the pedestrian image is input into the first feature extraction module to obtain a first intermediate feature map, wherein the first intermediate feature map is input into the contour space attention module to obtain a first attention map, wherein the first attention map is applied to the first intermediate feature map to obtain a first fine feature map, wherein the first fine feature map is input into the first MBN network to obtain a first contour embedding feature and a second contour embedding feature, specifically including: The pedestrian image passes through the first feature extraction module, wherein the first feature extraction module uses OSNet to obtain a first intermediate feature map, as shown in the following formula: F out =OS(X g ); Among them, os(·) represents the function corresponding to OSNet, X g represents the pedestrian image, F out Represents the first intermediate feature map, whose size is and Respectively represent the height, width and number of channels of the first intermediate feature map; The first intermediate feature map is passed through the contour space attention module to obtain a first attention map, as shown in the following formula: Among them, * represents the convolution operator, x represents the convolution kernel of size 1×1, sigmoid(·) represents the sigmoid activation function, represents the first attention map; The first attention map is applied to the first intermediate feature map using the following formula to obtain a first refined feature map: in, represents the Hadamard matrix product, F out_att represents the first fine feature map; The first fine feature map F out_att After the first MBN network, the first contour embedding feature is obtained and the second contour embedding feature The exposure unit includes a second feature extraction module, an exposure space attention module and a second MBN network. The pedestrian image is input into the second feature extraction module to obtain a second intermediate feature map. The second intermediate feature map is input into the exposure space attention module to obtain a second attention map. The second attention map is applied to the second intermediate feature map to obtain a second fine feature map. The second fine feature map is input into the second MBN network to obtain a first exposure embedding feature and a second exposure embedding feature. During the training process of the pedestrian re-identification model, the contour space attention module is used as the first student model and is distilled with the first teacher model to construct an attention loss guided by a mask in the contour flow. The exposure space attention module is used as the second student model and is distilled with the second teacher model. The model performs distillation learning to construct an attention loss guided by a mask in an exposure stream; constructs a facial information loss based on a first student model and a first teacher module and a second student model and a second teacher model; constructs a first cross entropy loss and a second cross entropy loss based on the first contour embedding feature and the first exposure embedding feature, respectively, constructs a batch hard triplet loss based on the second contour embedding feature and the second exposure embedding feature, and constructs a REID loss based on the first cross entropy loss, the second cross entropy loss, and the batch hard triplet loss; constructs a total loss function based on the attention loss guided by a mask in the contour stream, the attention loss guided by a mask in the exposure stream, the facial information loss, and the REID loss, and trains the pedestrian re-identification model based on the total loss function to obtain a trained pedestrian re-identification model; The trained pedestrian re-identification model is used in combination with the pedestrian image to be queried and the pedestrian image to be searched in the image set to perform pedestrian re-identification to obtain a pedestrian re-identification result.

2. The method for person re-identification based on mask-guided dual-stream network according to claim 1, characterized in that: The contour space attention module is used as the first student model and distilled with the first teacher model to construct a mask-guided attention loss in the contour flow, specifically including: The pedestrian image is input into the first teacher model, and the first teacher model includes a pre-trained HED network, an open operator, a closed operator and a Gaussian filter connected in sequence. The pedestrian image passes through the pre-trained HED network to obtain a third intermediate feature map, as shown in the following formula: G out =H(X g ); Among them, X g represents the pedestrian image, G out represents the third intermediate feature map, H(·) represents the function corresponding to the pre-trained HED network; The third intermediate feature map G out After passing through the opening operator, closing operator and Gaussian filter in sequence, the first mask feature map is obtained, and the calculation formula is as follows: C = O·B; in, is the opening operator corresponding to the opening operator, "·" is the closing operator corresponding to the closing operator, B represents a 5×5 differential operator, O represents the output feature of the opening operator, C represents the output feature of the closing operator, G(x1,x2,σ) represents a two-dimensional Gaussian function, x1 represents the horizontal coordinate, x2 represents the vertical coordinate, σ represents the standard deviation, * represents the convolution operator, represents the first mask feature map; Mask-guided attention loss in the contour flow Defined as: Wherein, i and j represent the horizontal coordinate and vertical coordinate of the pixel point in the first attention map or the first mask feature map, respectively. and Respectively represent the height, width and number of channels of the first intermediate feature map, Denotes the first attention map.

3. The method for person re-identification based on mask-guided dual-stream network according to claim 1, characterized in that: The pedestrian image is input into the second feature extraction module to obtain a second intermediate feature map, the second intermediate feature map is input into the exposure space attention module to obtain a second attention map, the second attention map is applied to the second intermediate feature map to obtain a second fine feature map, the second fine feature map is input into the second MBN network to obtain a first exposure embedding feature and a second exposure embedding feature, specifically including: The pedestrian image passes through the second feature extraction module, wherein the second feature extraction module adopts CNN to obtain a second intermediate feature map, as shown in the following formula: F exp =CN(X g ); Among them, Cn(·) represents the function corresponding to CNN, X g represents the pedestrian image, F exp Represents the second intermediate feature map, whose size is and Respectively represent the height, width and number of channels of the second intermediate feature map; The second intermediate feature map passes through the exposed spatial attention module to obtain a second attention map, as shown in the following formula: Among them, * represents the convolution operator, x represents the convolution kernel of size 1×1, sigmoid(·) represents the sigmoid activation function, represents the second attention map; The second attention map is applied to the second intermediate feature map using the following formula to obtain a second refined feature map: in, represents the Hadamard matrix product, F exp_att represents the second fine feature map; The second fine feature map F exp_att After the second MBN network, the first exposed embedding feature is obtained and the second exposed embedded feature 4. The method for person re-identification based on mask-guided dual-stream network according to claim 1, characterized in that: The exposure spatial attention module is used as the second student model and distilled with the second teacher model to construct a mask-guided attention loss in the exposure stream, specifically including: The pedestrian image is input into the second teacher model, and the second teacher model includes a pre-trained SAM encoder and a pre-trained SAM decoder connected in sequence. The pedestrian image passes through the pre-trained SAM encoder to obtain a fourth intermediate feature map, as shown in the following formula: G exp =E(X g ); Among them, X g represents the pedestrian image, G exp represents the fourth intermediate feature map, E(·) represents the function corresponding to the pre-trained SAM encoder; The fourth intermediate feature map G exp After the pre-trained SAM decoder, the second mask feature map is obtained, and the calculation formula is as follows: Where D(·) represents the function corresponding to the pre-trained SAM decoder, represents the second mask feature map; Mask-guided attention loss in the exposure stream Defined as: Wherein, m and n represent the horizontal coordinate and vertical coordinate of the pixel point in the second attention map or the second mask feature map, respectively. and Respectively represent the height, width and number of channels of the second intermediate feature map, Denotes the second attention map.

5. The method for person re-identification based on mask-guided dual-stream network according to claim 1, characterized in that: The facial information loss for: Among them, KL represents Kullback-Leibler divergence, S represents Softmax function, τ represents parameter, represents the first mask feature map output by the first teacher model or the second mask feature map output by the second teacher model, represents the first attention map output by the first student model or the second attention map output by the second student model, k represents the kth category, then k∈[1,N], N represents the total number of categories, and |·| represents the difference between the two probability distributions; The first cross entropy loss for: Among them, CE represents the cross entropy loss function, y represents the true identity label of the pedestrian image, represents the predicted identity label of the pedestrian image corresponding to the first contour embedding feature, represents the first contour embedding feature; The second cross entropy loss for: in, represents the predicted identity label of the pedestrian image corresponding to the first exposed embedded feature, represents the first exposed embedded feature; The batch hard triplet loss for: Wherein, Trip represents a batch hard triplet loss function, f1 represents the predicted identity label of the pedestrian image corresponding to the second contour embedding feature, and f2 represents the predicted identity label of the pedestrian image corresponding to the second exposure embedding feature; The REID loss for: The total loss function for: Among them, λ1, λ2, λ3 and λ4 represent the first weight, the second weight, the third weight and the fourth weight respectively.

6. The method for person re-identification based on mask-guided dual-stream network according to claim 1, characterized in that: The trained person re-identification model is used in combination with the pedestrian image to be queried and the pedestrian image to be searched in the image set to perform pedestrian re-identification to obtain a pedestrian re-identification result, specifically including: Inputting the pedestrian image to be queried and the pedestrian image to be searched in the image set into the trained pedestrian re-identification model respectively to obtain query features and search features; In response to determining that a pedestrian in the exposed area is detected in the pedestrian image to be queried and the pedestrian image to be searched, a first embedding vector obtained before the last fully connected layer in the first MBN network of the contour unit and a second embedding vector obtained before the last fully connected layer in the second MBN network of the exposure unit are added to obtain the query feature and the search feature; In response to determining that no pedestrian in the exposed area is detected in the pedestrian image to be queried and the pedestrian image to be searched, taking the first embedding vector obtained before the last fully connected layer in the first MBN network of the contour unit as the query feature and the search feature; The similarity between the query feature and each search feature is calculated, and the identity label corresponding to the search feature having the greatest similarity with the query feature is used as the identity information of the target pedestrian in the pedestrian image to be queried.

7. A pedestrian re-identification device based on mask-guided dual-stream network, characterized in that: include: A data acquisition module is configured to acquire a pedestrian image to be queried containing a target pedestrian and an image set containing at least one pedestrian image to be searched; The model building module is configured to build a pedestrian re-identification model, a first teacher model and a second teacher model. The pedestrian re-identification model includes a contour unit and an exposure unit. The contour unit includes a first feature extraction module, a contour space attention module and a first MBN network. The pedestrian image is input into the first feature extraction module to obtain a first intermediate feature map. The first intermediate feature map is input into the contour space attention module to obtain a first attention map. The first attention map is applied to the first intermediate feature map to obtain a first fine feature map. The first fine feature map is input into the first MBN network to obtain a first contour embedding feature and a second contour embedding feature. Specifically, it includes: The pedestrian image passes through the first feature extraction module, wherein the first feature extraction module uses OSNet to obtain a first intermediate feature map, as shown in the following formula: F out =OS(X g ); Among them, OS(·) represents the function corresponding to OSNet, X g represents the pedestrian image, F out Represents the first intermediate feature map, whose size is and Respectively represent the height, width and number of channels of the first intermediate feature map; The first intermediate feature map is passed through the contour space attention module to obtain a first attention map, as shown in the following formula: Among them, * represents the convolution operator, x represents the convolution kernel of size 1×1, sigmoid(·) represents the sigmoid activation function, represents the first attention map; The first attention map is applied to the first intermediate feature map using the following formula to obtain a first refined feature map: in, represents the Hadamard matrix product, F out_att represents the first fine feature map; The first fine feature map F out_ott After the first MBN network, the first contour embedding feature is obtained and the second contour embedding feature The exposure unit includes a second feature extraction module, an exposure space attention module and a second MBN network. The pedestrian image is input into the second feature extraction module to obtain a second intermediate feature map. The second intermediate feature map is input into the exposure space attention module to obtain a second attention map. The second attention map is applied to the second intermediate feature map to obtain a second fine feature map. The second fine feature map is input into the second MBN network to obtain a first exposure embedding feature and a second exposure embedding feature. During the training process of the pedestrian re-identification model, the contour space attention module is used as the first student model and is distilled with the first teacher model to construct an attention loss guided by a mask in the contour flow. The exposure space attention module is used as the second student model and is distilled with the second teacher model. The model performs distillation learning to construct an attention loss guided by a mask in an exposure stream; constructs a facial information loss based on a first student model and a first teacher module and a second student model and a second teacher model; constructs a first cross entropy loss and a second cross entropy loss based on the first contour embedding feature and the first exposure embedding feature, respectively, constructs a batch hard triplet loss based on the second contour embedding feature and the second exposure embedding feature, and constructs a REID loss based on the first cross entropy loss, the second cross entropy loss, and the batch hard triplet loss; constructs a total loss function based on the attention loss guided by a mask in the contour stream, the attention loss guided by a mask in the exposure stream, the facial information loss, and the REID loss, and trains the pedestrian re-identification model based on the total loss function to obtain a trained pedestrian re-identification model; The re-identification module is configured to use the trained pedestrian re-identification model in combination with the pedestrian image to be queried and the pedestrian image to be searched in the image set to perform pedestrian re-identification to obtain a pedestrian re-identification result.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images