Pedestrian re-identification method and device

Abstract

The present disclosure relates to the technical field of computers, and provides a pedestrian re-identification method and device. The method comprises: obtaining pedestrian image data to be identified; inputting the pedestrian image data into a preset pedestrian re-identification model to obtain first description features of the pedestrian image data output by the pedestrian re-identification model, wherein the description features of sample image data in a training data set used for training the pedestrian re-identification model are obtained through normalization style removal processing and context content recovery processing; and determining a pedestrian re-identification result according to the first description features and in-database description features in an image database.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for pedestrian re-identification. Background Technology

[0002] In existing person re-identification (ReID) algorithms, the same dataset is generally used for both training and testing of the person re-identification model. However, most person re-identification technologies are characterized by varying poses and viewpoints. Therefore, after changing the test set, the recognition performance of person re-identification models lacking domain adaptation capabilities will drop significantly.

[0003] Unsupervised domain adaptation methods can help pedestrian re-identification models adapt from the source domain to the target domain. These methods use unlabeled target domain data for model updates, but still require data collection and model updates, increasing the cost of pedestrian re-identification.

[0004] Improving the domain adaptability of pedestrian re-identification models is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present disclosure provide a pedestrian re-identification method, apparatus, electronic device, and computer-readable storage medium to solve the problem that pedestrian re-identification models in the prior art lack domain adaptation capabilities.

[0006] A first aspect of this disclosure provides a pedestrian re-identification method, the method comprising: acquiring pedestrian image data to be identified; inputting the pedestrian image data into a preset pedestrian re-identification model to obtain a first descriptive feature of the pedestrian image data output by the pedestrian re-identification model, wherein the descriptive features of sample image data in the training dataset used to train the pedestrian re-identification model are obtained through normalized style removal processing and context content restoration processing; and determining a pedestrian re-identification result based on the first descriptive feature and in-database descriptive features in an image database.

[0007] A second aspect of this disclosure provides a pedestrian re-identification apparatus, comprising: an acquisition module for acquiring pedestrian image data to be identified; an identification module for inputting the pedestrian image data into a preset pedestrian re-identification model to obtain a first descriptive feature of the pedestrian image data output by the pedestrian re-identification model, wherein the descriptive features of sample image data in the training dataset used to train the pedestrian re-identification model are obtained through normalized style removal processing and contextual content restoration processing; and a determination module for determining a pedestrian re-identification result based on the first descriptive feature and in-database descriptive features in an image database.

[0008] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0009] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0010] The beneficial effects of this embodiment compared with the prior art are: by performing normalization style removal processing and context content restoration processing on the descriptive features of sample image data, the style differences between training data in different domains in the training dataset are eliminated, so that the trained person re-identification model has domain adaptation capability, thereby making the person re-identification results obtained by applying the person re-identification model more accurate.

[0011] Specifically, this disclosure proposes a person re-identification algorithm based on style removal and context recovery. It first removes style features from different domains of the original image, then recovers the information lost due to style removal. Simultaneously, it learns useful and useless information from this lost data, thus solving the domain adaptation problem in the field of person re-identification. Furthermore, the technical solution of this disclosure employs dual loss to constrain the network's learning of useful feature information in the input image for the person re-identification task. This results in image pairs with the same ID having a smaller distance after feature enhancement, while image pairs with different IDs have a larger distance. Image pairs with the same ID have a larger distance after feature weakening, while image pairs with different IDs have a smaller distance, thereby enhancing the domain adaptation capability of the person re-identification model. Attached Figure Description To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure; Figure 2 This is a flowchart illustrating a pedestrian re-identification method provided in an embodiment of this disclosure; Figure 3 This is a schematic flowchart illustrating the processing of descriptive features of sample image data according to an embodiment of this disclosure; Figure 4This is a schematic diagram of another process for processing descriptive features of sample image data provided in this embodiment of the disclosure; Figure 5 This is a schematic diagram of the structure of a pedestrian re-identification device provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0013] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0014] The pedestrian re-identification method and apparatus according to embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0015] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure. The application scenario may include terminal devices 101, 102, and 103, server 104, and network 105.

[0016] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays and supporting communication with server 104, including but not limited to smartphones, robots, laptops, and desktop computers (for example, 102 can be a robot). When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. Terminal devices 101, 102, and 103 can be implemented as multiple software programs or software modules, or as a single software program or software module; this disclosure does not limit this. Furthermore, various applications can be installed on terminal devices 101, 102, and 103, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0017] Server 104 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. This backend server can receive and analyze the requests sent by the terminal devices and generate processing results. Server 104 can be a single server, a server cluster consisting of several servers, or a cloud computing service center. This embodiment of the disclosure does not impose any limitations on these aspects.

[0018] It should be noted that server 104 can be either hardware or software. When server 104 is hardware, it can be various electronic devices that provide various services to terminal devices 101, 102, and 103. When server 104 is software, it can be multiple software programs or software modules that provide various services to terminal devices 101, 102, and 103, or it can be a single software program or software module that provides various services to terminal devices 101, 102, and 103. This disclosure does not limit the scope of the embodiments.

[0019] Network 105 can be a wired network using coaxial cable, twisted pair, and fiber optic connection, or it can be a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), Infrared, etc. This disclosure does not limit the scope of the network.

[0020] Target users can establish a communication connection with server 104 via network 105 through terminal devices 101, 102, and 103 to receive or send information, etc. It should be noted that the specific types, quantities, and combinations of terminal devices 101, 102, and 103, server 104, and network 105 can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not impose any limitations on this.

[0021] Figure 2 This is a flowchart illustrating a pedestrian re-identification method provided in an embodiment of this disclosure. The method provided in this embodiment can be executed by any electronic device with computer processing capabilities, such as a terminal or server. Figure 2 As shown, the pedestrian re-identification method includes: Step S201: Obtain the pedestrian image data to be identified.

[0022] Specifically, the pedestrian image data to be identified is the image data prepared for pedestrian re-identification. Pedestrian re-identification (ReID) is a technique that uses computer vision technology to determine whether a specific pedestrian or a specific category of pedestrian exists in an image or video sequence.

[0023] Step S202: Input pedestrian image data into a preset pedestrian re-identification model to obtain the first descriptive feature of the pedestrian image data output by the pedestrian re-identification model. The descriptive features of the sample image data in the training dataset used to train the pedestrian re-identification model are obtained through normalization style removal processing and context content restoration processing. The first descriptive feature is the descriptive feature of the pedestrian image data.

[0024] Specifically, the pedestrian re-identification model can extract features from pedestrian image data, compare the extracted features with the descriptive features in the image database, and obtain the similarity between the extracted features and the descriptive features in the database, thereby obtaining the pedestrian re-identification result.

[0025] Step S203: Determine the pedestrian re-identification result based on the first descriptive feature and the in-database descriptive features in the image database.

[0026] Specifically, the purpose of pedestrian re-identification is to classify or identify pedestrians. The in-database descriptive features provided in this embodiment can be a target pedestrian of a certain category or a specified target pedestrian. Based on the similarity between the first descriptive feature and the in-database descriptive features in the image database, pedestrians in the current pedestrian image to be identified can be further classified, or it can be determined whether the current pedestrian image to be identified contains a target pedestrian.

[0027] The pedestrian re-identification algorithm provided in this embodiment is a pedestrian re-identification algorithm based on style removal processing and context restoration processing. The pedestrian re-identification algorithm first eliminates style differences between different domains through style removal processing, and then performs context restoration processing on the style-removed features to restore the lost image content. In this way, the domain-independent pedestrian features can be extracted to form training data to train the pedestrian re-identification model, so that the trained pedestrian re-identification model has domain adaptation capability. Applying this pedestrian re-identification model to perform pedestrian re-identification can obtain relatively accurate pedestrian re-identification results.

[0028] In this embodiment of the disclosure, when training the pedestrian re-identification model, the descriptive features of the original sample image data can be processed first, and the loss function value can be calculated based on the processed data group. The initial model can be trained based on the loss function value until the loss function converges to obtain the pedestrian re-identification model.

[0029] Specifically, the descriptive features of the original sample image data are processed, including the following steps: Normalization style removal and contextual content restoration are sequentially performed on the first, second, and third sample image data to obtain processed data sets, and loss function values ​​are obtained based on these processed data sets. The first, second, and third sample image data are sample image data from the same batch in the training dataset. The second sample image data is similar to the first sample image data, and the third sample image data is dissimilar to the first sample image data. Similar sample image data refers to sample image data with the same ID as the first sample image data (i.e., containing the same pedestrian), while similar sample image data refers to sample image data with different IDs than the first sample image data (i.e., containing different pedestrians).

[0030] like Figure 3 As shown, when processing the descriptive features of sample image data, the normalization style removal and contextual content restoration steps include: Step S301: Obtain the second descriptive feature of the current sample image data. The second descriptive feature is the descriptive feature of the current sample image data.

[0031] Step S302: Input the second descriptive feature into the residual network for normalization style removal processing to obtain the third descriptive feature.

[0032] Step S303: The difference between the second descriptive feature and the third descriptive feature is used as input and input into the first attention branch and the second attention branch of the convolutional neural network respectively to obtain the output features of the first useful branch and the output features of the first useless branch.

[0033] Step S304: Add the first useful branch output feature and the first useless branch output feature to the third descriptive feature respectively to obtain the second useful branch output feature and the second useless branch output feature, and add the second useful branch output feature and the second useless branch output feature to the processing data group.

[0034] Specifically, steps S301 to S304 can be performed using the first, second, and third sample image data as the current sample image data to obtain the second useful branch output features and the second useless branch output features corresponding to the three sample image data respectively. The processed data in the processed data group are the second useful branch output features and the second useless branch output features corresponding to the three sample image data respectively. The loss function value can be calculated using the second useful branch output features and the second useless branch output features corresponding to the three sample image data respectively.

[0035] In step S301, the residual network includes a residual branch, a short-circuit branch, and three 3×3 convolutional kernels. The first convolution of the residual branch is a 1×1 convolution with a stride of 1, and the second convolution is a 3×3 convolution with a stride of 2. An average pooling layer is set before the convolutional layer of the short-circuit branch.

[0036] Specifically, the residual network in this embodiment is an improvement upon the ResNet34 network. ResNet34 is a convolutional neural network whose scaled-down backbone includes a 7×7 convolutional layer with a stride of 2. Since the computational cost of convolution is the square of its length and width, the 7×7 convolutional structure is 5.4 times more computationally expensive than a 3×3 convolutional structure. To extract richer features, this embodiment replaces the 7×7 convolutional kernel with three traditional 3×3 convolutional kernels, reducing computational cost while maintaining the same receptive field as the 7x7 kernel.

[0037] In the original ResNet34 network, the residual branch contains a 1×1 convolution, a 3×3 convolution, and a 1×1 convolution, with the first 1×1 convolution having a stride of 2 and the second 3×3 convolution having a stride of 1. To avoid information loss, the technical solution of this disclosure swaps these two strides, setting the stride of the first 1×1 convolution to 1 and the stride of the second 3×3 convolution to 2.

[0038] The original ResNet34 network has a 1×1 convolution with a stride of 2 in the short-circuit branch. In order to further reduce the information loss of sample image data in the residual branch, the technical solution of this disclosure replaces the 1×1 convolution with a stride of 2 in the short-circuit branch with an average pooling layer and a 1×1 convolution.

[0039] In step S302, the second descriptive feature can be processed using the following normalization formula (1): (1) Where x is the input feature, IN(x) is the output feature, μ(x) is the mean, σ(x) is the variance, β is the translation factor, and γ is the scaling factor.

[0040] Specifically, the technical solution of this disclosure employs Instance Normalization (IN) to remove style differences between different domains. In computer vision, IN is essentially a style normalization that can unify different images into a single style. Instance normalization can remove the domain style of the descriptive features of the original image.

[0041] Specifically, in step S303, contextual content recovery is performed on the difference between the second descriptive feature and the third descriptive feature, which can recover the content information lost in the style removal stage.

[0042] like Figure 4As shown, the second descriptive feature F is input into the residual network and then into the normalized style removal module IN for normalized style removal processing, resulting in the third descriptive feature F_in. Subtracting the second descriptive feature F from the third descriptive feature F_in yields the difference F_res. This difference F_res is then input into different attention branches of the convolutional neural network, namely the first attention branch and the second attention branch, to obtain the first useful branch output feature F_s1 and the first useless branch output feature F_s2. Adding the first useful branch output feature F_s1 and the first useless branch output feature F_s2 to the style-removed third descriptive feature F_in yields the second useful branch output feature F_p and the second useless branch output feature F_n.

[0043] Specifically, the difference can represent the information lost during the instance normalization phase. Step S303, based on the difference, can recover useful contextual content. Here, the first attention branch is used to extract useful content information, and the second attention branch is used to extract useless content information. Both the first and second attention branches can be simple SE structures.

[0044] The SE structure can include two fully connected layers and an activation function layer connected in sequence. To accelerate computation, the fully connected layers can be designed as 1×1 convolutional structures.

[0045] In this embodiment, each of the first attention branch and the second attention branch includes a first fully connected layer, a second fully connected layer, and an activation function layer. Both the first and second fully connected layers are 1×1 convolutional structures. In step S303, after the difference between the second descriptive feature and the third descriptive feature is input into the first attention branch of the convolutional neural network, the difference is processed using the first fully connected layer, the second fully connected layer, and the activation function layer of the first attention branch. Similarly, after the difference between the second descriptive feature and the third descriptive feature is input into the second attention branch of the convolutional neural network, the difference is processed using the first fully connected layer, the second fully connected layer, and the activation function layer of the second attention branch.

[0046] The first useful branch output feature F_s1 can represent useful content information, and the first useless branch output feature F_s2 can represent useless content information. By adding the first useful branch output feature F_s1 and the first useless branch output feature F_s2 to the style-removed third descriptive feature F_in respectively, we can obtain the useful and useless information extracted from the original image. Specifically, the second useful branch output feature F_p and the second useless branch output feature F_n represent the useful and useless information extracted from the original image, respectively. Among them, the second useful branch output feature F_p is the enhanced original feature, which is more discriminative, while the second useless branch output feature F_n is the weakened original feature, which is more difficult to distinguish.

[0047] To constrain the network to learn useful feature information from the input images for the person re-identification task, in this embodiment of the disclosure, during the training of the person re-identification model, three images can be extracted from the same batch of the training dataset for processing at once. These three images are: the original image a, a similar sample image p, and a dissimilar sample image n. The similar sample image refers to a sample image with the same ID as the original image, and the dissimilar sample image refers to a sample image with a different ID than the original image.

[0048] Specifically, in this embodiment, the loss function corresponding to the loss function value includes a pedestrian re-identification loss component and a dual loss component. The calculation formula for the dual loss component includes the following formula (2): L dual = ( ( + , + ) ( , )) + ( ( , ) ( + , + (2) in, ( ) = ln(1 + exp( )), For distance, + The useful branch output features are the first sample image data. + The useful branch output features are the data corresponding to the second sample image data. The useless branch output features are the first sample image data. - The useless branch output features are the second sample image data. n - The useless branch output features are the third sample image data. n + The useful branch output features are the third sample image data.

[0049] In this embodiment of the disclosure, when obtaining the loss function value based on the processed data set, it is necessary to obtain the loss function value of the dual loss component based on the processed data set, including: obtaining the first distance between the second useful branch output feature corresponding to the first sample image data and the second useful branch output feature corresponding to the second sample image data. ( + , + ); Obtain the second distance between the second useless branch output feature corresponding to the first sample image data and the second useless branch output feature corresponding to the second sample image data. ( , ); Obtain the third distance between the output features of the second useless branch corresponding to the first sample image data and the output features of the second useless branch corresponding to the third sample image data. ( , ); Obtain the fourth distance between the second useful branch output features corresponding to the first sample image data and the second useful branch output features corresponding to the third sample image data. ( + , + The dual loss function value can be obtained based on the first distance, second distance, third distance, and fourth distance; the loss function value can be obtained based on the re-identification loss function value and the dual loss function value.

[0050] In this embodiment of the disclosure, the final total loss function formula is: L total = L ReID + L dual L total For the total loss value, L ReID L is the loss value for the pedestrian re-identification loss portion. dual This represents the loss value of the dual loss component. In this embodiment of the disclosure, the loss function for the pedestrian re-identification loss component can be a triplet loss function, but is not limited to this.

[0051] This disclosure proposes a person re-identification algorithm based on style removal and context recovery. It first removes style features from different domains of the original image, then recovers the information lost due to style removal. Simultaneously, it learns useful and useless information from this lost data, thus solving the domain adaptation problem in the field of person re-identification. Furthermore, the technical solution of this disclosure employs dual loss to constrain the feature information useful for person re-identification in the input image learned by the network. This results in image pairs with the same ID having a smaller distance after feature enhancement, while image pairs with different IDs have a larger distance. Image pairs with the same ID have a larger distance after feature weakening, while image pairs with different IDs have a smaller distance, thereby enhancing the domain adaptation capability of the person re-identification model.

[0052] According to the pedestrian re-identification method of this disclosure, by performing normalization style removal processing and context content restoration processing on the descriptive features of sample image data, the style differences between different domains in the training dataset are eliminated, so that the trained pedestrian re-identification model has domain adaptation capability, thereby making the pedestrian re-identification results obtained by applying the pedestrian re-identification model more accurate.

[0053] The following are embodiments of the apparatus disclosed herein, which can be used to execute the embodiments of the method disclosed herein. The pedestrian re-identification apparatus described below and the pedestrian re-identification method described above can be referred to each other. For details not disclosed in the embodiments of the apparatus disclosed herein, please refer to the embodiments of the method disclosed herein.

[0054] Figure 5 This is a schematic diagram of a pedestrian re-identification device provided in an embodiment of this disclosure. Figure 5 As shown, the pedestrian re-identification device includes: The acquisition module 501 can be used to acquire pedestrian image data to be identified.

[0055] Specifically, the pedestrian image data to be identified is the image data prepared for pedestrian re-identification. Pedestrian re-identification is a technique that uses computer vision technology to determine whether a specific pedestrian or a specific category of pedestrian exists in an image or video sequence.

[0056] The recognition module 502 can be used to input pedestrian image data into a preset pedestrian re-identification model to obtain the first descriptive features of the pedestrian image data output by the pedestrian re-identification model. The descriptive features of the sample image data in the training dataset used to train the pedestrian re-identification model are obtained through normalization style removal processing and context content restoration processing.

[0057] Specifically, the pedestrian re-identification model can extract features from pedestrian image data, compare the extracted features with the descriptive features in the image database, and obtain the similarity between the extracted features and the descriptive features in the database, thereby obtaining the pedestrian re-identification result.

[0058] The determination module 503 can be used to determine the pedestrian re-identification result based on the first descriptive feature and the in-database descriptive features in the image database.

[0059] Specifically, the purpose of pedestrian re-identification is to classify or identify pedestrians. The in-database descriptive features provided in this embodiment can be a target pedestrian of a certain category or a specified target pedestrian. Based on the similarity between the first descriptive feature and the in-database descriptive features in the image database, pedestrians in the current pedestrian image to be identified can be further classified, or it can be determined whether the current pedestrian image to be identified contains a target pedestrian.

[0060] The pedestrian re-identification algorithm provided in this embodiment is a pedestrian re-identification algorithm based on style removal processing and context restoration processing. The pedestrian re-identification algorithm first eliminates style differences between different domains through style removal processing, and then performs context restoration processing on the style-removed features to restore the lost image content. In this way, the domain-independent pedestrian features can be extracted to form training data to train the pedestrian re-identification model, so that the trained pedestrian re-identification model has domain adaptation capability. Applying this pedestrian re-identification model to perform pedestrian re-identification can obtain relatively accurate pedestrian re-identification results.

[0061] In this embodiment of the present disclosure, the pedestrian re-identification device may further include a training module for training a pedestrian re-identification model. Further, the training module may be used to first process the descriptive features of the original sample image data, calculate a loss function value based on the processed data set, and train the initial model based on the loss function value until the loss function converges, thereby obtaining the pedestrian re-identification model.

[0062] Specifically, when the training module processes the descriptive features of the original sample image data, it performs normalization style removal and context content restoration processing on the first sample image data, the second sample image data, and the third sample image data in sequence to obtain a processed data set and obtain the loss function value based on the processed data set. The first sample image data, the second sample image data, and the third sample image data are sample image data from the same batch in the training dataset. The second sample image data is sample image data of the same type as the first sample image data, and the third sample image data is sample image data of a different type than the first sample image data.

[0063] In this embodiment of the present disclosure, the pedestrian re-identification device may further include a processing module, configured to: acquire a second descriptive feature of the current sample image data; input the second descriptive feature into a residual network for normalization style removal processing to obtain a third descriptive feature; input the difference between the second descriptive feature and the third descriptive feature into a first attention branch and a second attention branch of a convolutional neural network, respectively, to obtain a first useful branch output feature and a first useless branch output feature; add the first useful branch output feature and the first useless branch output feature to the third descriptive feature, respectively, to obtain a second useful branch output feature and a second useless branch output feature, and add the second useful branch output feature and the second useless branch output feature to the processing data group.

[0064] In this embodiment of the disclosure, the loss function corresponding to the loss function value includes a pedestrian re-identification loss component and a dual loss component. The processing module can also be used to: obtain a first distance between the second useful branch output feature corresponding to the first sample image data and the second useful branch output feature corresponding to the second sample image data; obtain a second distance between the second useless branch output feature corresponding to the first sample image data and the second useless branch output feature corresponding to the second sample image data; obtain a third distance between the second useless branch output feature corresponding to the first sample image data and the second useless branch output feature corresponding to the third sample image data; obtain a fourth distance between the second useful branch output feature corresponding to the first sample image data and the second useful branch output feature corresponding to the third sample image data; obtain the dual loss component function value based on the first distance, the second distance, the third distance, and the fourth distance; and obtain the loss function value based on the re-identification loss component function value and the dual loss component function value.

[0065] In this embodiment of the disclosure, the calculation formula for the dual loss component of the loss function includes the following formula (2): L dual = ( ( + , + ) ( , )) + ( ( , ) ( + , + )) (2) in, ( ) = ln(1 + exp( )), For distance, + The useful branch output features are the first sample image data. + The useful branch output features are the data corresponding to the second sample image data. The useless branch output features are the first sample image data. - The useless branch output features are the second sample image data. n - The useless branch output features are the third sample image data. n + The useful branch output features are the third sample image data.

[0066] The first useful branch output feature F_s1 can represent useful content information, and the first useless branch output feature F_s2 can represent useless content information. By adding the first useful branch output feature F_s1 and the first useless branch output feature F_s2 to the style-removed third descriptive feature F_in respectively, we can obtain the useful and useless information extracted from the original image. Specifically, the second useful branch output feature F_p and the second useless branch output feature F_n represent the useful and useless information extracted from the original image, respectively. Among them, the second useful branch output feature F_p is the enhanced original feature, which is more discriminative, while the second useless branch output feature F_n is the weakened original feature, which is more difficult to distinguish.

[0067] In this embodiment of the disclosure, the first attention branch or the second attention branch each includes a first fully connected layer, a second fully connected layer and an activation function layer, wherein the first fully connected layer and the second fully connected layer are both 1×1 convolutional structures; the processing module can also be used to process the difference using the first fully connected layer, the second fully connected layer and the activation function layer.

[0068] After the difference between the second and third descriptive features is input into the first attention branch of the convolutional neural network, the processing module can further process the difference using the first fully connected layer, the second fully connected layer, and the activation function layer of the first attention branch.

[0069] In this embodiment of the disclosure, the processing module can also be used to process the second descriptive feature using the following normalization formula (1): (1) Where x is the input feature, IN(x) is the output feature, μ(x) is the mean, σ(x) is the variance, β is the translation factor, and γ is the scaling factor.

[0070] Specifically, the technical solution of this disclosure employs instance normalization technology to remove style differences between different domains. In computer vision, IN is essentially a style normalization method that can unify different images into a single style. Instance normalization technology can remove the domain style of the descriptive features of the original image.

[0071] In this embodiment of the disclosure, the residual network includes a residual branch, a short-circuit branch, and three 3×3 convolutional kernels. The first convolution of the residual branch is a 1×1 convolution with a stride of 1, and the second convolution is a 3×3 convolution with a stride of 2. An average pooling layer is provided before the convolutional layer of the short-circuit branch.

[0072] Specifically, the residual network in this embodiment is an improvement upon the ResNet34 network. ResNet34 is a convolutional neural network whose scaled-down backbone includes a 7×7 convolutional layer with a stride of 2. Since the computational cost of convolution is the square of its length and width, the 7×7 convolutional structure is 5.4 times more computationally expensive than a 3×3 convolutional structure. To extract richer features, this embodiment replaces the 7×7 convolutional kernel with three traditional 3×3 convolutional kernels, reducing computational cost while maintaining the same receptive field as the 7x7 kernel.

[0073] In the original ResNet34 network, the residual branch contains a 1×1 convolution, a 3×3 convolution, and a 1×1 convolution, with the first 1×1 convolution having a stride of 2 and the second 3×3 convolution having a stride of 1. To avoid information loss, the technical solution of this disclosure swaps these two strides, setting the stride of the first 1×1 convolution to 1 and the stride of the second 3×3 convolution to 2.

[0074] The original ResNet34 network has a 1×1 convolution with a stride of 2 in the short-circuit branch. In order to further reduce the information loss of sample image data in the residual branch, the technical solution of this disclosure replaces the 1×1 convolution with a stride of 2 in the short-circuit branch with an average pooling layer and a 1×1 convolution.

[0075] This disclosure proposes a person re-identification algorithm based on style removal and context recovery. It first removes style features from different domains of the original image, then recovers the information lost due to style removal. Simultaneously, it learns useful and useless information from this lost data, thus solving the domain adaptation problem in the field of person re-identification. Furthermore, the technical solution of this disclosure employs dual loss to constrain the feature information useful for person re-identification in the input image learned by the network. This results in image pairs with the same ID having a smaller distance after feature enhancement, while image pairs with different IDs have a larger distance. Image pairs with the same ID have a larger distance after feature weakening, while image pairs with different IDs have a smaller distance, thereby enhancing the domain adaptation capability of the person re-identification model.

[0076] Since the functional modules of the pedestrian re-identification device in the example embodiments of this disclosure correspond to the steps of the example embodiments of the pedestrian re-identification method described above, for details not disclosed in the device embodiments of this disclosure, please refer to the embodiments of the pedestrian re-identification method described above.

[0077] According to the pedestrian re-identification apparatus of this disclosure, by performing normalization style removal processing and context content restoration processing on the descriptive features of sample image data, the style differences between different domains in the training dataset are eliminated, so that the trained pedestrian re-identification model has domain adaptation capability, thereby making the pedestrian re-identification result obtained by applying the pedestrian re-identification model more accurate.

[0078] Figure 6 This is a schematic diagram of the electronic device 6 provided in an embodiment of this disclosure. Figure 6As shown, the electronic device 6 of this embodiment includes a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, it implements the steps in the various method embodiments described above. Alternatively, when the processor 601 executes the computer program 603, it implements the functions of each module in the various device embodiments described above.

[0079] Electronic device 6 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 6 may include, but is not limited to, processor 601 and memory 602. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or different components.

[0080] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0081] The memory 602 can be an internal storage unit of the electronic device 6, such as a hard disk or RAM of the electronic device 6. The memory 602 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 6. The memory 602 can also include both internal and external storage units of the electronic device 6. The memory 602 is used to store computer programs and other programs and data required by the electronic device.

[0082] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0083] If the integrated module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0084] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A pedestrian re-identification method, characterized in that, The method includes: Acquire pedestrian image data to be identified; The pedestrian image data is input into a preset pedestrian re-identification model to obtain the first descriptive feature of the pedestrian image data output by the pedestrian re-identification model. The first descriptive feature is the descriptive feature of the pedestrian image data. The pedestrian re-identification result is determined based on the first descriptive feature and the in-database descriptive features in the image database. The training method for the pedestrian re-identification model includes: Normalization style removal and context content restoration are performed sequentially on the first sample image data, the second sample image data, and the third sample image data to obtain a processed data set. The loss function value is obtained based on the processed data set. The first sample image data, the second sample image data, and the third sample image data are sample image data from the same batch in the training dataset. The second sample image data is sample image data of the same type as the first sample image data. The third sample image data is sample image data of a different type than the first sample image data. The initial model is trained based on the loss function value until the loss function converges, thus obtaining the pedestrian re-identification model. The normalization style removal and context content restoration steps include: Obtain the second descriptive feature of the current sample image data, where the second descriptive feature is a descriptive feature of the current sample image data; The second descriptive feature is input into the residual network for normalization and style removal processing to obtain the third descriptive feature; The difference between the second descriptive feature and the third descriptive feature is used as input, and then input into the first attention branch and the second attention branch of the convolutional neural network to obtain the first useful branch output feature and the first useless branch output feature. The first useful branch output feature and the first useless branch output feature are added to the third descriptive feature to obtain the second useful branch output feature and the second useless branch output feature, and the second useful branch output feature and the second useless branch output feature are added to the processing data group.

2. The method according to claim 1, characterized in that, The loss function corresponding to the loss function value includes a person re-identification loss component and a dual loss component. The loss function value is obtained based on the processed data set, including: Obtain the first distance between the second useful branch output feature corresponding to the first sample image data and the second useful branch output feature corresponding to the second sample image data; Obtain the second distance between the second useless branch output feature corresponding to the first sample image data and the second useless branch output feature corresponding to the second sample image data; Obtain the third distance between the second useless branch output feature corresponding to the first sample image data and the second useless branch output feature corresponding to the third sample image data; Obtain the fourth distance between the second useful branch output feature corresponding to the first sample image data and the second useful branch output feature corresponding to the third sample image data; The dual loss function value is obtained based on the first distance, the second distance, the third distance, and the fourth distance; The loss function value is obtained based on the re-identification loss function value and the dual loss function value.

3. The method according to claim 1, characterized in that, The formula for calculating the dual loss component of the loss function includes the following formula: L dual = ( ( + , + ) ( , )) + ( ( , ) ( + , + )) in, ( ) = ln(1 + exp( )), For distance, + The useful branch output features are the first sample image data. + The useful branch output features are the data corresponding to the second sample image data. The useless branch output features are the first sample image data. - The useless branch output features are the second sample image data. n - The useless branch output features are the third sample image data. n + The useful branch output features are the third sample image data.

4. The method according to claim 1, characterized in that, The first attention branch or the second attention branch each includes a first fully connected layer, a second fully connected layer and an activation function layer, wherein the first fully connected layer and the second fully connected layer are both 1×1 convolutional structures; After inputting the difference between the second descriptive feature and the third descriptive feature into the first attention branch and the second attention branch of the convolutional neural network, respectively, the method further includes: The difference is processed using the first fully connected layer, the second fully connected layer, and the activation function layer.

5. The method according to claim 1, characterized in that, The second descriptive feature is input into the residual network for normalization style removal processing, including: processing the second descriptive feature using the following normalization formula: Where x is the input feature, IN(x) is the output feature, μ(x) is the mean, σ(x) is the variance, β is the translation factor, and γ is the scaling factor.

6. The method according to claim 1, characterized in that, The residual network includes a residual branch, a short-circuit branch, and three 3×3 convolutional kernels. The first convolution of the residual branch is a 1×1 convolution with a stride of 1, and the second convolution is a 3×3 convolution with a stride of 2. An average pooling layer is set before the convolutional layer of the short-circuit branch.

7. A pedestrian re-identification device, characterized in that, The device includes: The acquisition module is used to acquire pedestrian image data to be identified; The recognition module is used to input the pedestrian image data into a preset pedestrian re-identification model to obtain a first descriptive feature of the pedestrian image data output by the pedestrian re-identification model, wherein the first descriptive feature is a descriptive feature of the pedestrian image data; The pedestrian re-identification model was trained as follows: Normalization style removal and context content restoration are sequentially performed on the first, second, and third sample image data to obtain a processed data set. A loss function value is then obtained based on the processed data set. The first, second, and third sample image data are sample image data from the same batch in the training dataset. The second sample image data is similar to the first sample image data, and the third sample image data is dissimilar to the first sample image data. The normalization style removal and context content restoration steps include: Obtain the second descriptive feature of the current sample image data, where the second descriptive feature is a descriptive feature of the current sample image data; The second descriptive feature is input into the residual network for normalization and style removal processing to obtain the third descriptive feature; The difference between the second descriptive feature and the third descriptive feature is used as input, and then input into the first attention branch and the second attention branch of the convolutional neural network to obtain the first useful branch output feature and the first useless branch output feature. The first useful branch output feature and the first useless branch output feature are added to the third descriptive feature to obtain the second useful branch output feature and the second useless branch output feature, and the second useful branch output feature and the second useless branch output feature are added to the processing data group. The initial model is trained based on the loss function value until the loss function converges, thus obtaining the pedestrian re-identification model. The determination module is used to determine the pedestrian re-identification result based on the first descriptive feature and the in-database descriptive features in the image database.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images