Pedestrian re-identification model generation method, system, device and storage medium

Abstract

The application discloses a pedestrian re-identification model generation method, system, device and storage medium, and the generation method comprises the following steps: obtaining a pedestrian label, a visible light image and an infrared image; all visible light images and infrared images are preprocessed; a pedestrian re-identification model is created, the pedestrian re-identification model comprises: a shallow double-branch network architecture for processing visible light images, a shallow double-branch network architecture for processing infrared images, a high-level feature extraction subnetwork, and a high-level feature extraction subnetwork; based on a plurality of pedestrian labels and a plurality of visible light images and a plurality of infrared images corresponding to each pedestrian label, the pedestrian re-identification model is trained. The generation method can generate a pedestrian re-identification model.

Description

Technical Field

[0001] This invention relates to the field of pedestrian re-identification technology, and in particular to a method, system, device and storage medium for generating a pedestrian re-identification model. Background Technology

[0002] To conduct continuous video surveillance in public places, surveillance cameras are being deployed more and more widely in subways, stations, squares, and other locations. The video footage captured by these cameras allows for the rapid retrieval, screening, or continuous tracking of specific pedestrian targets, providing clues for investigations and evidence collection, or aiding in the analysis of suspects' behavior. However, because surveillance cameras are typically installed at relatively high positions to achieve a wider monitoring range, and it's difficult to ensure complete overlap between camera views, facial information of pedestrian targets in the surveillance footage is often blurry, making accurate identification impossible. Continuous monitoring or tracking often relies on clothing and appearance details. When a pedestrian disappears from the current camera's view, they need to be quickly and accurately identified and retrieved from other camera feeds. Traditional manual visual retrieval methods for this task suffer from high labor costs, high energy consumption, and difficulty in passing on experience. Summary of the Invention

[0003] In view of this, the main objective of the present invention is to provide a method, system, device and storage medium for generating a pedestrian re-identification model.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows: a method for generating a pedestrian re-identification model, comprising the following steps: acquiring a plurality of pedestrian tags, and a plurality of visible light images and a plurality of infrared images corresponding to each pedestrian tag, wherein each visible light image contains a corresponding pedestrian and the image region where the pedestrian is located is marked, and each infrared image contains a corresponding pedestrian and the image region where the pedestrian is located is marked; preprocessing all visible light images and infrared images; and creating a pedestrian re-identification model, wherein the pedestrian re-identification model comprises: a shallow dual-branch network architecture for processing visible light images. A shallow dual-branch network architecture for processing infrared images High-level feature extraction subnetwork and high-level feature extraction subnetwork The pedestrian re-identification model is trained based on several pedestrian labels and several visible light images and several infrared images corresponding to each pedestrian label.

[0005] As an improvement to this embodiment of the invention, the "preprocessing of all visible light images and infrared images" specifically includes: scaling all infrared images to a preset size; performing the following processing on each visible light image according to a first preset probability: first, filling the width and height of the visible light image with Num pixels of 0 values; then, randomly cropping a sub-image of a preset size from the visible light image; then, randomly horizontally flipping the sub-image according to a second preset probability; then, standardizing the RGB channel data in the visible light image; and the processing is complete; where Num is a natural number.

[0006] As an improvement to this embodiment of the invention, the shallow dual-branch network architecture It is used to receive visible light images, then perform forward operations to obtain shallow visible light feature tensors. The shallow dual-branch network architecture Used to receive infrared images, then perform forward operations to obtain shallow infrared feature tensors. The pedestrian re-identification model is also used to: convert the visible light feature tensor Divide the sample evenly into three groups along the channel dimension and obtain the results. , infrared feature tensor Divide the sample evenly into three groups along the channel dimension and obtain the results. ,in, and All correspond to the R channel. and All correspond to the G channel. and All correspond to channel B.

[0007] As an improvement to this embodiment of the invention, the pedestrian re-identification model is further configured to: for each pedestrian label p and the K corresponding to the pedestrian label p... The following processing is performed: Calculate the center point of the visible light features of the pedestrian in the feature space. , R, G, B} correspond to the R / G / B color channels of the image, and the pedestrian label and the K corresponding to the pedestrian label are... Calculate the center point of the pedestrian's infrared features in the feature space. .

[0008] As an improvement to this embodiment of the invention, the pedestrian re-identification model is further used to: calculate channel alignment loss. ,in, , , Refers to visible light data modes and Refers to infrared data modes; calculates pedestrian center separation loss within a mode. ,in, For the p-th pedestrian in modal The center point below, By and It is obtained by concatenating the tensors corresponding to the features of different color channels. For hinge functions, For manually specified distance thresholds; calculate intermodal distance shrinkage loss. Where m1 and m2 represent different data modes, For manually specified distance thresholds; calculate the total loss for feature compensation and alignment in shallow visible and infrared images. ,in, and All of these are loss weight hyperparameters.

[0009] As an improvement to this embodiment of the invention, the pedestrian re-identification model is further used to: convert the visible light feature tensor Input high-level feature extraction subnetwork In the middle, the high-level visible light feature tensor was obtained. ; Infrared feature tensor Input high-level feature extraction subnetwork In the middle, and obtain the high-level infrared feature tensor. Global max pooling is applied to obtain the global feature representation of visible light images. And global feature representation of infrared images. ;Will and The data is divided into 8 units based on the height dimension, and then generalized average pooling is applied to the 8 units to obtain 8 level feature representations. and Then, based on the expression of 8 level features and Calculate the enhanced feature representation of each local feature in the visible light mode. And the enhanced feature representation of various local features in the infrared mode. ,in, .

[0010] As an improvement to this embodiment of the invention, the pedestrian re-identification model is further used for: global feature representation in the obtained visible light mode and infrared mode. Local feature representation Enhanced feature representation Based on global feature representation Local feature representation and enhanced feature representation Calculate the loss function.

[0011] This invention also provides a device for generating a pedestrian re-identification model, comprising the following modules: an information acquisition module, used to acquire a plurality of pedestrian tags, and a plurality of visible light images and a plurality of infrared images corresponding to each pedestrian tag, wherein each visible light image contains a corresponding pedestrian and the image region where the pedestrian is located is marked, and each infrared image contains a corresponding pedestrian and the image region where the pedestrian is located is marked; preprocessing all visible light images and infrared images; and a model creation module, used to create a pedestrian re-identification model, wherein the pedestrian re-identification model includes: a shallow dual-branch network architecture for processing visible light images. A shallow dual-branch network architecture for processing infrared images High-level feature extraction subnetwork and high-level feature extraction subnetwork The model training module is used to train the pedestrian re-identification model based on several pedestrian labels and several visible light images and several infrared images corresponding to each pedestrian label.

[0012] This invention also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the generation method described above.

[0013] This invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the generation method described above.

[0014] The pedestrian re-identification model generation method, system, device, and storage medium provided in this invention have the following advantages: This invention discloses a pedestrian re-identification model generation method, system, device, and storage medium. The generation method includes the following steps: acquiring pedestrian tags, visible light images, and infrared images; preprocessing all visible light and infrared images; and creating a pedestrian re-identification model, wherein the pedestrian re-identification model includes a shallow dual-branch network architecture for processing visible light images. A shallow dual-branch network architecture for processing infrared images High-level feature extraction subnetwork and high-level feature extraction subnetwork The pedestrian re-identification model is trained based on several pedestrian tags, as well as several visible light images and several infrared images corresponding to each pedestrian tag. This generation method can generate a human re-identification model. Attached Figure Description

[0015] Figure 1 and Figure 2 This is a schematic diagram of the pedestrian re-identification model in the embodiment; Figure 3 This is a flowchart illustrating the generation method in the embodiment; Figure 4 The figure shows the experimental results of the pedestrian re-identification model in the embodiment. Detailed Implementation

[0016] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0017] The following description and accompanying drawings fully illustrate specific embodiments described herein to enable those skilled in the art to practice them. Some embodiments may include or substitute parts and features of other embodiments. The scope of the embodiments herein encompasses the entire scope of the claims and all available equivalents thereof. Throughout this document, the terms “first,” “second,” etc., are used only to distinguish one element from another without requiring or implying any actual relationship or order between the elements. Indeed, a first element can also be referred to as a second element, and vice versa. Furthermore, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a structure, apparatus, or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a structure, apparatus, or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the structure, apparatus, or device that includes said element. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0018] The terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer" used in this document to indicate orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings and are used only for the convenience of describing this document and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention. In the description herein, unless otherwise specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two elements, or direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0019] Embodiment 1 of the present invention provides a method for generating a pedestrian re-identification model, such as... Figure 3 As shown, it includes the following steps: Step 301: Obtain several pedestrian tags, and several visible light images and several infrared images corresponding to each pedestrian tag. Each visible light image contains a corresponding pedestrian and the image area where the pedestrian is located is marked. Each infrared image contains a corresponding pedestrian and the image area where the pedestrian is located is marked. Preprocess all visible light images and infrared images. Here, visible light cameras and infrared cameras can be installed in specific locations (such as public places), and these cameras can be controlled to capture images of those specific locations, obtaining visible light and infrared images containing pedestrians. Then, pedestrians can be manually marked in these visible light and infrared images (for example, by using a box to enclose the image area containing pedestrians), and pedestrian information (such as age, height, body type, or ID number) can be obtained, thus producing several visible light images and several infrared images corresponding to each pedestrian.

[0020] Specifically, in the experiment, the inventors randomly selected P (e.g., P=4) pedestrians from their no-replacement sampling strategy, and then randomly extracted K (e.g., K=8) images from the visible light and infrared images of each pedestrian. This sampling process was repeated to form the aforementioned pedestrian tags, and for each pedestrian tag, several visible light images and several infrared images, totaling [amount missing]. Zhang Visible Light Images and Zhang infrared image.

[0021] Step 302: Create a pedestrian re-identification model, which includes a shallow dual-branch network architecture for processing visible light images. A shallow dual-branch network architecture for processing infrared images High-level feature extraction subnetwork and high-level feature extraction subnetwork ; Step 303: Train the pedestrian re-identification model based on several pedestrian labels and several visible light images and several infrared images corresponding to each pedestrian label.

[0022] In this embodiment, the "preprocessing of all visible light and infrared images" specifically includes: Scale all infrared images to the preset size; Each visible light image is processed according to a first preset probability as follows: First, the width and height of the visible light image are padded with Num pixels of 0 values. Then, a sub-image of a preset size is randomly cropped from the visible light image. Next, the sub-image is randomly horizontally flipped according to a second preset probability. Finally, the RGB channel data in the visible light image are standardized. This completes the processing. Wherein, Num is a natural number.

[0023] Here, the preset size can be For each pixel, the first preset probability can be 50%, and the second preset probability can also be 50%. The standardization operation can be: using the three-channel average as... The standard deviation is Perform the following operations on the RGB channel data: .

[0024] In this embodiment, the shallow dual-branch network architecture It is used to receive visible light images, then perform forward operations to obtain shallow visible light feature tensors. The shallow dual-branch network architecture Used to receive infrared images, then perform forward operations to obtain shallow infrared feature tensors. ; The pedestrian re-identification model is also used to: convert the visible light feature tensor Divide the sample evenly into three groups along the channel dimension and obtain the results. , infrared feature tensor Divide the sample evenly into three groups along the channel dimension and obtain the results. ,in, and All correspond to the R channel. and All correspond to the G channel. and All correspond to channel B.

[0025] Here, the visible light feature tensor With infrared feature tensor Its shape can be 32×256×72×36.

[0026] Here, in the inventor's actual experiments, , , Since the number of channels is not divisible by 3, the remaining 256th channel dimension is discarded here; similarly, , , .

[0027] Here, a shallow dual-branch network architecture and Both can be composed of the first two blocks of ResNet-50, but a shallow dual-branch network architecture and They do not share parameters and perform calculations independently on visible light image input and infrared image input; in addition, the high-level feature extraction subnetwork With high-level feature extraction subnetwork They can all be composed of ResNet-50 Layer 2 to Layer 4, sharing parameters when processing visible light and infrared images.

[0028] In this embodiment, the pedestrian re-identification model is also used for: For each pedestrian tag p and the K corresponding pedestrian tags p The following processing is performed: Calculate the center point of the visible light features of the pedestrian in the feature space. , R, G, B} correspond to the R / G / B color channels of the image, and the pedestrian label and the K corresponding to the pedestrian label are... Calculate the center point of the pedestrian's infrared features in the feature space. .

[0029] In this embodiment, the pedestrian re-identification model is also used for: Calculate channel alignment loss ,in, , , Refers to visible light data modes and Refers to infrared data modes; calculates pedestrian center separation loss within a mode. ,in, For the p-th pedestrian in modal The center point below, By and It is obtained by concatenating the tensors corresponding to the features of different color channels. For hinge functions, For manually specified distance thresholds; calculate intermodal distance shrinkage loss. Where m1 and m2 represent different data modes, For manually specified distance thresholds; calculate the total loss for feature compensation and alignment in shallow visible and infrared images. ,in, and All of these are loss weight hyperparameters.

[0030] In obtaining and Then, using the G-channel feature tensor as an intermediate operation, the KL divergence of the feature tensors corresponding to each color channel of R / G / B in the visible and infrared modes is calculated, specifically as follows: ,in, , These refer to color channels and data modes, respectively.

[0031] right and Apply constraints that increase the distance between the feature centers of different pedestrians within the same modality. Its calculation formula is ,in, For the p-th pedestrian in modal The center point below, By and It is obtained by concatenating the tensors corresponding to the features of different color channels. For hinge functions, For manually specified distance thresholds, The value can be 1.

[0032] right and Further calculation of the cross-modal pedestrian feature tensor center contraction loss The loss value is used to constrain the feature tensor centers of pedestrians in multiple images in the visible light mode to be as close as possible to the feature tensor centers of pedestrians in multiple images in the infrared mode, and the feature tensor centers of pedestrians with different labels in both modes are lower than a certain specified threshold. The calculation expression is as follows Where m1 and m2 represent different data modes, For manually specified distance thresholds, The value can be 1.

[0033] Calculate the total loss for feature compensation and alignment in shallow visible and infrared images. ,in, and All of these are loss weight hyperparameters. and The value can be 0.5.

[0034] In this embodiment, the pedestrian re-identification model is also used for: visible light feature tensor Input high-level feature extraction subnetwork In the middle, the high-level visible light feature tensor was obtained. ; infrared feature tensor Input high-level feature extraction subnetwork In the middle, and obtain the high-level infrared feature tensor. ; Global max pooling is used to obtain the global feature representation of visible light images. And global feature representation of infrared images. ;Will and The data is divided into 8 units based on the height dimension, and then generalized average pooling is applied to the 8 units to obtain 8 level feature representations. and Then, based on the expression of 8 level features and Calculate the enhanced feature representation of each local feature in the visible light mode. And the enhanced feature representation of various local features in the infrared mode. ,in, .

[0035] Here, the high-level visible light feature tensor and high-level infrared feature tensor The shape can be 32×2048×18×9.

[0036] Global max pooling is used to obtain the global feature representation of visible light images. And global feature representation of infrared images. ;Will and The data is divided into 8 units based on the height dimension, and then generalized average pooling is applied to the 8 units to obtain 8 level feature representations. and ,in, .

[0037] Calculate the enhanced feature representation of each local feature in the visible light mode. And the enhanced feature representation of various local features in the infrared mode. ,in, .for First calculate and Cosine similarity between Then, a similarity vector is established. Then, the Softmax function is applied to s to obtain the normalized weighted coefficients. Execute again The operation yields a weighted representation of other local features. Next, the... The feature representation obtained after performing a 1×1 convolution is... By concatenating along the channel dimension, a temporary feature representation of length 2×2048 is obtained. After compressing its channels to 2048 dimensions using another 1×1 convolution, it is then combined with the original... Perform weighted fusion operations to obtain enhanced local feature representations that incorporate spatial information. The above process can be represented as: ,in, This represents the concatenation operation of features along the channel dimension. This is a preset hyperparameter, and its value can be 0.5.

[0038] for First calculate and Cosine similarity between Then, a similarity vector is established. Then, the Softmax function is applied to s to obtain the normalized weighted coefficients. Execute again The operation yields a weighted representation of other local features. Next, the... The feature representation obtained after performing a 1×1 convolution is... By concatenating along the channel dimension, a temporary feature representation of length 2×2048 is obtained. After compressing its channels to 2048 dimensions using another 1×1 convolution, it is then combined with the original... Perform weighted fusion operations to obtain enhanced local feature representations that incorporate spatial information. The above process can be represented as: ,in, This represents the concatenation operation of features along the channel dimension. This is a preset hyperparameter, and its value can be 0.5.

[0039] In this embodiment, the pedestrian re-identification model is also used for: Global feature representations obtained in visible light and infrared modes Local feature representation Enhanced feature representation Based on global feature representation Local feature representation and enhanced feature representation Calculate the loss function.

[0040] right Calculate pedestrian identification loss using all feature representations. The loss is calculated using the label-smoothed cross-entropy function, which is expressed as follows: Where M is the sum of the number of visible light images and the number of infrared images, The pedestrian label is the one corresponding to the i-th image. To use the Softmax function for feature representation The image output after calculation belongs to The probability of the label. For the smoothed label values, ,in, =0.1 is the label smoothing parameter.

[0041] right Calculate the ternary relative distance loss of all feature representations across modal pedestrian feature centers. ,in, , , Refers to modality The feature representation center of the pedestrian label p. Preset hyperparameters. Three-dimensional relative distance loss. The distance between the same pedestrian feature representation centers under different modalities should be lower than the distance between different pedestrian feature representation centers.

[0042] right All feature representations are used to compute cross-modal local alignment loss. This loss is used to improve the accuracy of similarity calculation for local image features in different modalities. For a specific local feature representation of an image in the visible light modality... Local feature representation of another infrared image First, calculate an 8×8 distance matrix, the nth distance of which is... element Let represent the distance from the i-th local region of the visible light image to the j-th local region of the infrared image. After obtaining the distance matrix, a dynamic programming method is used to find the optimal match between the visible light image and the infrared image, which is the shortest path from distance matrix (1, 1) to (8, 8). When searching for this shortest path, define . .

[0043] Find the shortest path from (1,1) to (i,j) in the distance matrix. After completing the entire search, This represents the final shortest path, which also indicates the optimal matching process between various localities of the two cross-modal images. For the enhanced local feature representation of a certain image in the visible light modality... Local feature representation of another infrared image The same process is applied to obtain the optimal matching process.

[0044] Using the shortest path after optimal matching as defined above as the distance similarity between two cross-modal images, we further calculate their loss after spatial alignment. The formula for its calculation is: ,in, Refers to the local feature representation of an image in the m-modal. or enhanced local feature expression , For the selected anchor point, Indicates and Feature representation of an image with the same pedestrian label but in a different modality. Indicates and The pedestrian labels are different and represent the features of a certain image in another modality. This represents the distance between image features after the optimal match.

[0045] The loss of the pedestrian re-identification model ,in and These are all preset loss weighting coefficients. Optional, =2 and =1.

[0046] Here, based on the loss value The parameters of the shallow network are updated by backpropagation using the Stochastic Gradient Descent (SGD) optimization method, and then based on the loss value... The parameters of the entire network are updated using backpropagation with the SGD optimization method. The initial step size of SGD is 0.01, and it is reduced by a factor of 10 after every 10 epochs.

[0047] The optimization process is repeated, with each epoch representing the exhaustion of all sequence batches of data, until the maximum number of training epochs is reached. During model training, the model parameters achieving optimal performance need to be saved to disk.

[0048] After training the model, the saved optimal model is used to extract feature representations from all images in the test set, calculate the Euclidean distance between any two images in different modalities, and sort the images in order of distance from low to high to obtain cross-modal pedestrian image retrieval results.

[0049] The generation method in this embodiment of the invention has the following advantages: Existing generative adversarial network (GAN)-based methods generate image instances that are inconsistent in quality, impose a heavy training burden on the model, and inevitably introduce noise. Other methods for learning image invariance features across different modalities tend to lose modality-specific information. In the cross-modal pedestrian image retrieval method based on channel feature compensation and alignment proposed in this invention, the distribution of channel features in shallow features of different modalities is constrained, the correlation between different color channels is mined, and cross-modal channel feature alignment is achieved. This reduces the differences in feature representation between different modalities without increasing the model training burden. Furthermore, the proposed method enhances the discriminative ability by embedding spatial structure information into local image features and introduces dynamic search in cross-modal image similarity calculation to achieve spatial alignment, enabling more accurate local image matching and thus improving the accuracy of cross-modal pedestrian image retrieval.

[0050] To test the performance of the cross-modal pedestrian image retrieval method proposed in this invention, performance verification was performed on the publicly available SYSU-MM01 dataset. The experimental environment was an Ubuntu 20.04 system, using the PyTorch 1.2 deep learning framework, and hardware acceleration was achieved using an Nvidia GTX 3090 GPU. The SYSU-MM01 dataset contains 30,701 visible light images and 15,792 infrared images of 491 pedestrians, captured by four visible light cameras and two infrared cameras. The SYSU-MM01 dataset provides a training and test set partitioning scheme. The training set contains 22,258 visible light images and 11,909 infrared images of 395 pedestrians; the test set contains 3,803 infrared images of the remaining 96 pedestrians used as retrieval images, and an additional 301 images randomly selected from the visible light images serve as the match set. Two evaluation settings are used to assess model performance: a full retrieval mode, where images from all visible light cameras are used as the match set; and an indoor retrieval mode, where only visible light images from two indoor cameras are used as the match set. Rank-n matching accuracy and MAP (mean Average Precision) are used as performance metrics for cross-modal image retrieval. Rank-n matching accuracy represents the probability of a correct match at position n in the search results, while MAP represents the average retrieval accuracy for each pedestrian and is suitable for comprehensively evaluating retrieval performance in multi-camera environments. Figure 4As shown, on the SYSU-MM01 dataset, the model of this invention achieved a Rank-1 accuracy of 79.26% and a MAP of 75.58% in all retrieval modes, and an Rank-1 accuracy of 83.31% and a MAP of 85.06% in indoor retrieval mode. These results are superior to those reported by existing methods, verifying the effectiveness of the cross-modal pedestrian image retrieval method proposed in this invention.

[0051] Embodiment 2 of the present invention provides a device for generating a pedestrian re-identification model, comprising the following modules: an information acquisition module, used to acquire a plurality of pedestrian tags, and a plurality of visible light images and a plurality of infrared images corresponding to each pedestrian tag, wherein each visible light image contains a corresponding pedestrian and the image region in which the pedestrian is located is marked, and each infrared image contains a corresponding pedestrian and the image region in which the pedestrian is located is marked; preprocessing is performed on all visible light images and infrared images; and a model creation module, used to create a pedestrian re-identification model, wherein the pedestrian re-identification model includes: a shallow dual-branch network architecture for processing visible light images. A shallow dual-branch network architecture for processing infrared images High-level feature extraction subnetwork and high-level feature extraction subnetwork The model training module is used to train the pedestrian re-identification model based on several pedestrian labels and several visible light images and several infrared images corresponding to each pedestrian label.

[0052] Embodiment 3 of the present invention provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the generation method as described in Embodiment 1.

[0053] Embodiment 4 of the present invention provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the generation method as described in Embodiment 1.

[0054] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order, as long as the required function can be achieved.

[0055] This invention can be a system, method, and / or computer program product. A computer program product may include a readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0056] A readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. Readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0057] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating a pedestrian re-identification model, characterized in that, Includes the following steps: Acquire several pedestrian tags, as well as several visible light images and several infrared images corresponding to each pedestrian tag. Each visible light image contains a corresponding pedestrian and the image area where the pedestrian is located is marked. Each infrared image contains a corresponding pedestrian and the image area where the pedestrian is located is marked. All visible light and infrared images are preprocessed; Create a pedestrian re-identification model, the pedestrian re-identification model comprising: a shallow dual-branch network architecture for processing visible light images. A shallow dual-branch network architecture for processing infrared images High-level feature extraction subnetwork and high-level feature extraction subnetwork ; The pedestrian re-identification model is trained based on several pedestrian tags, as well as several visible light images and several infrared images corresponding to each pedestrian tag.

2. The generation method according to claim 1, characterized in that, The phrase "preprocessing all visible light and infrared images" specifically includes: Scale all infrared images to the preset size; Each visible light image is processed according to a first preset probability as follows: First, the width and height of the visible light image are padded with Num pixels of 0 values. Then, a sub-image of a preset size is randomly cropped from the visible light image. Next, the sub-image is randomly horizontally flipped according to a second preset probability. Finally, the RGB channel data in the visible light image are standardized. This completes the processing. Wherein, Num is a natural number.

3. The generation method according to claim 1, characterized in that, The shallow dual-branch network architecture It is used to receive visible light images, then perform forward operations to obtain shallow visible light feature tensors. The shallow dual-branch network architecture Used to receive infrared images, then perform forward operations to obtain shallow infrared feature tensors. ; The pedestrian re-identification model is also used to: convert the visible light feature tensor Divide the sample evenly into three groups along the channel dimension and obtain the results. , infrared feature tensor Divide the sample evenly into three groups along the channel dimension and obtain the results. ,in, and All correspond to the R channel. and All correspond to the G channel. and All correspond to channel B.

4. The generation method according to claim 3, characterized in that, The pedestrian re-identification model is also used for: For each pedestrian tag p and the K corresponding pedestrian tags p The following processing is performed: Calculate the center point of the visible light features of the pedestrian in the feature space. , R, G, B} correspond to the R / G / B color channels of the image, and the pedestrian label and the K corresponding to the pedestrian label are... Calculate the center point of the pedestrian's infrared features in the feature space. .

5. The generation method according to claim 4, characterized in that, The pedestrian re-identification model is also used for: Calculate channel alignment loss ,in, , , Refers to visible light data modes and Refers to infrared data modes; calculates pedestrian center separation loss within a mode. ,in, For the p-th pedestrian in modal The center point below, By and It is obtained by concatenating the tensors corresponding to the features of different color channels. For hinge functions, For manually specified distance thresholds; calculate intermodal distance shrinkage loss. Where m1 and m2 represent different data modes, For manually specified distance thresholds; calculate the total loss for feature compensation and alignment in shallow visible and infrared images. ,in, and All of these are loss weight hyperparameters.

6. The generation method according to claim 5, characterized in that, The pedestrian re-identification model is also used for: visible light feature tensor Input high-level feature extraction subnetwork In the middle, the high-level visible light feature tensor was obtained. ; infrared feature tensor Input high-level feature extraction subnetwork In the middle, and obtain the high-level infrared feature tensor. ; Global max pooling is used to obtain the global feature representation of visible light images. And global feature representation of infrared images. ;Will and The data is divided into 8 units based on the height dimension, and then generalized average pooling is applied to the 8 units to obtain 8 level feature representations. and Then, based on the expression of 8 level features and Calculate the enhanced feature representation of each local feature in the visible light mode. And the enhanced feature representation of various local features in the infrared mode. ,in, .

7. The generation method according to claim 6, characterized in that, The pedestrian re-identification model is also used for: Global feature representations obtained in visible light and infrared modes Local feature representation Enhanced feature representation Based on global feature representation Local feature representation and enhanced feature representation Calculate the loss function.

8. A device for generating a pedestrian re-identification model, characterized in that, Includes the following modules: The information acquisition module is used to acquire several pedestrian tags, as well as several visible light images and several infrared images corresponding to each pedestrian tag. Each visible light image contains a corresponding pedestrian and the image area where the pedestrian is located is marked. Each infrared image contains a corresponding pedestrian and the image area where the pedestrian is located is marked. All visible light and infrared images are preprocessed; The model creation module is used to create a pedestrian re-identification model, which includes a shallow dual-branch network architecture for processing visible light images. A shallow dual-branch network architecture for processing infrared images High-level feature extraction subnetwork and high-level feature extraction subnetwork ; The model training module is used to train the pedestrian re-identification model based on several pedestrian labels and several visible light images and several infrared images corresponding to each pedestrian label.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the generation method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the generation method as described in any one of claims 1 to 7.

Images