Pedestrian re-identification method and device, algorithm server and pedestrian re-identification system

Abstract

This application relates to a pedestrian re-identification method, a pedestrian re-identification device, an algorithm server, and a pedestrian re-identification system. The method includes: extracting feature maps of different depths from an image to be processed; fusing the feature maps of different depths to generate a first fused feature map; generating a weight map based on the deep feature map to highlight potential small target regions, and modulating the first fused feature map based on the weight map to enhance the features corresponding to small target pedestrians, forming a second fused feature map; extracting feature representations incorporating contextual information from the second fused feature map, and performing pedestrian re-identification based on these feature representations. This method can effectively preserve and enhance detailed information in low-level features without introducing excessive computational complexity, actively focus on and strengthen the features of small targets, and fully utilize contextual information to improve feature discrimination power, thereby comprehensively improving the performance of the pedestrian re-identification model in complex scenes.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a pedestrian re-identification method, a pedestrian re-identification device, an algorithm server, and a pedestrian re-identification system. Background Technology

[0002] Person re-identification is a key technology for matching people across different camera scenarios, and it is widely used in fields such as intelligent video surveillance, security management, and industrial safety monitoring. Its core objective is to identify and match images of the same pedestrian within non-overlapping fields of view by analyzing video images captured by different cameras.

[0003] A typical pedestrian re-identification system mainly includes the following three key steps: 1. Extracting original visual features from image or video sequences that can effectively describe a person's appearance, posture, and clothing; 2. Transforming the extracted high-dimensional original visual features into a low-dimensional feature vector with high discriminative power. This vector should be able to distinguish pedestrians of different identities to the greatest extent possible, while maintaining the feature stability of pedestrians of the same identity; 3. Determining whether pedestrians belong to the same identity by comparing pedestrian features captured by different cameras.

[0004] Extracting raw visual features and transforming them into discriminative feature vectors is the core and bottleneck of the entire system, and the strength of feature representation directly determines the upper limit of system performance. Although deep learning-based person re-identification technology has made significant progress, existing models still have a series of fundamental technical shortcomings in processing multi-scale features, especially small but crucial details in images. These include easy loss of feature details, insufficient multi-scale feature fusion, unreasonable detail enhancement methods, and low computational efficiency. Consequently, the recognition accuracy and robustness of the model significantly decrease when facing complex scenes such as similar appearances, occlusion, and resolution variations. Summary of the Invention

[0005] Therefore, it is necessary to provide a pedestrian re-identification method, device, system, algorithm server, storage medium, and computer program product to address the aforementioned technical problems.

[0006] Firstly, this application provides a pedestrian re-identification method, the method comprising: Extract feature maps of different depths from the image to be processed; The feature maps of different depths are fused to generate a first fused feature map; Based on a deep feature map, a weight map is generated to highlight potential small target regions, and the first fused feature map is modulated based on the weight map to enhance the features corresponding to small target pedestrians, forming a second fused feature map; Extract the feature representation that incorporates contextual information from the second fused feature map, and perform pedestrian re-identification based on the feature representation.

[0007] In at least one embodiment, the step of extracting feature maps of different depths from the image to be processed includes: The image to be processed is input into the feature extraction network, which outputs an initial feature map. The initial feature map is divided into multiple local region blocks in the spatial dimension; The initial feature map and the multiple local region blocks are respectively input into multiple subnets with independent parameters within the feature extraction network for deep feature extraction, resulting in feature maps of different depths.

[0008] In at least one embodiment, the subnet is a ResNet18 network with the first convolutional layer and the last classification layer removed.

[0009] In at least one embodiment, the step of fusing the feature maps of different depths to generate a first fused feature map includes: Select at least three feature maps of different depths from the aforementioned feature maps; Perform scale alignment on the selected feature maps; The scale-aligned feature maps are fused to generate the first fused feature map.

[0010] In at least one embodiment, the selection of the at least three feature maps of different depths is to select feature maps of the (L-1)th layer, the Lth layer, and the L+1th layer, where L is an integer greater than 1; The scale alignment operation involves downsampling the feature map of layer L-1 and upsampling the feature map of layer L+1 so that both feature maps have the same scale as the feature map of layer L. The fusion is a channel splicing.

[0011] In at least one embodiment, the step of generating the weight map based on the deep feature map, and modulating the first fused feature map based on the weight map to form a second fused feature map, includes: The second fused feature map is formed according to the following formula: F2 = (1 - δ(U(F0))) × F1 + F1; Wherein, F2 represents the second fused feature map, δ represents the Sigmoid activation function, U(·) represents the upsampling operation, F0 represents the deep feature map, and F1 represents the first fused feature map.

[0012] In at least one embodiment, the step of extracting the feature representation fused with context information from the second fused feature map includes: The second fused feature map is divided into multiple non-overlapping windows; Self-attention computation is performed within each window to capture local contextual information; By using window shifting operations, adjacent windows can interact and a new round of in-window self-attention calculations can be performed to capture global context information, thereby generating feature representations that integrate local and global context information.

[0013] In at least one embodiment, the pre-configured model used in the method is trained based on a triple loss function; The triple loss function is configured such that, in the same batch of training data, the maximum feature distance between all positive sample pairs and feature vectors is less than the minimum feature distance between all negative sample pairs and feature vectors.

[0014] In at least one embodiment, the same batch of training data consists of multiple triples, each triple including an anchor image, a positive sample image, and a negative sample image, wherein the positive sample image is associated with the same identity as the corresponding anchor image, and the negative sample image is associated with a different identity as the corresponding anchor image. And / or, the feature distance is a Euclidean distance. In one embodiment, Secondly, this application also provides a pedestrian re-identification device, the device comprising: The extraction module is used to extract feature maps of different depths from the image to be processed; The fusion module is used to fuse the feature maps of different depths to generate a first fused feature map; The weighting module is used to generate a weight map for highlighting potential small target regions based on a deep feature map, and modulate the first fused feature map based on the weight map to enhance the features corresponding to small target pedestrians, forming a second fused feature map. The execution module is used to extract a feature representation that incorporates contextual information from the second fused feature map and perform pedestrian re-identification based on the feature representation.

[0015] Thirdly, embodiments of this application provide an algorithm server, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of a pedestrian re-identification method.

[0016] Fourthly, embodiments of this application provide a pedestrian re-identification system, including the aforementioned algorithm server and at least one image acquisition device, wherein the image acquisition device communicates with the algorithm server.

[0017] One of the above technical solutions has the following advantages or beneficial effects: by using deep learning, it can automatically learn how to extract distinctive features of the human body; by using channel segmentation and low-level feature enhancement modules, it can better preserve low-level features of the person; without introducing excessive computational complexity, it can effectively preserve and enhance the detailed information in low-level features; actively focus on and strengthen the features of small targets; and make full use of contextual information to improve its feature discrimination power, thereby comprehensively improving the performance of the person re-identification model in complex scenarios and improving the accuracy and efficiency of person re-identification. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a pedestrian re-identification method in one embodiment; Figure 2 This is a schematic diagram of the model architecture of a pedestrian re-identification method in one embodiment; Figure 3 This is a schematic diagram of the feature extraction network channel division process in one embodiment; Figure 4 This is a schematic diagram of the low-level feature enhancement process in one embodiment; Figure 5 This is a schematic diagram of the process for forming a second fused feature map based on a feature attention mechanism in one embodiment; Figure 6 This is a diagram comparing the principles of in-window self-attention calculation and traditional self-attention calculation in one embodiment; Figure 7 This is a structural block diagram of a pedestrian re-identification device in one embodiment; Figure 8 This is a diagram of the internal structure of the algorithm server in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] Figure 1The diagram shown is a flowchart of a pedestrian re-identification method in one embodiment. This method can be integrated into an algorithm server, edge computing device, or cloud server via software, hardware, or a combination of both, and is implemented by a processor executing computer program instructions stored in a storage medium. The pedestrian re-identification method includes the following steps: S102, extract feature maps of different depths from the image to be processed.

[0022] Specifically, the image to be processed can be an image containing one or more pedestrian targets captured by an image acquisition device (such as a surveillance camera, smartphone, etc.). This step aims to extract multi-scale or different levels of abstraction visual feature representations from the input image using a feature extraction network.

[0023] These feature maps of different depths each have their own advantages. Specifically, shallow feature maps usually contain richer details and spatial information, such as edges and textures, which are more crucial for representing small-target pedestrians. Deep feature maps contain higher-level semantic information, such as clothing categories and overall outlines, but their ability to perceive details weakens as the network depth increases.

[0024] S104, fuse feature maps of different depths to generate the first fused feature map.

[0025] This step aims to effectively integrate the feature information of different depths and regions extracted in step S102 to form a unified feature representation containing multi-scale and multi-local region information, namely the first fused feature map, which provides a more comprehensive information foundation for subsequent processing.

[0026] S106, Based on a deep feature map, a weight map is generated to highlight potential small target regions, and the first fused feature map is modulated based on the weight map to enhance the features corresponding to small target pedestrians, forming a second fused feature map.

[0027] This step aims to address the problem that features of small-target pedestrians are easily submerged or lost. Specifically, it utilizes the semantic information of deep features to guide the recalibration of fused features, adaptively enhancing the response of feature regions that may correspond to small-target pedestrians. These feature regions that may correspond to small-target pedestrians can be understood as the aforementioned potential small-target regions.

[0028] The features corresponding to small-target pedestrians can be understood as tiny but crucial details, such as the general outline of the head and hairstyle, the lines of the shoulders and neck, the subtle silhouette of the torso and limbs, a small contrasting edge between clothing and background, and local high-contrast areas, such as a light-colored collar or buttons on a dark top. Although these features account for a very small percentage of pixels and have a poor signal-to-noise ratio, they can sometimes serve as decisive clues to distinguish them from background noise and other small objects.

[0029] Specifically, deep feature maps can be understood as outputs from deeper layers of the network, containing rich scene and target semantic information, and possessing a certain ability to perceive the importance of different regions in an image. Based on this deep feature map, a spatial weight map can be generated. The value of each spatial location in this weight map reflects the probability of a small pedestrian being present in the corresponding region, i.e., the degree to which it needs to be enhanced. Thus, by modulating the first fused feature map based on the weight map, the features in the first fused feature map corresponding to small pedestrians can be enhanced.

[0030] S108, extract the feature representation that incorporates contextual information from the second fused feature map, and perform pedestrian re-identification based on the feature representation.

[0031] The second fused feature map, enhanced by S106 modulation, already contains enhanced small target features. This step aims to further extract feature representations that can effectively distinguish pedestrian identities. These feature representations fully integrate local details and global contextual information and can ultimately be used as image fingerprints. By calculating cosine similarity, the differences between images are measured, and the images are sorted according to similarity to complete the pedestrian re-identification task.

[0032] This method addresses the fundamental shortcomings of existing technologies, such as easy loss of feature details, insufficient multi-scale feature fusion, and unreasonable detail enhancement methods, by effectively solving these problems through multi-scale feature fusion and adaptive detail enhancement mechanisms.

[0033] Specifically, the first step involves extracting and fusing original visual features from different depths to generate a first fused feature map. This helps integrate semantic and detail information from different layers of the network, overcoming the bottleneck problem of insufficient multi-scale feature fusion. Then, a weight map focused on potential small target regions is generated using deep features, and this weight map is used to perform data-driven adaptive modulation of the first fused feature map. This significantly enhances the expression of key detail features corresponding to small pedestrians, fundamentally avoiding the loss of important discriminative information. Finally, a discriminative feature vector incorporating rich contextual information is extracted from the modulated second fused feature map. This process greatly improves feature discrimination ability and the stability of identical identity features while ensuring computational efficiency, thereby significantly improving the model's recognition accuracy and robustness when dealing with complex challenges such as similar appearance, occlusion, and resolution variations.

[0034] In some embodiments, such as Figure 2As shown, step S102 may specifically include: inputting the image to be processed into a feature extraction network to output an initial feature map; dividing the initial feature map into multiple local region blocks in the spatial dimension; inputting the initial feature map and the multiple local region blocks into multiple subnetworks with independent parameters within the feature extraction network for deep feature extraction to obtain feature maps of different depths.

[0035] Specifically, feature extraction networks include, but are not limited to, ResNet and VGG. To obtain more refined local features, the initial feature map can be spatially divided into multiple non-overlapping local region blocks, such as dividing it into upper and lower parts, multiple horizontal or vertical stripe boxes, etc. The initial feature map and the multiple local region blocks are then input into multiple parameter-independent subnetworks within the feature extraction network for deeper feature extraction. These subnetworks can share the same underlying architecture but do not share parameters to ensure that the extracted features are independent and diverse, thus obtaining a set of feature maps with different depths and different focuses.

[0036] One specific implementation involves normalizing the input image (i.e., the image to be processed) to a uniform size of 256×128 pixels. Each image, after entering the network, passes through a first-layer convolutional layer with 64 channels, a kernel size of 7×7, and a stride of 2. Following this, it goes through pooling layers and ReLU layers, outputting 64 feature maps that have been reduced by half. Traditional deep convolutional networks increase network depth by continuously stacking convolutional and pooling modules, thereby improving the network's abstraction capabilities. However, considering the low resolution of pedestrian re-identification images, which require detailed comparison of low-level features and details, simply increasing network depth cannot blindly enhance the network's learning ability. Based on the above analysis, this application proposes... Figure 3 The channel partitioning mechanism is shown. Since most of the low-level features of the network are corner points or edges and other features that do not have semantic meaning, this application chooses not to divide the original image into blocks. Instead, after the image passes through a convolutional layer, the low-level features in the initial feature map are shared. This not only reduces the number of parameters and increases the network's computational efficiency, but also, because the change in pixel values ​​from the original image to the values ​​in the first feature map is not very drastic, not much information is lost.

[0037] Combination Figure 3 It can be seen that the above channel division can be roughly divided into four steps: First, the image to be processed enters the first convolutional layer and outputs the initial feature map; then, the initial feature map is copied and saved, representing the feature map with global significance; next, the initial feature map saved in the previous step is vertically divided into four parts (local region blocks), denoted as upper upper body, lower upper body, upper lower body, and lower lower body, respectively, combined with... Figure 2 and Figure 3Finally, the five feature maps are fed into subnets with independent parameters for deeper feature extraction.

[0038] In a further embodiment, the subnet can be a ResNet18 network with the first convolutional layer and the last classification layer removed, to reduce computation while maintaining feature extraction capabilities. Furthermore, the subnet can be obtained by removing the first convolutional layer and the last SoftMax layer and fully connected layer from the ResNet18 network. A specific structure is shown in the table below:

[0039] In some embodiments, such as Figure 4 As shown, step S104 may specifically include: selecting at least three feature maps of different depths; performing scale alignment on the selected feature maps; and fusing the scale-aligned feature maps to generate a first fused feature map.

[0040] From the obtained feature maps of different depths, select at least three feature maps with representative depths. For example, you can select feature maps of the (L-1)th layer (relatively shallow), the Lth layer (intermediate layer), and the L+1th layer (relatively deep), where L is an integer greater than 1.

[0041] Since feature maps at different depths may have different spatial dimensions, scale alignment is required. Specifically, the feature map of layer L-1 can be downsampled, such as by using strided convolution or pooling; and the feature map of layer L+1 can be upsampled, such as by using transposed convolution or interpolation, so that the spatial dimensions of both are consistent with the scale of the feature map of layer L.

[0042] Multiple scale-aligned feature maps are then fused. Specific fusion methods include channel concatenation, element-wise addition, or weighted fusion. In a preferred embodiment, channel concatenation can be used to preserve the original channel information of all feature maps, generating a first fused feature map.

[0043] A more specific implementation involves using multi-scale feature fusion. Three layers of features {C2, C3, C4} from the feature pyramid are directly fused to reduce information loss during transmission and enhance the multi-scale features of the target. In this case, the deep feature map can be C5 from the pyramid. Because the C3 feature map has higher resolution than C2, contains more texture information about small targets, and requires fewer computational resources, C3 can be the preferred target alignment scale for feature fusion. Figure 4 As shown.

[0044] After feature fusion, the final output is the first fused feature map, i.e. Figure 5 The C3' in the formula can be obtained from the fusion process: C3'=Concat([D(C2),C3,U(C4)]); Where D represents downsampling using average pooling, and U represents upsampling using bilinear interpolation.

[0045] In one embodiment, such as Figure 5 As shown, step S106 can be achieved using the following formula: F2=(1-δ(U(F0)))×F1+F1 Where F2 represents the second fused feature map obtained after modulation; F1 represents the first fused feature map; F0 represents the selected deep feature map; U(·) represents the upsampling operation, which aims to upsample the spatial size of the deep feature map F0 to be the same as F1; δ(·) represents the Sigmoid activation function, which maps the upsampled feature values ​​to the (0, 1) interval, thereby forming an effective weight map δ(U(F0)); the high-value region (close to 1) of the weight map generated by (1-δ(U(F0))) corresponds to the region that the deep feature map considers to be "background" or "non-significant large target", and these regions may be the areas where small targets such as pedestrians are located. Multiplying this weight map with F1 is equivalent to enhancing the feature intensity of these potential small target regions in F1. Finally, adding the original F1, i.e., performing residual connections, helps to stabilize the training process and preserve the original information.

[0046] exist Figure 5 In the embodiment shown, the second fused feature map is Fs, the first fused feature map is C3' mentioned above, and the deep feature map is C5 in the figure.

[0047] In some embodiments, the extraction of feature representations incorporating contextual information in step S108 can employ a Transformer-based approach, preferably using ideas from the Swing Transformer, such as... Figure 6 As shown.

[0048] Specifically, this involves dividing the second fused feature map into multiple non-overlapping windows. Self-attention computation is performed within each window. This local attention mechanism efficiently captures local contextual information within the window, significantly reducing computational complexity compared to global attention. A window shifting operation is used to offset the window partitioning position, enabling interaction between adjacent windows. Self-attention computation is then performed again within the newly shifted window partition. Through this progressive approach, the model effectively captures global contextual information. Ultimately, the generated feature representation integrates local details and global dependencies, providing a more comprehensive and robust depiction of pedestrian appearance.

[0049] It should be explained that, such as Figure 4 dashed box and Figure 6As shown, Swin-Transformer is a transformer-based neural network model. Its core idea is to divide the input image into a series of non-overlapping image patches, and then further divide the input image into multiple windows, each containing multiple patches. Multi-head self-attention within these windows is used to obtain local information. Then, multi-head self-attention through window shifting allows different windows to interact, obtaining global information. The difference between Swin-Transformer and traditional transformer models lies in the introduction of a window mechanism. This mechanism effectively reduces the computational complexity of the global self-attention mechanism used in transformers by confining attention calculations to local attention windows, focusing only on pixels within those windows. Then, by shifting the window and combining information from different windows, window attention is performed again, reducing computational complexity while still obtaining global information.

[0050] Optionally, the Swin-Transformer module divides the feature map into multiple windows, with the window size set to 16, to obtain information about the surroundings of the small target. Specifically, firstly, the enhanced feature map of the small target is divided into multiple windows. By performing self-attention on each window region containing the target, local contextual information of the small target is captured. Then, global contextual information is obtained through window interaction. Next, by combining the local and global information of the target in the feature map, the target can be identified and located more accurately. Finally, the small target features are transmitted to the C3 layer, where pixel addition is used on the basis of the original branch to enhance the target information in the original feature map.

[0051] Based on the final feature representation that incorporates contextual information, the pedestrian re-identification task can be performed by calculating the similarity between pedestrian features in the query image and the gallery image, such as cosine similarity or the reciprocal of Euclidean distance; in other words, the task is to find the most matching pedestrian image with the same identity in the gallery image for the pedestrian in the query image.

[0052] To train a model that implements the above method, a suitable loss function needs to be designed. In one embodiment, the pre-configured model used in the method is trained based on a triple loss function. The triple loss function is configured to constrain the maximum feature distance between feature vectors of all positive sample pairs in the same batch of training data to be less than the minimum feature distance between feature vectors of all negative sample pairs. Positive sample pairs can be understood as different images of the same person, and negative sample pairs can be understood as images of different people. This constrained triple loss function can effectively bring similar features closer together and push away dissimilar features.

[0053] Optionally, the same batch of training data can consist of multiple triples, each of which may include an anchor image, a positive sample image, and a negative sample image. A positive sample image is associated with the same identity as its corresponding anchor image, while a negative sample image is associated with a different identity. The feature distance in the maximum and minimum feature distances can be calculated using Euclidean distance.

[0054] In one specific implementation, the objective function of the constrained triple loss function can be written in the following form: ; in, This can be understood as all the parameters of the network. It is the distance interval. This represents the number of positive sample images in each batch. This represents the number of anchor point images in each batch. represent and Euclidean distance, This represents the mapping represented by the network parameters. Represents the anchor point image. This represents the negative sample image corresponding to the anchor point image. This represents the positive sample image corresponding to the anchor point image.

[0055] It should be noted that the physical meaning of the above formula can be understood as follows: in each batch of processed images, the maximum Euclidean distance between the feature vectors extracted from the positive samples and anchor images after mapping by the network should be less than the minimum Euclidean distance between the feature vectors of the negative samples and anchor images.

[0056] The pedestrian re-identification method proposed in this application, based on multi-channel and multi-scale feature fusion, can automatically learn how to extract distinctive features of the human body by using a deep learning mechanism. At the same time, through channel segmentation and low-level feature enhancement modules, it can better preserve the low-level features of the person, thereby effectively improving the accuracy of pedestrian re-identification.

[0057] The deficiencies of the above solutions and the proposed solutions are the result of the inventor's practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0058] It should be understood that, for the foregoing method embodiments, although the steps in the flowcharts are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the method embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0059] Based on the same inventive concept, this application also provides a pedestrian re-identification device for implementing the pedestrian re-identification method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more pedestrian re-identification device embodiments provided below can be found in the limitations of the pedestrian re-identification method described above, and will not be repeated here.

[0060] In one embodiment, such as Figure 7 As shown, a pedestrian re-identification device 700 is provided, including: an extraction module 701, a fusion module 702, a weighting module 703, and an execution module 704, wherein: The extraction module 701 is used to extract feature maps of different depths from the image to be processed; The fusion module 702 is used to fuse feature maps of different depths to generate a first fused feature map. The weighting module 703 is used to generate a weight map for highlighting potential small target regions based on a deep feature map, and modulate the first fused feature map based on the weight map to enhance the features corresponding to small target pedestrians to form a second fused feature map. The execution module 704 is used to extract a feature representation that incorporates contextual information from the second fused feature map and perform pedestrian re-identification based on the feature representation.

[0061] In at least one embodiment, the extraction module 701 is specifically used to input the image to be processed into a feature extraction network and output an initial feature map; divide the initial feature map into multiple local region blocks in the spatial dimension; and input the initial feature map and multiple local region blocks into multiple subnetworks with independent parameters within the feature extraction network for deep feature extraction to obtain feature maps of different depths.

[0062] In at least one embodiment, the aforementioned subnet is a ResNet18 network with the first convolutional layer and the last classification layer removed.

[0063] In at least one embodiment, the fusion module 702 is specifically used to select at least three feature maps of different depths; perform scale alignment on the selected feature maps; and fuse the scale-aligned feature maps to generate a first fused feature map.

[0064] In at least one embodiment, selecting at least three feature maps of different depths is to select feature maps of layer L-1, layer L, and layer L+1, where L is an integer greater than 1; the scale alignment operation is to downsample the feature map of layer L-1 and upsample the feature map of layer L+1 so that the scale of both is consistent with the scale of the feature map of layer L; fusion is channel stitching.

[0065] In at least one embodiment, the weighting module 703 is specifically configured to form a second fused feature map according to the following formula: F2 = (1 - δ(U(F0))) × F1 + F1; Where F2 represents the second fused feature map, δ represents the Sigmoid activation function, U(·) represents the upsampling operation, F0 represents the deep feature map, and F1 represents the first fused feature map.

[0066] In at least one embodiment, the execution module 704 is specifically used to divide the second fused feature map into multiple non-overlapping windows; perform self-attention calculation within each window to capture local context information; enable interaction between adjacent windows through window shifting operations, and perform a new round of in-window self-attention calculation to capture global context information, thereby generating a feature representation that fuses local and global context information.

[0067] In at least one embodiment, the pre-configured model used in the pedestrian re-identification device 700 is trained based on a triple loss function; wherein the triple loss function is configured to constrain the maximum feature distance between all positive sample pairs and feature vectors in the same batch of training data to be less than the minimum feature distance between all negative sample pairs and feature vectors.

[0068] In at least one embodiment, the same batch of training data consists of multiple triples, each triple including an anchor image, a positive sample image, and a negative sample image. The positive sample image is associated with the same identity as the corresponding anchor image, and the negative sample image is associated with a different identity as the corresponding anchor image.

[0069] In at least one embodiment, the feature distance is a Euclidean distance.

[0070] Specific limitations regarding the pedestrian re-identification device can be found in the limitations of the pedestrian re-identification method above, and will not be repeated here. Each module in the aforementioned pedestrian re-identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the algorithm server, or stored in software in the memory of the algorithm server, so that the processor can call and execute the corresponding operations of each module.

[0071] Furthermore, in the above-described implementation of the pedestrian re-identification device, the logical division of each program module is merely illustrative. In practical applications, the above functions can be assigned to different program modules as needed, for example, for the sake of corresponding hardware configuration requirements or the convenience of software implementation. That is, the internal structure of the pedestrian re-identification device can be divided into different program modules to complete all or part of the functions described above.

[0072] In one embodiment, an algorithm server is provided. This algorithm server can be a mobile terminal, or... The internal structure diagram of the algorithm server can be as follows... Figure 8 As shown, the algorithm server includes a processor, memory, input / output interfaces, and a communication interface. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a pedestrian re-identification method.

[0073] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the algorithm server on which the solution of this application is applied. A specific algorithm server may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0074] In one embodiment, a pedestrian re-identification system is provided, including the algorithm server described above and at least one image acquisition device, such as a network of surveillance cameras, a handheld device camera, etc. The image acquisition device communicates with the algorithm server via a network (such as a local area network or the Internet). The image acquisition device is responsible for capturing scene images and transmitting them to the algorithm server. The algorithm server is responsible for performing the pedestrian re-identification method described above on the received images and returning the identification results, such as the matched pedestrian ID and similarity score, to the corresponding user terminal or security platform.

[0075] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0076] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0077] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0078] It should be noted that, in the embodiments of this application, certain existing solutions in the industry, such as software, components, and models, may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions. All methods described in this application are entirely independently developed executable software algorithms. All software algorithms are implemented using general-purpose high-level languages, such as C++ and Python. The development environments, such as Visual Studio Community Edition and PyCharm Community Edition, are free and publicly available software, and do not involve software licensing or other intellectual property issues.

[0079] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0080] The terms “comprising” and “having”, and any variations thereof, in the embodiments herein are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or (module) units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0081] In this article, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0082] The terms "first" and "second" used herein are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permissible. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0083] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A pedestrian re-identification method, characterized in that, include: The image to be processed is input into the feature extraction network, which outputs an initial feature map. The initial feature map is divided into multiple local region blocks in the spatial dimension; The initial feature map and the multiple local region blocks are respectively input into multiple subnets with independent parameters within the feature extraction network for deep feature extraction to obtain feature maps of different depths. The feature maps of different depths are fused to generate a first fused feature map; Based on a deep feature map, a weight map is generated to highlight potential small target regions. The weight map is then multiplied element-wise with the first fused feature map, and the multiplication result is residually connected with the first fused feature map to enhance the features corresponding to small target pedestrians, thus forming a second fused feature map. The second fused feature map is divided into multiple non-overlapping windows. Self-attention calculation is performed within each window to capture local context information. Through window shifting operations, adjacent windows interact and a new round of in-window self-attention calculation is performed to capture global context information. This generates a feature representation that fuses local and global context information, and pedestrian re-identification is performed based on this feature representation.

2. The method according to claim 1, characterized in that, The subnet is a ResNet18 network with the first convolutional layer and the last classification layer removed.

3. The method according to claim 1, characterized in that, The step of fusing the feature maps of different depths to generate a first fused feature map includes: Select at least three feature maps of different depths from the aforementioned feature maps; Perform scale alignment on the selected feature maps; The scale-aligned feature maps are fused to generate the first fused feature map.

4. The method according to claim 3, characterized in that, The feature maps at least three different depths are selected as feature maps of the (L-1)th layer, the Lth layer, and the (L+1)th layer, where L is an integer greater than 1; The scale alignment operation involves downsampling the feature map of layer L-1 and upsampling the feature map of layer L+1 so that both feature maps have the same scale as the feature map of layer L. The fusion is a channel splicing.

5. The method according to any one of claims 1 to 4, characterized in that, The step of forming the second fused feature map includes: The second fused feature map is formed according to the following formula: F2 = (1 - δ(U(F0))) × F1 + F1; Wherein, F2 represents the second fused feature map, δ represents the Sigmoid activation function, U(·) represents the upsampling operation, F0 represents the deep feature map, and F1 represents the first fused feature map.

6. The method according to any one of claims 1 to 4, characterized in that, The method uses a pre-configured model trained based on a triple loss function; The triple loss function is configured such that, in the same batch of training data, the maximum feature distance between all positive sample pairs and feature vectors is less than the minimum feature distance between all negative sample pairs and feature vectors.

7. The method according to claim 6, characterized in that, The same batch of training data consists of multiple triples. Each triple includes an anchor image, a positive sample image, and a negative sample image. The positive sample image is associated with the same identity as the corresponding anchor image, and the negative sample image is associated with a different identity as the corresponding anchor image. And / or, the characteristic distance is a Euclidean distance.

8. A pedestrian re-identification device, characterized in that, The device includes: The extraction module is used to input the image to be processed into the feature extraction network and output an initial feature map; divide the initial feature map into multiple local region blocks in the spatial dimension; input the initial feature map and the multiple local region blocks into multiple subnetworks with independent parameters within the feature extraction network for deep feature extraction to obtain feature maps of different depths; The fusion module is used to fuse the feature maps of different depths to generate a first fused feature map; The weighting module is used to generate a weight map for highlighting potential small target regions based on a deep feature map, and multiply the weight map element-wise with the first fused feature map, and perform residual connection between the multiplication result and the first fused feature map to enhance the features corresponding to small target pedestrians, forming a second fused feature map. The execution module is used to divide the second fused feature map into multiple non-overlapping windows, perform self-attention calculation within each window to capture local context information, enable interaction between adjacent windows through window shifting operations, and perform a new round of in-window self-attention calculation to capture global context information, thereby generating a feature representation that fuses local and global context information, and performing pedestrian re-identification based on the feature representation.

9. An algorithm server, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A pedestrian re-identification system, characterized in that, It includes the algorithm server as described in claim 9 and at least one image acquisition device, wherein the image acquisition device communicates with the algorithm server.

Images