A pedestrian search system and method based on multi-scale deformable attention feature alignment

Abstract

The application provides a pedestrian search system and method based on multi-scale deformable attention feature alignment, and belongs to the technical field of pedestrian search based on computer vision; an anchor-free pedestrian search network framework based on a deep convolutional neural network is constructed; firstly, relative position and scale information is injected by using convolution position coding and hierarchical scale embedding injection through a context perception feature enhancement module; secondly, a geometric adaptive sampling module is introduced, feature extraction and fusion are carried out on multi-scale feature maps through a cross-scale sparse sampling strategy; finally, through an adaptive modulation mechanism based on a prediction box, the distribution of sampling points is dynamically adjusted by using the geometric prior of a detection branch, and topological alignment of feature receptive fields and pedestrian postures is realized. The application can effectively improve the feature alignment accuracy and multi-scale robustness in complex scenes, significantly reduce the calculation overhead, and meet the real-time processing requirements.

Description

Technical Field

[0001] This invention belongs to the field of pedestrian search technology based on computer vision, and particularly relates to a pedestrian search system and method based on multi-scale deformable attention feature alignment. Background Technology

[0002] With the deepening of smart city construction, video surveillance networks have widely covered various complex scenarios. Pedestrian search, as a core task in the field of intelligent video analytics, aims to quickly and accurately locate and identify specific pedestrians matching the identity of the target being queried from uncropped raw panoramic images. This technology involves the deep collaboration of two major tasks: detection and re-identification.

[0003] Currently, commonly used technologies in the industry include: 1. Anchor-based methods (such as Faster R-CNN+OIM): These methods extract candidate regions using predefined dense anchor boxes and a Region Proposal Network (RPN). Their drawback is the need to generate a large number of redundant candidate boxes, involving complex nonmaximum suppression (NMS) and IoU calculations, resulting in high computational overhead and making it difficult to meet the real-time processing requirements of large-scale video data.

[0004] 2. Anchor-free methods (such as AlignPS): These methods abandon anchor boxes and RPN, directly regressing on the pixel points of the feature map. Although this improves speed, the lack of explicit region of interest (ROI) cropping and correction operations makes it easy for background noise to be mixed in during feature extraction, resulting in a "feature misalignment" problem.

[0005] 3. Transformer-based methods (such as PSTR): These methods utilize global attention mechanisms to capture context. Their drawbacks include computational complexity that is quadratic with the feature map size, high memory usage, and slow inference speed. Furthermore, standard attention mechanisms lack adaptive sampling for human structure, resulting in insufficient feature discriminative power when faced with occlusion or irregular poses.

[0006] The difficulty in solving the above technical problems lies in: The challenge of balancing feature offset and feature accuracy in anchorless architectures: By eliminating anchor boxes, models lose explicit boundary constraints, making the receptive field of deep networks prone to divergence and drift. The core difficulty lies in designing a mechanism that allows the network to automatically perceive edges and achieve precise alignment between features and targets without predefined physical boxes, preventing feature aggregation failure in occluded or cluttered backgrounds.

[0007] The challenge of dynamic modeling of non-rigid targets isomorphic to their topology: Pedestrian postures vary greatly (e.g., cycling, bending over), causing drastic fluctuations in aspect ratio, while existing networks mostly have fixed or isotropic receptive fields. The core difficulty lies in endowing the network with adaptive deformation capabilities, enabling sampling points to be dynamically stretched or shrunk according to the physical topology of a specific posture, and this mechanism must meet the requirements of differentiability and lightweight design to support end-to-end training.

[0008] The challenge of balancing high-resolution panoramic feature processing with linear computational complexity: Pedestrian search needs to consider both shallow details and deep semantics on uncropped panoramic images, requiring a global Transformer... The complexity results in excessive memory consumption. The core challenge lies in designing a linear complexity solution. The sparse attention mechanism can capture multi-scale long-distance dependencies while breaking through the computing power bottleneck of real-time inference. Summary of the Invention

[0009] To address the aforementioned problems, this invention provides a pedestrian search system and method based on multi-scale deformable attention feature alignment. It decouples feature offset from pedestrian pose changes in an anchorless framework, and dynamically modulates the feature sampling distribution using the geometric prior of the detection branch. Even under challenging conditions such as extremely distorted pedestrian poses or occlusion, it can extract accurately aligned identity features. To balance computational efficiency and feature accuracy, the model employs a sparse sampling strategy, achieving deep fusion of multi-scale semantics while maintaining linear computational complexity.

[0010] The first aspect of this invention proposes a pedestrian search system based on multi-scale deformable attention feature alignment, which is constructed based on an anchor-free deep convolutional neural network and includes a feature extraction backbone network, a context-aware feature enhancement module, a geometry-adaptive sampling module, a detection branch, and a re-identification branch connected in sequence. The feature extraction backbone network takes the original complex panoramic image as input and uses multiple convolutional kernels and pooling layers of a deep convolutional neural network to perform forward propagation and downsampling processing on the input image layer by layer, extracting visual details at the bottom layer and semantic information at the top layer, and generating a multi-scale feature pyramid containing different spatial resolutions and semantic levels. The context-aware feature enhancement module takes a multi-scale feature pyramid as input and generates an enhanced multi-scale feature map by injecting relative position information and scale semantics. The geometric adaptive sampling module takes the enhanced multi-scale feature map as input and uses a cross-scale sparse sampling strategy based on the attention mechanism to sample and fuse multi-scale features. In this process, the geometric prior information obtained by the adaptive sampling mechanism based on the prediction box is used to dynamically adjust the offset of the sampling points, so as to achieve geometric alignment between the feature receptive field and the pedestrian pose, and generate an aligned feature map. The detection branch and the re-identification branch are connected in parallel after the geometric adaptive sampling module. The detection branch uses the aligned feature map to predict the center point position and geometric boundary information of the pedestrian, while the re-identification branch uses the aligned feature map to generate a discriminative pedestrian identity feature vector.

[0011] Preferably, the context-aware feature enhancement module includes a convolutional position encoding unit (CPE) and a scale-level embedding unit. The CPE uses a 3×3 deep convolutional layer (DWConv) to process the features of each layer of the multi-scale feature pyramid, implicitly modeling the relative positional relationship between feature points and their neighboring pixels using the sliding window property of the convolutional kernel to maintain translational equivariance. The scale-level embedding unit introduces a learnable scale embedding vector to each layer of the feature pyramid, and after spatial dimensional expansion through a broadcast mechanism, it is added element-wise to the feature map to construct a multi-scale feature space that distinguishes different levels of semantics.

[0012] Preferably, the specific processing flow of the prediction box based on the geometric adaptive sampling module is as follows: First, obtain the four-dimensional distance vector predicted by the detection branch for the current query element Query. ,in These represent the distances from the current point to the left, right, top, and bottom boundaries of the target, respectively; secondly, the true width of the target is calculated based on the four-dimensional distance vector. and height And calculate the modulation weight vector based on the aspect ratio. ; Finally, the modulation weight vector The original sampling offset learned by the network Perform Hadamard product operation to reconstruct the final sampling coordinate offset. ,Right now:

[0013] in, Represents element-wise multiplication; when At that time, the weight vector The component in the vertical direction is amplified; when At that time, the weight vector The horizontal component is amplified, forcing the sampling point distribution to adapt to the pedestrian's physical posture.

[0014] Preferably, the geometric adaptive sampling module incorporates a geometric prior initialization strategy; in the initial stage of model training, based on the pedestrian structure consistency prior, the geometric prior initialization strategy is applied to each query element. The offset of each sampling point is initialized to be uniformly distributed along the vertical axis Y of the reference point; specifically, the weight matrix of the fully connected layer LinearLayer used to generate the sampling offset is initialized to 0, and the bias vector is initialized to the expanded value of a preset set of vertical coordinates. The range of values ​​of the set of vertical coordinates covers the area from the human head to the feet, so as to force the region of interest to align with the human skeletal structure in the early stage of training and accelerate network convergence.

[0015] Preferably, the geometric adaptive sampling module employs a cross-scale sparse sampling strategy; for each query element on the feature map, sampling is performed only based on the reference point and the final sampling coordinate offset. A small number of key sampling points are determined on a multi-scale feature map; the feature values ​​of the key sampling points are obtained on the multi-scale feature map using bilinear interpolation, and the learned attention weights are used to adaptively aggregate information from different feature levels, thus fusing deep semantic features with shallow high-resolution geometric features.

[0016] Preferably, the model employs a joint loss function for end-to-end optimization during the training phase. The joint loss function includes a detection loss for optimizing the detection branch and a TOIM loss function for optimizing the re-identification branch.

[0017] A second aspect of this invention provides a pedestrian search method based on multi-scale deformable attention feature alignment, comprising the following steps: S1, Obtain a panoramic image of the scene to be searched; S2, input the image into the pedestrian search system based on multi-scale deformable attention feature alignment as described in the first aspect; extract features through the feature extraction backbone network, and inject position and scale information using the context-aware feature enhancement module; output the predicted bounding box of the pedestrian from the detection branch, and at the same time use the geometric adaptive sampling module to dynamically adjust the feature sampling point distribution of the re-identification branch according to the geometric shape of the predicted box, and extract the aligned identity features. S3 outputs the detected pedestrian bounding boxes and the corresponding identity recognition results.

[0018] A third aspect of the present invention provides a pedestrian search device based on multi-scale deformable attention feature alignment, the device comprising at least one processor and at least one memory coupled together; the memory stores a computer program for a pedestrian search system based on multi-scale deformable attention feature alignment as described in the first aspect; when the processor executes the computer program stored in the memory, the processor executes a pedestrian search method based on multi-scale deformable attention feature alignment.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program or instructions for a pedestrian search system based on multi-scale deformable attention feature alignment as described in the first aspect, wherein when the program or instructions are executed by a processor, the processor performs a pedestrian search method based on multi-scale deformable attention feature alignment.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. Significantly improved feature alignment accuracy and multi-scale robustness: It solves the problem of receptive field shift caused by the lack of explicit region clipping in anchor-free networks, effectively avoids background noise (such as occlusions), and achieves pixel-level accurate feature alignment.

[0021] 2. Strong adaptive modeling capability for irregular pedestrian postures: It breaks the limitation of fixed rectangular receptive field and can dynamically adjust the sampling range according to the target posture (such as cycling or bending over) to closely fit the human body contour.

[0022] 3. High inference efficiency and fast training convergence: By reducing the complexity to linear through sparse sampling and combining geometric prior initialization, the memory usage and computational overhead are greatly reduced, making it suitable for deployment on resource-constrained edge devices. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the following description is only one embodiment of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the system architecture based on multi-scale deformable attention feature alignment provided by the present invention; Figure 2 This is a schematic diagram illustrating the flow and principle of the context-aware enhancement module provided by the present invention; Figure 3 This is a schematic diagram illustrating the process and principle of the geometric adaptive sampling module provided by the present invention; Figure 4 This is a schematic diagram illustrating the principle of the geometric prior initialization strategy provided by the present invention; Figure 5 This is a visualization of an embodiment of the present invention. Figure 6 This is a simplified structural diagram of the pedestrian search device of the present invention. Detailed Implementation

[0025] The invention will be further described below with reference to specific embodiments.

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments.

[0027] This invention proposes a novel pedestrian search model based on an anchor-free deep learning architecture. Addressing the challenges of implicit feature alignment and poor adaptability to non-rigid poses in anchor-free pedestrian search, this model utilizes geometric priors in the detection branch to dynamically modulate the distribution of feature samples. It also explores the deep fusion of cross-scale semantic and geometric information by combining context-aware feature enhancement and cross-scale sparse sampling mechanisms to guide the model's learning process. The feature extraction and alignment branch of this invention comprises three key modules: a context-aware feature enhancement module that injects relative positional information using convolutional positional encoding; a geometrically adaptive sampling module that aggregates features at different resolution levels through a sparse sampling strategy while dynamically adjusting the sampling point distribution based on the adaptive modulation mechanism of the prediction box using geometric priors. This forces the model to generate features highly isomorphic to the physical topology of the human body, maintaining high-quality search accuracy even in complex situations with extremely distorted pedestrian poses or occlusion.

[0028] The overall framework of the model in this paper is as follows: Figure 1 As shown, this invention employs ResNet-50 as the feature extraction backbone to extract a multi-scale feature pyramid and constructs parallel-connected detection and re-identification branches. Specifically, the multi-level features (such as P3, P4, P5) output by the backbone network are first fed into a context-aware feature enhancement module to inject positional and scale semantics. Subsequently, the detection branch uses the enhanced features to predict the pedestrian's bounding box and centrality score. An adaptive modulation mechanism based on the predicted bounding box receives the geometric output of the detection branch, dynamically calculates the sampling modulation weights, and applies them to the sampling offset of the cross-scale sparse sampling. Finally, the re-identification branch uses the aligned features to generate an identity vector.

[0029] (a) Feature extraction backbone network In this architecture, ResNet-50 is used as the basic backbone network. The specific process is as follows: For an input size of... The surveillance panoramic images are processed by the backbone network to extract three feature maps at different scales, denoted as follows: The downsampling step sizes relative to the input image are 8, 16, and 32, respectively. To construct the feature pyramid, the following methods are used: Convolution maps the number of channels in these feature maps to a uniform value. To obtain the basic feature pyramid These feature maps preserve rich content, ranging from shallow geometric details to deep semantic information, providing a data foundation for subsequent feature enhancement and alignment.

[0030] (II) Context-Aware Feature Enhancement Module For anchor-free pedestrian search models, the lack of explicit bounding box clipping makes positional information in feature maps crucial for distinguishing dense targets. Furthermore, multi-scale feature interactions can easily lead to scale semantic ambiguity. Therefore, introducing a context-aware feature enhancement module helps the model clarify the relative position and scale level of feature points in their local neighborhood while maintaining translational equivariance. This module utilizes… Deep convolutions and learnable scale embedding vectors are used to enhance features, such as Figure 2 As shown. Specifically, Convolutional Position Encoding (CPE) implicitly models the relative positional relationship between feature points and their neighboring pixels using the sliding window property of the convolutional kernel. For each feature layer... Through a The feature map is processed using a depth-wise convolutional layer, which injects positional information while maintaining the feature map resolution. That is:

[0031] Simultaneously, scale-level embedding aims to distinguish the semantics of different feature layers. For each layer of the pyramid... Initialize a learnable scale embedding vector This vector is extended to the feature map via a broadcasting mechanism. After obtaining the same spatial dimensions, element-wise addition is performed to obtain the final enhanced features. .Right now:

[0032] (III) Geometric Adaptive Sampling Module Pedestrian postures typically exhibit high uncertainty (e.g., standing, cycling, lying down), leading to drastic variations in aspect ratio. Standard attention mechanisms often employ isotropic sampling point distributions, which can introduce background noise when processing non-upright pedestrians. Therefore, an adaptive modulation module based on prediction boxes is designed. This module utilizes the geometric prior of the detection branch prediction to dynamically adjust the distribution of sampling points, ensuring it remains isomorphic to the physical topology of the pedestrian. Figure 4 As shown.

[0033] like Figure 3 As shown, the specific processing flow of this module is as follows: First, obtain the four-dimensional distance vector predicted by the detection branch for the current query element (Query). Secondly, the true width of the target is calculated based on this vector. and height And construct a modulation weight vector based on aspect ratio. The design logic for this weight is: when When standing, increase the weight in the vertical direction. ;when (In non-upright posture) increase the horizontal weight. Finally, the modulation weight vector The original sampling offset learned by the network Perform the Hadamard product operation to reconstruct the final sampled coordinate offset. The formula is as follows:

[0034] in, This represents a step-by-step multiplication operation. In this way, the distribution range of the sampling points is forcibly constrained within the physical contour of the target.

[0035] The geometrically adaptive sampling module aims to address the excessive computational complexity in panoramic high-resolution image processing. It employs a sparse sampling strategy, focusing only on key sampling points around a reference point, thus reducing computational complexity to linear. This module receives enhanced multi-scale features. and the corrected sampling offset As input. Specifically, for each query element Its reference point is The module is in all Generate at each feature level 1 sampling point. Using bilinear interpolation function. The feature values ​​of these sampling points are extracted from the feature map and combined with the learned attention weights. Perform weighted aggregation. The formula is as follows:

[0036] in, For the number of attention heads, and This is the projection matrix. This operation enables deep fusion of cross-scale information, so that the final re-identified features contain both deep semantic information and retain shallow geometric details.

[0037] (iv) Loss Function The model is supervised by two types of tasks: detection and re-identification. For the detection task, FocalLoss is used. Optimize the classification branch to address the imbalance between positive and negative samples; use GIoU Loss. and Center-ness Loss Optimize bounding box regression and centrality prediction. For the re-identification task, the OIM (Online Instance Matching) loss function is adopted. This function maintains a lookup table to store the feature centers of labeled pedestrians and a circular queue to store the features of unlabeled pedestrians. The loss function calculation formula is as follows:

[0038] in, The current pedestrian feature vector after L2 normalization. To find the feature center corresponding to the real identity label in the table, To find the total number of identities that have been tagged in the table, For the first in the circular queue One unlabeled feature, This represents the maximum capacity of the circular queue. A temperature coefficient is used to control the smoothness of the probability distribution; and after each backpropagation of the network, the corresponding feature center in the lookup table is dynamically updated using the current feature, and the unlabeled features of the current batch are enqueued to update the circular queue. Therefore:

[0039] in, These are the balance coefficients for each loss term. and These represent the classification loss, regression loss, and centrality loss of the detection branch, respectively.

[0040] (v) Detection and Re-identification Branch The detection branch takes the aligned feature map output by the geometrically adaptive sampling module as input. This branch employs a fully convolutional network (FCN) structure, containing three parallel sub-network heads: classification, centrality, and bounding box regression. During processing, the detection branch performs forward inference on each pixel location in the feature map, outputting a classification score indicating whether the location belongs to a pedestrian, a center-ness prediction value for high-dimensional candidate boxes used to suppress deviations from the target center, and a four-dimensional distance vector (l, r, t, b). This four-dimensional distance vector is used not only to generate the final detection bounding box for spatial localization.

[0041] For the re-identification branch, the aligned feature map output by the geometric adaptive sampling module is used as input. A linear mapping layer (1×1 convolutional layer) is used to reduce the dimensionality and map these aligned features, followed by batch normalization and L2 normalization. The final output is a high-dimensional, highly discriminative pedestrian identity feature vector. This vector is fed into the TOIM loss function for end-to-end optimization and used during actual inference to perform cosine similarity comparison with target features in the retrieval database, thus completing the final pedestrian search and matching.

[0042] (V) Experimental Verification To verify the effectiveness of this invention, examples were performed on the standard pedestrian search datasets CUHK-SYSU and PRW: The experimental environment was as follows: a 64-bit Ubuntu 22.04 operating system, an NVIDIA GeForce RTX 3090 Ti graphics processor, and 12GB of video memory. The Adam optimizer was used, with an initial learning rate set to 0.0002. =0.1. This embodiment uses a RestNet-50 pre-trained on ImageNet as the backbone network, with a training batch size of 4. The learning rate is reduced by a factor of 10 at epochs 22 and 40, for a total of 50 epochs. In deformable attention, M=8 and K=4 by default, and these parameters are shared across different feature levels. The learning rate used for predicting the linear projection of the object query reference point and sampling point offset is multiplied by a factor of 0.1. During training, this embodiment employs a multi-scale training strategy, randomly resizing the longer side of the image from 667 to 2000 pixels, while using zero padding to adapt to images of different resolutions. During inference, the test set images are scaled to a fixed size of 1500x900.

[0043] This embodiment uses two datasets: one is CUHK-SYSU, jointly created by the Chinese University of Hong Kong and Sun Yat-sen University, containing 18,184 images and 96,143 pedestrian bounding box annotations, covering 8,432 labeled identities. The training set contains 11,206 images and 5,532 query persons, while the test set contains 6,978 images and 2,900 query persons. The images come from two sources: real street view snapshots and movie / television footage. The other dataset is PRW, a dataset of complex real-world scenes released by a team from the University of Science and Technology of China. It was collected by six campus surveillance cameras and contains a total of 11,816 video frames. The training set contains 5,704 images and 482 pedestrian identities, while the test set contains 6,112 images and 450 pedestrian identities. In addition, the IDs of 34,034 pedestrian identities in the dataset range from 0 to 932, and the rest are labeled with -2 to indicate uncertain pedestrians.

[0044] The evaluation metrics used in this experiment were mean average precision (mAP) and top-1 accuracy (Top-1). The experimental results are shown in Tables 1 and 2.

[0045] Table 1. Comparison on the CUHK-SYSU dataset

[0046] Table 2 Comparison on the PRW dataset

[0047] Experimental results show that the proposed method achieves mAP and Top-1 accuracy of 95.3% and 95.2%, respectively, which are improvements over current mainstream methods. In particular, compared with AlignPS, it achieves relative gains of 1.7% and 1.3%, respectively. This fully demonstrates the effectiveness of the proposed method in pedestrian search on large-scale video datasets. Experimental results on the PRW dataset show that the proposed method achieves mAP and Top-1 accuracy of 54.6% and 88.7%, respectively, which are significant performance improvements over current mainstream methods. Especially compared with AlignPS, it achieves relative gains of 3.3% and 3.0%, respectively, fully demonstrating the superior performance of the proposed method in real-world, complex scenarios.

[0048] To visually demonstrate the effect of feature alignment using this module, feature maps at different resolutions were obtained from the geometric adaptive sampling module, and sampling points and attention weights were plotted. Each sampling point was marked with a solid circle, the color of which represents the corresponding attention weight. The reference point is indicated by a green cross in the image, as shown below. Figure 5As shown in the figure, the hierarchical position encoding module provides relative position information. Simultaneously, the adaptive modulation mechanism based on the prediction box allows the method to dynamically adjust the sampling position according to the pedestrian's posture. Therefore, the visualization results show that the sampling point positions and weights are dynamically adjusted according to different reference points for different pedestrians. Pedestrian features achieve effective alignment in both region and posture, strongly demonstrating the technical superiority of the proposed deformable attention multi-scale feature alignment method in eliminating region feature alignment errors and improving the adaptability of pedestrian posture changes.

[0049] like Figure 6 As shown, this invention also provides a pedestrian search device based on multi-scale deformable attention feature alignment. The device includes at least one processor and at least one memory, as well as a communication interface and an internal bus. The memory stores a computer-executable program for the pedestrian search system based on multi-scale deformable attention feature alignment as described above. When the processor executes the computer-executable program stored in the memory, it can execute a pedestrian search method based on multi-scale deformable attention feature alignment. The internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus in the accompanying drawings is not limited to only one bus or one type of bus. The memory may include high-speed RAM memory, and may also include non-volatile memory (NVM), such as at least one disk storage device, or a USB flash drive, portable hard drive, read-only memory, disk, or optical disk, etc.

[0050] The device may be provided as a terminal, server, or other type of device. In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0051] The present invention also provides a computer-readable storage medium storing a computer executable program of the pedestrian search system based on multi-scale deformable attention feature alignment as described above. When the computer executable program is executed by a processor, the processor can execute a pedestrian search method based on multi-scale deformable attention feature alignment.

[0052] Specifically, a system, apparatus, or device may be provided equipped with a readable storage medium on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system, apparatus, or device reads and executes the instructions stored in the readable storage medium. In this case, the program code read from the readable medium itself can implement the functions of any of the embodiments described above, therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0053] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0054] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A pedestrian search system based on multi-scale deformable attention feature alignment, characterized in that: The system is built on an anchor-free deep convolutional neural network, which includes a feature extraction backbone network, a context-aware feature enhancement module, a geometry-adaptive sampling module, a detection branch, and a re-identification branch connected in sequence. The feature extraction backbone network takes the original complex panoramic image as input and uses multiple convolutional kernels and pooling layers of a deep convolutional neural network to perform forward propagation and downsampling processing on the input image layer by layer, extracting visual details at the bottom layer and semantic information at the top layer, and generating a multi-scale feature pyramid containing different spatial resolutions and semantic levels. The context-aware feature enhancement module takes a multi-scale feature pyramid as input and generates an enhanced multi-scale feature map by injecting relative position information and scale semantics. The geometric adaptive sampling module takes the enhanced multi-scale feature map as input and uses a cross-scale sparse sampling strategy based on the attention mechanism to sample and fuse multi-scale features. In this process, the geometric prior information obtained by the adaptive sampling mechanism based on the prediction box is used to dynamically adjust the offset of the sampling points, so as to achieve geometric alignment between the feature receptive field and the pedestrian pose, and generate an aligned feature map. The detection branch and the re-identification branch are connected in parallel after the geometric adaptive sampling module. The detection branch uses the aligned feature map to predict the center point position and geometric boundary information of the pedestrian, while the re-identification branch uses the aligned feature map to generate a discriminative pedestrian identity feature vector.

2. The pedestrian search system based on multi-scale deformable attention feature alignment as described in claim 1, characterized in that: The context-aware feature enhancement module includes a convolutional position encoding unit (CPE) and a scale-level embedding unit. The convolutional position encoding unit uses a 3×3 deep convolutional layer (DWConv) to process the features of each layer of the multi-scale feature pyramid. It utilizes the sliding window property of the convolutional kernel to implicitly model the relative positional relationship between feature points and their neighboring pixels in order to maintain translational equivariance. The scale-level embedding unit introduces a learnable scale embedding vector for each layer of the feature pyramid. After the spatial dimension is expanded through a broadcast mechanism, it is added element-wise to the feature map to construct a multi-scale feature space that distinguishes different levels of semantics.

3. The pedestrian search system based on multi-scale deformable attention feature alignment as described in claim 1, characterized in that: The specific processing flow of the prediction box based on the geometric adaptive sampling module is as follows: First, obtain the four-dimensional distance vector predicted by the detection branch for the current query element Query. ,in These represent the distances from the current point to the left, right, top, and bottom boundaries of the target, respectively; secondly, the true width of the target is calculated based on the four-dimensional distance vector. and height And calculate the modulation weight vector based on the aspect ratio. ; Finally, the modulation weight vector The original sampling offset learned by the network Perform Hadamard product operation to reconstruct the final sampling coordinate offset. ,Right now: in, Represents element-wise multiplication; when At that time, the weight vector The component in the vertical direction is amplified; when At that time, the weight vector The horizontal component is amplified, forcing the sampling point distribution to adapt to the pedestrian's physical posture.

4. The pedestrian search system based on multi-scale deformable attention feature alignment as described in claim 1, characterized in that: The geometric adaptive sampling module introduces a geometric prior initialization strategy. In the initial stage of model training, based on the prior knowledge of pedestrian structure consistency, each query element is assigned a corresponding... The offset of each sampling point is initialized to be uniformly distributed along the vertical axis Y of the reference point; specifically, the weight matrix of the fully connected Linear Layer used to generate the sampling offset is initialized to 0, and the bias vector is initialized to the expanded value of a preset set of vertical coordinates. The range of values ​​of the set of vertical coordinates covers the area from the head to the feet of the human body, so as to force the region of interest to align with the human skeletal structure in the early stage of training and accelerate network convergence.

5. The pedestrian search system based on multi-scale deformable attention feature alignment as described in claim 3, characterized in that: The geometric adaptive sampling module employs a cross-scale sparse sampling strategy; for each query element on the feature map, it is based solely on the reference point and the final sampling coordinate offset. A small number of key sampling points are determined on a multi-scale feature map; the feature values ​​of the key sampling points are obtained on the multi-scale feature map using bilinear interpolation, and the learned attention weights are used to adaptively aggregate information from different feature levels, thus fusing deep semantic features with shallow high-resolution geometric features.

6. The pedestrian search system based on multi-scale deformable attention feature alignment as described in claim 1, characterized in that: The model employs a joint loss function for end-to-end optimization during the training phase. The joint loss function includes a detection loss for optimizing the detection branch and a TOIM loss function for optimizing the re-identification branch.

7. A pedestrian search method based on multi-scale deformable attention feature alignment, characterized in that, Includes the following processes: S1, Obtain a panoramic image of the scene to be searched; S2, input the image into the pedestrian search system based on multi-scale deformable attention feature alignment as described in any one of claims 1 to 6; Features are extracted through a feature extraction backbone network, and location and scale information are injected using a context-aware feature enhancement module. The detection branch outputs the predicted bounding box of the pedestrian, and at the same time, the geometric adaptive sampling module dynamically adjusts the distribution of feature sampling points of the re-identification branch according to the geometric shape of the predicted box to extract the aligned identity features. S3 outputs the detected pedestrian bounding boxes and the corresponding identity recognition results.

8. A pedestrian search device based on multi-scale deformable attention feature alignment, characterized in that: The device includes at least one processor and at least one memory, the processor and the memory being coupled; the memory stores a computer-executable program for the pedestrian search system based on multi-scale deformable attention feature alignment as described in any one of claims 1 to 6; When the processor executes a computer program stored in the memory, it causes the processor to execute a pedestrian search method based on multi-scale deformable attention feature alignment.

9. A computer-readable storage medium storing a computer program or instructions for a pedestrian search system based on multi-scale deformable attention feature alignment as described in any one of claims 1 to 6, wherein when the program or instructions are executed by a processor, the processor performs a pedestrian search method based on multi-scale deformable attention feature alignment.

Images