A dense pedestrian detection method and system based on an improved D-FINE-N network

Abstract

This application belongs to the field of computer vision technology and discloses a dense pedestrian detection method and system based on an improved D-FINE-N network. It acquires and divides the image dataset of the scene to be detected, and uses the D-FINE-N network as a foundation. Multi-scale gating feature enhancement operations are introduced in the encoder feature extraction to improve the local feature extraction unit, and residual fusion upsampling operations are introduced in the feature fusion to improve the upsampling structure, resulting in an improved model. A dense pedestrian detection model is trained using the dataset, and the image to be detected is input into the model for detection, outputting the dense pedestrian detection result. This application can improve the accuracy and robustness of detection in complex scenes while maintaining lightweight design.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to image detection technology, and particularly to a dense pedestrian detection method and system based on an improved D-FINE-N network. Background Technology

[0002] Dense pedestrian detection is one of the important research topics in intelligent security, intelligent transportation, video surveillance of public places, and urban governance.

[0003] Traditionally, pedestrian detection in scenes has relied primarily on manual inspection or image analysis methods based on empirical rules. This approach is highly dependent on human experience, resulting in low efficiency, strong subjectivity, difficulty adapting to complex environments, and inability to meet the demands of large-scale real-time detection. With the development of deep learning technology, object detection methods based on convolutional neural networks and end-to-end detection frameworks have been widely applied to pedestrian detection tasks, achieving certain results in general scenarios.

[0004] Existing dense pedestrian detection methods mainly fall into the following categories: The first category consists of detection methods based on sliding windows, handcrafted features, and traditional classifiers. These methods typically rely on handcrafted features such as HOG and Haar, as well as classifiers such as support vector machines. Their feature representation capabilities are limited, making them difficult to adapt to complex occlusion and scale changes. The second category consists of methods based on two-stage detection frameworks. These methods typically generate candidate regions first, and then classify and regress the candidate regions. The detection accuracy is relatively high, but the computation process is complex, and in dense scenes, they are easily affected by overlapping candidate boxes and blurred target boundaries due to occlusion. The third category consists of detection methods based on one-stage convolutional neural networks. These methods directly predict the category and bounding box on the feature map, resulting in fast inference speed. However, they still have shortcomings in distinguishing small-scale pedestrians, overlapping pedestrians, and neighboring targets. The fourth category consists of detection methods based on Transformer or end-to-end query mechanisms. These methods can utilize global attention to model the relationships between targets, but in lightweight deployment scenarios, they still need to balance detection accuracy, model parameter count, and computational cost.

[0005] Therefore, existing methods for detecting dense pedestrians still have certain limitations, mainly in the following aspects: (1) In complex and dense scenes, pedestrian targets usually have characteristics such as large scale variation, dense arrangement, severe mutual occlusion and complex background interference. Existing methods are still insufficient in extracting local detail features, occlusion edge features and differences between neighboring targets, which easily leads to missed detection, false detection and difficulty in distinguishing adjacent targets.

[0006] (2) In the feature fusion process, existing upsampling methods usually adopt relatively simple structures such as nearest neighbor interpolation, bilinear interpolation, or transposed convolution. Specifically, the common approach is to first restore the high-level semantic features to the same spatial size as the low-level features by upsampling at a fixed ratio, and then fuse them with the low-level spatial detail features by feature addition or channel concatenation. Finally, 1×1 convolution or 3×3 convolution is used for channel adjustment and feature smoothing. Although the above structures are simple to implement and have low computational cost, most upsampling operations themselves do not have adaptive feature selection capabilities, and it is difficult to reweight the restored features according to the differences between occluded edges, small-scale targets, and neighboring targets in dense pedestrian scenes. At the same time, simple concatenation or addition can easily lead to insufficient fusion between high-level semantic information and low-level spatial detail information, resulting in a significant loss of detail information during feature restoration, thereby affecting the detection performance of small targets, occluded targets, and neighboring targets in dense pedestrian scenes. Summary of the Invention

[0007] This invention aims to address the problems of insufficient local detail feature extraction capability and loss of detail information and insufficient fusion during multi-scale feature fusion in existing dense pedestrian detection methods in complex and dense scenes. It provides a dense pedestrian detection method and system based on an improved D-FINE-N network.

[0008] In a first aspect, the present invention provides a dense pedestrian detection method based on an improved D-FINE-N network, comprising the following steps: A dataset of dense pedestrian scene images was obtained, and after annotation processing, it was divided into training set, validation set and test set; An improved D-FINE-N network is constructed, which includes an encoder. The encoder is equipped with a multi-scale gated feature enhancement module for receiving input features. Local features are extracted through parallel multi-scale convolutional branches and the feature response is adaptively enhanced by a gated weighting mechanism. The parallel multi-scale convolutional branches include a first depthwise separable convolutional branch and a second depthwise separable convolutional branch. The improved D-FINE-N network's feature fusion path includes a residual fusion upsampling module. This module receives high-level features and, while restoring the feature space resolution, adaptively filters the high-level features through a gated fusion mechanism. The improved D-FINE-N network is iteratively optimized and trained using the training set, validation set, and test set. During the training process, the loss is calculated based on the predicted bounding boxes and the labeled ground truth bounding boxes in the network output to update the network parameters, thereby obtaining a dense pedestrian detection model. The image to be detected is input into the dense pedestrian detection model for detection, and the dense pedestrian detection result containing pedestrian target category information and location information is output.

[0009] As an optional implementation of the first aspect of this application, the acquisition of the dense pedestrian scene image dataset further includes: The acquired image dataset is subjected to data augmentation processing, which includes: In the early stage of training, a combined data augmentation strategy is adopted, which includes at least one of random shrink augmentation, random intersection-over-union pruning augmentation, and random horizontal flip augmentation. The combined data augmentation strategy was discontinued in the later stages of training, and only the basic size transformation and normalization were retained.

[0010] As an optional implementation of the first aspect of this application, the improved D-FINE-N network further includes a backbone network and a decoding prediction head, wherein: The backbone network uses HGNetv2_B0 to extract basic features of the input image and output multi-scale features. The encoder uses a HybridEncoder, and the multi-scale gated feature enhancement module is located inside the RepNCSPELAN4 module included in the encoder. The decoding prediction head is used to receive multi-scale features encoded by the encoder and fused with features, and output the prediction box.

[0011] As an optional implementation of the first aspect of this application, the multi-scale gated feature enhancement module further includes: The input features are fed into the first depthwise separable convolution branch and the second depthwise separable convolution branch respectively for parallel feature extraction. The first depthwise separable convolution branch uses a 3×3 kernel with a dilation rate of 1 to extract detailed information in the conventional local receptive field. The second depthwise separable convolution branch uses a 3×3 kernel with a dilation rate of 2 to extract contextual information in the extended receptive field without reducing spatial resolution. The output features of the first depthwise separable convolutional branch and the second depthwise separable convolutional branch are concatenated along the channel dimension to obtain a fused feature map; The fused feature map is subjected to pointwise convolution with a kernel of 1×1 to obtain an intermediate feature map. Then, the intermediate feature map is subjected to global average pooling and pointwise convolution with a kernel of 1×1. The Sigmoid activation function is used to generate gate weights with values ​​ranging from 0 to 1. The gating weights are multiplied element-wise with the intermediate feature map to obtain a weighted feature map; The weighted feature map is added to the input feature by residual addition to obtain the output feature of the multi-scale gated feature enhancement module.

[0012] As an optional implementation of the first aspect of this application, the residual fusion upsampling module further includes: The gated fusion mechanism is implemented by setting up parallel first and second convolutional branches; The high-level features are spatially magnified by a nearest neighbor upsampling operation with an upsampling factor of 2 to obtain upsampled features; The upsampled features are processed in parallel through the first convolutional branch and the second convolutional branch. The first convolutional branch includes a depthwise separable convolution with a 3×3 kernel and a pointwise convolution with a 1×1 kernel, used to extract local spatial details from the upsampled features and obtain reconstructed features. The second convolutional branch includes a depthwise separable convolution with a 3×3 kernel, a pointwise convolution with a 1×1 kernel, and a sigmoid activation function, used to generate corresponding gating weights. The reconstructed features output from the first convolutional branch are multiplied element-wise with the gating weights output from the second convolutional branch to obtain the gated fusion features. The residuals of the gated fusion features and the upsampled features are added together to obtain the output features of the residual fusion upsampling module.

[0013] As an optional implementation of the first aspect of this application, the step of calculating the loss based on the predicted bounding boxes and labeled ground truth bounding boxes during training to update the network parameters includes: The training set is input into the improved D-FINE-N network for forward propagation to obtain the category prediction results and bounding box prediction results of the pedestrian target, so as to generate the prediction box; A Hungarian matching strategy is used to establish a one-to-one correspondence between the predicted bounding boxes and the ground truth bounding boxes. The comprehensive matching cost of the Hungarian matching strategy includes classification cost and bounding box cost. L 1. Cost and GIoU cost; A joint loss function is constructed based on the category prediction results and the bounding box prediction results. The joint loss function includes a classification loss function, a bounding box regression loss function, and a center point loss function. The improved D-FINE-N network is backpropagated and its parameters are updated according to the joint loss function, and the model with the best detection performance is selected as the dense pedestrian detection model using the validation set.

[0014] As an optional implementation of the first aspect of this application, the classification loss function, the bounding box regression loss function, and the center point loss function include: The classification loss function uses the Varifocal loss function, and the bounding box regression loss function includes...L 1 loss and GIoU The loss, the center point loss function includes local loss and DDF loss.

[0015] Secondly, embodiments of this application provide a dense pedestrian detection system based on an improved D-FINE-N network, comprising: The data acquisition module is used to acquire dense pedestrian scene image datasets, and after annotation processing, divides them into training set, validation set and test set; An encoder improvement module is used to construct an improved D-FINE-N network. The improved D-FINE-N network includes an encoder. The encoder is equipped with a multi-scale gated feature enhancement module for receiving input features. Local features are extracted through parallel multi-scale convolutional branches and the feature response is adaptively enhanced by combining a gated weighting mechanism. The parallel multi-scale convolutional branches include a first depthwise separable convolutional branch and a second depthwise separable convolutional branch. The fusion path improvement module is used to set a residual fusion upsampling module in the feature fusion path of the improved D-FINE-N network. The residual fusion upsampling module is used to receive high-level features and adaptively filter the high-level features through a gating fusion mechanism while restoring the feature space resolution. The model training module is used to iteratively optimize and train the improved D-FINE-N network using the training set, validation set, and test set. During the training process, the loss is calculated based on the predicted bounding boxes and the labeled ground truth bounding boxes output by the network to update the network parameters and obtain a dense pedestrian detection model. The target detection module is used to input the image to be detected into the dense pedestrian detection model for detection, and output dense pedestrian detection results containing pedestrian target category information and location information.

[0016] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0017] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0018] Compared with existing technologies, this invention proposes a dense pedestrian detection method based on an improved D-FINE-N network, which has the following beneficial effects: In the multi-scale gated feature enhancement network, local features under different receptive fields are extracted by parallel convolutional branches, and the feature response is adaptively enhanced by combining gated weighting and residual connections. This can improve the modeling ability of occlusion edges, fine-grained details and differences in neighboring targets in dense pedestrian scenes, thereby improving the problem of missed detection and false detection in complex scenes. By using a residual fusion upsampling network, the high-level semantic features after upsampling are reconstructed and enhanced while restoring the feature space resolution. This provides a more sufficient feature expression for subsequent fusion with low-level spatial detail features, which can reduce the problem of loss of detail information in the traditional upsampling process and improve the feature expression effect of pedestrian targets at different scales. Therefore, the present invention proposes a dense pedestrian detection method based on an improved D-FINE-N network, which not only enhances the ability to extract local features and fuse features in complex scenes, but also improves the accuracy and robustness of dense pedestrian detection while maintaining low computational overhead. Attached Figure Description

[0019] Figure 1 This is a flowchart of a dense pedestrian detection method based on an improved D-FINE-N network; Figure 2 This is a network diagram illustrating the dense pedestrian detection method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the multi-scale gating feature enhancement module in the dense pedestrian detection method according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the residual fusion upsampling module in the dense pedestrian detection method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a dense pedestrian detection system based on an improved D-FINE-N network. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0022] Example 1 In this application, D-FINE-N is the name of the basic object detection network, used as the basic detection model for dense pedestrian detection tasks; HybridEncoder represents the hybrid encoder in the D-FINE-N network, used to encode and fuse the multi-scale features output by the backbone network; RepNCSPELAN4 represents the local feature extraction structure in the basic network encoder, used for convolutional feature extraction and feature channel fusion; MSG-RepNCSPELAN4 represents the local feature extraction structure with an embedded multi-scale gated feature enhancement module; and RA-CASAB represents the residual fusion upsampling module.

[0023] MSGBlock is a custom module name (Multi-Scale Gated Block), used to enhance local features through regular depthwise separable convolution branches, dilated depthwise separable convolution branches, global average pooling, and the Sigmoid gating mechanism; Sigmoid refers to the Sigmoid activation function. The English names DCRU, Transformer encoder, etc., appearing in the attached figures and below refer to existing functional units in the basic network related to feature transformation and encoding processing.

[0024] Please see Figure 1 This is a flowchart illustrating a dense pedestrian detection method based on an improved D-FINE-N network, provided by an embodiment of the present invention. The method may include the following steps: S1: Obtain a dataset of dense pedestrian scene images, perform annotation processing, and divide it into training set, validation set, and test set.

[0025] In this step, images are selected from publicly available dense pedestrian scene datasets, ensuring coverage of pedestrian samples with varying densities, degrees of occlusion, scales, and background conditions. Next, the label files are organized and converted, transforming the original dataset annotation format into an object detection annotation format suitable for subsequent training and validation. Based on the size of the image set and the distribution of target samples, the image dataset is divided into training, validation, and test image sets, ensuring a reasonable sample-to-target ratio among the three.

[0026] To obtain a more accurate and efficient training image set, data cleaning was performed on both the training and validation sets. Images without labeled boxes, images with duplicate labeled boxes, and data with obviously unreasonable labels were removed. Furthermore, some target boxes with significant labeling errors were manually corrected. The test set required no additional processing.

[0027] In optimizing the training image set, various image enhancement techniques were employed to expand the diversity and richness of the data. These techniques include: (1) Mosaic enhancement method, which involves randomly selecting multiple images from the dataset, performing independent augmentation, and then stitching them together into one image; (2) Image blending technology, which involves randomly selecting two sample images and blending them in a certain proportion, and the target information is also distributed in a corresponding proportion; (3) Random flipping and random scaling techniques: randomly flip the image horizontally or vertically, and randomly adjust the image size to simulate different scale changes; (4) Random affine transformation technology involves operations such as rotation, translation, scaling and shearing to simulate various geometric changes in complex scenes.

[0028] In this specific implementation example, the WiderPerson dataset is used. The images in this dataset are all from real-world dense pedestrian detection tasks. The images contain pedestrian targets of different scales, occlusion levels, shooting distances, and density distributions, effectively reflecting the needs of dense pedestrian detection in complex scenes. The WiderPerson dataset contains 13,382 images and approximately 399,786 annotations, with 8,000 images in the training set, 1,000 in the validation set, and 4,382 in the test set. The diverse scene images ensure comprehensive coverage of pedestrian targets at long distances, small scales, with mutual occlusion, and complex backgrounds, enhancing the robustness and generalization ability of the model trained on this dataset.

[0029] Table 1. Pedestrian Detection Dataset Partitioning Table The pedestrian detection dataset is divided as shown in Table 1 above.

[0030] In this embodiment, the input size of the training images is uniformly adjusted to 640×640, with 32 images per training run and 200 training epochs. To improve generalization ability, a combined data augmentation strategy is adopted in the early stage of training, while some strong augmentation operations are turned off in the later stage. Specifically, random scaling-up augmentation, random intersection-over-union (IoU) cropping augmentation, and random horizontal flipping augmentation are enabled in the first 128 training epochs; random scaling-up augmentation and random IoU cropping augmentation are stopped in the last 72 training epochs, and only the basic size transformation and necessary normalization processing are retained to reduce the localization error caused by excessive perturbation in the later stage of training.

[0031] Random scaling enhancement: The original image is randomly scaled down with a certain probability and placed in a larger background canvas, with the background area filled with preset pixel values ​​or the dataset mean; then the target bounding box coordinates are adjusted synchronously according to the scaling ratio and placement offset. This enhancement can increase the proportion of small target samples, making it easier for the model to learn the spatial distribution characteristics of dense pedestrians at a distance.

[0032] Random Intersection Over Union (IoU) Cropping Enhancement: A threshold is randomly selected from a preset set of minimum IoU thresholds to randomly generate candidate cropping regions. When a cropping region satisfies the corresponding IoU constraint with at least one ground truth bounding box, and the target's center point is located within the cropping region, the cropping region is retained, and both the image and the target bounding box are cropped simultaneously. For target bounding boxes that exceed the image boundary after cropping, their coordinates are truncated to the range of the cropping region. This enhancement improves the model's adaptability to local occlusion, boundary truncation, and dense target distributions.

[0033] Random Horizontal Flip Enhancement: The input image is horizontally flipped with a probability of 0.5, and the horizontal and vertical coordinates of the target bounding box are updated simultaneously. If the original target bounding box is represented as... The image width is W Then the horizontal and vertical coordinates of the target bounding box after horizontal flipping are updated to... ;in This represents the left boundary of the original target bounding box. This represents the right boundary of the original target bounding box. This represents the upper boundary of the original target bounding box. This represents the lower boundary of the original target bounding box.

[0034] After the above enhancements are performed, the images are uniformly scaled or padded to 640×640, and the pixel values ​​are normalized to meet the input requirements of the improved D-FINE-N network. This data augmentation strategy can increase the scale variation and spatial distribution diversity of dense pedestrian targets while maintaining the realism of the training samples.

[0035] S2: Construct an improved D-FINE-N network, which includes an encoder. The encoder is equipped with a multi-scale gated feature enhancement module for receiving input features. Local features are extracted through parallel multi-scale convolutional branches and the feature response is adaptively enhanced by a gated weighting mechanism. The parallel multi-scale convolutional branches include a first depthwise separable convolutional branch and a second depthwise separable convolutional branch.

[0036] The improved D-FINE-N network includes a backbone network, an encoder, a feature fusion structure, and a decoding prediction head. The backbone network uses HGNetv2_B0; the encoder uses HybridEncoder.

[0037] See Figure 2 The multi-scale gated feature enhancement module (MSGBlock) is located within the RepNCSPELAN4 module of the HybridEncoder (forming the MSG-RepNCSPELAN4 structure) and is used to improve the local feature extraction unit, where k is the convolution kernel size and s is the stride. Let the input features be... ,in C Indicates the number of input channels. H and W These represent the feature map height and width, respectively. The input features are first processed through a 1×1 convolution with a stride of 1 for channel adjustment, resulting in intermediate features. These intermediate features are then divided into two parts along the channel dimension, each with approximately 0.5 times the number of output channels. One part is directly retained as short-path features, while the other part is sequentially processed through MSGBlock, a 3×3 convolution with a stride of 1, a second MSGBlock, and another 3×3 convolution with a stride of 1 to achieve local feature enhancement.

[0038] Inside MSGBlock, input features X They are simultaneously fed into two parallel depthwise separable convolution branches: First depthwise separable convolution branch (first branch): Employs a depthwise separable convolution with a kernel of 3×3, a stride of 1, and an inflation rate of 1 to extract detailed information from the conventional local receptive field; The second depthwise separable convolution branch (second branch): employs a 3×3 kernel, a stride of 1, and a dilation rate of 2 to expand the effective receptive field without reducing spatial resolution, extracting contextual information about differences between neighboring objects and occlusion edges. Both branches maintain the output feature map size through appropriate padding operations. H×W .

[0039] The features output from the two branches are concatenated along the channel dimension to obtain 2. CThe channel fusion features are then used to perform channel compression and cross-channel information fusion through pointwise convolution with a 1×1 kernel and a stride of 1, restoring the number of channels to [value missing]. C The intermediate feature map is obtained. Then, global average pooling is performed on this fused feature map to obtain a channel-level global context vector. Next, a 1×1 convolution and a sigmoid activation function are applied to generate gate weights with values ​​ranging from 0 to 1. These gate weights are then multiplied element-wise with the intermediate feature map processed by the 1×1 convolution to obtain a weighted feature map. Finally, the weighted feature map is added to the residual of the MSGBlock input features to obtain the module output feature map.

[0040] Through the aforementioned data flow, MSGBlock maintains the same number of input and output channels while simultaneously enhancing local features of dense pedestrians using both conventional and extended local receptive fields. Furthermore, it employs a gating mechanism to adaptively filter effective feature responses, thereby improving the model's ability to model occlusion edges, differences between neighboring targets, and fine-grained local information.

[0041] Setting the dilation rate of the second depthwise separable convolutional branch to 2 is to align with the feature map downsampling factor of the D-FINE-N network in dense pedestrian detection tasks. Taking an input image size of 640×640 as an example, the main fused features output by the D-FINE-N backbone and HybridEncoder include scales of 40×40 and 20×20, corresponding to downsampling ratios of approximately 16x and 32x respectively. When using a standard 3×3 convolution on a 40×40 feature map, its single local modeling range corresponds to a neighborhood of approximately 3×16 pixels in the original image; when the dilation rate is set to 2, the effective kernel size of the 3×3 convolution expands to 5×5, corresponding to a neighborhood of approximately 5×16 pixels in the original image. This setting expands the perceptual range for nearby pedestrian targets, occlusion edges, and local background context without further reducing feature map resolution or introducing a large number of additional convolutional kernel parameters. If the dilation rate is set to only 1, the receptive field difference between the second branch and the regular local branch is small, making it difficult to reflect the multi-scale feature enhancement effect. If the dilation rate is set too large, it is easy to introduce too much irrelevant background information and may weaken the boundary details of dense small targets. Therefore, a dilation rate of 2 can achieve a good balance between local detail preservation and contextual range expansion, and has a strong coupling relationship with the specific downsampling factor of the D-FINE-N network.

[0042] S3: The feature fusion path of the improved D-FINE-N network is equipped with a residual fusion upsampling module. The residual fusion upsampling module is used to receive high-level features and, while restoring the feature space resolution, adaptively filter the high-level features through a gating fusion mechanism.

[0043] See Figure 3 The residual fusion upsampling module replaces the original upsampling structure in the feature fusion path of the basic detection network. Its input is high-level semantic features. The processing includes: First, high-level semantic features Spatial scale recovery is performed using nearest neighbor upsampling with an upsampling factor of 2, resulting in upsampled features. Nearest neighbor upsampling does not introduce additional learnable parameters, enabling scaling while maintaining lightweight design.

[0044] Subsequently, upsampling features U The data is fed into two branches in parallel: The first convolutional branch (feature reconstruction branch) includes a depthwise separable convolution with a 3×3 kernel and a stride of 1, and a pointwise convolution with a 1×1 kernel and a stride of 1. These are used to extract local spatial detail information from the upsampled features, resulting in the reconstructed features. R .

[0045] The second convolutional branch (gated weight generation branch) includes, in sequence, a depthwise separable convolution with a 3×3 kernel and a stride of 1, a pointwise convolution with a 1×1 kernel and a stride of 1, and a sigmoid activation function, used to generate gated weights with the same size as the reconstructed features. M .

[0046] Then, the reconstructed features output from the first branch are... R Gating weights of the second branch output M Element-wise multiplication is performed to obtain the gated fusion features. R ⊙ M (⊙ indicates element-wise multiplication). Finally, the gated fusion features are combined with the upsampled features. U The residuals are summed to obtain the output features of the residual fusion upsampling module. Y ,Right now Y=U + R ⊙ M When the input and output channels are inconsistent, a 1×1 convolution can be added at the residual path or branch output for channel alignment; in this embodiment... Figure 3 In the structure shown, the input channel and the output channel are consistent.

[0047] Through the aforementioned data flow, the residual fusion upsampling module not only completes the scale recovery of high-level semantic features, but also uses bi-branch convolution and gating mechanisms to adaptively filter the upsampled detail responses, and retains the original high-level semantic information through residual connections, thereby improving the fusion effect of high-level semantic features and low-level spatial detail features.

[0048] See Figure 4 The decoder and prediction head are positioned after the HybridEncoder and feature fusion structure, receiving the multi-scale fused features processed by the multi-scale gated feature enhancement module and the residual fusion upsampling module. Specifically, the 40×40×128 and 20×20×128 features output from the HybridEncoder are fed into the decoder and prediction head. The 40×40 features retain more spatial detail information, while the 20×20 features contain stronger high-level semantic information. These multi-scale features, after being processed through a unified channel, serve as the input features for the decoding and prediction head.

[0049] The decoding prediction head includes a query vector generation section, a decoding interaction section, a classification prediction branch, and a bounding box regression branch. Specifically, several target query vectors (e.g., 300 queries) are set according to the target quantity requirements of the dense pedestrian scene. Each query vector interacts with the encoded image features to obtain the instance features of the corresponding candidate target. The classification prediction branch outputs the pedestrian target category confidence, and the bounding box regression branch outputs the target bounding box coordinates. For the dense pedestrian detection task, the category prediction result is used to determine whether the candidate target is a pedestrian, and the bounding box prediction result is used to represent the position of the pedestrian target in the image.

[0050] The input channels of the decoder prediction head are consistent with the output channels of the encoder (e.g., 128). Each query ultimately outputs a class confidence score and bounding box coordinates. The bounding box can be represented as the normalized center point coordinates, width, and height. , and These are the coordinates of the center point of the actual bounding box. w The actual bounding box width. h This represents the actual bounding box height.

[0051] Through the above connections, the backbone network is responsible for extracting basic visual features, the HybridEncoder and the multi-scale gating feature enhancement module are responsible for enhancing local details and differences between local targets, the residual fusion upsampling module is responsible for improving the quality of multi-scale feature fusion, and the decoding prediction head completes category judgment and position regression based on the fused features.

[0052] S4: The improved D-FINE-N network is iteratively optimized and trained using the training set, validation set, and test set. During the training process, the loss is calculated based on the predicted bounding boxes and the labeled ground truth bounding boxes in the network output to update the network parameters, thereby obtaining a dense pedestrian detection model.

[0053] The training image set is input into the improved D-FINE-N network for forward propagation to obtain category prediction results and bounding box prediction results. In this embodiment, the positive and negative sample matching strategy adopts the Hungarian matching strategy. The matching cost is adapted to the task of this application to establish a one-to-one correspondence between predicted boxes and ground truth boxes, thereby determining positive and negative samples. The comprehensive matching cost is expressed as: ,in This represents the classification cost, used to measure the difference between the predicted class confidence and the true class. Represents the coordinates between the predicted bounding box and the ground truth bounding box. L 1 distance, Indicates the distance between the predicted bounding box and the ground truth bounding box. GIoU Geometric cost, , , These represent the weights of the corresponding cost terms.

[0054] A loss function is constructed based on the prediction results, and backpropagation and parameter updates are performed. The total loss includes classification loss, bounding box loss, and center point loss.

[0055] (1) The classification loss uses the Varifoccal loss function: ,in Represents the Varifocal classification loss. p This represents the confidence level of the class predicted by the model. q This represents the quality score of the supervised target (when a match is successful, the intersection is performed and the quality scores are compared; when no match is found, the comparison is performed). q =0), and As a hyperparameter, the classification loss weight is set to 1.0, allowing the classification loss and localization loss to participate in the model parameter update together. This loss function enables high-quality localization boxes to receive higher classification supervision weights, thereby prompting the model to prioritize learning the prediction results of higher-quality localized targets in dense pedestrian scenes and reducing the interference of low-quality candidate boxes on the training of the classification branch.

[0056] (2) Bounding box loss includes L 1 loss and GIoU Loss. Let the true bounding box be... The predicted bounding box is , and To predict the coordinates of the bounding box center point, To predict the bounding box width, To predict the bounding box height. L 1. Loss is used to measure the absolute deviation of the predicted bounding box from the ground truth bounding box in terms of coordinate values, and can be expressed as: . GIoUThe loss function is used to constrain the spatial relationship between the predicted bounding box and the ground truth bounding box from the perspective of region overlap. Let... A Indicates the predicted bounding box area. B Represents the actual bounding box region. C Indicates simultaneous encirclement A and B The smallest closure region, IoU express A and B The intersection-to-union ratio between them, then GIoU It can be represented as , ,in A , B These are the predicted and ground truth bounding boxes, respectively. C The smallest closure region. Indicates the area of ​​the region. This represents the union of the predicted bounding box and the ground truth bounding box. This represents the region within the closure that is not covered by either the predicted bounding box or the ground truth bounding box. L 1. Loss can directly constrain the accuracy of coordinate regression. GIoU The loss function can provide an effective direction for geometric optimization when the predicted bounding box and the ground truth bounding box do not completely overlap or even intersect. The box loss is... , and They represent L 1 loss and GIoU The weights for the loss are configured as 5.0 and 2.0 respectively.

[0057] (3) The center point loss includes local loss and DDF Loss: Localized loss , DDF loss ,in Representing the four boundary directions: left, top, right, and bottom, let the model be relative to the first... i The first direction K The prediction probability for each discrete interval is The corresponding target distribution is , K Indicates the discrete distance interval number. It is a minimal constant. The total loss at the center point is... . and Representing local loss and DDF The weights of the loss. Local loss can enhance the model's ability to locate the center position of dense pedestrian targets and reduce mismatches between neighboring targets caused by center point offset. DDF The loss function imposes fine-grained constraints on the boundary distance distribution, enabling the predicted bounding box to achieve more stable boundary localization results in densely occluded scenarios.

[0058] Since the detection category is mainly pedestrian targets, the classification cost is used to distinguish pedestrian targets from background candidates; the bounding box cost is used to ensure that the matching result prioritizes candidate boxes with higher localization quality. By keeping the matching cost consistent with the classification loss and bounding box loss of this application, the sample allocation during the training phase can be coordinated with the model optimization objective, thereby improving the localization stability in dense pedestrian scenes.

[0059] The model performance during training was validated using a validation image set, and the model with the best object detection performance was selected as the dense pedestrian detection model.

[0060] Table 2 Configuration Table for Model Training The configuration information for model training in this embodiment is shown in Table 2 above. The training strategy adopted is consistent with D-FINE-N, where the box loss weights of 5.0 / 2.0 correspond to... L 1. Loss weights GIoU Loss weights.

[0061] Table 3 Comparison of Experimental Results To verify the effectiveness of the technology, a test image set was used for testing. The final results are detailed in Table 3.

[0062] Table 4 Ablation Experiment Further ablation experiments are shown in Table 4. The results show that when only the multi-scale gating feature enhancement module is added, the model's ability to express local details, occlusion edges, and differences in neighboring targets in dense pedestrian scenes is enhanced, resulting in improved detection accuracy compared to the basic model. When only the residual fusion upsampling module is added, the model can more fully preserve high-level semantic information during feature space resolution restoration and adaptively filter upsampled spatial details, thus improving the detection performance for small-scale pedestrians and occluded targets.

[0063] When a multi-scale gating feature enhancement module and a residual fusion upsampling module are introduced simultaneously, the former mainly enhances the encoder's local feature extraction capability, while the latter mainly improves the fusion effect of high-level semantic features and low-level spatial detail features. These two modules respectively address the technical problems of insufficient local detail feature extraction and inadequate upsampling feature fusion. Therefore, the combined model D-FINE-NH achieves higher detection metrics than the basic D-FINE-N model while maintaining relatively stable parameter counts, demonstrating that the structural improvements proposed in this application can effectively improve the accuracy and robustness of pedestrian detection in complex and dense scenes.

[0064] S5: Input the image to be detected into the dense pedestrian detection model for detection, and output the dense pedestrian detection result containing pedestrian target category information and location information.

[0065] Specifically, the scene image to be tested is input into the trained dense pedestrian detection model. After feature extraction by the backbone network, local feature enhancement by the encoder, and feature fusion path upsampling, the target query result is output by the decoding prediction head. Background candidates are filtered according to category confidence, and the final output is a dense pedestrian detection result containing pedestrian target category information and bounding box location information.

[0066] In summary, this invention proposes a dense pedestrian detection method based on an improved D-FINE-N network. This method employs an improved network structure combining a multi-scale gated feature enhancement module and a residual fusion upsampling module, aiming to address the problems of insufficient extraction of local detail features and inadequate upsampling feature fusion in dense pedestrian scenes. By introducing a multi-scale gated feature enhancement module into the encoder, and utilizing parallel multi-scale convolutional branches and a gated weighting mechanism, the network's ability to model local details, occlusion edges, and differences in neighboring targets is effectively enhanced. By setting a residual fusion upsampling module in the feature fusion path, the feature space resolution is restored while the gated fusion mechanism is used to adaptively filter high-level features, improving the quality of multi-scale feature fusion. This method significantly improves the accuracy and robustness of dense pedestrian detection while maintaining a relatively stable number of model parameters.

[0067] Example 2 Please see Figure 5 The diagram shows a schematic of a dense pedestrian detection system based on an improved D-FINE-N network proposed in the second embodiment of this application. The system includes the following key modules: The data acquisition module 100 is used to acquire dense pedestrian scene image datasets, and after annotation processing, divides them into training set, validation set and test set; The encoder improvement module 200 is used to construct an improved D-FINE-N network. The improved D-FINE-N network includes an encoder. The encoder is provided with a multi-scale gated feature enhancement module for receiving input features. Local features are extracted through parallel multi-scale convolutional branches and the feature response is adaptively enhanced by combining a gated weighting mechanism. The parallel multi-scale convolutional branches include a first depthwise separable convolutional branch and a second depthwise separable convolutional branch. The fusion path improvement module 300 is used to set a residual fusion upsampling module in the feature fusion path of the improved D-FINE-N network. The residual fusion upsampling module is used to receive high-level features and adaptively filter the high-level features through a gating fusion mechanism while restoring the feature space resolution. The model training module 400 is used to iteratively optimize and train the improved D-FINE-N network using the training set, validation set and test set. During the training process, the loss is calculated based on the predicted boxes and labeled ground truth boxes output by the network to update the network parameters and obtain a dense pedestrian detection model. The target detection module 500 is used to input the image to be detected into the dense pedestrian detection model for detection, and output dense pedestrian detection results containing pedestrian target category information and location information.

[0068] The dense pedestrian detection system based on an improved D-FINE-N network in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), etc. This application embodiment does not impose specific limitations.

[0069] The dense pedestrian detection system based on an improved D-FINE-N network in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0070] This application provides a dense pedestrian detection system based on an improved D-FINE-N network, which can achieve... Figure 1 The various processes implemented in the method embodiment based on a dense pedestrian detection method using an improved D-FINE-N network will not be described again here to avoid repetition.

[0071] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiment of a dense pedestrian detection method based on an improved D-FINE-N network and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0072] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of a dense pedestrian detection method based on an improved D-FINE-N network and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0073] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0074] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0075] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0076] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A dense pedestrian detection method based on an improved D-FINE-N network, characterized in that, Includes the following steps: A dataset of dense pedestrian scene images was obtained, and after annotation processing, it was divided into training set, validation set and test set; An improved D-FINE-N network is constructed, which includes an encoder. The encoder is equipped with a multi-scale gated feature enhancement module for receiving input features. Local features are extracted through parallel multi-scale convolutional branches and the feature response is adaptively enhanced by a gated weighting mechanism. The parallel multi-scale convolutional branches include a first depthwise separable convolutional branch and a second depthwise separable convolutional branch. The improved D-FINE-N network's feature fusion path includes a residual fusion upsampling module. This module receives high-level features and, while restoring the feature space resolution, adaptively filters the high-level features through a gated fusion mechanism. The improved D-FINE-N network is iteratively optimized and trained using the training set, validation set, and test set. During the training process, the loss is calculated based on the predicted bounding boxes and the labeled ground truth bounding boxes in the network output to update the network parameters, thereby obtaining a dense pedestrian detection model. The image to be detected is input into the dense pedestrian detection model for detection, and the dense pedestrian detection result containing pedestrian target category information and location information is output.

2. The dense pedestrian detection method based on an improved D-FINE-N network according to claim 1, characterized in that, The acquisition of the dense pedestrian scene image dataset also includes: The acquired image dataset is subjected to data augmentation processing, which includes: In the early stage of training, a combined data augmentation strategy is adopted, which includes at least one of random shrink augmentation, random intersection-over-union pruning augmentation, and random horizontal flip augmentation. The combined data augmentation strategy was discontinued in the later stages of training, and only the basic size transformation and normalization were retained.

3. The dense pedestrian detection method based on an improved D-FINE-N network according to claim 1, characterized in that, The improved D-FINE-N network also includes a backbone network and a decoding prediction head, wherein: The backbone network uses HGNetv2_B0 to extract basic features of the input image and output multi-scale features; The encoder uses a HybridEncoder, and the multi-scale gated feature enhancement module is located inside the RepNCSPELAN4 module included in the encoder. The decoding prediction head is used to receive multi-scale features encoded by the encoder and fused with features, and output the prediction box.

4. A dense pedestrian detection method based on an improved D-FINE-N network according to claim 1 or 3, characterized in that, The multi-scale gated feature enhancement module also includes: The input features are fed into the first depthwise separable convolution branch and the second depthwise separable convolution branch respectively for parallel feature extraction. The first depthwise separable convolution branch uses a 3×3 kernel with a dilation rate of 1 to extract detailed information in the conventional local receptive field. The second depthwise separable convolution branch uses a 3×3 kernel with a dilation rate of 2 to extract contextual information in the extended receptive field without reducing spatial resolution. The output features of the first depthwise separable convolutional branch and the second depthwise separable convolutional branch are concatenated along the channel dimension to obtain a fused feature map; The fused feature map is subjected to pointwise convolution with a kernel of 1×1 to obtain an intermediate feature map. Then, the intermediate feature map is subjected to global average pooling and pointwise convolution with a kernel of 1×1. The Sigmoid activation function is used to generate gate weights with values ​​ranging from 0 to 1. The gating weights are multiplied element-wise with the intermediate feature map to obtain a weighted feature map; The weighted feature map is added to the input feature by residual addition to obtain the output feature of the multi-scale gated feature enhancement module.

5. The dense pedestrian detection method based on an improved D-FINE-N network according to claim 1, characterized in that, The residual fusion upsampling module further includes: The gated fusion mechanism is implemented by setting up parallel first and second convolutional branches; The high-level features are spatially magnified by a nearest neighbor upsampling operation with an upsampling factor of 2 to obtain upsampled features; The upsampled features are processed in parallel through the first convolutional branch and the second convolutional branch. The first convolutional branch includes a depthwise separable convolution with a 3×3 kernel and a pointwise convolution with a 1×1 kernel, used to extract local spatial details from the upsampled features and obtain reconstructed features. The second convolutional branch includes a depthwise separable convolution with a 3×3 kernel, a pointwise convolution with a 1×1 kernel, and a sigmoid activation function, used to generate corresponding gating weights. The reconstructed features output from the first convolutional branch are multiplied element-wise with the gating weights output from the second convolutional branch to obtain the gated fusion features. The residuals of the gated fusion features and the upsampled features are added together to obtain the output features of the residual fusion upsampling module.

6. The dense pedestrian detection method based on an improved D-FINE-N network according to claim 1, characterized in that, The step of calculating the loss based on the predicted bounding boxes and labeled ground truth bounding boxes during training to update network parameters includes: The training set is input into the improved D-FINE-N network for forward propagation to obtain the category prediction results and bounding box prediction results of the pedestrian target, so as to generate the prediction box; A Hungarian matching strategy is used to establish a one-to-one correspondence between the predicted bounding boxes and the ground truth bounding boxes. The comprehensive matching cost of the Hungarian matching strategy includes classification cost and bounding box cost. L 1. Cost and GIoU cost; A joint loss function is constructed based on the category prediction results and the bounding box prediction results. The joint loss function includes a classification loss function, a bounding box regression loss function, and a center point loss function. The improved D-FINE-N network is backpropagated and its parameters are updated according to the joint loss function, and the model with the best detection performance is selected as the dense pedestrian detection model using the validation set.

7. A dense pedestrian detection method based on an improved D-FINE-N network according to claim 6, characterized in that, The classification loss function, bounding box regression loss function, and center point loss function include: The classification loss function uses the Varifocal loss function, and the bounding box regression loss function includes... L 1 loss and GIoU The loss, the center point loss function includes local loss and DDF loss.

8. A dense pedestrian detection system based on an improved D-FINE-N network, characterized in that, include: The data acquisition module is used to acquire dense pedestrian scene image datasets, and after annotation processing, divides them into training set, validation set and test set; An encoder improvement module is used to construct an improved D-FINE-N network. The improved D-FINE-N network includes an encoder. The encoder is equipped with a multi-scale gated feature enhancement module for receiving input features. Local features are extracted through parallel multi-scale convolutional branches and the feature response is adaptively enhanced by combining a gated weighting mechanism. The parallel multi-scale convolutional branches include a first depthwise separable convolutional branch and a second depthwise separable convolutional branch. The fusion path improvement module is used to set a residual fusion upsampling module in the feature fusion path of the improved D-FINE-N network. The residual fusion upsampling module is used to receive high-level features and adaptively filter the high-level features through a gating fusion mechanism while restoring the feature space resolution. The model training module is used to iteratively optimize and train the improved D-FINE-N network using the training set, validation set, and test set. During the training process, the loss is calculated based on the predicted bounding boxes and the labeled ground truth bounding boxes output by the network to update the network parameters and obtain a dense pedestrian detection model. The target detection module is used to input the image to be detected into the dense pedestrian detection model for detection, and output dense pedestrian detection results containing pedestrian target category information and location information.

9. An electronic device, characterized in that, The method includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of a dense pedestrian detection method based on an improved D-FINE-N network as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of a dense pedestrian detection method based on an improved D-FINE-N network as described in any one of claims 1-7.

Images