A pedestrian detection method and device for a fence occluded scene

Abstract

The application relates to the field of video and image segmentation, and provides a pedestrian detection method and device for a fence shielding scene, wherein a preset detection model is trained by using an enhanced pedestrian data set to learn global features of complete pedestrians; the preset detection model is trained by using the enhanced pedestrian data set and a fence shielding data set to learn feature differences between pedestrians and fences; the preset detection model is optimized in a fine-grained optimization mode focusing on a preset shielding mode by using the fence shielding data set; a semantic segmentation branch in the preset detection model is removed to obtain a light detection model; features of a to-be-detected image are extracted by using the light detection model to obtain a pedestrian candidate frame; and the quality of the pedestrian candidate frame is optimized according to pedestrian coordinates and confidence by using an enhanced post-processing algorithm, and an optimized candidate frame is output; the application improves the detection capability of the model by constructing two types of complementary data sets and adopting a multi-section training strategy and data enhancement, and the detection precision is relatively high.

Description

Technical Field

[0001] This invention relates to the field of video and image segmentation, and more particularly to a method and apparatus for pedestrian detection in scenes with fence occlusion. Background Technology

[0002] In the field of security monitoring, pedestrian detection is one of the core technologies of intelligent video analytics. It is widely used in public area security and control, such as stations, squares, and park and community management. Its core objective is to identify pedestrian targets in the monitoring screen in real time and accurately, providing basic support for abnormal behavior warnings, such as intrusion and loitering; personnel trajectory tracking; and security risk assessment, which directly affects the reliability and response efficiency of the security system.

[0003] However, in actual monitoring scenarios, pedestrian targets often exhibit incomplete features due to environmental obstructions, severely impacting detection accuracy. Among these, fence-like objects, such as metal railings, mesh fences, and wrought iron railings, are a particularly representative and frequently occurring problem in security scenarios. These obstructions are widely present in various security areas requiring physical isolation, such as residential perimeter walls, station perimeter barriers, and factory boundaries. Their structures are often regular vertical stripes, grids, or intersecting textures, and spatially, they often form a foreground-obstruction-background or obstruction-foreground relationship with the pedestrian target. That is, the pedestrian's body may be partially or mostly obscured by the fence, with only the head, arms, legs, or other localized areas exposed, or the fence texture may directly cover the pedestrian's outline.

[0004] Accurate detection of pedestrians obscured by fences is crucial to the effectiveness of security systems. From a security perspective, fences, as core physical barriers, are often associated with pedestrian behavior that could lead to boundary crossings or intrusions. If obstruction results in missed detections, individuals illegally climbing or scaling fences may go undetected, or suspicious individuals inside the fence, such as potential vandals, may be mistaken for background figures, creating security vulnerabilities. From a system reliability perspective, fence obstruction can easily lead to false negatives (missed detection of obscured pedestrians) or false positives (misinterpreting fence textures as local pedestrian features). This directly reduces the accuracy of subsequent behavior analysis and trajectory tracking, impacting the effectiveness of security decisions. From a scenario adaptability perspective, with the widespread adoption of high-definition surveillance and large-scale security coverage, the proportion of scenarios involving fence obstruction continues to increase. The limitations of traditional detection methods have become a significant bottleneck restricting the upgrade of security systems to all-weather, all-scenario capabilities.

[0005] Methods based on handcrafted features, such as HOG+SVM, rely on global features like complete edges and contours of pedestrians, making them sensitive to occlusion. The regular texture of fences can easily be confused with local edges of pedestrians, such as arm lines and clothing folds, leading to an inability to effectively distinguish between occluded objects and pedestrian information during feature extraction, resulting in a significant decrease in detection accuracy.

[0006] While mainstream deep learning-based methods, such as YOLO and Faster R-CNN, have improved robustness to partial occlusion through deep feature learning, they haven't optimized for the specific characteristics of fence-type occlusion. On one hand, the rigid texture of fences, such as vertical stripe spacing and grid density, exhibits strong regularity and is easily misidentified by the model as key pedestrian features, such as torso contours, leading to false positives. On the other hand, when a pedestrian is occluded by a fence, the proportion of effective feature regions, such as the face and torso, decreases, making it difficult for the model to anchor the pedestrian target from the incomplete features, resulting in false negatives.

[0007] Current improvements to pedestrian occlusion detection, such as attention-based feature enhancement and context-based completion, primarily focus on mutual occlusion between pedestrians, such as partial occlusion within a crowd. However, the occluders are flexible, irregular human body regions, which differ significantly from rigid, regularly textured occluders like fences, making direct application impossible and resulting in insufficient adaptability to fence occlusion. Summary of the Invention

[0008] This invention provides a pedestrian detection method and apparatus for fence-obstructed scenarios, which solves the shortcomings of existing technologies that cannot effectively distinguish between obstruction and pedestrian information and have low detection accuracy. It can effectively separate fence texture and pedestrian features and accurately identify obstructed pedestrians, thereby improving the reliability and practicality of security systems in complex obstruction environments.

[0009] This invention provides a pedestrian detection method for scenarios where fences obstruct the view, comprising the following steps.

[0010] Using an augmented pedestrian dataset, a pre-defined detection model is trained to learn the global features of a complete pedestrian; using an augmented pedestrian dataset and a fence occlusion dataset, a pre-defined detection model is trained to learn the feature differences between pedestrians and fences.

[0011] Using a fence occlusion dataset, the preset detection model is optimized by focusing on fine-grained optimization of preset occlusion patterns; the semantic segmentation branch in the preset detection model is removed to obtain a lightweight detection model.

[0012] A lightweight detection model is used to extract features from the image to be detected, resulting in pedestrian candidate boxes. An enhanced post-processing algorithm is then used to optimize the quality of the pedestrian candidate boxes based on pedestrian coordinates and confidence scores, outputting optimized candidate boxes. The pedestrian candidate boxes include pedestrian coordinates and confidence scores.

[0013] A pedestrian detection method for a fence-occluded scene according to the present invention further includes:

[0014] The pedestrian dataset was obtained by constructing a method of labeling pedestrian detection boxes in the collected pedestrian images;

[0015] By annotating the pedestrian detection box labels and fence semantic segmentation labels in the collected pedestrian images occluded by fences, a fence occlusion dataset is obtained.

[0016] Augmentation processing is performed on the pedestrian dataset to obtain the augmented pedestrian dataset;

[0017] The augmentation process for the pedestrian dataset includes:

[0018] Construct a virtual fence template that includes multiple texture types; the texture types include: vertical stripe texture, grid texture, and cross texture;

[0019] After undergoing geometric transformations, the virtual fence template is superimposed onto the images in the pedestrian dataset, and corresponding fence semantic segmentation labels are automatically generated. The geometric transformations include rotation transformation, scaling transformation, and perspective distortion transformation.

[0020] According to the present invention, a pedestrian detection method for fence occlusion scenarios includes a preset detection model comprising: a pedestrian detection branch and a semantic segmentation branch; training the preset detection model using an augmented pedestrian dataset to learn the global features of a complete pedestrian; and training the preset detection model using both the augmented pedestrian dataset and the fence occlusion dataset to learn the feature differences between pedestrians and fences, including:

[0021] Using an augmented pedestrian dataset, train the pedestrian detection branch to learn the global features of complete pedestrians;

[0022] Using an augmented pedestrian dataset and a fence occlusion dataset, the pedestrian detection branch and the semantic segmentation branch are simultaneously optimized based on a multi-task loss function to learn the feature differences between pedestrians and fences.

[0023] The pedestrian detection branch and the semantic segmentation branch share the feature pyramid output of the backbone network. The semantic segmentation branch extracts the global structural features of the fence through a group of convolutional layers containing dilated convolutions and outputs the fence probability mask. The multi-task loss function is composed of a weighted sum of detection loss and segmentation loss, and the segmentation loss is a weighted sum of Dice loss and binary cross-entropy loss.

[0024] According to the present invention, a pedestrian detection method for fence-occluded scenes includes an enhanced post-processing algorithm that optimizes the quality of pedestrian candidate boxes based on pedestrian coordinates and confidence levels, and outputs optimized candidate boxes, including:

[0025] By using a confidence-based hierarchical mechanism, multiple pedestrian candidate boxes are classified according to their confidence levels to obtain strong and weak boxes.

[0026] The weak bounding boxes are spatially compensated using a weak bounding box space compensation method to obtain merged candidate boxes;

[0027] The confidence of the candidate boxes to be merged is attenuated by a soft nonmaximum suppression method to obtain the retained boxes;

[0028] The strong bounding boxes and the reserved bounding boxes are identified as candidate boxes for optimization, and the optimized candidate boxes are output.

[0029] According to the present invention, a pedestrian detection method for fence-occluded scenarios is provided. This method classifies multiple pedestrian candidate boxes based on confidence levels using a confidence-based hierarchical mechanism to obtain strong and weak boxes, including:

[0030] Pedestrian candidate boxes with a confidence level greater than or equal to a preset strong confidence threshold are identified as strong boxes and used as high-confidence base boxes.

[0031] Pedestrian candidate boxes with a confidence level less than a preset strong confidence threshold but greater than or equal to a preset weak confidence threshold are identified as weak boxes and used as potential occlusion boxes to be repaired.

[0032] Discard pedestrian candidate boxes with a confidence level lower than the preset weak confidence threshold.

[0033] According to the present invention, a pedestrian detection method for fence-occluded scenarios is provided, which uses a weak bounding box spatial compensation method to perform spatial compensation on weak bounding boxes to obtain merged candidate boxes, including:

[0034] Calculate the intersection-union ratio of the weak box and any strong box, and calculate the ratio of the projected intersection to the projected union of the weak box and the strong box in the vertical direction to obtain the vertical overlap rate.

[0035] If the intersection-union ratio is greater than or equal to the first threshold, or the vertical overlap ratio is not less than the second threshold, the weak box and the strong box are determined to be local boxes of the same pedestrian separated by the fence.

[0036] The weak and strong bounding boxes are merged using the bounding rectangle to generate merge candidate boxes;

[0037] The confidence of the merged candidate boxes is calculated by weighting the confidence of weak boxes and strong boxes together, and by using the cross-union ratio of weak boxes and strong boxes as a decay factor.

[0038] According to the present invention, a pedestrian detection method for fence-occluded scenes is provided, which uses a soft nonmaximum suppression method to perform confidence decay on merged candidate boxes to obtain retained boxes, including:

[0039] Multiple candidate boxes for merging are sorted according to confidence level. The sorted candidate boxes are then classified according to a preset retention threshold to obtain retained boxes and boxes to be deleted.

[0040] For each bounding box to be deleted, calculate the intersection-union ratio (IU) between the bounding box to be deleted and the boxes to be kept; for bounding boxes to be deleted whose IU is greater than a preset retention threshold, calculate the confidence level of the bounding box to be deleted according to a preset attenuation formula, and obtain the attenuated confidence level.

[0041] Boxes whose confidence level after decay is greater than or equal to the retention threshold are identified as retention boxes.

[0042] The present invention also provides a pedestrian detection device for a fence-obstructed scene, comprising the following modules:

[0043] The model training module is used to train a preset detection model using the augmented pedestrian dataset to learn the global features of a complete pedestrian; and to train a preset detection model using the augmented pedestrian dataset and the fence occlusion dataset to learn the feature differences between pedestrians and fences.

[0044] The model optimization module is used to optimize the preset detection model by focusing on fine-grained optimization of the preset occlusion pattern using the fence occlusion dataset; and removes the semantic segmentation branch in the preset detection model to obtain a lightweight detection model.

[0045] The detection module is used to extract features from the image to be detected using a lightweight detection model to obtain pedestrian candidate boxes. The post-processing algorithm is used to optimize the quality of the pedestrian candidate boxes based on pedestrian coordinates and confidence scores, and output the optimized candidate boxes. The pedestrian candidate boxes include pedestrian coordinates and confidence scores.

[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the pedestrian detection method for fence occlusion scenarios as described above.

[0047] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the pedestrian detection method for fence occlusion scenarios as described above.

[0048] The present invention provides a pedestrian detection method and apparatus for fence-occluded scenarios. It trains a pre-defined detection model using an enhanced pedestrian dataset to learn the global features of a complete pedestrian; trains the pre-defined detection model using both the enhanced pedestrian dataset and the fence-occlusion dataset to learn the feature differences between pedestrians and fences; optimizes the pre-defined detection model using the fence-occlusion dataset through fine-grained optimization focusing on pre-defined occlusion patterns; removes the semantic segmentation branch from the pre-defined detection model to obtain a lightweight detection model; extracts features from the image to be detected using the lightweight detection model to obtain pedestrian candidate boxes; and optimizes the quality of the pedestrian candidate boxes using a post-processing algorithm based on pedestrian coordinates and confidence levels, outputting the optimized candidate boxes. Compared to existing technologies that cannot effectively distinguish between occluded objects and pedestrian information, the present invention improves the model's detection capability and achieves higher detection accuracy by constructing two complementary datasets—a pedestrian dataset and a fence-occlusion dataset—and employing a multi-stage training strategy and data augmentation. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the pedestrian detection method for fence-occluded scenarios provided by the present invention.

[0051] Figure 2 This is a schematic diagram of the pedestrian detection device for fence-obstructed scenarios provided by the present invention.

[0052] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] The following is combined Figures 1-3 This invention is described.

[0055] Figure 1 This is a flowchart illustrating the pedestrian detection method for fence-occluded scenarios provided by the present invention, as shown below. Figure 1 As shown, the method includes the following:

[0056] Step 101: Using the augmented pedestrian dataset, train a preset detection model to learn the global features of a complete pedestrian; using the augmented pedestrian dataset and the fence occlusion dataset, train a preset detection model to learn the feature differences between pedestrians and fences.

[0057] In step 101 above, the present invention first constructs a basic dataset and a fence occlusion dataset, namely, an enhanced pedestrian dataset and a fence occlusion dataset. The two datasets contain ordinary pedestrian images and pedestrian images occluded by fences, respectively. The process of constructing the datasets includes image acquisition and image annotation. Both datasets require annotation of pedestrian detection boxes, and the fence occlusion dataset also requires additional annotation of semantic segmentation of the fences.

[0058] In practical annotation, images are often captured from multiple fixed cameras, and the fences are positioned in a fixed location within the image. A single fence segmentation annotation can be shared for the same camera position. A YOLOv8 detection model with an added segmentation branch was trained using images and annotations. During training, randomly generated fences were added above the pedestrian layer on the base dataset—a targeted data augmentation method—and semantic segmentation annotations for the fences were generated simultaneously. The model was trained in stages.

[0059] Optionally, before step 101, steps A1 to A3 are also included:

[0060] Step A1: Obtain the pedestrian dataset by annotating the pedestrian detection box labels in the collected pedestrian images.

[0061] Step A2: By labeling the pedestrian detection box labels and fence semantic segmentation labels in the collected pedestrian images occluded by fences, a fence occlusion dataset is obtained.

[0062] Step A3: Enhance the pedestrian dataset to obtain an enhanced pedestrian dataset; the enhancement process includes: constructing a virtual fence template containing multiple texture types; the texture types include: vertical stripe texture, grid texture, and cross texture; after geometric transformation, the virtual fence template is superimposed onto the image of the pedestrian dataset, and corresponding fence semantic segmentation labels are automatically generated; the geometric transformations include: rotation transformation, scale transformation, and perspective distortion transformation.

[0063] In steps A1 to A3 above, the images in the pedestrian dataset are ordinary pedestrian images collected without fence obstruction. For example, the collection scenarios cover 6 types of security scenarios, including train stations, squares, and residential areas, including 4 weather conditions (sunny, cloudy, rainy, and foggy), and 6 time periods from 6:00 AM to 10:00 PM, totaling 6431 images; at the same time, 10,000 unobstructed pedestrian images from the COCO public dataset are added, bringing the total size to 16,431 images to ensure sample diversity.

[0064] The images in the fence occlusion dataset are pedestrian images collected with fence occlusion. The scenes correspond to the basic dataset and cover three typical fence types: vertical stripes with a spacing of 5-20cm; grids with apertures of 5×5-30×30cm; and intersections with angles of 30°-90°, totaling 5561 images. For fixed camera locations, since the fence positions are fixed, the semantic segmentation annotations of fences from the same camera position can be reused, reducing annotation costs.

[0065] Both datasets are labeled with pedestrian detection boxes in COCO format, with coordinates accurate to the pixel level, covering the entire area from the pedestrian's head to their feet.

[0066] The fence occlusion dataset is additionally labeled with fence semantic segmentation labels, using a binary mask where 1 represents a fence pixel and 0 represents a non-fence pixel; the labeling accuracy requirement is an edge error ≤ 2 pixels.

[0067] The basic dataset is divided into training and testing sets in an 8:2 ratio.

[0068] The fence occlusion dataset was divided into training and testing sets in an 8:2 ratio.

[0069] Geometric transformations include rotational transformations, scaling transformations, and perspective distortion transformations to simulate the occlusion patterns of fences from different perspectives.

[0070] Optionally, the preset detection model includes: a pedestrian detection branch and a semantic segmentation branch; step 101 includes steps B1 to B2:

[0071] Step B1: Using the augmented pedestrian dataset, train the pedestrian detection branch to learn the global features of complete pedestrians.

[0072] Step B2: Using the augmented pedestrian dataset and the fence occlusion dataset, simultaneously optimize the pedestrian detection branch and the semantic segmentation branch based on the multi-task loss function to learn the feature differences between pedestrians and fences.

[0073] The pedestrian detection branch and the semantic segmentation branch share the feature pyramid output of the backbone network. The semantic segmentation branch extracts the global structural features of the fence through a group of convolutional layers containing dilated convolutions and outputs a fence probability mask. The multi-task loss function is composed of a weighted sum of detection loss and segmentation loss. The segmentation loss is a weighted sum of Dice loss and binary cross-entropy loss to balance the sample ratio between the fence region and the background region.

[0074] In steps B1 to B2 above, this invention employs segmentation branches to enhance detection capabilities. For each fixed camera location in the field application, only one fence segmentation annotation is required. The segmentation branches are removed during the inference phase to accelerate inference. The YOLOv8 model with fence semantic segmentation branches is trained in stages. Based on the YOLOv8 architecture, the fence semantic segmentation branches are extended, and a three-stage progressive training strategy is adopted to guide the model to gradually learn the ability to distinguish between pedestrian and fence features.

[0075] The first phase, which can consist of 30 rounds, lays the foundation for basic features. Training data uses only the basic dataset, which does not contain images occluded by fences. Data augmentation employs only basic geometric transformations, including: random rotation ±15°, horizontal flip, scaling by 0.8-1.2 times, and translation ±10% pixels. The training objective is to activate only the pedestrian detection branch and learn the global features of a complete pedestrian, including contours and limb structure. The initial learning rate is set to 0.01.

[0076] The second stage, which can consist of 40 rounds, focuses on occlusion recognition and multi-task collaboration. Training data includes a base dataset training set and a fence occlusion dataset training set. The dynamic enhancement strategy involves randomly overlaying fence templates (including vertical bars, grids, and intersections) onto unoccluded pedestrians in the base dataset. Corresponding segmentation labels are generated. The fence templates can be rotated (e.g., ±30°), scaled (e.g., 0.5-1.5x), and subjected to perspective distortion (e.g., ±10°), simulating real-world occlusion. The training objective is to simultaneously activate the detection and segmentation branches, learning the "pedestrian-fence" feature differences through multi-task loss, with the learning rate reduced to 0.005.

[0077] The YOLOv8 model architecture design includes a backbone network, a fence semantic segmentation branch, and a multi-task loss function.

[0078] The backbone network adopts the YOLOv8 structure, retaining the Feature Pyramid Network (FPN) and Path Aggregation Network (PAN), and generates four-scale feature layers: P2 (1 / 4 resolution), P3 (1 / 8), P4 (1 / 16), and P5 (1 / 32).

[0079] The fence semantic segmentation branch, at layer P2, aims to achieve high resolution and rich detail, followed by the construction of a lightweight segmentation head. Specifically, it includes: 1×1 convolution, compressing the number of channels from 256 to 128 → 3×3 dilated convolution, with a dilation rate of 2, capturing the global fence structure → 3×3 ordinary convolution, enhancing local details → 1×1 convolution, reducing the number of channels to 1 → deconvolution, with a stride of 2, restoring the resolution to half the original level → sigmoid activation, outputting a fence probability mask with the same resolution as the input.

[0080] The multi-task loss function is: Total Loss = Detection Loss + 0.5 × Segmentation Loss.

[0081] The detection loss employs YOLOv8's native loss formula, which includes classification loss, localization loss, and confidence loss. The segmentation loss is a weighted sum of Dice loss (weight 0.6) and BCE loss (weight 0.4) to mitigate class imbalance in fenced regions.

[0082] Step 102: Using the fence occlusion dataset, optimize the preset detection model by focusing on fine-grained optimization of the preset occlusion pattern; remove the semantic segmentation branch in the preset detection model to obtain a lightweight detection model.

[0083] In step 102 above, the third stage can be 10 rounds, which is occlusion scene-specific reinforcement. The training data uses only the fence occlusion dataset. The training objective is to focus on fine-grained optimization of complex occlusion patterns, including high-density fences, large-area occlusion, and lighting interference; and to reduce the learning rate to 0.001 to enhance the model's adaptability to real-world occlusion scenes.

[0084] In the lightweight decoding inference stage, the fence semantic segmentation branch is removed, retaining only the pedestrian detection branch, which reduces computational overhead while maintaining accuracy. After removing the segmentation branch, the number of model parameters is reduced by 11%, the inference speed is improved by 13%, and the detection accuracy is not affected, since the segmentation branch only assists feature learning in the training stage.

[0085] Step 103: Using a lightweight detection model, feature extraction is performed on the image to be detected to obtain pedestrian candidate boxes; using an enhanced post-processing algorithm, the quality of the pedestrian candidate boxes is optimized based on pedestrian coordinates and confidence scores, and the optimized candidate boxes are output; wherein, the pedestrian candidate boxes include: pedestrian coordinates and confidence scores.

[0086] In step 103 above, the input image is first preprocessed. For example, a 1080×1920 image is resized to 544×940 and normalized to [0,1]. Feature extraction is performed, generating P3-P5 features through the backbone network and FPN / PAN. The detection head output directly generates pedestrian candidate boxes through the YOLOv8 detection branch.

[0087] The enhanced post-processing algorithm integrates a confidence-based hierarchical mechanism, weak bounding box space compensation, and soft nonmaximum suppression to optimize candidate box quality and improve the detection rate of occluded pedestrians.

[0088] Optionally, the enhanced post-processing algorithm in step 103 optimizes the quality of the pedestrian candidate boxes based on the pedestrian coordinates and confidence level, and outputs the optimized candidate boxes, including steps C1 to C4:

[0089] Step C1: Using a confidence-based hierarchical mechanism, classify multiple pedestrian candidate boxes according to their confidence levels to obtain strong and weak boxes.

[0090] Step C2: Use the weak box space compensation method to perform space compensation on the weak boxes to obtain merged candidate boxes.

[0091] Step C3: Use the soft nonmaximum suppression method to reduce the confidence of the merged candidate boxes and obtain the retained boxes.

[0092] Step C4: Determine the strong bounding boxes and the reserved bounding boxes as optimized candidate boxes, and output the optimized candidate boxes.

[0093] In steps C1 to C4 above, the present invention employs a post-processing method that integrates confidence leveling mechanism, weak box space compensation and soft nonmaximum suppression to improve the detection rate of occluded pedestrians.

[0094] Optionally, step C1 includes steps C11 to C13:

[0095] Step C11: Pedestrian candidate boxes with a confidence level greater than or equal to the preset strong confidence threshold are identified as strong boxes and used as high-confidence base boxes.

[0096] Step C12: Pedestrian candidate boxes with confidence scores less than the preset strong confidence threshold but greater than or equal to the preset weak confidence threshold are identified as weak boxes and used as potential occlusion boxes to be repaired.

[0097] Step C13: Discard pedestrian candidate boxes with confidence scores lower than the preset weak confidence threshold to filter background noise.

[0098] In steps C11 to C13 above, for example, the confidence level stratification mechanism includes setting dual thresholds and classification rules. The dual thresholds include a strong confidence threshold (0.55) and a weak confidence threshold (0.25). The classification rules include: candidate boxes with a confidence level ≥ 0.55 are "strong boxes" and are directly retained; boxes with a confidence level ≤ 0.25 and < 0.55 are "weak boxes" and enter the compensation stage; boxes with a confidence level < 0.25 are directly discarded to reduce noise interference.

[0099] Optionally, step C2 includes steps C21 to C24, comprising:

[0100] C21: Calculate the intersection-union ratio of the weak box and any strong box, and calculate the ratio of the projection intersection and projection union of the weak box and the strong box in the vertical direction to obtain the vertical overlap rate.

[0101] C22: If the intersection-union ratio is greater than or equal to the first threshold, or the vertical overlap ratio is not less than the second threshold, the weak box and the strong box are determined to be local boxes of the same pedestrian separated by the fence.

[0102] C23: Merge weak and strong boxes using an outer rectangle to generate merge candidate boxes.

[0103] C24: Calculate the confidence of the merged candidate boxes by weighting the confidence of weak boxes and strong boxes together, and by combining the cross-union ratio of weak boxes and strong boxes as a decay factor.

[0104] In steps C21 to C24 above, weak box space compensation includes spatial similarity analysis, merging rules, and confidence reassessment.

[0105] Spatial similarity analysis involves calculating the intersection-union ratio (IOU) and vertical overlap rate between strong and weak bounding boxes. The vertical overlap rate includes the intersection / union in the vertical direction.

[0106] The merging rule is that if the IOU is ≥0.3 or the vertical overlap rate is ≥0.5, it is determined that the same pedestrian is a local box separated by the fence, and a merged box is generated by the bounding rectangle.

[0107] The confidence level is recalculated as follows: Confidence level of merged boxes = (confidence level of strong boxes × 0.7 + confidence level of weak boxes × 0.3) × IOU, balancing the original confidence level with spatial correlation.

[0108] Optionally, step C3 includes steps C31 to C33:

[0109] Step C31: Sort the multiple candidate boxes for merging according to their confidence scores. Then, classify the sorted candidate boxes according to a preset retention threshold to obtain the retained boxes and the boxes to be deleted.

[0110] Step C32: For each box to be deleted, calculate the intersection-union ratio (IUR) between the box to be deleted and the boxes to be retained; for boxes to be deleted whose IUR is greater than the preset retention threshold, calculate the confidence level of the box to be deleted according to the preset attenuation formula, and obtain the attenuated confidence level.

[0111] Step C33: Select boxes to be deleted with a confidence level greater than or equal to the retention threshold after attenuation as retained boxes.

[0112] In steps C31 to C33 above, the merged candidate boxes are sorted by confidence level, and high-confidence boxes are traversed and retained. For overlapping boxes with IOU ≥ 0.5 with the retained boxes, they are not directly deleted, but their confidence level is reduced according to the formula: confidence level = confidence level × exp (-IOU² / σ) (σ = 0.5), to avoid box splitting caused by fence occlusion being mistakenly deleted. Finally, boxes with confidence level ≥ 0.5 are retained as output.

[0113] On the test set, this method achieved a pedestrian detection mAP@0.5 score of 0.91, which is significantly higher than the 0.75 mAP@0.5 score achieved by YOLOv8n directly, demonstrating a substantial improvement in detection performance.

[0114] This invention provides a pedestrian detection method for fence-occluded scenes. It trains a pre-defined detection model using an augmented pedestrian dataset to learn the global features of a complete pedestrian. It then trains the same pre-defined detection model using both the augmented pedestrian dataset and the fence-occlusion dataset to learn the feature differences between pedestrians and fences. Using the fence-occlusion dataset, it optimizes the pre-defined detection model through fine-grained optimization focusing on pre-defined occlusion patterns. The semantic segmentation branch in the pre-defined detection model is removed to obtain a lightweight detection model. The lightweight detection model is used to extract features from the image to be detected, obtaining pedestrian candidate boxes. An augmented post-processing algorithm is then used to optimize the quality of the pedestrian candidate boxes based on pedestrian coordinates and confidence levels, outputting the optimized candidate boxes. Compared to existing technologies that cannot effectively distinguish between occluded objects and pedestrian information, this invention improves the model's detection capability and achieves higher detection accuracy by constructing two complementary datasets—a pedestrian dataset and a fence-occlusion dataset—and employing a multi-stage training strategy and data augmentation.

[0115] The pedestrian detection device for fence-obscured scenarios provided by the present invention will be described below. The pedestrian detection device for fence-obscured scenarios described below can be referred to in correspondence with the pedestrian detection method for fence-obscured scenarios described above.

[0116] Figure 2 This is a flowchart illustrating the pedestrian detection device for fence-obstructed scenarios provided by the present invention, as shown below. Figure 2 As shown, the device includes the following:

[0117] The model training module 201 is used to train a preset detection model using the augmented pedestrian dataset to learn the global features of a complete pedestrian; and to train a preset detection model using the augmented pedestrian dataset and the fence occlusion dataset to learn the feature differences between pedestrians and fences.

[0118] The model optimization module 202 is used to optimize the preset detection model by focusing on fine-grained optimization of the preset occlusion mode using the fence occlusion dataset; and remove the semantic segmentation branch in the preset detection model to obtain a lightweight detection model.

[0119] The detection module 203 is used to extract features from the image to be detected using a lightweight detection model to obtain pedestrian candidate boxes; and to optimize the quality of the pedestrian candidate boxes based on pedestrian coordinates and confidence scores using an enhanced post-processing algorithm, and output the optimized candidate boxes; wherein, the pedestrian candidate boxes include: pedestrian coordinates and confidence scores.

[0120] This invention provides a pedestrian detection device for fence-occluded scenarios. It trains a pre-defined detection model using an enhanced pedestrian dataset to learn the global features of a complete pedestrian. It then trains the same pre-defined detection model using both the enhanced pedestrian dataset and the fence-occlusion dataset to learn the feature differences between pedestrians and fences. Using the fence-occlusion dataset, it optimizes the pre-defined detection model through fine-grained optimization focusing on pre-defined occlusion patterns. It removes the semantic segmentation branch from the pre-defined detection model to obtain a lightweight detection model. The lightweight detection model extracts features from the image to be detected, obtaining pedestrian candidate boxes. An enhanced post-processing algorithm optimizes the quality of the pedestrian candidate boxes based on pedestrian coordinates and confidence levels, outputting the optimized candidate boxes. Compared to existing technologies that cannot effectively distinguish between occluded objects and pedestrian information, this invention improves the model's detection capability and achieves higher detection accuracy by constructing two complementary datasets—a pedestrian dataset and a fence-occlusion dataset—and employing a multi-stage training strategy and data augmentation.

[0121] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions from the memory 830 to execute a pedestrian detection method for fence-occlusion scenarios.

[0122] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the pedestrian detection method for fence occlusion scenarios provided by the above methods.

[0124] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the pedestrian detection methods for fence-occluded scenarios provided by the methods described above.

[0125] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pedestrian detection method for scenarios with fence occlusion, characterized in that, include: Using an augmented pedestrian dataset, a pre-defined detection model is trained to learn the global features of a complete pedestrian. Using the augmented pedestrian dataset and the fence occlusion dataset, the preset detection model is trained to learn the feature differences between pedestrians and fences; Using the fence occlusion dataset, the preset detection model is optimized by focusing on fine-grained optimization of preset occlusion patterns; Remove the semantic segmentation branch from the preset detection model to obtain a lightweight detection model; The lightweight detection model is used to extract features from the image to be detected, and pedestrian candidate boxes are obtained. By using an enhanced post-processing algorithm, the quality of the pedestrian candidate boxes is optimized based on pedestrian coordinates and confidence scores, and optimized candidate boxes are output; wherein, the pedestrian candidate boxes include: pedestrian coordinates and confidence scores; The enhanced pedestrian dataset consists of ordinary pedestrian images; The preset detection model includes a pedestrian detection branch and a semantic segmentation branch; the pedestrian detection branch and the semantic segmentation branch share the feature pyramid output of the backbone network; the preset detection model is trained using an enhanced pedestrian dataset to learn the global features of a complete pedestrian; the preset detection model is trained using the enhanced pedestrian dataset and the fence occlusion dataset to learn the feature differences between pedestrians and fences, including: Using the augmented pedestrian dataset, the pedestrian detection branch is trained to learn the global features of a complete pedestrian. Using the augmented pedestrian dataset and the fence occlusion dataset, the pedestrian detection branch and the semantic segmentation branch are simultaneously optimized based on a multi-task loss function to learn the feature differences between pedestrians and fences. The semantic segmentation branch extracts the global structural features of the fence through a group of convolutional layers containing dilated convolutions and outputs a fence probability mask; the multi-task loss function is composed of a weighted sum of detection loss and segmentation loss, and the segmentation loss is a weighted sum of Dice loss and binary cross-entropy loss. The pedestrian detection method for fence-occlusion scenarios further includes: The pedestrian dataset was obtained by constructing a method of labeling pedestrian detection boxes in the collected pedestrian images; By annotating the pedestrian detection box labels and fence semantic segmentation labels in the collected pedestrian images occluded by fences, a fence occlusion dataset is obtained. The pedestrian dataset is augmented to obtain an augmented pedestrian dataset; The augmentation process for the pedestrian dataset includes: Construct a virtual fence template containing multiple texture types; wherein, the texture types include: vertical stripe texture, grid texture, and cross texture; The virtual fence template is geometrically transformed and then superimposed onto the images of the pedestrian dataset, and corresponding fence semantic segmentation labels are automatically generated; wherein, the geometric transformation includes: rotation transformation, scale transformation and perspective distortion transformation.

2. The pedestrian detection method for fence-occluded scenarios according to claim 1, characterized in that, The enhanced post-processing algorithm optimizes the quality of the pedestrian candidate boxes based on pedestrian coordinates and confidence scores, and outputs optimized candidate boxes, including: By using a confidence-based hierarchical mechanism, multiple pedestrian candidate boxes are classified according to their confidence levels to obtain strong boxes and weak boxes. The weak boxes are spatially compensated using a weak box space compensation method to obtain merged candidate boxes; The confidence level of the merged candidate boxes is reduced by a soft nonmaximum suppression method to obtain the retained boxes; The strong bounding box and the reserved bounding box are determined as optimized candidate bounding boxes, and the optimized candidate bounding boxes are output.

3. The pedestrian detection method for fence-occluded scenarios according to claim 2, characterized in that, The confidence-based hierarchical mechanism classifies multiple pedestrian candidate boxes according to their confidence levels to obtain strong and weak boxes, including: Pedestrian candidate boxes with a confidence level greater than or equal to a preset strong confidence threshold are identified as strong boxes and used as high-confidence base boxes. Pedestrian candidate boxes with a confidence level less than the preset strong confidence threshold but greater than or equal to the preset weak confidence threshold are identified as weak boxes and used as potential occlusion boxes to be repaired. Pedestrian candidate boxes with a confidence level lower than the preset weak confidence threshold are discarded.

4. The pedestrian detection method for fence-occluded scenarios according to claim 3, characterized in that, The step of performing spatial compensation on the weak bounding boxes using a weak bounding box spatial compensation method to obtain merged candidate boxes includes: Calculate the intersection-union ratio of the weak box and any strong box, and calculate the ratio of the projection intersection and projection union of the weak box and the strong box in the vertical direction to obtain the vertical overlap rate. If the intersection-union ratio is greater than or equal to the first threshold, or the vertical overlap ratio is not less than the second threshold, the weak box and the strong box are determined to be local boxes of the same pedestrian separated by the fence. The weak bounding box and the strong bounding box are merged by using an outer rectangle to generate a merge candidate box; The confidence of the merged candidate box is calculated by weighting the confidence of the weak box and the confidence of the strong box, and by using the intersection-union ratio of the weak box and the strong box as a decay factor.

5. The pedestrian detection method for fence-occluded scenarios according to claim 4, characterized in that, The process of using a soft nonmaximum suppression method to attenuate the confidence of the merged candidate boxes to obtain the retained boxes includes: The multiple candidate boxes to be merged are sorted according to their confidence levels. The sorted candidate boxes are then classified according to a preset retention threshold to obtain retained boxes and boxes to be deleted. For each box to be deleted, calculate the intersection-union ratio (IUR) between the box to be deleted and the boxes to be retained; for boxes to be deleted whose IUR is greater than a preset retention threshold, calculate the confidence level of the box to be deleted according to a preset attenuation formula, and obtain the attenuated confidence level. Boxes whose confidence level after decay is greater than or equal to the retention threshold are identified as retention boxes.

6. A pedestrian detection device for scenarios obscured by fences, characterized in that, include: The model training module is used to train a preset detection model using an augmented pedestrian dataset and learn the global features of a complete pedestrian. Using the augmented pedestrian dataset and the fence occlusion dataset, the preset detection model is trained to learn the feature differences between pedestrians and fences; The model optimization module is used to optimize the preset detection model by focusing on fine-grained optimization of the preset occlusion mode using the fence occlusion dataset. During the inference phase, the semantic segmentation branch in the preset detection model is removed to obtain a lightweight detection model; The detection module is used to extract features from the image to be detected using the lightweight detection model to obtain pedestrian candidate boxes; By using an enhanced post-processing algorithm, the quality of the pedestrian candidate boxes is optimized based on pedestrian coordinates and confidence scores, and optimized candidate boxes are output; wherein, the pedestrian candidate boxes include: pedestrian coordinates and confidence scores; The enhanced pedestrian dataset consists of ordinary pedestrian images; The preset detection model includes a pedestrian detection branch and a semantic segmentation branch; the model training module is used to train the pedestrian detection branch using the enhanced pedestrian dataset to learn the global features of a complete pedestrian; and to simultaneously optimize the pedestrian detection branch and the semantic segmentation branch based on a multi-task loss function using the enhanced pedestrian dataset and the fence occlusion dataset to learn the feature differences between pedestrians and fences. The semantic segmentation branch extracts the global structural features of the fence through a group of convolutional layers containing dilated convolutions and outputs a fence probability mask; the multi-task loss function is composed of a weighted sum of detection loss and segmentation loss, and the segmentation loss is a weighted sum of Dice loss and binary cross-entropy loss. The pedestrian detection device for fence-obstructed scenarios further includes: The pedestrian dataset was obtained by constructing a method of labeling pedestrian detection boxes in the collected pedestrian images; By annotating the pedestrian detection box labels and fence semantic segmentation labels in the collected pedestrian images occluded by fences, a fence occlusion dataset is obtained. The pedestrian dataset is augmented to obtain an augmented pedestrian dataset; The augmentation process for the pedestrian dataset includes: Construct a virtual fence template containing multiple texture types; wherein, the texture types include: vertical stripe texture, grid texture, and cross texture; The virtual fence template is geometrically transformed and then superimposed onto the images of the pedestrian dataset, and corresponding fence semantic segmentation labels are automatically generated; wherein, the geometric transformation includes: rotation transformation, scale transformation and perspective distortion transformation.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the pedestrian detection method for fence occlusion scenarios as described in any one of claims 1 to 5.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the pedestrian detection method for fence occlusion scenarios as described in any one of claims 1 to 5.

Images