Method and device for detecting pedestrians in dangerous area of power station and computer equipment
By improving the YOLOv11 model and combining high-frequency feature extraction, multi-scale feature fusion, and a lightweight detection head, the problem of low pedestrian detection accuracy in dangerous areas of power stations was solved, achieving efficient safety warning and detection.
Patent Information
- Application Number
- CN202510772368.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-11
- Publication Date
- 2025-09-05
AI Technical Summary
Existing general target detection algorithms have reduced detection accuracy and serious false detection and missed detection in complex industrial scenarios such as power plants, and are not suitable for pedestrian detection in dangerous areas of power plants.
An improved YOLOv11 model, including a high-frequency feature extraction network, a multi-scale feature fusion network, and a lightweight detection head, was used to construct a pedestrian dataset in a power station scenario, and pedestrian detection was performed through the training model.
The system significantly improves the accuracy of pedestrian detection in dangerous areas of power plants, taking into account both detection performance and inference speed, and is suitable for pedestrian detection in dangerous areas of power plants.
Smart Images
Figure CN120599544A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision detection, and in particular to a method, a detection device and a computer device for detecting pedestrians in a dangerous area of a power station. Background Art
[0002] With the development and increasing intelligence of the power industry, power plants, as a vital component of the national energy system, are receiving high attention for their safety management. During power plant operation, particularly in coal-fired power plants, waste-to-energy plants, and hydropower stations where large machinery and vehicles frequently operate, there are numerous potentially high-risk areas, such as fuel transfer corridors, unloading platforms, and areas surrounding high-voltage power equipment. If personnel mistakenly enter or operate in violation of regulations in these areas, accidents such as electric shock, falls, crushing, and vehicle collisions can easily occur, resulting in serious personal injury and economic losses.
[0003] In recent years, with the advancement of computer vision and deep learning technologies, video-based pedestrian detection methods have been widely used in intelligent surveillance, security warning, and other fields. However, existing general-purpose object detection algorithms, such as the YOLO series and Faster R-CNN, are primarily developed for urban street scenes or indoor environments. In power plants, factors such as drastic lighting variations, severe human occlusion, and large variations in object size can lead to reduced detection accuracy and severe false detection and missed detection issues. Therefore, existing general-purpose object detection algorithms are not suitable for pedestrian detection in complex industrial environments such as power plants.
[0004] Therefore, there is an urgent need for a pedestrian detection method that can adapt to the complex environment of power plants, so as to achieve efficient safety warnings, reduce the probability of safety accidents, and thus improve the intelligent safety management level of power plants. Summary of the Invention
[0005] In order to overcome the technical defect in the prior art of lacking a pedestrian detection method applicable to dangerous areas of power plants, the present invention provides a pedestrian detection method, a detection device and a computer device in dangerous areas of power plants.
[0006] The present invention provides a method for detecting pedestrians in a dangerous area of a power station, comprising:
[0007] S1. Create a pedestrian dataset in a power station scenario and label the dataset;
[0008] S2. Construct an improved YOLOv11 model, wherein the improved YOLOv11 model uses a high-frequency feature extraction network as the backbone network, a multi-scale aggregation and expansion feature fusion network as the neck network, and a lightweight detection head as the head network;
[0009] S3. randomly divide the dataset into a training set and a validation set, and train the improved YOLOv11 model, saving the best model parameters based on performance to obtain the target model;
[0010] S4. Input the data to be detected into the target model and output the pedestrian detection result.
[0011] Optionally, the high-frequency feature extraction network is obtained by replacing the first convolution layer and the second convolution layer in the backbone network of the YOLOv11 model with a high-frequency feature enhancement module.
[0012] Optionally, the high-frequency feature enhancement module performs the following operations:
[0013] 1) Input image Perform preliminary processing through a 3×3 convolution layer and reduce the channel dimension to obtain features ;
[0014] 2) The features The Sobel operator is used to enhance high-frequency information and obtain the feature , and the features MaxPool is used to perform maximum pooling to obtain features ;
[0015] 3) Use feature splicing to combine features and features The fusion is performed, and a 3×3 convolutional layer is used to further mine the shallow features after splicing;
[0016] 4) The output features of the previous step are passed through a 1×1 convolution layer to adjust the number of channels, and batch normalization is used for normalization. The ReLU activation function is used to complete feature activation to obtain the feature .
[0017] Optionally, the multi-scale aggregation and expansion feature fusion network includes a feature aggregation module and a feature diffusion module;
[0018] The feature aggregation module is used to integrate cross-scale information of features at different scales to aggregate feature information at different scales;
[0019] The feature diffusion module is used to diffuse the aggregated features so that features at each scale can obtain partial context information of other scales.
[0020] Optionally, the feature aggregation module performs the following operations:
[0021] 1) The features extracted by the high-frequency feature extraction network and Enhance the feature representation capability through efficient local attention to obtain the corresponding features and ;
[0022] 2) The features and and the features extracted by the high-frequency feature extraction network Input into the dynamic adaptive selection fusion module to obtain the fusion features of P4 .
[0023] Optionally, the feature diffusion module performs the following operations:
[0024] On the one hand, the fusion features of P4 Through dynamic learnable upsampling operation, upsampling to the spatial scale of P3 and combining with feature Perform feature splicing and then input it into the C3K2 module to fuse the semantic information from the middle layer and the low layer to obtain the fused features of P3 ;
[0025] On the other hand, the fusion features of P4 The spatial context information is extracted by depthwise separable convolution and then combined with the feature The fusion is performed and finally sent to the C3K2 module to extract deeper semantic features to obtain the fusion features of P5 .
[0026] Optionally, the lightweight detection head performs the following operations:
[0027] 1) The features 、 and The input is sent to the parameter-sharing common feature extraction module, and independent convolution operations and group convolution operations are performed in sequence. The convolution output results of each group are then spliced together.
[0028] 2) The features extracted in the previous step are processed through 1×1 convolution to generate a tensor with a channel number of 4×self.reg_max, which serves as the regression branch. At the same time, they are processed through 1×1 convolution to generate a tensor with a channel number of nc, which serves as the classification branch.
[0029] 3) Output the bounding box information of pedestrians in the image.
[0030] The present invention provides a pedestrian detection device for monitoring dangerous areas in power stations, comprising:
[0031] An image acquisition module, which is used to acquire real-time images of hazardous areas of the power station;
[0032] A data processing module, configured to pre-process the images acquired by the image acquisition module;
[0033] A pedestrian detection module, which has the aforementioned improved YOLOv11 model built in. The pedestrian detection module is used to input the image preprocessed by the data processing module into the improved YOLOv11 model and output the pedestrian detection result;
[0034] Dangerous area demarcation module, which is used to calibrate the coordinates of dangerous areas of concern in the power station video surveillance images and clarify the dangerous boundaries;
[0035] The early warning module is connected to the pedestrian detection module and the danger zone demarcation module, and is used to trigger an alarm signal when a pedestrian is detected entering the demarcated danger zone.
[0036] The computer device provided by the present invention includes:
[0037] A processor having the aforementioned data processing module, pedestrian detection module, danger zone demarcation module, and warning module built therein, the processor being configured to execute a computer program;
[0038] A read-write memory used to store programs or data;
[0039] Image acquisition interface, which is used to connect to the image acquisition module.
[0040] Optionally, the computer device also includes:
[0041] The display control interface is used to connect to the display terminal to output pedestrian detection results, dangerous area intrusion determination results and alarm signals to the display terminal.
[0042] The technical solution provided by the present invention has the following advantages compared with the prior art:
[0043] The present invention provides a method, device, and computer equipment for detecting pedestrians in hazardous areas of power plants. These methods utilize an improved YOLOv11 model, comprising a high-frequency feature extraction network, a multi-scale convergence and expansion feature fusion network, and a lightweight detection head. The high-frequency feature extraction network improves the ability to extract high-frequency information from shallow features, while the multi-scale convergence and expansion feature fusion network effectively fuses multi-scale features, significantly improving detection accuracy. Furthermore, the lightweight detection head effectively controls the scale of model parameters, balancing detection performance with inference speed and deployment efficiency. This makes this solution suitable for pedestrian detection in hazardous areas of power plants. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0046] Figure 1 A schematic diagram showing a flow chart of a detection method according to an embodiment of the present invention;
[0047] Figure 2 Schematic diagram showing an improved YOLOv11 model according to an embodiment of the present invention;
[0048] Figure 3 A schematic diagram showing the structure of a high-frequency feature enhancement module according to an embodiment of the present invention;
[0049] Figure 4 A schematic diagram showing the structure of a multi-scale convergence and expansion feature fusion network according to an embodiment of the present invention;
[0050] Figure 5 A schematic diagram showing the structure of a lightweight detection head according to an embodiment of the present invention;
[0051] Figure 6 A schematic structural diagram of a detection device according to an embodiment of the present invention is shown;
[0052] Figure 7 A schematic diagram showing the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION
[0053] In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the scheme of the present invention will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features therein can be combined with each other.
[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present invention, rather than all the embodiments.
[0055] The specific embodiments of the present invention are described in detail below with reference to the drawings.
[0056] Example 1
[0057] This embodiment provides a method for detecting pedestrians in a dangerous area of a power station, including steps S1 to S4.
[0058] S1. Create a pedestrian dataset in a power station scenario and label the dataset.
[0059] Specifically, this example captures surveillance footage from a surveillance camera at the waste power plant's unloading platform and extracts an image every 50 frames to construct a pedestrian detection dataset. This method yields approximately 2,000 image samples, covering a variety of typical scenes within the unloading area and environmental conditions under varying lighting conditions.
[0060] Specifically, Labelme annotation software is used to select and annotate pedestrian targets in the image, generating a standard annotation file format for input of subsequent training samples.
[0061] It should be noted that the dataset is not limited to the waste power plant unloading platform scenario, but can also be extended to other high-risk areas within the power plant, including but not limited to fuel transfer channels, high-voltage equipment areas, and maintenance channels. The specific collection and annotation scope can be flexibly selected based on the target hazardous area of concern.
[0062] S2. Build an improved YOLOv11 model. The improved YOLOv11 model uses a high-frequency feature extraction network as the backbone network, a multi-scale convergence and expansion feature fusion network (MS-FDFF) as the neck network, and a lightweight detection head (DSL-Head) as the head network.
[0063] Specifically, the high-frequency feature extraction network is obtained by replacing the first and second convolutional layers in the backbone network of the YOLOv11 model with the high-frequency feature enhancement module (EFEM).
[0064] It should be noted that after the image is input into the high-frequency feature extraction network, it is first processed by EFEM to enhance the high-frequency features of the target in the shallow feature extraction stage, thereby providing richer high-frequency information for subsequent feature processing. The subsequent feature extraction network is used to extract features of three different scales: P3, P4, and P5. 、 and .
[0065] Specifically, the high-frequency feature enhancement module performs the following operations:
[0066] 1) Input image Perform preliminary processing through a 3×3 convolution layer and reduce the channel dimension to obtain features ;
[0067] 2) The features The Sobel operator is used to enhance high-frequency information and obtain the feature , so that the model focuses on the edge contours of the target, The calculation formula is as follows:
[0068]
[0069]
[0070] And the features MaxPool is used to perform maximum pooling to obtain features , to capture information with a larger receptive field;
[0071] 3) Use feature splicing to combine features and features The fusion is performed, and a 3×3 convolutional layer is used to further mine the shallow features after splicing;
[0072] 4) The output features of the previous step are passed through a 1×1 convolution layer to adjust the number of channels, and batch normalization is used for normalization. The ReLU activation function is used to complete feature activation to obtain the feature , The calculation formula is as follows:
[0073]
[0074] Where, represents a 3×3 convolution operation, represents a 1×1 convolution operation, Represents a batch normalization operation.
[0075] Specifically, the multi-scale aggregation and expansion feature fusion network includes a feature aggregation module and a feature diffusion module; the feature aggregation module is used to integrate cross-scale information of features at different scales to aggregate feature information at different scales; the feature diffusion module is used to diffuse the aggregated features so that features at each scale can obtain partial contextual information at other scales.
[0076] More specifically, the feature aggregation module performs the following operations:
[0077] 1) Extract the features extracted by the high-frequency feature extraction network and Enhance the feature representation capability through efficient local attention (ELA) to obtain the corresponding features and ;
[0078] 2) The features and and features extracted by the high-frequency feature extraction network Input into the dynamic adaptive selection fusion module (DASI) to obtain the fusion features of P4 .
[0079] In detail, DASI divides high-dimensional, low-dimensional and current layer features in the channel dimension, and fuses appropriate feature information according to the size and characteristics of the target through an adaptive weight selection mechanism to obtain the fusion feature of P4 :
[0080] DASI converts high-dimensional features and low-dimensional features t is aligned to the features of the current layer through convolution and interpolation operations , ensuring the spatial and semantic consistency of multi-scale features; the aligned features are divided into four parts along the channel dimension, and the 、 and ( =1, 2, 3, 4); each part is merged through an adaptive weight selection mechanism, The calculation formula is as follows:
[0081]
[0082] Where, It is through Apply the weights generated by the Sigmoid function for balancing and Contribution, represents the batch normalization operation, represents the ReLU activation function, represents a 1×1 convolution operation, Represents a feature concatenation operation.
[0083] More specifically, the feature diffusion module performs the following operations:
[0084] On the one hand, the fusion features of P4 Through the dynamic learnable upsampling operation (Dysample), upsampling to the spatial scale of P3 and combining with the feature Perform feature splicing and then input it into the C3K2 module to fuse the semantic information from the middle layer and the low layer to obtain the fused features of P3 ;
[0085] On the other hand, the fusion features of P4 The spatial context information is extracted by depthwise separable convolution and then combined with the feature The fusion is performed and finally sent to the C3K2 module to extract deeper semantic features to obtain the fusion features of P5 .
[0086] Specifically, the lightweight detection head performs the following operations:
[0087] 1) The features 、 and The input is sent to the parameter-sharing common feature extraction module, and independent convolution operations and group convolution operations are performed in sequence. The convolution output results of each group are then spliced together.
[0088] Independent convolution operations can enhance the feature expression within each channel. In grouped convolution operations, each group contains a fixed number of 16 channels. The features within each group are processed independently through standard convolution, and then the convolution output results of each group are spliced together.
[0089] 2) The features extracted in the previous step are processed through 1×1 convolution to generate a tensor with a channel number of 4×self.reg_max, which serves as the regression branch. At the same time, they are processed through 1×1 convolution to generate a tensor with a channel number of nc, which serves as the classification branch.
[0090] 3) Output the bounding box information of pedestrians in the image.
[0091] S3. Randomly divide the dataset into training and validation sets, and train the improved YOLOv11 model. Save the best model parameters based on performance to obtain the target model.
[0092] Specifically, the training set and validation set are randomly divided in a ratio of 8:2 for model training and performance evaluation.
[0093] Specifically, the training parameters include but are not limited to batch size, training cycle, initial learning rate, weight decay coefficient, etc. The specific values can be set according to the actual application scenario and model requirements. For example: in this embodiment, the batch size is 32, the training cycle is 200, the initial learning rate is 0.01, and the weight decay coefficient is 0.0005.
[0094] Specifically, the training environment is built based on PyTorch 2.6.0 and Python 3.9.21, and the implementation process completes the model training and validation process in this environment. The training environment is configured with the following hardware: an Intel(R) Xeon(R) Silver 4316 CPU, an NVIDIA GeForce RTX 4090 graphics card (24GB of video memory), and the operating system OpenEuler 22.03.
[0095] S4. Input the data to be detected into the target model and output the pedestrian detection results.
[0096] Specifically, the image and video data to be detected are input into the target model to generate a set of prediction boxes for pedestrian detection, thereby determining the pedestrian detection results.
[0097] Example 2
[0098] This embodiment provides a pedestrian detection device for monitoring dangerous areas in a power station, comprising an image acquisition module, a data processing module, a pedestrian detection module, a dangerous area demarcation module, and an early warning module.
[0099] Among them, the image acquisition module is used to collect real-time images of dangerous areas of the power station.
[0100] The data processing module is used to pre-process the images collected by the image collection module.
[0101] Specifically, preprocessing includes but is not limited to image enhancement, noise removal, etc., with the purpose of improving the accuracy of subsequent detection.
[0102] Among them, the pedestrian detection module has a built-in improved YOLOv11 model described in Example 1. The pedestrian detection module is used to input the image preprocessed by the data processing module into the improved YOLOv11 model and output the pedestrian detection result.
[0103] Among them, the dangerous area delineation module is used to calibrate the coordinates of the dangerous areas of concern in the power station video surveillance images and clarify the dangerous boundaries.
[0104] Specifically, for the unloading platform of a garbage power plant, the dangerous area includes but is not limited to the garbage unloading port, etc.
[0105] It is easy to understand that clarifying the danger boundary mainly provides a basis for subsequent pedestrian intrusion determination in dangerous areas.
[0106] Among them, the early warning module is connected to the pedestrian detection module and the danger zone demarcation module, and is used to trigger an alarm signal when a pedestrian is detected entering the designated danger zone.
[0107] Example 3
[0108] This embodiment provides a computer device including a processor, a read-write memory, and an image acquisition interface.
[0109] The processor has built-in data processing module, pedestrian detection module, danger zone demarcation module and warning module described in Example 2, and is used to execute computer programs.
[0110] It should be noted that the number of processors can be one or more to achieve functions such as pedestrian detection, danger zone demarcation, intrusion judgment and warning information generation.
[0111] The read-write memory is used to store programs or data.
[0112] Specifically, the program or data includes but is not limited to one or more computer programs or data such as an operating system, a pedestrian detection method, model parameters, and historical image data or log information related to security monitoring.
[0113] Among them, the image acquisition interface is used to connect the image acquisition module.
[0114] Specifically, the image acquisition interface is connected to the image acquisition module by wire or wirelessly to receive and acquire real-time video stream or image data of the power station.
[0115] Furthermore, the computer device also includes a display control interface, which is used to connect to a display terminal to output pedestrian detection results, dangerous area intrusion determination results and alarm signals to the display terminal.
[0116] Specifically, the display terminal includes but is not limited to a large monitoring screen, a visualization panel, a mobile terminal, etc., which is used to assist operators in understanding the on-site safety status in real time.
[0117] The above is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Although detailed descriptions have been made with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents; and such modifications or replacements do not deviate from the essence of the corresponding technical solutions within the scope of the technical solutions of the embodiments, and they should all be covered by the scope of protection of the claims.
Claims
1. A method for detecting pedestrians in dangerous areas of a power station, characterized in that: The steps include: S1. Create a pedestrian dataset in a power station scenario and label the dataset; S2. Construct an improved YOLOv11 model, wherein the improved YOLOv11 model uses a high-frequency feature extraction network as the backbone network, a multi-scale aggregation and expansion feature fusion network as the neck network, and a lightweight detection head as the head network; S3. randomly divide the dataset into a training set and a validation set, and train the improved YOLOv11 model, saving the best model parameters based on performance to obtain the target model; S4. Input the data to be detected into the target model and output the pedestrian detection result.
2. The method for detecting pedestrians in dangerous areas of a power station according to claim 1, characterized in that: The high-frequency feature extraction network is obtained by replacing the first convolutional layer and the second convolutional layer in the backbone network of the YOLOv11 model with a high-frequency feature enhancement module.
3. The pedestrian detection method in a dangerous area of a power station according to claim 2, characterized in that: The high-frequency feature enhancement module performs the following operations: 1) Input image Perform preliminary processing through a 3×3 convolution layer and reduce the channel dimension to obtain features ; 2) The features The Sobel operator is used to enhance high-frequency information and obtain the feature , and the features MaxPool is used to perform maximum pooling to obtain features ; 3) Use feature splicing to combine features and features The fusion is performed, and a 3×3 convolutional layer is used to further mine the shallow features after splicing; 4) The output features of the previous step are passed through a 1×1 convolution layer to adjust the number of channels, and batch normalization is used for normalization. The ReLU activation function is used to complete feature activation to obtain the feature .
4. The method for detecting pedestrians in dangerous areas of a power station according to claim 1, characterized in that: The multi-scale aggregation and expansion feature fusion network includes a feature aggregation module and a feature diffusion module; The feature aggregation module is used to integrate cross-scale information of features at different scales to aggregate feature information at different scales; The feature diffusion module is used to diffuse the aggregated features so that features at each scale can obtain partial context information of other scales.
5. The method for detecting pedestrians in dangerous areas of a power station according to claim 4, characterized in that: The feature aggregation module performs the following operations: 1) The features extracted by the high-frequency feature extraction network and Enhance the feature representation capability through efficient local attention to obtain the corresponding features and ; 2) The features and and the features extracted by the high-frequency feature extraction network Input into the dynamic adaptive selection fusion module to obtain the fusion features of P4 .
6. The method for detecting pedestrians in dangerous areas of a power station according to claim 5, characterized in that: The feature diffusion module performs the following operations: On the one hand, the fusion features of P4 Through dynamic learnable upsampling operation, upsampling to the spatial scale of P3 and combining with feature Perform feature splicing and then input it into the C3K2 module to fuse the semantic information from the middle and low layers to obtain the fused features of P3 ; On the other hand, the fusion features of P4 The spatial context information is extracted by depthwise separable convolution and then combined with the feature The fusion is performed and finally sent to the C3K2 module to extract deeper semantic features to obtain the fusion features of P5 .
7. The method for detecting pedestrians in dangerous areas of a power station according to claim 6, characterized in that: The lightweight detection head performs the following operations: 1) The features 、 and The input is sent to the parameter-sharing common feature extraction module, and independent convolution operations and group convolution operations are performed in sequence. The convolution output results of each group are then spliced together. 2) The features extracted in the previous step are processed through 1×1 convolution to generate a tensor with a channel number of 4×self.reg_max, which serves as the regression branch. At the same time, they are processed through 1×1 convolution to generate a tensor with a channel number of nc, which serves as the classification branch. 3) Output the bounding box information of pedestrians in the image.
8. A pedestrian detection device for monitoring dangerous areas in power stations, characterized in that: include: An image acquisition module, which is used to acquire real-time images of hazardous areas of the power station; A data processing module, configured to pre-process the images acquired by the image acquisition module; A pedestrian detection module having a built-in improved YOLOv11 model according to any one of claims 1 to 7, wherein the pedestrian detection module is configured to input the image preprocessed by the data processing module into the improved YOLOv11 model and output a pedestrian detection result; Dangerous area demarcation module, which is used to calibrate the coordinates of dangerous areas of concern in the power station video surveillance images and clarify the dangerous boundaries; The early warning module is connected to the pedestrian detection module and the danger zone demarcation module, and is used to trigger an alarm signal when a pedestrian is detected entering the demarcated danger zone.
9. A computer device, characterized in that: include: a processor having a built-in data processing module, a pedestrian detection module, a danger zone delineation module, and an early warning module as claimed in claim 8, wherein the processor is configured to execute a computer program; A read-write memory used to store programs or data; Image acquisition interface, which is used to connect to the image acquisition module.
10. The computer device according to claim 9, wherein: Also includes: The display control interface is used to connect to the display terminal to output pedestrian detection results, dangerous area intrusion determination results and alarm signals to the display terminal.