Image detection method, device and equipment of pumped storage power station and storage medium

Abstract

This application provides an image detection method, apparatus, equipment, and storage medium for pumped storage power stations, relating to the field of image processing, to improve the accuracy and reliability of detecting whether workers are wearing target objects, thereby ensuring worker safety. The method includes: firstly, extracting and fusing multi-level features of the image to enhance the representation ability of targets at different scales; then, outputting the target's wearing status, location information, and pixel-level instance segmentation mask based on the fused features; and finally, comprehensively utilizing the above multi-dimensional output information for a comprehensive judgment, thereby achieving highly accurate and reliable automated identification of the wearing status of target objects in complex environments.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an image detection method, apparatus, equipment, and storage medium for pumped storage power stations. Background Technology

[0002] As an important component of the power system, pumped storage power stations have deep underground caverns, intersecting corridors, and uneven lighting. During unit maintenance, multiple trades work simultaneously, resulting in problems such as dense personnel, small target size, and mutual obstruction in the work area. In such a complex working environment, the automatic and accurate identification of the wearing status of workers' safety protective equipment (such as safety helmets) is one of the core requirements for ensuring production safety.

[0003] In existing technologies, dedicated sensors (such as RFID tags) are pre-integrated on the target object, and corresponding signal receivers are deployed in the work area. By analyzing the signal reception, it is determined whether the operator is wearing the target object.

[0004] However, this detection method relies on additional dedicated hardware, resulting in high deployment and maintenance costs; moreover, in complex environments such as complex metal structures or large spatial depths, signal transmission is easily interfered with, leading to insufficient detection reliability. Summary of the Invention

[0005] This application provides an image detection method, apparatus, device, and storage medium for pumped storage power stations, which improves the accuracy and reliability of detecting whether a target object is wearing a target object.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions: In a first aspect, embodiments of this application provide an image detection method for a pumped storage power station, the method comprising: Acquire the image to be processed; the image to be processed is the image frame data of the working area of ​​the pumped storage power station, and the image frame data contains the target object to be detected. The image to be processed is subjected to feature extraction to obtain feature maps of multiple levels with different spatial resolutions. Based on the spatial resolution differences and hierarchical relationships between feature maps of multiple levels, feature fusion processing is performed on feature maps of different levels to generate multiple fused feature maps. Based on multiple fused feature maps, the detection process is performed to determine whether the target object in the image to be processed is wearing a target object, and the detection result is obtained. The detection result includes the target object wearing status corresponding to the target object, the target object's location information, and the target object's instance segmentation mask. Based on the target object's wearing status, location information, and instance segmentation mask in the detection results, it is determined whether the target object is wearing a target object.

[0007] In this embodiment, an image of the pumped storage power station's operating area is acquired and multi-level feature maps are extracted to provide a multi-scale information foundation for subsequent fusion and detection. The feature maps are then fused based on hierarchical relationships and resolution differences to generate multiple fused feature maps. This process retains the semantic information of high-level features while incorporating the detailed information of shallow features, enhancing the model's ability to represent targets at different scales. Detection is then performed based on the fused feature maps, simultaneously outputting wearing status, location information, and instance segmentation masks to achieve joint output of detection and segmentation. Finally, the wearing status, location information, and mask information are combined to comprehensively determine whether a target object is being worn, avoiding misjudgments caused by a single criterion. Overall, this method, through a mechanism of "multi-scale feature fusion + joint output determination," improves the accuracy and reliability of target object wearing status detection in complex environments, thereby better ensuring the safety of the target object.

[0008] In one possible implementation of the first aspect, the feature maps of multiple levels include a k-th level feature map, where k is an integer from 1 to N, and N is a positive integer greater than 1; feature extraction processing is performed on the image to be processed to obtain feature maps of multiple levels with different spatial resolutions, including: The (k-1)th level feature map is subjected to the first convolution process to obtain the first intermediate feature map. Based on the channel dimension, the first intermediate feature map is divided to obtain multiple sub-feature maps with different channel dimensions. The convolution kernel of the first convolution process is 1×1. The second convolution process is performed on each sub-feature map, and the sub-feature maps processed by the second convolution are merged along the channel dimension to obtain the second intermediate feature map; the convolution kernel for the second convolution process is 3×3; The k-1 level feature map is divided into a first feature sub-map and a second feature sub-map in the channel dimension according to a preset ratio. The first feature sub-map and the second intermediate feature map are then added element by element to obtain the third intermediate feature map. The third intermediate feature map is subjected to nonlinear activation processing to obtain the k-th level feature map; When k is 1, the (k-1)th level feature map is the image to be processed; the spatial resolution decreases from the first level feature map to the kth level feature map.

[0009] Based on the above technical content, in this embodiment, recursive feature extraction is used to generate feature maps with decreasing resolution layer by layer. In feature extraction, 1×1 convolutional kernels are used for channel compression to reduce computational load, followed by grouped 3×3 convolutional kernels to enhance feature diversity. Then, cross-channel residual connections are introduced to preserve input information, and finally, nonlinear activation is used to enhance expressive power. This extraction method ensures feature richness while maintaining high computational efficiency, providing a stable and high-quality feature foundation for subsequent multi-level feature fusion and accurate detection.

[0010] In one possible implementation of the first aspect, the multiple fused feature maps include the k-th fused feature map; k is an integer from 1 to N, and N is a positive integer greater than 1; Based on the spatial resolution differences and hierarchical relationships between feature maps of multiple levels, feature fusion processing is performed on feature maps of different levels to generate multiple fused feature maps, including: When k is N, feature extraction is performed on the first-level feature map to obtain the Nth fused feature map; When k is less than N, the reference feature map is upsampled to obtain an upsampled feature map; the upsampled feature map is concatenated with the (Nk)th level feature map to obtain a concatenated feature map; the concatenated feature map is subjected to feature extraction to obtain the kth fused feature map; where, when k is 1, the reference feature map is the Nth level feature map; when k is greater than 1 and less than N, the reference feature map is the (k-1)th fused feature map.

[0011] Here, multi-scale feature fusion is achieved through layer-by-layer upsampling and feature concatenation: high-level features are upsampled and concatenated with adjacent low-level features, and then fused features are obtained through feature extraction. This operation effectively combines high-level semantic information with low-level detail information, improving the model's ability to perceive small targets and detailed features. Simultaneously, the shallowest layer features are extracted independently to retain the highest resolution information, further enhancing the detection capability for extremely small targets. Overall, this fusion strategy constructs a feature pyramid covering multiple scales, providing rich and structured feature inputs for subsequent detection heads.

[0012] In one possible implementation of the first aspect, the location information is the bounding box coordinates; based on multiple fused feature maps, the detection process is performed to determine whether the target object in the image to be processed is wearing a target object, and the detection result is obtained, including: Each fused feature map is convolved to generate an initial instance segmentation mask for the target object, and the mask quality corresponding to each instance segmentation mask is determined. Adjust the spatial offset of deformable convolution during feature sampling based on mask quality; The adjusted deformable convolution is used to sample features from each fused feature map to obtain a sampled feature map. Based on the sampled feature map, the wearing state and position of the target object are identified. The initial bounding box coordinates of the target object, the confidence level of the initial bounding box coordinates, and the initial wearing state category of the target object are obtained at the spatial resolution corresponding to the fused feature map. The wearing state category of the target object includes the wearing state and the unwearing state. Based on the initial bounding box coordinates and confidence scores of the target object obtained from all fused feature maps, non-maximum suppression is performed on multiple initial bounding box coordinates of the target object to obtain the bounding box coordinates of the target object in the image to be processed. The initial target object wearing state category associated with the bounding box coordinates is determined as the target object wearing state, and the initial instance segmentation mask associated with the bounding box coordinates is determined as the instance segmentation mask.

[0013] In this embodiment, the detection process first generates an instance segmentation mask and evaluates its quality. Based on this, the sampling offset of the deformable convolution is dynamically adjusted to achieve adaptive feature focusing on occluded or deformed regions. Then, the adjusted convolution is used for feature sampling and state recognition to obtain preliminary bounding boxes, confidence scores, and wear categories. Finally, non-maximum suppression is used to integrate the preliminary results from multiple scales to determine the final bounding box coordinates, wear status, and instance mask. This "mask quality-guided dynamic sampling + multi-scale result aggregation" mechanism effectively improves the consistency and accuracy of detection and segmentation results in complex scenarios such as personnel occlusion and dense target areas.

[0014] In one possible implementation of the first aspect, determining whether the target object is wearing a target object based on the target object's wearing state, bounding box coordinates, and instance segmentation mask in the detection results includes: If the target object's corresponding target object wearing state category is not wearing, and the confidence level corresponding to the bounding box coordinates of the target object is greater than or equal to a preset first threshold, and the effective pixel area of ​​the instance segmentation mask corresponding to the target object is greater than or equal to a preset second threshold, then it is determined that the target object is not wearing a target object. If the target object's corresponding target item wearing status category is "wearing", then it is determined that the target object has worn the target item.

[0015] Based on the above technical content, this embodiment establishes a multi-condition joint judgment logic: only when the target is classified as "not wearing", its location confidence level meets the standard, and its instance mask area is valid, is it ultimately determined to be an unwearing event. This triple filtering mechanism can effectively identify and eliminate false alarms caused by image noise, partial occlusion, or detection fluctuations, thereby significantly improving the reliability and credibility of the final judgment result and reducing invalid interference with security supervision work.

[0016] In one possible implementation of the first aspect, the method further includes: Obtain relevant information about the target object that is not wearing a target object; the relevant information includes one or more of the following: the target object's location information, timestamp, corresponding instance segmentation mask, and mask quality; The relevant information is output to the preset platform; the preset platform is used to display the number, frequency or distribution of events where no target object is worn under multiple preset dimensions; the multiple preset dimensions include time dimension and region dimension.

[0017] In this embodiment, by extracting relevant information (such as location, time, mask, etc.) of non-wearing events and outputting it to the management platform, multi-dimensional statistics and analysis are supported, which helps managers quickly grasp the spatiotemporal distribution and frequency characteristics of non-wearing events and provides data support for the formulation of safety early warning and control strategies.

[0018] In one possible implementation of the first aspect, the image detection method for pumped storage power stations is implemented using a pre-trained model, which is trained by the following steps: Repeat the following steps until the adjusted model meets the preset conditions and you obtain a trained model: Input the training image into the initial model to obtain the predicted wearing state category of each target object in the training image output by the initial model, the predicted bounding box coordinates of each target object, and the predicted instance segmentation mask of each target object; Determine the classification loss between the predicted wearing state category and the actual wearing state category for each target object in the training image; Determine the aspect ratio difference between the predicted bounding box and the true bounding box for each target object, the distance between their center points, and the intersection-union ratio (IU) between the predicted bounding box and the true bounding box for each target object. Based on the aspect ratio difference between the predicted bounding box and the true bounding box and the distance between their center points, determine the loss weighting coefficient for each target object. Based on the corresponding loss weighting coefficient and IU, determine the bounding box regression loss for each target object. Determine the segmentation loss between the predicted instance segmentation mask and the actual segmentation mask for each target object; Calculate the total loss of the initial model with respect to the training image based on the classification loss, bounding box regression loss, and segmentation loss for each target object; Based on the total loss, adjust the parameters of the initial model to obtain the adjusted model.

[0019] Here, in model training, this embodiment enables the model to simultaneously learn the ability to classify wear status, regress position, and segment at the pixel level by jointly optimizing the classification loss, bounding box regression loss, and segmentation loss. In particular, a dynamic weighting coefficient based on aspect ratio and center point distance is introduced in the bounding box regression, which makes the training pay more attention to difficult samples and improves the detection stability and convergence efficiency of the model in complex scenes.

[0020] In one possible implementation of the first aspect, acquiring the image to be processed includes: Video stream data is acquired using an image acquisition terminal deployed on the top of the underground cavern of a pumped storage power station; Each image frame in the video stream is subjected to illumination equalization and contrast enhancement processing to obtain the image to be processed.

[0021] This embodiment constructs an image input chain that conforms to the actual working environment by deploying image acquisition terminals on the top of the underground cavern of a pumped storage power station and implementing preprocessing: the top deployment can obtain a panoramic view from above, covering the depth of the cavern and intersecting corridors, avoiding ground view obstruction; after acquiring the video stream, illumination equalization and contrast enhancement processing are performed to address typical problems such as uneven lighting, metal reflection, and dust interference within the cavern, significantly improving the detail discernibility of the image in low-light and high-contrast areas. This scenario-adaptive acquisition and preprocessing mechanism suppresses the impact of the complex industrial environment on image quality from the source, providing clear, stable, and scenario-specific image input for subsequent feature extraction and target detection, thereby significantly improving the system's detection reliability and environmental adaptability in real cavern environments without affecting real-time performance.

[0022] Secondly, an image detection device for a pumped storage power station is provided, the device comprising: The acquisition unit is used to acquire the image to be processed; the image to be processed is image frame data of the working area of ​​the pumped storage power station, and the image frame data contains the target object to be detected. The first processing unit is used to perform feature extraction processing on the image to be processed, and obtain feature maps of multiple levels with different spatial resolutions. The second processing unit is used to perform feature fusion processing on feature maps of different levels based on the spatial resolution differences and hierarchical relationships between feature maps of multiple levels, and generate multiple fused feature maps. The third processing unit is used to detect whether the target object in the image to be processed is wearing a target object based on multiple fused feature maps, and to obtain the detection result. The detection result includes the target object wearing status corresponding to the target object, the target object's location information, and the target object's instance segmentation mask. The determination unit is used to determine whether the target object is wearing a target object based on the target object wearing status, location information and instance segmentation mask corresponding to the target object in the detection results.

[0023] Thirdly, an electronic device is provided, the method comprising: a transceiver, a memory, and at least one processor. The memory is communicatively connected to the processor. The memory is used to store computer program code, the computer program code including computer instructions. When the processor executes the computer instructions, it causes the electronic device to perform the method of the first aspect and any possible implementation thereof.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions. When these computer instructions are executed by a processor, they are used to implement the method as described in the first aspect and any possible implementation thereof.

[0025] Fifthly, embodiments of this application provide a computer program product that, when running on a computer / executed by the computer's processor, implements the method described in the first aspect and any possible design thereof. The computer may be an electronic device as described in the third aspect and any possible implementation thereof.

[0026] Understandably, the beneficial effects achieved by the image detection device for the pumped storage power station of the second aspect, the electronic device of the third aspect, the computer-readable storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above can be referred to as the beneficial effects of the first aspect and any possible implementation thereof, which will not be repeated here. Attached Figure Description

[0027] Figure 1 This is a schematic flowchart of an image detection method for a pumped storage power station provided in an embodiment of this application; Figure 2 This is a flowchart illustrating another image detection method for a pumped storage power station provided in an embodiment of this application. Figure 3 This is a flowchart illustrating another image detection method for a pumped storage power station provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an image detection device for a pumped storage power station provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0030] The collection, storage, use, processing, transmission, provision, and disclosure of user data and other information involved in the technical solutions provided in this application comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0031] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0032] In semi-enclosed spaces such as pumped-storage power stations and mine tunnels, which are deep, structurally complex, and subject to variable lighting conditions, work sites are scattered, multiple trades frequently work together, and personnel and equipment are highly mobile. This results in visual challenges in monitoring images, including densely packed targets, small individual sizes, and severe mutual obstruction. Traditional manual inspection methods are not only limited by labor costs but also suffer from limited visibility in the narrow underground spaces and the inability to maintain continuous 24-hour on-duty presence, making it difficult to achieve all-weather, comprehensive safety status coverage. This leads to blind spots and delayed responses in safety supervision. Therefore, developing an automatic, accurate, and adaptable personnel safety status (such as the wearing of personal protective equipment) identification technology that can adapt to complex industrial environments has urgent practical significance and important application value.

[0033] In related technologies, dedicated sensors (such as RFID tags) are pre-integrated onto targets such as safety helmets, and a corresponding signal receiving network is deployed in the work area. By analyzing the reception and interpretation of signals, it is indirectly determined whether the worker is wearing the target. However, this method is highly dependent on additional dedicated hardware, which significantly increases the cost of system deployment and subsequent maintenance. More importantly, in complex environments with numerous metal structures (such as the walls of power plant tunnels or steel frames of factory buildings) or large spatial depths, wireless signals are susceptible to multipath interference and shielding attenuation, resulting in unstable signal reception, large positioning drift, and difficulty in ensuring the reliability of detection. This makes it impossible to meet the requirements for high-precision and high-robust continuous monitoring.

[0034] To improve the accuracy and reliability of detecting whether workers are wearing target objects, in this embodiment, multi-level features of the image are first extracted and fused to enhance the representation ability of targets at different scales. Then, based on the fused features, the wearing status, location information and pixel-level instance segmentation mask of the target are output. Finally, the above multi-dimensional output information is comprehensively used to make a comprehensive judgment, thereby achieving highly accurate and reliable automated recognition of the wearing status of target objects in complex environments.

[0035] This application provides an image detection method for pumped storage power stations, which can be applied to electronic devices. These electronic devices can be computer equipment, a single server or a server cluster consisting of multiple servers, or edge computing devices with image detection capabilities, cloud computing platforms, chips, or devices with computing capabilities. This application does not limit the specific form of the electronic device.

[0036] Figure 1 This is a schematic flowchart of an image detection method for a pumped storage power station provided in an embodiment of this application. Figure 1 As shown, the method in the embodiments of this application may include: S101. Obtain the image to be processed; the image to be processed is the image frame data of the pumped storage power station's operating area, and the image frame data contains the target object to be detected.

[0037] For example, in this embodiment, an image frame containing the target object is obtained by using an image acquisition device (such as a camera) or by reading from an image storage system.

[0038] For example, in some work scenarios, the image to be processed may be a frame from a real-time video stream of the work site, which contains construction workers (the target object).

[0039] This embodiment provides raw input for subsequent wearable status detection by acquiring image data containing the target object, ensuring that the data source of the detection process is authentic and reliable.

[0040] S102. Perform feature extraction processing on the image to be processed to obtain feature maps of multiple levels with different spatial resolutions.

[0041] For example, this embodiment uses a pre-trained neural network or feature extraction network to perform forward propagation on the image to be processed, extracting feature maps at multiple levels. For instance, by stacking convolutional layers and pooling layers, multiple feature maps are obtained from shallow to deep layers, with the spatial resolution decreasing layer by layer and the semantic information increasing layer by layer. For example, the first layer feature map has a higher resolution and contains detailed textures; the deeper feature maps have a lower resolution and contain the overall semantic information of the target.

[0042] This embodiment captures visual information at different scales in an image by extracting multi-level feature maps, including both local detail features and global semantic features. This provides a rich feature base for subsequent multi-scale fusion and object detection, and enhances the model's adaptability to scale changes and occlusion.

[0043] S103. Based on the spatial resolution differences and hierarchical relationships between feature maps of multiple levels, feature fusion processing is performed on feature maps of different levels to generate multiple fused feature maps.

[0044] For example, this embodiment can employ a top-down or bottom-up feature fusion strategy to fuse feature maps from different levels. For instance, for deep feature maps with low resolution, an upsampling operation is first performed to match the spatial size of shallow feature maps, and then element-wise addition or concatenation is performed with the feature maps of the corresponding levels to generate a fused feature map.

[0045] During the fusion process, this embodiment may introduce an attention mechanism or learnable fusion weights to adaptively adjust the contribution of features at each level.

[0046] In one feasible implementation, this embodiment can use structures such as Feature Pyramid Network (FPN) or Path Aggregation Network (PAN) to achieve multi-scale feature fusion, ensuring that the fused feature map has both high semantic information and high spatial detail.

[0047] This embodiment effectively integrates high-level semantic information and low-level detailed information by fusing multi-level features, enhancing the multi-scale and robustness of feature representation. This enables the model to maintain high feature discrimination ability when dealing with targets of different sizes, varied poses, and local occlusion, laying the foundation for subsequent accurate detection and segmentation.

[0048] S104. Based on multiple fused feature maps, detect whether the target object in the image to be processed is wearing a target object, and obtain the detection result. The detection result includes the target object wearing status, the target object's location information, and the target object's instance segmentation mask.

[0049] For example, in this embodiment, multiple fused feature maps are input into a joint detection and segmentation network for parallel or serial processing. For instance, a detection head (such as an Anchor-Based or Anchor-Free detector) outputs the bounding box position of the target object and the classification confidence of its wearing state; simultaneously, a segmentation head (such as the mask branch in Mask R-CNN) generates a pixel-level instance segmentation mask for each detected target object. The wearing state can be categorized as "worn," "not worn," or "partially worn."

[0050] This embodiment achieves a comprehensive description of the target object's wearing status by performing joint detection and segmentation based on fused feature maps, simultaneously outputting wearing status, location information, and instance segmentation masks. This approach avoids the information inconsistency problem caused by the separation of detection and segmentation in traditional methods, improves the completeness and reliability of the output results, and provides multi-dimensional basis for subsequent judgment.

[0051] S105. Based on the target object's wearing status, location information, and instance segmentation mask in the detection results, determine whether the target object is wearing a target object.

[0052] For example, this embodiment can make a final determination by comprehensively considering the confidence level of the wearing status, whether the target object's location is reasonable, and the coverage area and integrity of the instance segmentation mask.

[0053] For example, if the confidence level of the wearing status is higher than the preset threshold, the target object is located in a reasonable area of ​​the human body (such as the head, hands, etc.), and the segmentation mask coverage area is continuous and the area meets the requirements, it is determined as "wearing"; otherwise, it is determined as "not wearing" or "abnormal wearing".

[0054] This embodiment achieves a comprehensive judgment by fusing wearable status, location information, and instance segmentation masks, avoiding misjudgments that may result from relying solely on classification confidence or location information. This enhances the reliability and scenario adaptability of the judgment results.

[0055] In summary, this embodiment acquires images of the pumped storage power station's operating area and extracts multi-level feature maps, providing a multi-scale information foundation for subsequent fusion and detection. Then, the feature maps are fused based on hierarchical relationships and resolution differences to generate multiple fused feature maps. This preserves the semantic information of high-level features while incorporating the detailed information of shallow features, enhancing the model's ability to represent targets at different scales. Detection is then performed based on the fused feature maps, simultaneously outputting wearing status, location information, and instance segmentation masks, achieving joint output of detection and segmentation. Finally, the wearing status, location information, and mask information are combined to comprehensively determine whether a target object is being worn, avoiding misjudgments caused by a single criterion. Overall, this method, through a mechanism of "multi-scale feature fusion + joint output determination," improves the accuracy and reliability of target object wearing status detection in complex environments, thereby better ensuring the safety of workers.

[0056] Figure 2 This is a schematic flowchart of another image detection method for pumped storage power stations provided in an embodiment of this application. Figure 2 As shown, the method in the embodiments of this application may include: S201. Using the image acquisition terminal deployed on the top of the underground cavern of the pumped storage power station, video stream data is acquired; each image frame in the video stream data is processed for illumination equalization and contrast enhancement to obtain the image to be processed.

[0057] For example, in this embodiment, explosion-proof panoramic cameras are deployed at key locations on the top of the underground cavern of a pumped storage power station to form an image acquisition terminal. The real-time video stream is transmitted to a computing inference terminal deployed at the edge or in a data center via an industrial Ethernet network. The terminal has a built-in adaptive light source preprocessing module that automatically equalizes and enhances the image quality interference caused by smoke, dark light, and metallic reflections in the video stream to obtain a clear image with appropriate contrast.

[0058] This embodiment ensures the acquisition of high-quality input images in complex and harsh underground industrial environments by deploying dedicated image acquisition equipment and performing adaptive preprocessing. This provides a reliable visual data foundation for subsequent high-precision detection and meets on-site deployment requirements such as explosion-proof and low power consumption.

[0059] S202. Perform feature extraction processing on the image to be processed to obtain feature maps of multiple levels with different spatial resolutions; the feature maps of multiple levels include the k-th level feature map, where k is an integer from 1 to N, and N is a positive integer greater than 1.

[0060] Wherein, N is a preset value. In some embodiments, N can be 5, or a positive integer greater than 5, or an integer less than 5 but greater than 1.

[0061] In one feasible implementation, S202 includes the following steps: The (k-1)th level feature map is subjected to the first convolution process to obtain the first intermediate feature map. Based on the channel dimension, the first intermediate feature map is divided to obtain multiple sub-feature maps with different channel dimensions. The convolution kernel of the first convolution process is 1×1. The second convolution process is performed on each sub-feature map, and the sub-feature maps processed by the second convolution are merged along the channel dimension to obtain the second intermediate feature map; the convolution kernel for the second convolution process is 3×3; The k-1 level feature map is divided into a first feature sub-map and a second feature sub-map in the channel dimension according to a preset ratio. The first feature sub-map and the second intermediate feature map are then added element by element to obtain the third intermediate feature map. The third intermediate feature map is subjected to nonlinear activation processing to obtain the k-th level feature map; When k is 1, the (k-1)th level feature map is the image to be processed; the spatial resolution decreases from the first level feature map to the kth level feature map.

[0062] For example, this embodiment uses the image to be processed as the initial input. By continuously applying the feature extraction module K times, the first-level feature map, the second-level feature map, ..., up to the Kth-level feature map are generated sequentially.

[0063] The feature extraction module sequentially performs 1×1 convolution, channel partitioning, 3×3 group convolution, channel merging, residual addition with a portion of the input feature map, and activation processing on the image.

[0064] In one example, when k=1, the (k-1)th level feature map is the image to be processed. First, a 1×1 convolution kernel is used to perform a first convolution on the image to be processed to achieve channel compression, resulting in a first intermediate feature map. Then, the first intermediate feature map is divided into sub-feature maps based on the channel dimension. Next, a 3×3 convolution kernel is used to perform a second convolution on each sub-feature map to effectively suppress periodic noise caused by environmental factors such as metal hole walls in the image. After that, the sub-feature maps processed by the second convolution are merged again in the channel dimension to obtain a second intermediate feature map. At the same time, the original image to be processed is divided in the channel dimension according to a preset ratio (e.g., 1:1) to obtain a first feature sub-map and a second feature sub-map. Subsequently, the first feature sub-map and the previously obtained second intermediate feature map are added element-wise to achieve cross-stage residual connection, resulting in a third intermediate feature map. Finally, a non-linear activation process (such as the SiLU activation function) is performed on the third intermediate feature map to output the first level feature map.

[0065] For k=2, the first-level feature map is used as input (i.e., the (k-1)th-level feature map), and the same sequence of steps as the first-level generation process is repeated: 1×1 convolution → channel splitting → 3×3 group convolution → channel merging → residual addition with a portion of the input feature map → activation. The output is a second-level feature map with further reduced spatial resolution.

[0066] This process is recursively performed. For the k-th level (k>2), the feature map of the (k-1)-th level is taken as input, and a feature map of the k-th level with decreasing spatial resolution is generated by 1×1 convolution → channel splitting → 3×3 group convolution → channel merging → residual addition with a part of the input feature map → activation processing.

[0067] This embodiment can use an improved lightweight object detection model (based on the YOLOv8n framework) as a feature extractor.

[0068] This embodiment combines techniques such as channel compression, grouped convolutional noise reduction, and cross-stage residual connections for feature extraction. It can extract more robust and detailed multi-scale features under complex lighting and noise environments, laying a solid feature foundation for subsequent detection of small targets and occluded targets.

[0069] S203. Based on the spatial resolution differences and hierarchical relationships between feature maps of multiple levels, feature fusion processing is performed on feature maps of different levels to generate multiple fused feature maps; the multiple fused feature maps include the k-th fused feature map; k is an integer from 1 to N, and N is a positive integer greater than 1.

[0070] In one feasible implementation, S203 includes the following steps: When k is N, feature extraction is performed on the first-level feature map to obtain the Nth fused feature map.

[0071] When k is less than N, the reference feature map is upsampled to obtain an upsampled feature map; the upsampled feature map is concatenated with the (Nk)th level feature map to obtain a concatenated feature map; the concatenated feature map is subjected to feature extraction to obtain the kth fused feature map; where, when k is 1, the reference feature map is the Nth level feature map; when k is greater than 1 and less than N, the reference feature map is the (k-1)th fused feature map.

[0072] For example, this embodiment uses N layers of feature maps to generate N fused feature maps (e.g., N=5, constructing a five-scale feature pyramid). The fusion process begins with the deepest semantic features, and through iterative sampling and combination with shallow features of corresponding resolution, a sequence from the Nth fused feature map (corresponding to the deepest layer and lowest resolution) to the first fused feature map (corresponding to the shallowest layer and highest resolution) is gradually generated.

[0073] In one example, when k = N, the first-level feature map with the highest spatial resolution is used as the input. Feature extraction processing (such as the feature extraction processing in S202) is performed on this feature map to refine the features and adapt to the subsequent fusion structure. After the processing is completed, the Nth fusion feature map is output. This feature map has the lowest resolution but carries rich semantic information.

[0074] Regarding the generation of the (N - 1)th fusion feature map: The reference feature map (here it is the Nth fusion feature map) is upsampled (for example, using bilinear interpolation) to increase its spatial size, obtaining an upsampled feature map. Then, the upsampled feature map is concatenated with the shallow feature map with the corresponding spatial resolution (for example, the (N - (N - 1)) = 1st-level feature map) in the channel dimension, obtaining a concatenated feature map. Finally, feature extraction processing is performed on the concatenated feature map to fuse the deep semantics and shallow details, and the (N - 1)th fusion feature map is output. Its spatial resolution is higher than that of the Nth fusion feature map.

[0075] For the kth fusion feature map (k < N - 1), its generation process is recursive and includes the following steps: Upsampling: The reference feature map (that is, the (k + 1)th fusion feature map generated in the previous step) is upsampled to obtain an upsampled feature map that matches the resolution of the target level.

[0076] Concatenation: The upsampled feature map is concatenated with the (N - k)th-level feature map with the corresponding spatial resolution output by the feature extraction module in the channel dimension, obtaining a concatenated feature map.

[0077] Extraction: Feature extraction processing is performed on the concatenated feature map to output the kth fusion feature map.

[0078] In this embodiment, by constructing a multi-scale feature pyramid and performing fusion based on the improved feature extraction module, the rich semantic information of high-level features and the fine spatial details of low-level features are effectively integrated, enhancing the representation ability for targets of different scales (especially small targets), and solving the problem of premature loss of small target features caused by downsampling.

[0079] S204. Perform convolution processing on each fusion feature map to generate an initial instance segmentation mask for the target object corresponding to the target object, and determine the mask quality corresponding to each instance segmentation mask.

[0080] Among them, the target object can be a safety helmet.

[0081] Exemplarily, in this embodiment, in the LADH (Location-Aware Dynamic Head) dual-task dynamic head, the detection branch and the mask branch are executed in parallel.

[0082] For the mask branch, it operates on each fused feature map (e.g., each layer in a five-scale feature pyramid). Specifically, this embodiment uses lightweight convolutional layers (e.g., 1×1 convolutions or small 3×3 depthwise separable convolutions) to process the input fused feature map. The number of channels output by this convolutional layer corresponds to the number of mask prototypes and classes predicted for each spatial location. Through subsequent activation and upsampling operations, a corresponding initial instance segmentation mask is generated for each anchor point or location detected as a potential safety helmet on the fused feature map. This mask is a binary map indicating the region belonging to the safety helmet at the pixel level. Simultaneously, this embodiment calculates a mask quality score for each generated initial instance segmentation mask. This score can be obtained by combining indicators such as the confidence score of the predicted mask, the sharpness of the mask edges (e.g., by calculating the gradient magnitude of the predicted mask), or the compactness of the mask region.

[0083] This embodiment directly generates instance segmentation masks and evaluates their quality through a mask branch, achieving pixel-level fine target contour extraction. The mask quality score quantifies the credibility and completeness of each predicted mask, providing a crucial guiding signal for adaptive feature sampling in subsequent detection branches. This is the foundation for jointly optimizing detection and segmentation tasks and improving perception capabilities in occluded scenes.

[0084] S205. Adjust the spatial offset of deformable convolution during feature sampling based on mask quality.

[0085] For example, this embodiment designs a dynamic offset prediction mechanism in the detection branch of the LADH detection head. Specifically, for each target location (anchor point) to be detected, this embodiment uses the "mask quality score" output by its associated mask branch as an input signal, feeding it into a lightweight fully connected network or a small convolutional module (i.e., an offset predictor). The predictor dynamically predicts a set of spatial offsets based on the input mask quality score. For instance, if the associated mask quality score is high (indicating that the predicted mask is clear and complete, and the target may not be severely occluded), the predictor may output a smaller offset, causing the deformable convolution's sampling points to be more concentrated in the target's central region, thus capturing more refined discriminative features. Conversely, if the mask quality score is low (indicating that the predicted mask is blurry or incomplete, and the target may be occluded), the predictor may output a set of offsets with a larger range and more diverse directions, guiding the deformable convolution's sampling points to expand to the area surrounding the target, thereby capturing more contextual information to infer the occluded portion.

[0086] This embodiment achieves adaptive adjustment of the feature sampling strategy by dynamically modulating the sampling offset of deformable convolution using mask quality. This mechanism enables the model to "intelligently" decide whether to focus on the details of the target subject or expand the receptive field to seek contextual assistance when facing targets with different degrees of occlusion, thereby significantly enhancing the robustness of feature extraction and the accuracy of target recognition in complex occlusion environments.

[0087] S206. Use the adjusted deformable convolution to sample features from each fused feature map to obtain a sampled feature map. Based on the sampled feature map, identify the target object's wearing state and position. Obtain the initial bounding box coordinates of the target object, the confidence level corresponding to the initial bounding box coordinates, and the initial target object wearing state category at the spatial resolution corresponding to the fused feature map. The target object wearing state category includes the worn state and the unworn state.

[0088] In this embodiment, for each scale of the fused feature map, a deformable convolutional kernel with dynamically adjusted spatial offsets (step S205) is used to resample the original fused feature map. This process allows the convolutional operation to extract features from the most relevant spatial locations based on the estimated occlusion of each specific target, thus obtaining a "sampled feature map" that better represents the characteristics of the current target. Subsequently, this embodiment inputs the sampled feature map into the subsequent classification and regression sub-networks. The classification sub-network (e.g., a fully connected layer or a convolutional layer) outputs the probability that each target location belongs to either "wearing a helmet" or "not wearing a helmet," taking the higher probability as the initial target's wearing status category, and using the probability value corresponding to that category as the confidence level. The bounding box regression sub-network (which can be a fully connected layer or a convolutional layer) outputs the fine-tuning amount (e.g., center point offset, width and height scaling) of each target location relative to a preset anchor box, thereby decoding the initial bounding box coordinates of the target at the current location scale.

[0089] This embodiment combines adaptive feature sampling with target state recognition, ensuring that the features used for classification and regression are specifically optimized. This enables the model to make more accurate wear state judgments and more precise bounding box localizations based on effective contextual information, even when the target is partially occluded, thus improving the accuracy of the detection results.

[0090] S207. Based on the initial bounding box coordinates and confidence scores of the target object obtained from all fused feature maps, non-maximum suppression processing is performed on the multiple initial bounding box coordinates of the target object to obtain the bounding box coordinates of the target object in the image to be processed. The initial target object wearing state category associated with the bounding box coordinates is determined as the target object wearing state, and the initial instance segmentation mask associated with the bounding box coordinates is determined as the instance segmentation mask.

[0091] For example, after completing parallel detection on all N scale fused feature maps, this embodiment will collect a large number of overlapping initial detection results (each result includes bounding box, confidence, category and associated mask).

[0092] To obtain a concise and accurate final output, this embodiment employs a non-maximum suppression algorithm. First, all initial detection boxes are sorted from highest to lowest confidence level. Then, the box with the highest confidence level is selected as the "retained box," and its intersection-union (IU) ratio with all remaining boxes is calculated. Boxes with an IU ratio exceeding a preset threshold (considered as duplicate detections of the same target) are suppressed (removed). This process is iterated until all boxes have been processed. The bounding box coordinates retained after non-maximum suppression represent the helmet location information for each target object (worker). The initial target wearing status category ("worn" or "not worn") associated with these final bounding boxes at the corresponding scale and location is determined as the final wearing status of the target. Simultaneously, the initial instance segmentation mask generated by the mask branch associated with these final bounding boxes is determined as the final instance segmentation mask for the target. Thus, for each target object in the image to be processed, this embodiment outputs its unique bounding box coordinates, wearing status, and pixel-level segmentation mask.

[0093] This embodiment effectively eliminates redundant bounding boxes caused by multi-scale detection and overlapping prediction by fusing detection results across scales and performing non-maximum suppression, ensuring that each target is represented by only one optimal detection result. This not only makes the output clear and unambiguous, but also further improves the overall accuracy and reliability of the detection results by selecting the result with the highest confidence as the final output.

[0094] S208. Based on the target object's wearing status, location information, and instance segmentation mask in the detection results, determine whether the target object is wearing a target object.

[0095] In one feasible implementation, if the target object's corresponding target object wearing state category is unwearing state, and the confidence level corresponding to the bounding box coordinates of the target object is greater than or equal to a preset first threshold, and the effective pixel area of ​​the instance segmentation mask corresponding to the target object is greater than or equal to a preset second threshold, then it is determined that the target object is not wearing a target object; if the target object's corresponding target object wearing state category is worn state, then it is determined that the target object is wearing a target object.

[0096] For example, this embodiment implements a joint decision logic. When the wearing status of the target object is classified as "not wearing" in the detection result, to further eliminate possible false detections (such as misjudging other circular objects as not wearing a helmet), this embodiment adds two verification conditions: 1) The confidence level of the detection box must be higher than a preset first threshold (to ensure the reliability of the detection itself); 2) In the instance segmentation mask associated with the target, the total area of ​​effective pixels belonging to the target object must be greater than or equal to a preset second threshold (to ensure the target has a certain size and eliminate noise or fragmentation false detections). Only when the three conditions of "not wearing", high confidence level and reasonable mask area are met simultaneously, is the target object ultimately determined to be "not wearing" a helmet. If the wearing status is directly classified as "wearing", it is directly determined to be wearing.

[0097] This embodiment introduces a joint decision mechanism based on confidence level and mask area. While utilizing the classification results, it also uses location and segmentation information for secondary verification, which greatly reduces the false alarm rate in dense and complex scenarios and improves the reliability of the security monitoring system.

[0098] In one feasible implementation, the image detection method for the above-mentioned pumped storage power station further includes: Obtain relevant information about the target object that is not wearing a target object; the relevant information includes one or more of the following: the target object's location information, timestamp, corresponding instance segmentation mask, and mask quality; output the relevant information to a preset platform; the preset platform is used to display the number, frequency, or distribution of events without wearing a target object under multiple preset dimensions; the multiple preset dimensions include time dimension and region dimension.

[0099] For example, in this embodiment, after determining that a valid "not wearing" safety helmet event has occurred, an alarm and reporting process is automatically triggered. The computational inference terminal drives the local audible and visual alarm to issue a real-time alarm. Simultaneously, the terminal encapsulates relevant information about the event, including but not limited to: the location (boundary box coordinates) of the unwearing target in the image, the timestamp of the event, the corresponding instance segmentation mask image, and the mask quality score. Subsequently, this structured information is uploaded to the cloud management platform via the network. The cloud platform provides a visual interface, supporting safety managers to query, statistically analyze, and trend-analyze historical alarm records by multiple dimensions, such as cavern passages (area dimension), time periods (time dimension), and even the severity of occlusion combined with mask quality analysis.

[0100] This embodiment achieves a closed loop from local real-time alarms to centralized cloud-based data management. It not only provides immediate on-site alerts but also offers precise data support for power plant safety management decisions (such as strengthening regional patrols and enhancing supervision during specific time periods) through information reporting and platform statistical analysis, thus realizing the digitalization and intelligentization of safety management.

[0101] In one feasible implementation, the image detection method for pumped storage power stations provided in this application is implemented using a pre-trained model, which is trained by the following steps: Repeat the following steps until the adjusted model meets the preset conditions and you obtain a trained model: Input the training image into the initial model to obtain the predicted wearing state category of each target object, the predicted bounding box coordinates of each target object, and the predicted instance segmentation mask of each target object in the training image output by the initial model.

[0102] Determine the classification loss between the predicted wearing state category and the actual wearing state category for each target object in the training image.

[0103] Determine the aspect ratio difference between the predicted bounding box and the true bounding box for each target object, the distance between their center points, and the intersection-union ratio (IU) between the predicted bounding box and the true bounding box for each target object. Based on the aspect ratio difference between the predicted bounding box and the true bounding box and the distance between their center points, determine the loss weighting coefficient for each target object. Based on the corresponding loss weighting coefficient and IU, determine the bounding box regression loss for each target object.

[0104] Determine the segmentation loss between the predicted instance segmentation mask and the true segmentation mask for each target object.

[0105] Calculate the total loss of the initial model with respect to the training images based on the classification loss, bounding box regression loss, and segmentation loss for each target object.

[0106] Based on the total loss, adjust the parameters of the initial model to obtain the adjusted model.

[0107] For example, the loss function used to train the detection model in this embodiment is specifically designed. Cross-entropy loss is used for classification tasks. Dice loss is used for instance segmentation tasks. The key improvement lies in the bounding box regression loss; this embodiment employs the Wise-IoUv3 loss function. This function first calculates the intersection-union ratio (IoU) between the predicted and ground truth boxes. Simultaneously, it dynamically analyzes the aspect ratio difference between each predicted and ground truth box and the Euclidean distance between their center points. Based on these geometric differences, a dynamic loss weighting coefficient is calculated: for "hard samples" with abnormal aspect ratios or large center point deviations (e.g., a helmet with significant deformation at a top-down view), this coefficient increases, causing the model to pay more attention to these difficult-to-regress samples during training; for "normal samples" with geometric characteristics similar to the ground truth boxes, the coefficient is smaller. Ultimately, the bounding box regression loss is determined by this dynamic weighting coefficient and the IoU calculation term. The model's total loss is a weighted sum of the classification loss, the Wise-IoUv3 bounding box regression loss, and the Dice segmentation loss. The model parameters are adjusted iteratively through backpropagation and optimizer until the model's performance converges on the validation set.

[0108] This embodiment employs the Wise-IoUv3 dynamic bounding box regression loss, enabling the model training process to adaptively focus on samples that are difficult to locate (such as safety helmets deformed from a specific viewpoint). This promotes the rapid and stable convergence of bounding box regression in complex scenes, improving the localization accuracy of targets with unconventional angles and scales, without the need for cumbersome hyperparameter adjustments.

[0109] In one example, when the detection model needs to be upgraded to adapt to a new scenario or optimize performance, this embodiment securely distributes a differential update package (containing only the newly added or modified model weights and structure configuration files) from the cloud management platform to each edge terminal. After receiving the update package, the terminal can complete the incremental fusion and loading of the model in the background. The entire process does not require stopping the normal detection service, thus achieving business continuity and agility in model iteration.

[0110] Figure 3 This is a flowchart illustrating another image detection method for a pumped storage power station provided in an embodiment of this application, as shown below. Figure 3 As shown, in this embodiment, firstly, an initial image set is constructed based on historical monitoring videos through frame-by-frame processing. The image set undergoes data preprocessing, including data annotation, image filtering, and image enhancement, to build a sample dataset for model training. Simultaneously, an inference model is constructed, including a feature extraction module and an LADH detection head, and configured with the Wise-IoUv3 loss function for training. Subsequently, the inference model is trained using the sample dataset to obtain the trained helmet-wearing detection model.

[0111] After training is complete, this embodiment inputs the real-time collected monitoring video stream into the trained helmet-wearing detection model for real-time inference and detection. Finally, the model's output of the helmet-wearing status detection result is displayed in real time, thereby achieving continuous and automatic monitoring of the safety compliance status of workers. This method optimizes model performance through offline training and achieves efficient real-time detection through online deployment, forming a complete technical closed loop.

[0112] Figure 4 This is a schematic diagram of the structure of an image detection device for a pumped storage power station provided in an embodiment of this application. Figure 4 As shown, the image detection device of the pumped storage power station includes an acquisition unit 401, a first processing unit 402, a second processing unit 403, a third processing unit 404, and a determination unit 405.

[0113] The acquisition unit 401 is used to acquire the image to be processed; the image to be processed is the image frame data of the working area of ​​the pumped storage power station, and the image frame data contains the target object to be detected.

[0114] The first processing unit 402 is used to perform feature extraction processing on the image to be processed, and obtain feature maps of multiple levels with different spatial resolutions.

[0115] The second processing unit 403 is used to perform feature fusion processing on feature maps of different levels according to the spatial resolution differences and hierarchical relationships between feature maps of multiple levels, and generate multiple fused feature maps.

[0116] The third processing unit 404 is used to detect whether the target object in the image to be processed is wearing a target object based on multiple fused feature maps, and obtain the detection result. The detection result includes the target object wearing status corresponding to the target object, the target object's location information, and the target object's instance segmentation mask.

[0117] The determination unit 405 is used to determine whether the target object is wearing a target object based on the target object wearing status, location information and instance segmentation mask corresponding to the target object in the detection result.

[0118] In some embodiments, the feature maps of multiple levels include a k-th level feature map, where k is an integer from 1 to N, and N is a positive integer greater than 1; the first processing unit 402 is specifically used for: The (k-1)th level feature map is subjected to the first convolution process to obtain the first intermediate feature map. Based on the channel dimension, the first intermediate feature map is divided to obtain multiple sub-feature maps with different channel dimensions. The convolution kernel of the first convolution process is 1×1. The second convolution process is performed on each sub-feature map, and the sub-feature maps processed by the second convolution are merged along the channel dimension to obtain the second intermediate feature map; the convolution kernel for the second convolution process is 3×3; The k-1 level feature map is divided into a first feature sub-map and a second feature sub-map in the channel dimension according to a preset ratio. The first feature sub-map and the second intermediate feature map are then added element by element to obtain the third intermediate feature map. The third intermediate feature map is subjected to nonlinear activation processing to obtain the k-th level feature map; When k is 1, the (k-1)th level feature map is the image to be processed; the spatial resolution decreases from the first level feature map to the kth level feature map.

[0119] In some embodiments, the plurality of fused feature maps includes the k-th fused feature map; k is an integer from 1 to N, and N is a positive integer greater than 1; the second processing unit 403 is specifically used for: Based on the spatial resolution differences and hierarchical relationships between feature maps of multiple levels, feature fusion processing is performed on feature maps of different levels to generate multiple fused feature maps, including: When k is N, feature extraction is performed on the first-level feature map to obtain the Nth fused feature map.

[0120] When k is less than N, the reference feature map is upsampled to obtain an upsampled feature map; the upsampled feature map is concatenated with the (Nk)th level feature map to obtain a concatenated feature map; the concatenated feature map is subjected to feature extraction to obtain the kth fused feature map; where, when k is 1, the reference feature map is the Nth level feature map; when k is greater than 1 and less than N, the reference feature map is the (k-1)th fused feature map.

[0121] In some embodiments, the location information is bounding box coordinates; the third processing unit 404 is specifically used for: Each fused feature map is convolved to generate an initial instance segmentation mask for the target object, and the mask quality for each instance segmentation mask is determined.

[0122] Spatial offset of deformable convolution during feature sampling is adjusted based on mask quality.

[0123] The adjusted deformable convolution is used to sample features from each fused feature map to obtain a sampled feature map. Based on the sampled feature map, the wearing state and position of the target object are identified. The initial bounding box coordinates of the target object, the confidence level of the initial bounding box coordinates, and the initial wearing state category of the target object are obtained at the spatial resolution corresponding to the fused feature map. The wearing state category of the target object includes the wearing state and the unwearing state.

[0124] Based on the initial bounding box coordinates and confidence scores of the target object obtained from all fused feature maps, non-maximum suppression is performed on multiple initial bounding box coordinates of the target object to obtain the bounding box coordinates of the target object in the image to be processed. The initial target object wearing state category associated with the bounding box coordinates is determined as the target object wearing state, and the initial instance segmentation mask associated with the bounding box coordinates is determined as the instance segmentation mask.

[0125] In some embodiments, the determination unit 405 is specifically used to: if the target object's corresponding target object wearing state category is unwearing state, and the confidence level corresponding to the bounding box coordinates of the target object is greater than or equal to a preset first threshold, and the effective pixel area of ​​the instance segmentation mask corresponding to the target object is greater than or equal to a preset second threshold, then determine that the target object is not wearing a target object.

[0126] If the target object's corresponding target item wearing status category is "wearing", then it is determined that the target object has worn the target item.

[0127] In some embodiments, the apparatus further includes an output unit for: Obtain relevant information about the target object that is not wearing a target object; the relevant information includes one or more of the following: the target object's location information, timestamp, corresponding instance segmentation mask, and mask quality.

[0128] The relevant information is output to the preset platform; the preset platform is used to display the number, frequency or distribution of events where no target object is worn under multiple preset dimensions; the multiple preset dimensions include time dimension and region dimension.

[0129] In some embodiments, the image detection method for pumped storage power stations is implemented using a pre-trained model, which is trained by the following steps: The device further includes a training unit for: Repeat the following steps until the adjusted model meets the preset conditions and you obtain a trained model: Input the training image into the initial model to obtain the predicted wearing state category of each target object, the predicted bounding box coordinates of each target object, and the predicted instance segmentation mask of each target object in the training image output by the initial model.

[0130] Determine the classification loss between the predicted wearing state category and the actual wearing state category for each target object in the training image.

[0131] Determine the aspect ratio difference between the predicted bounding box and the true bounding box for each target object, the distance between their center points, and the intersection-union ratio (IU) between the predicted bounding box and the true bounding box for each target object. Based on the aspect ratio difference between the predicted bounding box and the true bounding box and the distance between their center points, determine the loss weighting coefficient for each target object. Based on the corresponding loss weighting coefficient and IU, determine the bounding box regression loss for each target object.

[0132] Determine the segmentation loss between the predicted instance segmentation mask and the true segmentation mask for each target object.

[0133] Calculate the total loss of the initial model with respect to the training images based on the classification loss, bounding box regression loss, and segmentation loss for each target object.

[0134] Based on the total loss, adjust the parameters of the initial model to obtain the adjusted model.

[0135] In some embodiments, the obtaining unit 401 is specifically used for: Video stream data is acquired using an image acquisition device deployed on the top of the underground cavern of a pumped storage power station.

[0136] Each image frame in the video stream is subjected to illumination equalization and contrast enhancement processing to obtain the image to be processed.

[0137] The image detection device for pumped storage power stations provided in this application embodiment can execute the method shown in the above method embodiment. Its implementation principle and beneficial effects can be referred to the relevant description in the method embodiment, and will not be repeated here.

[0138] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 5 As shown, the electronic device includes: a memory 501, a transceiver 502, and at least one processor 503.

[0139] Transceiver 502 is used to interact with other devices to send and receive data.

[0140] The memory 501 is used to store computer program code, which includes computer instructions. These computer instructions run in the aforementioned electronic device to implement the method shown in the above-described method embodiments. For example, the memory may include high-speed random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk storage device, or a USB flash drive, portable hard drive, read-only memory, magnetic disk, or optical disk, etc.

[0141] Processor 503 can be a general-purpose processor, including a Central Processing Unit (CPU), a network processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. Processor 503 can also be other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor.

[0142] The memory 501, transceiver 502, and processor 503 are communicatively connected. For example, the memory 501 and transceiver 502 can be connected to the processor 503 via a system bus and communicate with each other. The system bus can be a peripheral component interconnect (PCI) bus, an extended industry standard architecture (EISA) bus, an industry standard architecture (ISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the figure, but this does not mean that there is only one bus or one type of bus.

[0143] Optionally, the memory 501 can be either standalone or integrated with the processor 503. When the memory 501 is set up independently, it is connected to the processor 503 via a system bus.

[0144] This application also provides a chip for executing instructions, which is used to execute the technical solution of the image detection method for pumped storage power stations described in the above embodiments.

[0145] This application also provides a computer-readable storage medium storing computer instructions. When these computer instructions are executed by a processor, they are used to implement the technical solution of the image detection method for pumped-storage power stations described in the above embodiments. Specifically, when the computer instructions are executed by a processor, the electronic device can execute the technical solution of the image detection method for pumped-storage power stations described in the above embodiments.

[0146] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the image detection method for pumped storage power stations described in the above embodiments.

[0147] The aforementioned computer-readable storage media can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0148] An exemplary computer-readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the computer-readable storage medium can also be a component of the processor. The processor and the computer-readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the computer-readable storage medium can exist as discrete components in an electronic control unit or main control device; this application does not limit this.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0150] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0151] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0152] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0153] It should be understood that the steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0154] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image detection method for a pumped storage power station, characterized in that, The method includes: Acquire an image to be processed; the image to be processed is image frame data of the working area of ​​a pumped storage power station, and the image frame data contains the target object to be detected. The image to be processed is subjected to feature extraction processing to obtain feature maps of multiple levels with different spatial resolutions; Based on the spatial resolution differences and hierarchical relationships between the feature maps of the multiple levels, feature fusion processing is performed on the feature maps of different levels to generate multiple fused feature maps. Based on the multiple fused feature maps, the detection process is performed to determine whether the target object in the image to be processed is wearing a target object, and the detection result is obtained. The detection result includes the target object wearing status corresponding to the target object, the location information of the target object, and the instance segmentation mask of the target object. Based on the target object's wearing status, location information, and instance segmentation mask in the detection results, it is determined whether the target object is wearing the target object.

2. The image detection method for a pumped storage power station according to claim 1, characterized in that, The multiple layers of feature maps include a k-th layer feature map, where k is an integer from 1 to N, and N is a positive integer greater than 1; the feature extraction process on the image to be processed to obtain multiple layers of feature maps with different spatial resolutions includes: The (k-1)th level feature map is subjected to a first convolutional process to obtain a first intermediate feature map. Based on the channel dimension, the first intermediate feature map is divided to obtain multiple sub-feature maps with different channel dimensions. The convolutional kernel of the first convolutional process is 1×1. The second convolution process is performed on each of the sub-feature maps, and the sub-feature maps that have undergone the second convolution process are merged along the channel dimension to obtain the second intermediate feature map; the convolution kernel of the second convolution process is 3×3; The (k-1)th level feature map is divided into a first feature sub-map and a second feature sub-map in the channel dimension according to a preset ratio. The first feature sub-map and the second intermediate feature map are then added element by element to obtain a third intermediate feature map. The third intermediate feature map is subjected to nonlinear activation processing to obtain the k-th level feature map; Where k is 1, the (k-1)th level feature map is the image to be processed; from the first level feature map to the kth level feature map, the corresponding spatial resolution decreases.

3. The image detection method for pumped storage power stations according to claim 2, characterized in that, The plurality of fused feature maps include the k-th fused feature map, where k is an integer from 1 to N, and N is a positive integer greater than 1; The step involves performing feature fusion processing on feature maps of different levels based on the spatial resolution differences and hierarchical relationships between the multiple levels of feature maps, generating multiple fused feature maps, including: When k is N, feature extraction processing is performed on the first-level feature map to obtain the Nth fused feature map; When k is less than N, the reference feature map is upsampled to obtain an upsampled feature map; the upsampled feature map is concatenated with the (Nk)th level feature map to obtain a concatenated feature map; the concatenated feature map is subjected to feature extraction to obtain the kth fused feature map; wherein, when k is 1, the reference feature map is the Nth level feature map; when k is greater than 1 and less than N, the reference feature map is the (k-1)th fused feature map.

4. The image detection method for a pumped storage power station according to claim 1, characterized in that, The location information is the bounding box coordinates; the detection process based on the multiple fused feature maps to determine whether the target object in the image to be processed is wearing a target object, and the resulting detection result includes: Each of the fused feature maps is convolved to generate an initial instance segmentation mask for the target object, and the mask quality corresponding to each instance segmentation mask is determined. The spatial offset of the deformable convolution during feature sampling is adjusted based on the mask quality. The adjusted deformable convolution is used to sample features from each fused feature map to obtain a sampled feature map. Based on the sampled feature map, the wearing state and position of the target object are identified, and the initial bounding box coordinates, the confidence level corresponding to the initial bounding box coordinates, and the initial wearing state category of the target object are obtained at the spatial resolution corresponding to the fused feature map. The wearing state category of the target object includes a worn state and an unworn state. Based on the initial bounding box coordinates and confidence scores of the target object obtained from all fused feature maps, non-maximum suppression processing is performed on multiple initial bounding box coordinates of the target object to obtain the bounding box coordinates of the target object in the image to be processed. The initial target object wearing state category associated with the bounding box coordinates is determined as the target object wearing state, and the initial instance segmentation mask associated with the bounding box coordinates is determined as the instance segmentation mask.

5. The image detection method for a pumped storage power station according to claim 4, characterized in that, The step of determining whether the target object is wearing the target object based on the target object wearing status, bounding box coordinates, and instance segmentation mask corresponding to the target object in the detection results includes: If the target object's corresponding target item wearing state category is not wearing state, and the confidence level corresponding to the bounding box coordinates of the target item is greater than or equal to a preset first threshold, and the effective pixel area of ​​the instance segmentation mask corresponding to the target item is greater than or equal to a preset second threshold, then it is determined that the target object is not wearing the target item. If the target object's corresponding target item wearing status category is "worn", then it is determined that the target object has worn the target item.

6. The image detection method for a pumped storage power station according to claim 5, characterized in that, The method further includes: Obtain relevant information about the target object that is not wearing the target object; the relevant information includes one or more of the following: the target object's location information, timestamp, corresponding instance segmentation mask, and mask quality; The relevant information is output to a preset platform; the preset platform is used to display the number, frequency or distribution of events where no target object is worn under multiple preset dimensions; the multiple preset dimensions include time dimension and region dimension.

7. The image detection method for a pumped storage power station according to claim 1, characterized in that, The image detection method for the pumped storage power station is implemented using a pre-trained model, which is trained through the following steps: Repeat the following steps until the adjusted model meets the preset conditions and you obtain a trained model: The training image is input into the initial model to obtain the predicted wearing state category of each target object in the training image, the predicted bounding box coordinates of each target object, and the predicted instance segmentation mask of each target object. Determine the classification loss between the predicted wearing state category and the actual wearing state category for each target object in the training image; The aspect ratio difference between the predicted bounding box and the true bounding box and the distance between their center points are determined for each target object, as well as the intersection-union ratio (IU) between the predicted bounding box and the true bounding box for each target object. Based on the aspect ratio difference between the predicted bounding box and the true bounding box and the distance between their center points, the loss weighting coefficient for each target object is determined. Based on the corresponding loss weighting coefficient and the IU, the bounding box regression loss for each target object is determined. Determine the segmentation loss between the predicted instance segmentation mask and the actual segmentation mask for each target object; The total loss of the initial model with respect to the training image is calculated based on the classification loss, the bounding box regression loss, and the segmentation loss corresponding to each of the target objects. Based on the total loss, the parameters of the initial model are adjusted to obtain the adjusted model.

8. The image detection method for a pumped storage power station according to claim 1, characterized in that, The process of acquiring the image to be processed includes: Video stream data is acquired using an image acquisition terminal deployed on the top of the underground cavern of a pumped storage power station; Each image frame in the video stream data is subjected to illumination equalization and contrast enhancement processing to obtain the image to be processed.

9. An image detection device for a pumped storage power station, characterized in that, The device includes: An acquisition unit is used to acquire an image to be processed; the image to be processed is image frame data of the working area of ​​a pumped storage power station, and the image frame data contains a target object to be detected. The first processing unit is used to perform feature extraction processing on the image to be processed to obtain feature maps of multiple levels with different spatial resolutions; The second processing unit is used to perform feature fusion processing on feature maps of different levels according to the spatial resolution differences and hierarchical relationships between the feature maps of the multiple levels, and generate multiple fused feature maps. The third processing unit is used to detect whether the target object in the image to be processed is wearing a target object based on the multiple fused feature maps, and to obtain a detection result. The detection result includes the target object wearing status corresponding to the target object, the location information of the target object, and the instance segmentation mask of the target object. The determination unit is used to determine whether the target object is wearing the target object based on the target object wearing status, location information and instance segmentation mask corresponding to the target object in the detection result.

10. An electronic device, characterized in that, include: A transceiver, a memory, and at least one processor; the memory is communicatively connected to the processor; the memory is used to store computer program code, the computer program code including computer instructions; when the processor executes the computer instructions, the electronic device causes the electronic device to perform the image detection method for a pumped storage power station as described in any one of claims 1-8.

Images