Hydropower station floater detection method based on global relation modeling

Abstract

The invention provides a hydropower station floating object detection method based on global relation modeling, and aims to improve the detection precision and efficiency of hydropower station water surface floating objects, and the method comprises the steps: collecting hydropower station water surface scene data through a multi-view device; extracting shallow multi-scale features through a lightweight convolutional network; modeling a global relationship between the floaters through a multi-head self-attention mechanism; carrying out the classification and positioning of floating objects through a target detection head; network training is performed through a multi-task loss function to optimize the model; and reasoning by using the trained floating object detection network, inputting a to-be-detected water surface image into the network, outputting the category and the position of the floating object, and finally generating a detection result. According to the method, the global relation modeling method is introduced, the detection precision of the hydropower station water surface floating objects is effectively improved, detection errors caused by insufficient local features in a traditional method are reduced, and the method has low calculation complexity and is suitable for real-time detection in practical application.

Description

Technical Field

[0001] The present invention relates to the technical field of hydropower station environmental monitoring, and in particular to a method for detecting floating objects in a hydropower station based on global relationship modeling, which aims to improve the detection accuracy and system adaptability of floating objects in a complex environment. Background Art

[0002] As an important clean energy production facility, hydropower stations occupy an increasingly important position in the global energy structure. Their operating efficiency and safety play a vital role in the stability of power supply. However, in the daily operation of hydropower stations, floating objects on the water surface (such as branches, plastic waste, industrial waste, etc.) have become a problem that cannot be ignored, bringing many negative impacts to the operation of hydropower stations. For example, floating objects gathered at the water inlet and trash rack of the hydropower station may cause the water inlet to be blocked or the water flow to be reduced, which will not only reduce the power generation efficiency of the hydropower station, but also increase the burden of the subsequent cleaning work, resulting in increased downtime, and thus affecting the power supply; floating objects enter the key equipment area of ​​the hydropower station with the water flow, which may cause wear, corrosion or blockage of the equipment, resulting in unstable equipment operation and even equipment failure, which will not only increase maintenance costs, but also cause equipment downtime and production interruption, affecting the stable supply of electricity; in addition, the long-term accumulation of floating objects on the water surface will affect the water quality and pollute the water environment, especially plastic waste and other wastes that are not easily degradable. To address these issues, hydropower stations need to monitor floating objects in real time and accurately, remove floating objects from water inlets and key equipment areas in a timely manner, and ensure the health of the water environment and the safe operation of the hydropower station.

[0003] At present, the monitoring methods of floating objects in hydropower stations mainly rely on manual inspections and traditional monitoring equipment. Manual inspections are not only labor-intensive, but also easily limited by the scope of the monitoring area and the reaction speed of the staff, and there are monitoring blind spots. In addition, although traditional monitoring equipment can provide video monitoring, it often lacks real-time and efficient processing capabilities, and cannot quickly and accurately identify and locate floating objects. With the rapid development of deep learning technology, object detection has been widely used in the automated analysis of images and videos, especially in methods such as YOLO and Faster R-CNN, deep learning has made significant breakthroughs in object recognition and classification. Although deep learning has achieved good results in many object detection tasks, there are still some problems in the monitoring of floating objects in hydropower stations. For example, in the process of feature extraction, traditional convolutional neural networks (CNNs) usually focus on feature capture in local areas, and cannot fully explore the global semantic relationship between floating objects, especially in the case of floating objects clustering or complex backgrounds. In addition, deep learning models, especially complex models such as Faster R-CNN, usually require a lot of computing resources and have slow inference speeds, which makes it difficult to meet the needs of real-time monitoring. Monitoring floating objects in hydropower stations requires fast, real-time responses, but traditional deep learning methods may not run efficiently on resource-limited devices due to excessive computational overhead, especially on edge computing or low-power devices. At the same time, the types and sizes of floating objects on the water surface vary greatly, especially small floating objects (such as small plastic pieces, paper scraps, etc.) are difficult to accurately identify in images. Existing deep learning models generally have accuracy problems in small target detection, especially in low-resolution images or long-distance monitoring, the detection performance of small floating objects is poor.

[0004] CN118537291A discloses an intelligent detection and early warning method for floating objects on the water surface of a hydropower station dam reservoir area. The method trains an intelligent detection model through a water surface floating object data set, deploys the trained model to a server, and builds a corresponding detection system. First, the video of floating objects on the water surface in the hydropower station dam area is made into a VOC data set, and the data set is enhanced to a certain extent. Then, the model is trained according to a certain strategy, and then the algorithm model is combined with the constructed detection system. Finally, the complete water surface floating object detection system is deployed to the server. The user can obtain the corresponding video information after detection by inputting the original video to be detected through the system platform; it realizes the automatic detection of common floating objects in the water area under complex environmental conditions. It also has the defects of the prior art. Summary of the invention

[0005] The present invention proposes a method for detecting floating objects in a hydropower station based on global relationship modeling. Aiming at the deficiencies of the prior art, the following innovative measures are taken:

[0006] 1) In the detection of floating objects in hydropower stations, the sizes of floating objects vary greatly. Using a single-scale feature map cannot effectively detect floating objects with large scale differences. Therefore, the problem can be effectively solved by introducing a multi-scale feature map. In addition, considering the increase in computational complexity caused by the introduction of multi-scale features, the lightweight neural network MobileNet is used to meet the demand for inference speed;

[0007] 2) Floating objects in a hydropower station environment may have complex spatial and semantic relationships. Traditional convolutional networks are difficult to effectively capture the global relationship between floating objects, which affects detection accuracy. Therefore, a sparse self-attention mechanism is introduced to model the spatial and semantic relationships between floating objects. Compared with the traditional self-attention mechanism, the sparse processing only focuses on key areas (such as the edge area of ​​floating objects), thereby greatly reducing the computational complexity and improving detection efficiency.

[0008] 3) The types of floating objects in the hydropower station are unevenly distributed, with more samples of common categories (such as branches) and fewer samples of minority categories (such as small debris). The traditional cross entropy loss function cannot effectively handle this category imbalance, resulting in overfitting of the model to the majority class. Focal Loss is used to solve the category imbalance problem. Focal Loss introduces a focusing factor and category weights , reducing the loss contribution of easy-to-classify samples, so that the model pays more attention to difficult-to-classify samples (i.e., minority class floaters) during training.

[0009] The present invention provides a method for detecting floating objects in a hydropower station based on global relationship modeling, and the specific steps are as follows:

[0010] S1: Obtain training data set. Collect water surface scene data of hydropower stations through multi-view equipment, covering various types of floating objects (such as branches, plastic bags) and different environmental conditions (such as lighting changes, water surface reflections). Label the categories of floating objects and bounding box . Apply data enhancement strategies: including lighting simulation, dynamic floating object trajectory generation, and water surface reflection texture overlay to build a diverse training data set;

[0011] S2: The floating object detection network performs shallow feature extraction. The input image is sent to the lightweight convolutional network MobileNet, and multi-scale image features are extracted through its efficient depth-separable convolutional structure. This step can effectively capture the local texture information of floating objects while significantly reducing the computational complexity;

[0012] S3: Deep feature extraction and global relationship modeling of floating object detection network. Deep feature extraction is performed on shallow features, and the global relationship between floating objects is captured by introducing a self-attention mechanism. The following steps are included: Use a multi-head self-attention module to model the floating object features in different regions; Fuse global features with shallow features to improve the expression ability of the detection network;

[0013] S4: Object detection output. The fused features are sent to the object detection head to classify and locate floating objects and optimize the classification confidence and bounding box regression accuracy in combination with the loss function;

[0014] S5: Train the floating object detection network based on the training data set. Use a multi-task loss function for training, which consists of classification loss and bounding box regression loss;

[0015] S6: Use the trained floating object detection network for inference. The water surface image to be detected is input into the trained network, and the network outputs the category and location of the floating object to generate the detection result.

[0016] Improvement of the above technical solution: The shallow feature extraction of the floating object detection network in step S2 includes the following three aspects:

[0017] S21: Using the MobileNet network based on depthwise separable convolution, the traditional convolution operation is decomposed into two stages: channel-by-channel convolution and point-by-point convolution. Compared with traditional convolution, depthwise separable convolution decomposes the convolution operation into two steps, which can reduce a lot of calculations. Traditional convolution needs to calculate the combination of all input channels and convolution kernels, while depthwise separable convolution only applies independent convolution kernels on each input channel, and then fuses them through 1×1 convolution.

[0018] S22: Introducing multi-scale features, for real-world scenarios with floating objects of varying sizes, the ability to extract features is enhanced by fusing features of different scales.

[0019] S23: The results are obtained by weighted summation of features of different scales, which enriches the feature extraction capability of the network and enables the network to emphasize the ability to extract important features.

[0020] Improvement of the above technical solution: The deep feature extraction and global relationship modeling of the floating object detection network in step S3 includes the following three aspects:

[0021] S31: Further refine the extracted shallow network features and use deep convolutional neural networks to extract image features with higher semantics. These features contain more content and context information, which is helpful for subsequent detection.

[0022] S32: Use the self-attention mechanism to model global relationships. The self-attention mechanism can model global relationships and obtain the relationships between different floating objects. These relationships can help us get better detection performance. Further considering the computational complexity, the sparse self-attention mechanism is used for calculation.

[0023] S33: The deep features and global relations are passed through two weight matrices to obtain fused weighted features.

[0024] Beneficial effects of the present invention:

[0025] 1. By introducing global relationship modeling and sparse self-attention mechanism, the present invention can effectively capture the spatial and semantic relationships between floating objects, significantly improving the accuracy of floating object detection in the hydropower station environment. Especially in complex scenes (such as water surface reflection, dynamic water flow, etc.), it can maintain high detection accuracy and reduce missed detection and false detection.

[0026] 2. The lightweight convolutional network MobileNet is used for feature extraction, and the sparse self-attention mechanism is combined to reduce the computational complexity of global relationship modeling, which greatly reduces the computational complexity of the model. Compared with the traditional target detection model, the present invention significantly reduces the demand for computing resources, making the method more suitable for real-time monitoring scenarios of hydropower stations, especially when deployed on edge devices, it can meet the needs of real-time processing.

[0027] 3. Effectively reduce the loss contribution of easy-to-classify samples, improve the detection ability of difficult-to-classify and minority floating objects (such as small floating objects), ensure the detection performance of the model for all categories, and avoid overfitting of the model for most types of floating objects.

[0028] 4. This method is suitable for the monitoring needs of floating objects of different scales in hydropower stations, and has the potential to be expanded to other water environment scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0030] Figure 1 Schematic diagram of depth-wise separable convolution.

[0031] Figure 2 Schematic diagram of shallow multi-scale feature extraction.

[0032] Figure 3 Schematic diagram for modeling deep features and global relationships.

[0033] Figure 4 Schematic diagram of target detection model training. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] See also Figure 1-4 The present invention provides a technical solution: a method for detecting floating objects in a hydropower station based on global relationship modeling, which is used for detecting floating objects in real-time video, and specifically includes the following steps:

[0036] S1: Obtain training data set. Collect water surface scene data of hydropower stations through multi-view equipment, covering various types of floating objects (such as branches, plastic bags) and different environmental conditions (such as lighting changes, water surface reflections). Label the categories of floating objects and bounding box . Apply data enhancement strategies: including lighting simulation, dynamic floating object trajectory generation, and water surface reflection texture overlay to build a diverse training dataset.

[0037] S2: Shallow feature extraction of floating object detection network. The input image is fed into the lightweight convolutional network MobileNet, which extracts multi-scale image features through its efficient depth-separable convolutional structure. This step can effectively capture the local texture information of floating objects while significantly reducing the computational complexity.

[0038] S3: Deep feature extraction and global relationship modeling of floating object detection network. Deep feature extraction is performed on shallow features, and the global relationship between floating objects is captured by introducing a self-attention mechanism. Specifically, the following steps are included: using a multi-head self-attention module to model the floating object features in different regions; fusing global features with shallow features to improve the expression ability of the detection network.

[0039] S4: The fused features are sent to the target detection head to classify and locate the floating objects to obtain the detection results and optimize the classification confidence and bounding box regression accuracy in combination with the loss function.

[0040] S5: Train the floating object detection network according to the training data set. Use the multi-task loss function for training. Until the loss function converges, the optimized detection model is obtained.

[0041] S6: Use the trained floating object detection network for inference. The water surface image to be detected is input into the trained network, and the network outputs the category and location of the floating object to generate the detection result.

[0042] Further, if Figure 2As shown, the shallow feature extraction of the floating object detection network in step S2 includes the following three aspects:

[0043] S21: In response to the need for edge algorithm deployment in hydropower plants, a lightweight convolutional neural network MobileNet is used. MobileNet is a lightweight convolutional neural network designed specifically for efficient image classification and target detection on devices with limited computing resources. Its main feature is the use of depthwise separable convolution, which greatly reduces the amount of computation and parameters while maintaining high model performance. This is different from the traditional convolution operation that convolves each input channel and then performs channel fusion. Figure 1 As shown in the figure, the depth separable convolution is divided into channel-by-channel convolution DWConV: for each input channel, a convolution operation is performed separately to capture the local features of each channel; point-by-point convolution PWConV: 1×1 convolution is used to fuse the features of multiple channels together. The depth separable convolution greatly reduces the amount of calculation, memory consumption and computational complexity.

[0044] S22: Multi-scale feature extraction generates feature maps of different scales through different convolutional layers. For example, capturing detailed features (such as small floating objects) in lower layers (high resolution) and capturing global information (such as large floating objects) in higher layers (low resolution).

[0045] S23: Use weighted summation to fuse features of each scale, further enriching the network's ability to perceive floating objects of different scales. Multi-scale feature extraction ensures that the network captures information about both large and small targets at the same time, improving the ability to detect various types of floating objects.

[0046] Further, if Figure 3 As shown, the deep feature extraction and global relationship modeling of step S3 include the following three aspects:

[0047] S31: In order to extract higher-level semantic information from shallow feature maps, the network uses multiple deep convolutional layers. Each convolutional layer further abstracts the input features to capture more complex features such as color, shape, texture, etc. of floating objects.

[0048] S32: The spatial relationship and relative position of floating objects are very important. The self-attention mechanism can capture the global relationship between floating objects by calculating the correlation between features. Multiple heads are used to calculate the attention values ​​of different regions in parallel, and each head learns different feature representations. Finally, the outputs of multiple heads are spliced ​​together to enhance the feature expression ability. As shown in Formula 1, Q, K represent the input query and key, V represents the value matrix, is the scaling factor:

[0049] ;

[0050] In addition, considering the high computational complexity in self-attention calculation, a sparse mask matrix M is introduced so that it only calculates the key areas, as shown in Formula 2:

[0051] ;

[0052] S33: Fusion of the global features obtained by the self-attention mechanism with the detail features extracted from the shallow layer. The fusion can be achieved through weighted summation, specifically by learning two weight matrices for summation.

[0053] Further, if Figure 4 As shown, the floating object detection network training in step S5 includes:

[0054] S51: The joint loss function of the classification loss and bounding box regression loss is used for training to optimize the category and position prediction of floating objects. The classification loss uses Focal Loss to solve the category imbalance problem as shown in Formula 3, which increases the attention to difficult-to-classify samples, where N is the number of samples. is the category weight, is the predicted category probability of the i-th sample, is a focusing factor used to increase the loss weight of difficult-to-classify samples.

[0055] ;

[0056] S52: The bounding box regression loss uses Smooth L1 Loss to improve the bounding box positioning accuracy, as shown in Formula 4-5, where is the predicted bounding box of the i-th target, is the ground-truth bounding box of the i-th object.

[0057] ;

[0058] ;

[0059] S53: The model optimizes both the classification loss and the bounding box regression loss. To ensure a balance between the two tasks, the two losses are combined into a total loss function. As shown in Formula 6, λ is a hyperparameter used to balance the weights of the classification loss and the regression loss:

[0060] .

[0061] The present invention is relatively easy to understand for those skilled in the art. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting floating objects in a hydropower station based on global relationship modeling, characterized in that: The following steps are involved: S1: Obtain training data sets, collect water surface scene data of hydropower stations through multi-view equipment, and annotate floating objects. At the same time, apply data enhancement strategies, including illumination simulation, dynamic floating object trajectory generation, and water surface reflection texture overlay, to build a diverse training data set; S2: Use a lightweight convolutional network for shallow feature extraction. Send the input image to the network and extract multi-scale image features through its deep separable convolutional structure, effectively capturing the local texture information of floating objects while significantly reducing the computational complexity. S3: Perform deep feature extraction and global relationship modeling, and capture the global relationship between floating objects by introducing the self-attention mechanism; S4: Send the fused features to the object detection head to classify and locate the floating objects, and optimize the classification confidence and bounding box regression accuracy in combination with the loss function; S5: Train the floating object detection network based on the training dataset, using a multi-task loss function consisting of classification loss and bounding box regression loss; S6: Use the trained floating object detection network for inference, input the water surface image to be detected into the trained network, and the network outputs the category and location of the floating object to generate the detection result.

2. The method for detecting floating objects in a hydropower station based on global relationship modeling according to claim 1 is characterized in that: The data enhancement strategy in step S1 includes: S11: Simulate changes in the illumination in the image to enhance the network’s ability to identify floating objects in different illumination environments; S12: Generate the trajectory of floating objects on the water surface based on physical simulation to increase the model's adaptability to the movement of floating objects; S13: Simulate the effect of water surface reflection so that the network can better handle the visual interference caused by water surface reflection.

3. The method for detecting floating objects in a hydropower station based on global relationship modeling according to claim 2 is characterized in that: The shallow feature extraction in step S2 includes: S21: The input image is fed into the lightweight convolutional network MobileNet to extract shallow features through its deep separable convolutional structure; S22: Introducing multi-scale features, for real-world scenarios with floating objects of varying sizes, the ability to extract features is enhanced by fusing features of different scales.

4. The method for detecting floating objects in a hydropower station based on global relationship modeling according to claim 3 is characterized in that: The deep feature extraction in step S3 includes: S31: Further process the shallow features and capture the global relationship between floating objects by introducing the self-attention mechanism; S32: Fusion of deep features and multi-scale features.

5. The method for detecting floating objects in a hydropower station based on global relationship modeling according to claim 4 is characterized in that: The detection result output in step S4 includes: S41: Send the fused features to the target detection head to classify and locate floating objects; S42: Use the classification detection head to classify the target and obtain the target category; S43: Use the frame detection head to locate the target and obtain the target position.

6. The method for detecting floating objects in a hydropower station based on global relationship modeling according to claim 5 is characterized in that: The loss function update in step S5 includes: S51: Train the floating object detection network according to the training data set. In this training process, a multi-task loss function is used for training; S52: Calculate the classification loss of the prediction and the actual result based on the classification loss; S53: Calculate the box regression loss between the prediction and the true result based on the box regression loss.

7. The method for detecting floating objects in a hydropower station based on global relationship modeling according to claim 6 is characterized in that: The reasoning deployment in step S6 includes: S61: Use the obtained model to perform detection reasoning on the hydropower station floating object picture or video; S62: Deploy the model algorithm to the edge device.

8. The method for detecting floating objects in a hydropower station based on global relationship modeling according to claim 2 is characterized in that: The shallow feature extraction in step S2 includes: S21: In response to the need for edge algorithm deployment in hydropower plants, a lightweight convolutional neural network MobileNet is used. MobileNet is a lightweight convolutional neural network specially designed for efficient image classification and target detection on devices with limited computing resources. Its main feature is the use of deep separable convolution, which reduces the amount of calculation and parameters while maintaining high model performance. Unlike traditional convolution operations that perform convolution on each input channel and then perform channel fusion, deep separable convolution is divided into channel-by-channel convolution DWConV: for each input channel, a convolution operation is performed separately to capture the local features of each channel; point-by-point convolution PWConV: 1×1 convolution is used to fuse the features of multiple channels together; S22: Multi-scale feature extraction generates feature maps of different scales through different convolutional layers, capturing detailed features at higher resolutions and global information at lower resolutions; S23: Use weighted summation to fuse features of each scale to further enrich the network's perception of floating objects of different scales. Multi-scale feature extraction can ensure that the network captures information on both large and small targets at the same time, improving the ability to detect various types of floating objects.

9. The method for detecting floating objects in a hydropower station based on global relationship modeling according to claim 8 is characterized in that: The deep feature extraction in step S3 includes: S31: In order to extract higher-level semantic information from shallow feature maps, the network uses multiple deep convolutional layers. Each convolutional layer further abstracts the input features to capture more complex features such as color, shape, and texture of floating objects. S32: The spatial relationship and relative position of floating objects are very important. The self-attention mechanism captures the global relationship between floating objects by calculating the correlation between features. The attention values ​​of different regions are calculated in parallel by multiple heads. Each head learns different feature representations. Finally, the outputs of multiple heads are spliced ​​together to enhance the feature expression ability; as shown in the following formula, where Q and K represent the input query and key, and V represents the value matrix. is the scaling factor: ； In addition, considering the high computational complexity in self-attention calculation, a sparse mask matrix M is introduced so that it only calculates the key areas, as shown in the following formula: ； S33: The global features obtained by the self-attention mechanism are fused with the detail features extracted in the shallow layer. The fusion is achieved through weighted summation, which is specifically achieved by learning two weight matrices for summation.

10. The method for detecting floating objects in a hydropower station based on global relationship modeling according to claim 9 is characterized in that: The loss function update in step S5 includes: S51: The joint loss function of the classification loss and bounding box regression loss is used for training to optimize the category and position prediction of floating objects. The classification loss uses Focal Loss to solve the category imbalance problem as shown in the following formula, which increases the attention to difficult-to-classify samples, where N is the number of samples. is the category weight, is the predicted category probability of the i-th sample, is the focusing factor, which is used to increase the loss weight of difficult-to-classify samples: ； S52: The bounding box regression loss uses Smooth L1 Loss to improve the bounding box positioning accuracy, as shown in the following two equations, where is the predicted bounding box of the i-th target, is the true bounding box of the i-th target: ； ； S53: The model optimizes both the classification loss and the bounding box regression loss. To ensure a balance between the two tasks, the two losses are combined into a total loss function as shown below, where λ is a hyperparameter used to balance the weights of the classification loss and the regression loss: 。

Images