Hydraulic power plant water-collecting well and pool semi-supervised flooding detection method based on DINO

Abstract

A hydraulic power plant water-collecting well and pool semi-supervised flooding detection method based on DINO constructs a semi-supervised target detection network through the steps of network construction, model construction and function optimization, and can realize high-quality detection of flooding detection tasks without performing complete manual annotation on unannotated images collected by a hydraulic power plant. The learning device is constructed by a target detection network DINO based on Transform, deep information contained in an image can be efficiently extracted by utilizing the strong long-range modeling capability of the learning device, and then object query with strong representation capability is generated. A cross-view-angle object query unification module designed based on a mean square error obtains effective features with an invariable extraction view angle by aligning the features extracted after executing different image enhancement operations on the same image, and enhances the generalization ability of the model to a flood detection task.

Description

Technical Field

[0001] The invention belongs to the technical fields of artificial intelligence and pattern recognition, and relates to a semi-supervised flooding detection method for a water collection well and a water pool in a hydropower plant based on DINO. Background Art

[0002] The purpose of the target detection task is to find all the targets (objects) of interest in the image and determine their categories and locations. It is one of the core issues in the field of computer vision. In the past, target detection models have made great progress and can be roughly divided into anchor-based and anchor-free. The anchor-based method regards the target detection task as a classification and regression problem for some candidate regions. More specifically, these candidate regions are generated by fixed-size anchor frames through sliding windows. The dense candidate regions improve the recall ability of the target and achieve good detection performance. In contrast, the anchor-free method directly predicts the center and four borders of the object, which is simpler and more flexible. In recent years, especially with the introduction of the target detection network DETR (Detection Transformer) based on the Transformer attention mechanism, the anchor-free method has gradually attracted the attention of scholars at home and abroad.

[0003] However, the performance of target detection tasks is often limited by the magnitude of labeled data. At the same time, image annotation is time-consuming and laborious and not easy to obtain. In order to efficiently utilize unlabeled image features, semi-supervised target detection tasks have received widespread attention. The invention with patent publication number CN118552738A proposes a dense pseudo-labeling method for semi-supervised rotation target detection, applying semi-supervised tasks to target detection tasks in aerial images. The patent with patent publication number CN117788999A proposes a semi-supervised target detection method that integrates smoothing constraints and contrastive learning. The contrastive learning method based on consistency constraints learns feature extraction capabilities by aligning the features extracted after performing different image enhancement operations on the same image.

[0004] However, the main part of the above methods is constructed by traditional convolutional neural networks. The performance of the target detection task is related to the amount of labeled data. However, image labeling is time-consuming and laborious, and it is necessary to design a suitable semi-supervised strategy to effectively utilize the large amount of unlabeled water collection wells and pools collected. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a semi-supervised flooding detection method for water collection wells and water pools in hydropower plants based on DINO. For a large amount of unlabeled water collection well and water pool image data, an efficient semi-supervised target detection network is designed to extract effective features, so as to realize flooding detection of water collection wells and water pools in hydropower plants.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a semi-supervised flooding detection method for water collection wells and pools in hydropower plants based on DINO, comprising the following steps: S1, build a network, build a semi-supervised flood detection network based on DINO; S2, build a model, build a semi-supervised data set for flooding detection based on the water collection well and pool scenes of the hydropower plant, and send it to the flooding detection model in S1 for model training; S3, function optimization, build loss function, and optimize model parameters.

[0007] In S1, the semi-supervised flood detection network based on DINO includes two branches: a teacher model and a student model. The model structures of the teacher model and the student model are exactly the same, including the encoder and decoder structures of DINO. The image passes through the teacher or student model and outputs an object query for predicting the information of the annotation box.

[0008] In S1, the teacher and student models adopt different strategies to update parameters during the training phase; the student model adopts a stage mixed matching strategy to match the predicted object with the true value label, and updates the parameters in combination with the loss function; the teacher model parameters are frozen, and the parameters are updated only after a complete round, after performing an exponential sliding average operation on the parameters of the student model.

[0009] In S1, the stage mixed matching strategy refers to using different matching strategies in different stages of training to achieve the matching of predicted objects and true value labels; using a one-to-many allocation strategy in the early stage of training and a one-to-one allocation strategy in the later stage of training; using a one-to-many allocation strategy in the early stage to increase the number of negative samples, improve the deduplication ability of model prediction, and solve the weak recall performance of DETR models due to their anchor-free structure.

[0010] In S2, the data set is fed into the flood detection model, including: S2-1, divide the flood detection dataset into two categories: unlabeled and labeled images; S2-2, the strong enhancement branches of labeled images and unlabeled images are sent to the student model; the weak enhancement branch of unlabeled images is sent to the teacher model; the corresponding object queries are output respectively, and then the predicted object annotation boxes are obtained.

[0011] In S2-1, for the unlabeled image, an additional image enhancement operation is performed; that is, the image enhancement includes two branches of different strengths that are performed simultaneously, namely strong enhancement and weak enhancement.

[0012] In S3, the loss function consists of three parts: supervised loss for calculating the labeled image prediction and the true value loss , calculate the unsupervised loss for unsupervised image prediction and the true value loss The consistency loss of object queries obtained by strong and weak enhancement branches of unsupervised images, respectively .

[0013] In S3, the unsupervised loss is the prediction result combined with the weak enhancement branch, and the objects involved in the calculation are the corresponding annotations in the strong enhancement branch whose confidence level in the weak enhancement branch is higher than the threshold.

[0014] In S3, the consistency loss is calculated by the unified cross-view object query module; the unified cross-view object query module is designed based on the mean square error (MSE). The more similar the two are, the smaller the loss value is.

[0015] The main beneficial effects of the present invention are: When faced with the water collection wells and pool flooding images collected by hydropower plants, there is no need to fully annotate the data set. Instead, a semi-supervised target detection network can be built to extract effective features from a large number of unlabeled images to achieve the target detection task.

[0016] The learner is built with the Transformer-based object detection network DINO. By leveraging its powerful long-range modeling capabilities, it can efficiently extract deep information contained in images and generate object queries with powerful representation capabilities.

[0017] A unified cross-view object query module designed based on mean square error aligns the features extracted after performing different image enhancement operations on the same image, obtains effective features that remain unchanged in terms of extraction perspective, and enhances the generalization ability of the model for flooding detection tasks.

[0018] Combined with the modules, this method can efficiently obtain useful features in the image and perform accurate flooding detection on the water collection wells and pools in the hydropower plant, thereby ensuring the stability of the visual algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0020] Figure 1 This is a structural diagram of the network model of the present invention. DETAILED DESCRIPTION

[0021] like Figure 1 In the present invention, a semi-supervised flooding detection method for water collection wells and water pools in a hydropower plant based on DINO comprises the following steps: S1, build a network, build a semi-supervised flood detection network based on DINO; S2, build a model, build a semi-supervised data set for flooding detection based on the water collection well and pool scenes of the hydropower plant, and send it to the flooding detection model in S1 for model training; S3, function optimization, build loss function, and optimize model parameters.

[0022] By constructing a semi-supervised object detection network, it is possible to achieve high-quality detection of flooding detection tasks without the need for complete manual annotation of unlabeled images collected by hydropower plants.

[0023] The learner is built with the Transformer-based object detection network DINO. By leveraging its powerful long-range modeling capabilities, it can efficiently extract deep information contained in images and generate object queries with powerful representation capabilities.

[0024] A unified cross-view object query module designed based on mean square error aligns the features extracted after performing different image enhancement operations on the same image, obtains effective features that remain unchanged in terms of extraction perspective, and enhances the generalization ability of the model for flooding detection tasks.

[0025] Embodiment 1, Figure 1 It is a model network structure diagram of an embodiment of the present invention. The network mainly includes three structures: a teacher model, a student model and a unified module for cross-view object query. The network structure of the teacher model is consistent with that of the student model, and both are Transformer-based target detection networks DINO.

[0026] Combination Figure 1 ,The semi-supervised flood detection network for water collection wells and pools in hydropower plants based on DINO includes the following steps: Step 1: Build a semi-supervised flood detection network based on DINO; Step 2: construct a semi-supervised data set for flooding detection based on the water collection well and pool scenes of the hydropower plant, and send it to the flooding detection model described in step 1 for model training; Step 3: Construct a loss function to optimize model parameters.

[0027] Furthermore, the DINO-based semi-supervised flood detection network includes two branches: a teacher model and a student model. More specifically, the model structure of the teacher model is completely consistent with that of the student model, including the encoder and decoder structure of DINO. The image is output through the teacher or student model, and the object query can be used to predict the information of the annotation box, such as the horizontal and vertical coordinates of the center of the annotation box, the width and height of the annotation box, the category and the confidence value.

[0028] Furthermore, the teacher and student models adopt different strategies to update parameters during the training phase. Specifically, the student model adopts a stage-mixed matching strategy to match the predicted object with the true value label and updates the parameters in combination with the loss function. The teacher model parameters are frozen and only after a complete round, the parameters of the student model are updated after performing an exponential sliding average operation.

[0029] Furthermore, the stage mixed matching strategy refers to using different matching strategies at different stages of training to achieve matching between the predicted object and the true value label. Specifically, a one-to-many allocation strategy is used in the early stage of training, and a one-to-one allocation strategy is used in the later stage of training. Using a one-to-many allocation strategy in the early stage and increasing the number of negative samples is conducive to improving the deduplication ability of the model prediction and solving the weak recall performance of the DETR model due to its anchor-free structure.

[0030] Furthermore, the data set is fed into a flood detection model, and the specific operation process is as follows: 1) The flood detection dataset is divided into two categories: unlabeled and labeled images. For unlabeled images, additional image enhancement operations need to be performed. Specifically, image enhancement includes two branches of different strengths that are performed simultaneously, namely strong enhancement and weak enhancement.

[0031] 2) The strong enhancement branches of labeled and unlabeled images are fed into the student model, and the weak enhancement branch of unlabeled images is fed into the teacher model, and the corresponding object queries are output respectively to obtain the predicted object annotation boxes.

[0032] Furthermore, the loss function consists of three parts: the supervised loss for calculating the labeled image prediction and the true value loss , calculate the unsupervised loss for unsupervised image prediction and the true value loss The consistency loss of object queries obtained by strong and weak enhancement branches of unsupervised images, respectively .

[0033] Furthermore, the unsupervised loss is specifically calculated in combination with the prediction result of the weak enhancement branch, and the objects involved in the calculation are the corresponding annotation boxes in the strong enhancement branch whose confidence level in the weak enhancement branch is higher than the threshold.

[0034] Furthermore, the consistency loss is calculated by a unified cross-view object query module. Specifically, the unified cross-view object query module is designed based on mean square error (MSE), and the more similar the two are, the smaller the loss value is.

[0035] In the above method, when facing the images of water collection wells and pools flooded by the hydropower plant, it is not necessary to fully annotate the data set, and a semi-supervised target detection network is constructed to extract effective features using a large number of unlabeled images, thereby achieving the target detection task. In this method, the learner is constructed by the Transformer-based target detection network DINO. By using its powerful long-range modeling ability, it can efficiently extract the deep information contained in the image, and then generate object queries with powerful representation capabilities; the cross-view object query unified module designed based on mean square error, by aligning the features extracted after performing different image enhancement operations on the same image, obtains effective features that are invariant to the extraction perspective, and enhances the model's generalization ability for flood detection tasks. Combined with the above module, this method can efficiently obtain beneficial features in the image, perform accurate flood detection on water collection wells and pools in hydropower plants, and thus ensure the stability of the visual algorithm.

[0036] The above-mentioned embodiments are only preferred technical solutions of the present invention and should not be regarded as limitations of the present invention. The embodiments and features in the embodiments of the present application can be arbitrarily combined with each other without conflict. The protection scope of the present invention shall be the technical solutions recorded in the claims, including the equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. A semi-supervised flooding detection method for water collection wells and pools in hydropower plants based on DINO, characterized in that: The steps include: S1, build a network, build a semi-supervised flood detection network based on DINO; S2, build a model, build a semi-supervised data set for flooding detection based on the water collection well and pool scenes of the hydropower plant, and send it to the flooding detection model in S1 for model training; S3, function optimization, build loss function, and optimize model parameters.

2. The semi-supervised flood detection method for water collection wells and pools in hydropower plants based on DINO according to claim 1 is characterized in that: In S1, the semi-supervised flood detection network based on DINO includes two branches: a teacher model and a student model. The model structure of the teacher model is exactly the same as that of the student model, including the encoder and decoder structure of DINO. The image passes through the teacher or student model and outputs an object query to predict the information of the annotation box.

3. The semi-supervised flood detection method for water collection wells and pools in hydropower plants based on DINO according to claim 2 is characterized in that: In S1, the teacher and student models adopt different strategies to update parameters during the training phase; the student model adopts a stage mixed matching strategy to match the predicted object with the true value label, and updates the parameters in combination with the loss function; the teacher model parameters are frozen, and the parameters are updated only after a complete round, after performing an exponential sliding average operation on the parameters of the student model.

4. The semi-supervised flood detection method for water collection wells and pools in hydropower plants based on DINO according to claim 3 is characterized in that: In S1, the stage mixed matching strategy refers to the use of different matching strategies in different stages of training to achieve the matching of predicted objects and true value labels; a one-to-many allocation strategy is used in the early stage of training, and a one-to-one allocation strategy is used in the later stage of training; a one-to-many allocation strategy is used in the early stage to increase the number of negative samples, improve the deduplication ability of model prediction, and solve the weak recall performance of DETR models due to their anchor-free structure.

5. The semi-supervised flooding detection method for water collection wells and pools in a hydropower plant based on DINO according to claim 1 is characterized by: In S2, the data set is fed into the flood detection model, including: S2-1, divide the flood detection dataset into two categories: unlabeled and labeled images; S2-2, the strong enhancement branches of labeled images and unlabeled images are sent to the student model; the weak enhancement branch of unlabeled images is sent to the teacher model; the corresponding object queries are output respectively, and then the predicted object annotation boxes are obtained.

6. The semi-supervised flooding detection method for water collection wells and pools in a hydropower plant based on DINO according to claim 5 is characterized by: In S2-1, for the unlabeled image, an additional image enhancement operation is performed; that is, the image enhancement includes two branches of different strengths that are performed simultaneously, namely strong enhancement and weak enhancement.

7. The semi-supervised flooding detection method for water collection wells and pools in a hydropower plant based on DINO according to claim 1 is characterized by: In S3, the loss function consists of three parts: supervised loss for calculating labeled image prediction and true value loss , calculate the unsupervised loss for unsupervised image prediction and the true value loss The consistency loss of object queries obtained by strong and weak enhancement branches of unsupervised images, respectively .

8. The DINO-based semi-supervised flooding detection method for water collection wells and pools in a hydropower plant according to claim 7 is characterized by: In S3, the unsupervised loss is the prediction result combined with the weak enhancement branch, and the objects involved in the calculation are the corresponding annotations in the strong enhancement branch whose confidence level in the weak enhancement branch is higher than the threshold.

9. The semi-supervised flooding detection method for water collection wells and pools in a hydropower plant based on DINO according to claim 7 is characterized by: In S3, the consistency loss is calculated by the cross-view object query unification module; The unified module for cross-view object query is designed based on the mean square error (MSE). The more similar the two are, the smaller the loss value is.

Images