Method and device for detecting sewage draining exit

Abstract

The invention discloses a sewage draining exit detection method and device. The detection method comprises the following steps: acquiring an image data set containing a sewage draining exit; preprocessing the image data set to obtain a training set; constructing a target detection network based on a YOLOv9 model; inputting the training set into a target detection network and the PGI module at the same time, and using the PGI module to assist gradient calculation of a back propagation process of a shallow network so as to complete training of the target detection network; removing the PGI module in the trained target detection network, and detecting the to-be-detected data by using the target detection network from which the PGI module is removed; according to the method, the problem of information loss of a traditional target detection model in small target detection is effectively solved, the detection precision is improved, the requirement for real-time detection is met through simplification processing in the reasoning stage, and the method is suitable for unmanned aerial vehicle sewage draining exit monitoring tasks in the complex environment.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a sewage outlet detection method and device. Background Art

[0002] As the main outlet for wastewater and pollutants to be discharged into rivers, lakes or oceans, sewage outlets are directly related to the health of the water environment. In order to effectively control the pollution sources, regular inspection and statistics of these sewage outlets are important means for environmental protection departments to understand the actual pollution discharge situation and crack down on illegal discharge. However, at this stage, the number, distribution location and operation status of sewage outlets are often not recorded in detail, which leads to blind spots in sewage management and environmental monitoring. Collecting these key data is a prerequisite for pollutant monitoring and tracing the source of pollution.

[0003] At present, manual inspection of sewage outlets is still the main method, that is, inspectors patrol along the river, manually record the location and status of each sewage outlet, and estimate the total amount. However, this method has obvious limitations: manual inspection is time-consuming and costly, and due to human resource constraints, it is difficult to cover the entire river or area, which often leads to missed inspections and incomplete information. In addition, faced with complex and changeable terrain, the efficiency of manual inspections often cannot meet the requirements of efficient and comprehensive environmental protection work. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a solution with high detection efficiency and high detection accuracy for sewage outlets.

[0005] In order to achieve the above-mentioned purpose, the present invention provides a method for detecting a sewage outlet, comprising: obtaining an image data set containing a sewage outlet; preprocessing the image data set to obtain a training set; constructing a target detection network based on a YOLOv9 model; removing the CBLinear module in the PGI module in the YOLOv9 model, directly splicing the feature layers selected in the backbone network and inputting them into the RepNCSPELAN module for feature fusion processing; inputting the training set into the target detection network and the PGI module at the same time, using the PGI module to assist the gradient calculation of the back propagation process of the shallow network, thereby completing the training of the target detection network; removing the PGI module in the trained target detection network, and using the target detection network after removing the PGI module to detect the data to be detected.

[0006] Optionally, the image data set is preprocessed to obtain a training set, including: compressing images in the image data set whose resolution is greater than a first preset value to a second preset value; annotating the compressed image data set; enhancing the annotated image data set using Copy-Paste; and spatially transforming the enhanced image data set using STN to obtain a training set.

[0007] Optionally, the compressed image dataset is enhanced using Copy-Paste, including: selecting an image of the sewage outlet to be copied from the annotated image dataset; using an image processing tool to crop a target area from the initial image dataset; selecting different background images from the image dataset to synthesize the copied image of the sewage outlet to obtain a synthetic image; the background image includes an image without a sewage outlet; color balancing, brightness and contrast adjustment of the synthetic image; annotating the generated image, and integrating the generated image and the annotation file into the original dataset to obtain an enhanced dataset.

[0008] Optionally, the backbone network of the YOLOv9 model is sampled three times.

[0009] Optionally, the loss function of the target detection network includes a BCE loss function, a CIOU loss function and a DFL loss function.

[0010] Optionally, the formula of the BCE loss function is:

[0011]

[0012] Where N is the number of samples, x i represents the true target category, Represents the predicted category.

[0013] Optionally, the formula of the CIOU loss function is:

[0014]

[0015] Among them, IoU represents; ρ(b,b gt ) is the distance between the center points of the predicted box and the real box; c is the diagonal distance of the minimum closure box; v represents the term used to measure the difference in aspect ratio, where

[0016]

[0017] Among them, w gt Indicates that h gt represents, w represents, h represents, and a represents the effect of aspect ratio consistency on the loss, which is defined as:

[0018]

[0019] Optionally, the formula of the DFL loss function is:

[0020] DFLLoss=-((y i+1 -y)log(S i )+(yy i)log(S i+1 ))

[0021] Among them, S i With S i+1 Represents the i-th and i+1-th discrete bounding box prediction scores output by the model, which represent the model's confidence in different discrete locations; y i and i+1 represents the actual label boundary value of the discrete position, and y represents the true continuous label boundary value of the target.

[0022] On the other hand, the present invention also provides a detection device for a sewage outlet, comprising:

[0023] An acquisition module, used for acquiring an image data set including a sewage outlet;

[0024] A preprocessing module, used for preprocessing the image data set to obtain a training set;

[0025] A construction module is used to construct a target detection network based on the YOLOv9 model; the CBLinear module in the PGI module in the YOLOv9 model is removed, and the selected feature layers in the backbone network are directly spliced ​​and input into the RepNCSPELAN module for feature fusion processing;

[0026] A training module, used to input the training set into the target detection network and the PGI module at the same time, and use the PGI module to assist the gradient calculation of the back propagation process of the shallow network, so as to complete the training of the target detection network;

[0027] The detection module is used to remove the PGI module from the trained target detection network, and use the target detection network after removing the PGI module to detect the data to be detected.

[0028] The beneficial effects of the present invention compared with the prior art are: the detection method optimizes the PGI module, removes the CBLinear module, and directly inputs the features extracted by the backbone into the RepNCSPELAN module for feature fusion, thereby enhancing the shallow feature extraction capability and improving the small target detection effect. Then, a joint optimization strategy is used. In the training phase, the improved PGI module is used to assist gradient descent reasoning, optimize the feature learning of the shallow network, and enable the model to better capture the detailed features of small targets at the sewage outlet; it effectively solves the problem of information loss in the traditional target detection model in small target detection, improves the detection accuracy, and realizes the real-time detection requirements by simplifying the processing in the reasoning phase, which is suitable for the UAV sewage outlet monitoring task in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1It is a flow chart of a sewage outlet detection method provided by the present invention;

[0030] Figure 2 It is a structural diagram of a sewage outlet detection device provided by the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Reference Figure 1 This embodiment provides a method for detecting a sewage outlet, comprising the following steps:

[0033] S10: Acquire an image dataset containing a sewage outlet.

[0034] Because drones have extremely high flexibility and maneuverability, they can obtain high-resolution images at different altitudes, provide more detailed observation data, and can quickly check over a large area. This makes drone aerial survey an ideal tool for detecting sewage outlets, greatly improving the accuracy and speed of data collection, and also providing environmental protection departments with a more efficient means of inspection. Therefore, in this embodiment, a drone shooting method is used to obtain an image data set containing sewage outlets.

[0035] Specifically, in this embodiment, the drone is responsible for flying and the camera is responsible for shooting. The shooting area covers the coastline and inland rivers, the flight altitude is 450 meters to 650 meters, and the image ground resolution is about 5.16 cm to 8.41 cm. At this altitude, the image width of the sewage outlet is about 11 to 96 pixels. A total of 39,500 images were collected.

[0036] S20: Preprocess the image data set to obtain a training set.

[0037] It should be understood that due to the complex flight environment, the quality of drone photography is affected by many factors, and there are mainly two types of abnormal situations: first, abnormal flight environment, such as cloud cover or blurred ground; second, flight interference, such as image blur caused by high-altitude wind speed. To ensure data quality, it is necessary to clean the data set and delete abnormal images. In addition, since the original image resolution is too high, it is not suitable for common target detection models. Therefore, the image size needs to be compressed. Since the average size of the sewage outlet in the original image is about 52×42 pixels, accounting for less than 0.005%, direct compression will result in information loss. To solve this problem, in this embodiment, the image is cropped into a small image of 640×640 pixels and segmented using an overlap rate of 15%. Images with edges less than 640×640 pixels are padded, and each original image is cropped into 150 small images of 640×640 pixels, resulting in 322,500 images.

[0038] In order to conduct deep learning training, the sewage outlet images need to be labeled. In this embodiment, the LabelMe tool is used to label the target of each sewage outlet with a rectangular frame, including the coordinates of the center point (x, y) and the width and height (w, h), and to mark whether it is blocked and the difficulty of detection. After the labeling is completed, the image without the sewage outlet is saved separately.

[0039] After labeling, the data needs to be enhanced. In this embodiment, the Copy-Paste data enhancement method is used to increase the diversity of the data set. This method can effectively increase the frequency of occurrence of targets and improve the robustness of the model in target detection tasks.

[0040] Specifically, first, select the sewage outlet to be copied from the labeled data, and the selection can be made based on the size of the sewage outlet.

[0041] Secondly, use image processing tools to crop the target region from the original image. Make sure the cropped region includes the complete boundary of the target and contains certain context information to maintain the naturalness of the target. Adjust the size of the cropped region according to the actual size of the target and the requirements of the target detection model.

[0042] Next, different background images are selected from the dataset. The background images should be diverse to increase the diversity of the synthesized images, including those without sewage outlets. When selecting background images, the texture, color, and lighting conditions of the background are taken into consideration to ensure that the generated images are natural.

[0043] Then, adjust the color balance, brightness, and contrast of the synthesized image to ensure that the visual effect of the generated image is consistent with the real scene. Check the visual effect of the generated image to ensure that the target blends naturally with the background and avoid obvious stitching marks.

[0044] Next, annotate the generated images using the same annotation tools as the original dataset. Integrate the generated images and annotation files into the original dataset to form the extended dataset. Make sure that the images and annotation files in the dataset match.

[0045] Finally, the expanded dataset is checked to ensure that the generated images and annotations are accurate.

[0046] In some other embodiments, in order to further improve the strength of the data, a spatial transformer network (STN) can be introduced to enhance the spatial transformation capability of the input image and the input feature map. Specifically, a 3-channel STN network is used to perform spatial transformation on the input image and feature map, and then successively spliced ​​with the original input data, so that the network can adaptively learn the spatial mixing of the input data, thereby enhancing the robustness and accuracy of the model.

[0047] S30: constructing a target detection network based on the YOLOv9 model; removing the CBLinear module in the PGI module in the YOLOv9 model, directly splicing the selected feature layers in the backbone network and inputting them into the RepNCSPELAN module for feature fusion processing.

[0048] In this embodiment, considering that the backbone network depth of the traditional YOLOv9 model is relatively large, which may cause the loss of small target information during the downsampling process, this embodiment reduces the number of downsampling times of the backbone network of the original YOLOv9 model from 5 to 3. This adjustment reduces the loss of small target information and enhances the model's sensitivity to details while maintaining the overall feature extraction capability of the network.

[0049] In addition, in order to further improve the detection effect of small targets, the present invention improves the PGI (Pyramid Grained Information) module in YOLOv9. The specific improvement scheme is to remove the CBLinear module in the PGI module, directly splice the selected feature layers in the backbone network and input them into the RepNCSPELAN module for feature fusion processing. This optimization method effectively enhances the feature extraction capability of the shallow network, enables the network to better capture the feature information of small targets, and enables the shallow network to obtain better gradient information.

[0050] It should be noted that, since the detection head of the PGI module is far away from the shallow network of the backbone network during the auxiliary training process, and the existence of the CBLinear module only extracts deeper semantic information, removing the CBLinear module can prevent the loss of small target semantic information of the PGI module during the training process, and increase the ability of PGI to assist gradient calculation during the training process; improve the small target recognition ability of yolov9.

[0051] S40: Input the training set into the target detection network and the PGI module at the same time, and use the PGI module to assist the gradient calculation of the back propagation process of the shallow network, so as to complete the training of the target detection network. Specifically, in the training stage of the model, the input image is sent to the target detection network and the improved PGI module at the same time. The PGI module not only participates in the small target detection task, but also optimizes the training process of the shallow network by assisting gradient descent reasoning. This joint optimization strategy enables the shallow network to more efficiently learn features that are helpful for small target detection during the training process, thereby improving the detection accuracy.

[0052] In this embodiment, the loss function of the target detection network includes a BCE loss function, a CIOU loss function, and a DFL loss function.

[0053] Specifically, the formula of the BCE loss function is:

[0054]

[0055] Where N is the number of samples, x i represents the true target category, Represents the predicted category.

[0056] In addition, the formula of CIOU loss function is:

[0057]

[0058] Among them, IoU represents; ρ(b,b gt ) is the distance between the center points of the predicted box and the real box; c is the diagonal distance of the minimum closure box; v represents the term used to measure the difference in aspect ratio, where

[0059]

[0060] Among them, w gt Indicates that h gt represents, w represents, h represents, and a represents the effect of aspect ratio consistency on the loss, which is defined as:

[0061]

[0062] In this embodiment, the formula of the DFL loss function is:

[0063] DFLLoss=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))

[0064] Among them, S i With S i+1 represents the i-th and i+1-th discrete bounding box prediction scores output by the model, which represent the model's confidence in different discrete locations; y i and i+1 represents the actual label boundary value of the discrete position, and y represents the true continuous label boundary value of the target.

[0065] S50: removing the PGI module from the trained target detection network, and using the target detection network after removing the PGI module to detect the data to be detected.

[0066] In this embodiment, considering the computational complexity of the PGI module in practical applications, the PGI module is removed during reasoning, and the optimized target detection network is retained to directly perform the detection task of small targets at the sewage outlet. This simplified processing greatly improves the reasoning efficiency without affecting the detection effect, and meets the needs of real-time detection.

[0067] The sewage outlet detection method provided in this embodiment solves the information loss problem in small target detection by optimizing data collection, preprocessing, data enhancement and model structure, while improving the detection efficiency of the model in the real-time reasoning process.

[0068] Specifically, we first collect and preprocess data, using drones to collect data from sewage outlets. Then we clean the images to remove images without targets and images with abnormalities, and then we crop and annotate the data to obtain a data set.

[0069] Then the YOLOv9 model was improved, including Backbone network optimization to reduce the number of downsampling times of the backbone network of the YOLOv9 model from 5 times to 3 times, reducing the loss of small target information during the downsampling process and improving the model's sensitivity to small targets. Then the PGI network was optimized, the CBLinear module was removed, and the features extracted by the backbone were directly input into the RepNCSPELAN module for feature fusion, thereby enhancing the shallow feature extraction capability and improving the small target detection effect. Then the joint optimization strategy was used. During the training phase, the improved PGI module was used to assist gradient descent reasoning, optimize the feature learning of the shallow network, and enable the model to better capture the detailed features of small targets at the sewage outlet.

[0070] During the training phase, a joint loss function combining classification, positioning and aspect ratio consistency optimization is used, including BCE Loss: binary cross entropy loss for target classification. CIoU Loss: used to optimize the IoU value between the predicted box and the true box, and enhance the consistency detection of the position and size of small targets. DFL Loss: used to optimize the accuracy of fine-grained prediction values ​​during positioning.

[0071] Simplified processing in the reasoning stage: To meet the needs of real-time detection, the present invention removes the PGI module in the reasoning stage and directly uses the optimized YOLOv9 network to detect small targets at sewage outlets, thereby improving the reasoning efficiency and ensuring the detection accuracy.

[0072] The sewage outlet small target detection method based on the improved YOLOv9 model provided by the present invention effectively solves the information loss problem of the traditional target detection model in small target detection, improves the detection accuracy, and realizes the real-time detection requirement by simplifying the processing in the reasoning stage. It is suitable for UAV sewage outlet monitoring tasks in complex environments.

[0073] Reference Figure 2 This embodiment also provides a sewage outlet detection device, including:

[0074] The acquisition module 100 is used to acquire an image data set including a sewage outlet. It should be noted that since the specific acquisition method and process have been described in detail in step S10 of the above sewage outlet detection method, they will not be repeated here.

[0075] The preprocessing module 200 is used to preprocess the image data set to obtain a training set. It should be noted that since the specific preprocessing method and process have been described in detail in step S20 of the above-mentioned sewage outlet detection method, they will not be repeated here.

[0076] A construction module 300 is used to construct a target detection network based on the YOLOv9 model; the CBLinear module in the PGI module in the YOLOv9 model is removed, and the feature layers selected in the backbone network are directly spliced ​​and input into the RepNCSPELAN module for feature fusion processing; it should be noted that since the specific construction method and process have been described in detail in step S30 of the above-mentioned sewage outlet detection method, they will not be repeated here.

[0077] The training module 400 is used to input the training set into the target detection network and the PGI module at the same time, and use the PGI module to assist the gradient calculation of the back propagation process of the shallow network, so as to complete the training of the target detection network; it should be noted that since the specific training method and process have been explained in detail in step S40 of the above-mentioned sewage outlet detection method, they will not be repeated here.

[0078] The detection module 500 is used to remove the PGI module in the trained target detection network, and use the target detection network after removing the PGI module to detect the data to be detected; it should be noted that since the specific detection method and process have been explained in detail in step S50 of the above-mentioned sewage outlet detection method, they will not be repeated here.

[0079] In addition, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium may store a program, and when the program is executed, the program includes part or all of the steps of any sewage outlet detection method recorded in the above method embodiment.

[0080] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0081] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0082] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0083] The above describes an exemplary flow chart for implementing a sewage outlet detection method according to an embodiment of the present invention with reference to the accompanying drawings. It should be noted that the large amount of details included in the above description are only exemplary illustrations of the present invention, rather than limitations of the present invention. In other embodiments of the present invention, the method may have more, fewer or different steps, and the order, inclusion, function, etc. of the steps may be different from those described and illustrated.

Claims

1. A method for detecting a sewage outlet, characterized in that: include: Obtain an image dataset containing sewage outlets; Preprocessing the image data set to obtain a training set; Construct a target detection network based on the YOLOv9 model; remove the CBLinear module in the PGI module in the YOLOv9 model, directly splice the selected feature layers in the backbone network and input them into the RepNCSPELAN module for feature fusion processing; The training set is input into the target detection network and the PGI module at the same time, and the PGI module is used to assist the gradient calculation of the back propagation process of the shallow network, thereby completing the training of the target detection network; The PGI module in the trained target detection network is removed, and the target detection network after the PGI module is removed is used to detect the data to be detected.

2. The detection method according to claim 1, characterized in that: Preprocessing the image data set to obtain a training set includes: Compressing images in the image data set with a resolution greater than a first preset value to a second preset value; Annotate the compressed image dataset; Use Copy-Paste to enhance the annotated image dataset; The STN is used to perform spatial transformation on the enhanced image dataset to obtain the training set.

3. The detection method according to claim 2, characterized in that: Use Copy-Paste to enhance the compressed image dataset, including: Select an image of the sewage outlet to be replicated from the annotated image dataset; Use image processing tools to crop the target area from the initial image dataset; Selecting different background images from the image data set and synthesizing them with the copied image of the sewage outlet to obtain a synthesized image; the background image includes an image without a sewage outlet; Make color balance, brightness, and contrast adjustments to composite images; The generated images are annotated, and the generated images and annotation files are integrated into the original dataset to obtain an enhanced dataset.

4. The detection method according to claim 1, characterized in that: The backbone network of the YOLOv9 model is sampled 3 times.

5. The detection method according to claim 1, characterized in that: The loss functions of the target detection network include BCE loss function, CIOU loss function and DFL loss function.

6. The detection method according to claim 5, characterized in that: The formula of the BCE loss function is: Where N is the number of samples, x i represents the true target category, Represents the predicted category.

7. The detection method according to claim 5, characterized in that: The formula of the CIOU loss function is: Among them, IoU represents; ρ(b,b gt ) is the distance between the center points of the predicted box and the real box; c is the diagonal distance of the minimum closure box; v represents the term used to measure the difference in aspect ratio, where Among them, w gt Indicates that h gt represents, w represents, h represents, and a represents the effect of aspect ratio consistency on the loss, which is defined as:

8. The detection method according to claim 5, characterized in that: The formula of the DFL loss function is: DFLLoss=-((y i+1 -y)log(S i )+(y-y i )log(S i+1 )) Among them, S i With S i+1 represents the i-th and i+1-th discrete bounding box prediction scores output by the model, which represent the model's confidence in different discrete locations; y i and i+1 represents the actual label boundary value of the discrete position, and y represents the true continuous label boundary value of the target.

9. A sewage outlet detection device, characterized in that: include: An acquisition module, used for acquiring an image data set including a sewage outlet; A preprocessing module, used for preprocessing the image data set to obtain a training set; A construction module is used to construct a target detection network based on the YOLOv9 model; the CBLinear module in the PGI module in the YOLOv9 model is removed, and the selected feature layers in the backbone network are directly spliced ​​and input into the RepNCSPELAN module for feature fusion processing; A training module, used to input the training set into the target detection network and the PGI module at the same time, and use the PGI module to assist the gradient calculation of the back propagation process of the shallow network, so as to complete the training of the target detection network; The detection module is used to remove the PGI module from the trained target detection network, and use the target detection network after removing the PGI module to detect the data to be detected.

10. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of a sewage outlet detection method as described in any one of claims 1 to 8.

Images