A river surface pollutant monitoring method, electronic device, and medium

Abstract

The application discloses a river surface pollutant monitoring method, an electronic device and a medium, and comprises the following steps: acquiring an original river surface image to be detected and performing pretreatment; inputting the pretreated river surface image into a trained image detection model to obtain a river surface pollutant detection result and an area distribution range; wherein the training process of the image detection model comprises the following steps: constructing a river background library and a river foreground library; inputting each river image in the river background library into a trained image segmentation model to obtain a river prediction map mask; calculating the centroid coordinates of each river prediction map; calculating the pasting scale according to the centroid coordinates of the river, and nonlinearly mapping the river foreground to the river background based on the pasting scale to obtain a river training data set; and training the image detection model based on the river training data set.

Description

Technical Field

[0001] This invention relates to the field of environmental pollution monitoring, and in particular to a method, electronic device, and medium for monitoring surface pollution in rivers. Background Technology

[0002] The management and protection of the water environment is a key area of ​​current social concern. Floating debris on the river surface not only directly pollutes water bodies and damages the aquatic ecosystem, but also, when it accumulates in large quantities, can cause river blockages and affect flood control safety. Therefore, effective monitoring and control of pollutants on the river surface is crucial for maintaining a healthy water environment.

[0003] Currently, monitoring technologies for river pollutants mainly include manual inspection, remote sensing, water quality sensor detection, multispectral / hyperspectral analysis, and image detection technology. However, in practical applications, each of these technologies has certain limitations: 1. Manual Inspection: This is the most traditional monitoring method, usually carried out by inspectors using handheld water quality analyzers and other equipment. While intuitive to operate, this method is only suitable for small-scale, short-distance monitoring. Its main drawbacks are high labor costs, low operational efficiency, and personnel safety risks in harsh environments or complex river sections, making it difficult to achieve long-term, large-scale, routine deployment.

[0004] 2. Remote sensing detection technology: Using satellites, drones and other equipment to acquire image data, although it can cover a large area of ​​river, its detection effect is easily affected by weather conditions (such as cloud cover) and changes in lighting, and the image resolution is limited, often making it difficult to accurately identify small floating pollutants.

[0005] 3. Water quality sensor detection technology: By deploying sensors in the river, water quality parameters such as dissolved oxygen, chemical oxygen demand (COD), and ammonia nitrogen can be monitored in real time, offering good real-time performance. However, this technology mainly targets the concentration of dissolved pollutants in the water and cannot detect or identify solid floating objects or garbage on the river surface, resulting in a monitoring blind spot.

[0006] 4. Multispectral / hyperspectral analysis technology: By analyzing the spectral characteristics of water surface reflectance, it is possible to distinguish types of pollutants such as oil and specific chemical substances with high precision. However, this technology has high equipment costs and complex data processing, making it difficult to widely apply in large-scale daily river inspections.

[0007] In contrast, image detection technology, by deploying cameras along the river or using drones to collect video images, combined with computer vision and deep learning techniques, can achieve automated identification and localization of floating objects on the water surface with high detection accuracy. However, existing vision technologies are prone to missed detections or false detections when faced with complex river environments (such as reflections, changes in light and shadow, and weakly textured backgrounds); and obtaining small target samples is difficult, resulting in insufficient training data.

[0008] In conclusion, in order to overcome the shortcomings of traditional manual inspection, such as low efficiency and high cost, as well as the inability of other sensor technologies to identify solid floating objects, applying machine vision technology to the detection of pollutants on the river surface is of great significance for improving the level of intelligence in river environmental monitoring. Summary of the Invention

[0009] To address the shortcomings of existing technologies, embodiments of the present invention provide a method, electronic device, and medium for monitoring river surface pollution.

[0010] In a first aspect, embodiments of the present invention provide a method for monitoring pollution on the surface of a river, the method comprising: Obtain the original surface image of the river channel to be inspected; The original river surface image is denoised and filtered, and the river and pollutants in the denoised river surface image are edge-sharpened to obtain the pre-processed river surface image. The preprocessed river surface image is input into the trained image detection model to obtain the detection results of pollutants on the river surface and the area distribution range. The training process of the image detection model includes: Construct a river background database and a river foreground database; Each river image in the river background database is input into the trained image segmentation model to obtain the river prediction map mask; For each predicted river channel, calculate the centroid coordinates of the river channel; The bonding scale is calculated based on the centroid coordinates of the river channel; The river foreground is nonlinearly mapped onto the river background based on the pasting scale to obtain the river training dataset. The image detection model is trained based on the river training dataset.

[0011] Secondly, embodiments of the present invention provide an electronic device, characterized in that it includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-described method for monitoring river surface pollution.

[0012] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the above-described method for monitoring pollution on the river surface.

[0013] Fourthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, characterized in that the computer program / instruction, when executed by a processor, implements the above-described method for monitoring river surface pollution.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a method for monitoring river surface pollution. By constructing a river background database and a river foreground database, and nonlinearly mapping the river foreground onto the river background based on a pasting scale, a river training dataset is synthesized. Compared with the traditional method that relies on manual collection and annotation of a large number of real river floating object images, this method can quickly expand the training dataset at a lower cost. It effectively solves the problems of difficulty in obtaining negative samples and uneven sample distribution in specific water scenarios (such as rare garbage types or specific lighting conditions), and provides sufficient and diverse data support for the training of image detection models.

[0015] Meanwhile, this invention employs a unique scene-aware strategy, avoiding the problems of target-background ratio imbalance (e.g., garbage in the distance is larger than that in the foreground) or unreasonable location (e.g., garbage is pasted on the shore) caused by simple random pasting in traditional data augmentation; this invention obtains the river prediction map mask by inputting the river background into the image segmentation model, accurately locking the water area in the image, ensuring that virtual pollutants are only mapped and pasted within the effective water surface area. Furthermore, this invention calculates the pasting scale by calculating the centroid coordinates of the river channel. This technique enables the synthesized pollutant image to simulate the "nearer is larger, farther is smaller" perspective rule of human vision, that is, automatically adjusting the size of the pollutant according to its relative position in the river channel. This makes the synthesized image closer to the real scene in terms of geometric features and visual perception, significantly improving the quality of the training data. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the method for monitoring pollutants on the river surface provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the median filter provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a Laplace operator use case provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of nonlinear mapping provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the image segmentation model training provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the image detection model provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a river surface scene provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0020] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for monitoring pollution on the surface of a river, the method comprising: Step S1: Obtain the original surface image of the river channel to be detected.

[0021] Step S2: Denoise filtering is applied to the original river surface image, and edge sharpening is performed on the river and pollutants in the denoised river surface image to obtain the preprocessed river surface image.

[0022] Specifically, a median filter is constructed; the median filter is used to denoise and filter the natural noise and electronic noise in the original river surface image. Figure 2 The middle represents the window size. The median filter. In the image The middle filter window is Formula (1) is as follows: In the formula, The coordinates in the image to be processed are Pixel value; The processed coordinates are Pixel value; For filter window size.

[0023] Furthermore, edge sharpening processing is performed on the river and pollutants in the denoised river surface image to further enhance the contour features of the pollutants on the river surface, making them distinct from the water surface features, thus obtaining the preprocessed river surface image.

[0024] like Figure 3 As shown, the Laplacian operator primarily uses partial differential equations to sharpen image features; its essence is to sharpen the image using the second derivative of the image, thereby improving contrast. For images... The calculation method is shown in the following formula (2): Since the Laplace transform is a linear transform, for discrete images, the Laplace operator is shown in equation (5), which can be derived from equations (3) and (4).

[0025] The Laplacian operator is used to sharpen the edges of the river and pollutants in the denoised river surface image, resulting in a preprocessed river surface image.

[0026] Step S3: Input the preprocessed river surface image into the trained image detection model to obtain the detection results of pollutants on the river surface and the area distribution range.

[0027] Among them, such as Figure 6 As shown, the image detection model is based on YOLOv11 and includes an input layer, a backbone network, a neck network, and a detection head that are coupled in sequence.

[0028] Furthermore, the convolutional layer at the bottleneck of the sixth layer residual feature learning module C3K2 in the backbone network of the image detection model is replaced with an Effective Pyramid Squeezing Attention Module (EPSA). It should be noted that by integrating the Effective Pyramid Squeezing Attention Module (EPSA) into the residual feature learning module C3K2, this invention enhances the feature extraction capability of the image detection model and alleviates the problem of missed detection of targets caused by small target size and weak background texture information of the river channel. The branch processing logic is adopted to both retain the original features and perform deep processing on the features to obtain deeper feature information. At the same time, the Effective Pyramid Squeezing Attention Module (EPSA) more effectively extracts multi-scale spatial information and performs multi-scale fusion. By solving the problem of information loss in the feature extraction process through branch processing and multi-scale information fusion, the image detection model can extract richer feature information.

[0029] Furthermore, an adaptive sparse attention mechanism (SSA) is added between each C3K2 module and CBS module in the neck network of the image detection model. It should be noted that this invention, through the adaptive sparse attention mechanism (SSA), improves the image detection model's attention to the target object, eliminates interference from objects with similar features to the target object, improves the false detection of pollutants caused by environmental interference such as river reflections and lighting, filters out more useful feature information for the target detection task, enhances the information analysis capability of the image detection model in complex scenes, and improves the target detection accuracy of the image detection model. The sparse attention mechanism calculates the attention weights of image patches using a fixed pattern, which may lose some important information; while the adaptive sparse attention mechanism dynamically learns a sparse mask based on the input data and calculates the attention weights for different image patches, resulting in lower complexity and greater flexibility compared to the sparse attention mechanism. The adaptive sparse attention mechanism reduces computational complexity while improving detection accuracy, enabling the image detection model to accurately detect pollutants on the river surface in natural and complex environments.

[0030] Furthermore, such as Figure 4 As shown, the training process of the image detection model includes: Step S100: Construct a river background database and a river foreground database.

[0031] In this example, the river background database uses the ISOOD (Images for Sewage Outfalls Objective Detection) dataset, which contains natural river images taken by drones and handheld cameras and released by research institutions such as Tsinghua University.

[0032] Furthermore, the construction process of the river foreground library includes: image segmentation of the river surface pollutant image dataset, detection of river surface pollutants, and obtaining river surface pollutant image slices to construct the river foreground library. The river surface pollutant image dataset can be self-collected river surface pollutant images or the existing open-source FloW dataset (Floating Waste dataset). The FloW dataset contains approximately 2000 images with 5271 bounding boxes, primarily targeting common floating objects such as bottles, plastic bags, and foam on the water surface. The FloW dataset is collected from real inland river environments, covering different lighting conditions, water wave interference, and reflection conditions.

[0033] Step S200: Input each river image in the river background database into the trained image segmentation model to obtain the river prediction map mask.

[0034] Furthermore, in this example, the image segmentation model uses the Deeplab image segmentation model and is trained using a "prediction-fusion-repetition" semi-supervised training strategy, such as... Figure 5 As shown, it specifically includes: The rivers in some river images in the river background database are labeled to obtain labeled river images and unlabeled river images; The Deeplab image segmentation model was trained using labeled river images to obtain a Deeplab image segmentation pre-trained model. The unlabeled river image is input into the Deeplab image segmentation pre-trained model to obtain the predicted mask image; The predicted mask image is used as the ground truth label for the unlabeled river image. The Deeplab image segmentation pre-training model is trained based on the labeled river image, the unlabeled river image and their ground truth labels. The training process uses mixed precision training and SGD optimizer, and adopts the cross-entropy loss function strategy. The initial learning rate is 1e-4 and a Poly decay strategy is used to obtain the trained Deeplab image segmentation model.

[0035] The trained Deeplab image segmentation model is used to segment each river image in the river background database to obtain river location information.

[0036] Step S300: For each predicted river channel map, calculate the centroid coordinates of the river channel.

[0037] It should be noted that, when the location information of the river channel is known, the river channel is decomposed into multiple triangles, and the area of ​​each triangle is calculated by equation (6). The horizontal and vertical coordinates of the centroid of the river channel are calculated by using the area of ​​the triangle as the weight. This can be derived from equations (7) and (8).

[0038] In the formula, The total area of ​​the river channel. The first river channel to be broken down The area of ​​each triangle, Let be the coordinates of the i-th vertex of the triangle, and n be the number of triangles. For the first The x-coordinate of the centroid of the triangle, For the first The ordinate of the centroid of the triangle, The x-coordinate of the river channel centroid is... The vertical coordinate of the river's centroid is given.

[0039] Step S400: Calculate the pasting scale based on the centroid coordinates of the river channel.

[0040] Furthermore, using the centroid ordinate of the river channel as the boundary, the river channel prediction map mask is divided into upper and lower parts, with the upper part representing the far field and the lower part representing the near field; the pasting scale is then calculated, as shown in the following expression: In the formula, The vertical coordinate of the foreground of the river channel to be pasted is [the coordinate of the foreground]. and The vertical coordinates of the river channel prediction map mask represent the upper and lower boundaries. and The range of the proportion of the near field in the river channel prediction map mask. and The range of the proportion of the far field to the river channel prediction map mask. It is a power function exponential factor. The ordinate of the centroid of the river channel prediction map mask.

[0041] In this example, the lower limit of the proportion of the near field in the river channel prediction map mask. Setting it to 0.18 represents the upper limit of the proportion of the near-field area in the river channel prediction map mask. Setting it to 0.12 is the lower limit of the proportion of the far field in the river channel prediction map mask. Setting it to 0.10 is the upper limit for the proportion of the far field in the river channel prediction map mask. Set to 0.04.

[0042] Step S500: Based on the pasting scale, the river foreground is nonlinearly mapped onto the river background to obtain the river training dataset.

[0043] Step S600: Train the image detection model based on the river training dataset. The training process uses mixed precision training and SGD optimizer, and adopts a loss function strategy that combines bounding box regression loss and classification loss. The initial learning rate is 1e-4 and a cosine annealing strategy is used to control the decay of the learning rate.

[0044] Example 1 The implementation process of the river surface pollutant monitoring method provided by the present invention will be described in further detail below with reference to Example 1.

[0045] In this embodiment, river surface waste generated during production and daily life is considered river surface pollutant. Scenes are categorized into three types: normal lighting and reflections, strong lighting and reflections, and complex scenes. Figure 7 As shown.

[0046] Table 1 below shows the engineering application results of this invention in real river scenarios, where AD is the scene-aware image data enhancement strategy, EPSA is the effective pyramid squeezing attention module, SSA is the adaptive sparse attention mechanism, and ES is the method using both the effective pyramid squeezing attention module and the adaptive sparse attention mechanism. The accuracy of the method of this invention can reach 89.8% under normal lighting and reflections, 90.4% under strong lighting and reflections, 72.5% under complex scenes, and 82.0% under all scenes. Regardless of whether it is under normal lighting and reflections, strong lighting and reflections, or complex scenes, the image detection model AD-ES-YOLOv11 used in this invention is superior to the YOLOv11 model in terms of accuracy, recall, F1 score, mAP@0.5, and mAP@0.5:0.95, showing excellent detection performance. This indicates that this method can detect pollutants on the surface of rivers in field river scenarios and can effectively monitor river pollution.

[0047] Table 1: Detection Results of Pollutants on River Surface In summary, this invention provides a method for monitoring river surface pollution. By enhancing river image data through scene awareness and training an image detection model on a high-quality river training dataset, the model learns richer correlation information between river background texture and pollutant features. This scene-aware data augmentation strategy effectively suppresses missed and false detections caused by weak background texture, light and shadow interference (such as reflections), or large variations in target size, significantly improving the detection accuracy and environmental adaptability of the image detection model in actual river monitoring tasks in the field.

[0048] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the above-described method for monitoring pollutants on the river surface. Figure 8 The diagram shown is a hardware structure diagram of any device with data processing capabilities used in the river surface pollutant monitoring method provided in this embodiment of the invention, except... Figure 8 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0049] Accordingly, this application also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the above-described method for monitoring pollutants on the river surface. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.

[0050] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only.

[0051] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for monitoring surface pollution in river channels, characterized in that, The method includes: Obtain the original surface image of the river channel to be inspected; The original river surface image is denoised and filtered, and the river and pollutants in the denoised river surface image are edge-sharpened to obtain the pre-processed river surface image. The preprocessed river surface image is input into the trained image detection model to obtain the detection results of pollutants on the river surface and the area distribution range. The training process of the image detection model includes: Construct a river background database and a river foreground database; Each river image in the river background database is input into the trained image segmentation model to obtain the river prediction map mask; For each predicted river channel, calculate the centroid coordinates of the river channel; The bonding scale is calculated based on the centroid coordinates of the river channel; The river foreground is nonlinearly mapped onto the river background based on the pasting scale to obtain the river training dataset. The image detection model is trained based on the river training dataset.

2. The method for monitoring river surface pollution according to claim 1, characterized in that, The process of performing denoising filtering on the original river surface image and edge sharpening on the river channel and pollutants in the denoised river surface image to obtain the preprocessed river surface image includes: Construct a median filter; Denoising filters are used to remove natural and electronic noise from the original river surface image using a median filter. The Laplacian operator is used to sharpen the edges of the river and pollutants in the denoised river surface image, resulting in a preprocessed river surface image.

3. The method for monitoring river surface pollution according to claim 1, characterized in that, The process of constructing river background and foreground databases includes: River images without surface pollutants were selected from the ISOOD dataset to construct a river background library; And / or, Image segmentation is performed on the river surface pollutant image dataset to detect pollutants on the river surface, resulting in river surface pollutant image slices, thereby constructing a river foreground library.

4. The method for monitoring river surface pollution according to claim 1 or 3, characterized in that, The training process of the image segmentation model includes: The rivers in some river images in the river background database are labeled to obtain labeled river images and unlabeled river images; Annotated river images were used to train the image segmentation model, resulting in a pre-trained image segmentation model. The unlabeled river image is input into the image segmentation pre-trained model to obtain the predicted mask image; The predicted mask image is used as the ground truth label for the unlabeled river image. Based on the labeled river image, the unlabeled river image and their ground truth labels, a pre-trained image segmentation model is trained to obtain the trained image segmentation model.

5. The method for monitoring river surface pollution according to claim 1, characterized in that, The process of calculating the bonding scale based on the centroid coordinates of the river channel includes: Using the centroid ordinate of the river channel as the boundary, the river channel prediction map mask is divided into upper and lower parts, with the upper part representing the far field and the lower part representing the near field; the pasting scale is then calculated, as shown in the following expression: ； In the formula, The vertical coordinate of the foreground of the river channel to be pasted is [the coordinate of the foreground]. and The vertical coordinates of the river channel prediction map mask represent the upper and lower boundaries. and The range of the proportion of the near field in the river channel prediction map mask. and The range of the proportion of the far field to the river channel prediction map mask. It is a power function exponential factor. The ordinate of the centroid of the river channel prediction map mask.

6. The method for monitoring river surface pollution according to claim 1 or 4, characterized in that, The image segmentation model used is the Deeplab image segmentation model.

7. The method for monitoring river surface pollution according to claim 1, characterized in that, The image detection model includes an input layer, a backbone network, a neck network, and a detection head, which are coupled in sequence. In particular, the convolutional layer at the bottleneck of the sixth layer residual feature learning module C3K2 in the backbone network of the image detection model is replaced with an effective pyramid squeeze attention module. The neck network of the image detection model is equipped with an adaptive sparse attention mechanism.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the river surface pollution monitoring method as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the river surface pollution monitoring method as described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the river surface pollution monitoring method according to any one of claims 1-7.

Images