Cooperative monitoring method and device for river drain outlet based on space-air-ground integration

Abstract

The invention provides a cooperative monitoring method and device for a river channel sewage draining exit based on space-air-ground integration, relates to the technical field of remote sensing, and solves the technical problem that large-range, high-precision and full-time monitoring of the river channel sewage draining exit cannot be realized at the same time in the prior art. The method specifically comprises the following steps: acquiring remote sensing image data of a first area, and preprocessing to obtain orthographic remote sensing image data; extracting a water area vector boundary of the first area by adopting a remote sensing water body index in combination with deep learning, training a model A for identifying water body pollution in combination with ground water quality data and remote sensing features, and performing inversion to obtain a second area; the dual-camera unmanned aerial vehicle is controlled to carry out aerial photography on the second area, and the position of a sewage draining exit is determined through K-means clustering in combination with visible light image data; taking the thermal infrared threshold exceeding area of the camera of the sewage draining exit as a sample, and training a model B for identifying a video; and monitoring the drain outlet day and night based on the model B and the second infrared image data, and identifying the pollution discharge information in real time. The method is used for monitoring the sewage draining exit of the riverway.

Description

Technical Field

[0001] This application relates to the field of remote sensing technology, and in particular to a method and device for collaborative monitoring of river sewage outlets based on an integrated air-space-ground system. Background Technology

[0002] As key points for pollutants entering surface water bodies, the monitoring of river sewage outlets is of great significance for protecting the ecological environment and safeguarding human health. Traditional monitoring of river sewage outlets relies on manual on-site inspections and sampling analysis of water quality indicators such as pH, chemical oxygen demand, and ammonia nitrogen. With technological advancements, existing technologies such as CN114781537B disclose a method for constructing sewage identification models based on high-resolution satellite remote sensing imagery to achieve large-scale monitoring, and CN119313532A discloses a method for improving monitoring accuracy by combining satellite remote sensing and UAV aerial surveys. Some solutions also attempt to introduce thermal infrared video monitoring to supplement monitoring. However, traditional manual methods are slow and costly, unable to meet the needs of real-time and large-area monitoring, and difficult to supervise sewage discharge at night. While the combination of remote sensing and UAVs can achieve large-scale, high-precision monitoring, it can only be carried out during the day, and video monitoring alone is insufficient to cover large areas. Therefore, existing technologies suffer from the technical problem of not being able to simultaneously achieve large-scale, high-precision, and all-weather monitoring of river sewage outlets. Summary of the Invention

[0003] This application provides a method and device for collaborative monitoring of river sewage outlets based on air-space-ground integration, which solves the technical problem that existing technologies cannot simultaneously achieve large-scale, high-precision, and all-time monitoring of river sewage outlets.

[0004] To achieve the above objectives, this application adopts the following technical solution: Firstly, remote sensing images of the area to be monitored are acquired and preprocessed to obtain orthorectified remote sensing images. A remote sensing water index combined with a deep learning model is used to extract the water vector boundary of the area to be monitored from the orthorectified remote sensing images. Using the water vector boundary as a mask, the orthorectified remote sensing images are inverted using model A, which is used to identify water pollution, to obtain the first region. Specifically, model A is obtained by training a random forest model based on a first sample set and a feature set. The first sample set consists of measured data from ground water quality monitoring stations in the first region. The feature set consists of the spectral values ​​of the orthorectified remote sensing images and remote sensing water quality monitoring data. The index; threshold segmentation of the first thermal infrared image to determine the second region, and verification of the sewage outlet location by combining it with the first visible light image; the first thermal infrared image and the first visible light image are thermal infrared and visible light images obtained by aerial photography of the first region; day and night monitoring of the sewage outlet location; the day and night monitoring process meets the following process: during the daytime, sewage information is identified by model B used for video recognition; during the nighttime, sewage information is identified based on the temperature threshold comparison of the sewage outlet thermal infrared image; wherein, model B is obtained by combining deep learning of the sewage outlet visible light image with threshold recognition of the sewage outlet thermal infrared image.

[0005] In conjunction with the first aspect mentioned above, one possible implementation involves using a remote sensing water index combined with a deep learning model to extract the water body vector boundary of the area to be monitored from orthorectified remote sensing images. This includes: calculating the NDWI index based on the green band and near-infrared band spectral values ​​of the orthorectified remote sensing image; setting an NDWI index threshold to perform preliminary water body extraction on the orthorectified remote sensing image, obtaining preliminary water body extraction vector patterns; post-processing the vector patterns to remove small areas smaller than a set area threshold; converting the post-processed vector patterns into raster data as a training sample set for the deep learning model; training the deep learning model using the training sample set; applying the trained deep learning model to the entire area of ​​the orthorectified remote sensing image for accurate water body extraction; and converting the extracted water body results for the entire area into vector format to obtain the water body vector boundary.

[0006] In conjunction with the first aspect mentioned above, in one possible implementation, a random forest model is trained based on a first sample set and a feature set to obtain model A, including: cleaning the first sample set; setting the core parameters of the random forest algorithm; using the cleaned first sample set as training labels and the feature set as input features to train each independent decision tree separately; and generating the final comprehensive decision tree by averaging the decision tree output results to obtain model A for identifying water pollution.

[0007] In conjunction with the first aspect mentioned above, in one possible implementation, using the water area vector boundary as a mask, the orthorectified remote sensing image is inverted using model A, which is used to identify water pollution, to obtain the first region. This includes: using the water area vector boundary as a mask, aligning it with the orthorectified remote sensing image, cropping and retaining the river water area image within the mask, and removing non-water areas; inputting the cropped water area image into model A, identifying polluted areas based on the image band spectral values ​​and remote sensing water quality monitoring index, and converting the identification results into vector format to obtain the first region.

[0008] In conjunction with the first aspect mentioned above, in one possible implementation, the first thermal infrared image is thresholded to determine the second region, and the location of the sewage outlet is verified and confirmed in conjunction with the first visible light image. This includes: performing histogram statistical analysis on the temperature values ​​of the first thermal infrared image, using K-means clustering for threshold segmentation, and locating the second region with abnormal temperature; retrieving the first visible light image from the same time period and location as the first thermal infrared image, and verifying and confirming the specific location of the sewage outlet within the second region in conjunction with its shape and texture information.

[0009] In conjunction with the first aspect mentioned above, one possible implementation involves using K-means clustering for threshold segmentation, including: S1. Select multiple random sample values ​​from the pixel temperature values ​​of the first thermal infrared image as the initial cluster centroids; S2. Calculate the distance between each pixel temperature value and each initial centroid based on the Euclidean distance, and assign the pixel temperature value to the category of the nearest centroid; the distance satisfies the following formula:

[0010] in, , Where n is the total number of pixels in the first thermal infrared image, K is the number of clusters, and m is the feature dimension. Let i be the temperature value of the i-th pixel in the first thermal infrared image. This represents the temperature value of the j-th initial cluster centroid in the K-means clustering algorithm. Let k be the feature value of the i-th pixel. Let k be the eigenvalue of the j-th centroid; S3. Calculate the average temperature value of all pixels in each cluster to obtain the updated centroid of that cluster. The centroid update process satisfies the following formula:

[0011] in, , The total number of pixels contained in cluster category G. The updated centroid for category G; S4. Repeat S2 and S3 until the values ​​of all cluster centroids no longer change.

[0012] In conjunction with the first aspect mentioned above, in one possible implementation, model B is obtained based on deep learning of visible light images of sewage outlets combined with threshold recognition of thermal infrared images of sewage outlets, including: setting a temperature threshold; the temperature threshold satisfies... ,in Here, x represents the temperature threshold, and x is the highest temperature in the thermal infrared image of the sewage outlet corresponding to the unpolluted water body at the same time the previous day at the current monitoring point. The temperature at the current time. The temperature was set for the same period the previous day. Frame-by-frame analysis of thermal infrared images captured by cameras at the sewage outlet locations was performed, marking areas with pixel temperature values ​​exceeding a set temperature threshold as polluted areas, thus creating polluted area labeling samples. These polluted area labeling samples were then spatially and temporally aligned with visible light images captured by cameras at the same time and location, ensuring that corresponding areas in the visible light images carried polluted area labeling information, thus constructing a training dataset for the deep learning model. The training dataset was used to train a deep learning recognition model for visible light images. After the model training was completed, the visible light deep learning model was superimposed with the temperature threshold judgment logic of the thermal infrared images to form Model B for video recognition.

[0013] In conjunction with the first aspect mentioned above, in one possible implementation, after monitoring the location of the sewage outlet day and night, the method further includes supplementing the first sample set with the sewage discharge information and the monitoring results of model B as a second sample set, and training and optimizing model A.

[0014] In conjunction with the first aspect mentioned above, in one possible implementation, the remote sensing water quality monitoring indices in the feature set include the Normalized Difference Vegetation Index (NDVI), the Black and Odorous Water Index (BOI), the Normalized Difference Suspended Solids Index (NDSSI), the Normalized Difference Turbidity Index (NDTI), the Dual-Band Water Index (DBWI), and the Water Cleanliness Index (WCI).

[0015] Secondly, a collaborative monitoring device for river sewage outlets based on an integrated air-space-ground system is provided, comprising: a communication unit and a processing unit; the communication unit is used to acquire remote sensing images of the area to be monitored; acquire a first thermal infrared image and a first visible light image; acquire thermal infrared images and visible light images of the sewage outlet; the processing unit is used to preprocess the remote sensing images to obtain orthorectified remote sensing images; extract the water area vector boundary of the area to be monitored from the orthorectified remote sensing images using a remote sensing water index combined with a deep learning model; use the water area vector boundary as a mask to invert the orthorectified remote sensing images using a model A for identifying water pollution to obtain a first region; perform threshold segmentation on the first thermal infrared image to determine a second region, and verify the location of the sewage outlet by combining it with the first visible light image; monitor the location of the sewage outlet day and night; the day and night monitoring process satisfies the following process: during the daytime, sewage discharge information is identified using a model B for identifying video; during the nighttime, sewage discharge information is identified based on the temperature threshold comparison of the thermal infrared images of the sewage outlet.

[0016] Thirdly, this application provides a space-air-ground integrated river discharge outlet collaborative monitoring device, comprising: a processor and a storage medium; the storage medium includes instructions, and the processor is used to execute the instructions to implement the method described in the first aspect and any possible implementation thereof. This space-air-ground integrated river discharge outlet collaborative monitoring device can be an electronic device or a chip within an electronic device.

[0017] Fourthly, this application provides a collaborative monitoring system for river sewage outlets based on an integrated air-space-ground system, comprising: remote sensing satellites, unmanned aerial vehicles (UAVs), video monitoring devices, and electronic equipment; wherein, the remote sensing satellites are used to acquire remote sensing images of the area to be monitored; the UAVs are used to acquire first thermal infrared images and first visible light images; the video monitoring devices are used to acquire thermal infrared images and visible light images of the sewage outlets; the electronic equipment is used to preprocess the remote sensing images to obtain orthorectified remote sensing images; using a remote sensing water index combined with a deep learning model, the water area vector boundary of the area to be monitored is extracted from the orthorectified remote sensing images; using the water area vector boundary as a mask, the orthorectified remote sensing images are inverted using model A, which is used to identify water pollution, to obtain a first region; the first thermal infrared images are thresholded to determine a second region, and the location of the sewage outlet is verified and confirmed in conjunction with the first visible light image; the location of the sewage outlet is monitored day and night; the day and night monitoring process satisfies the following process: during the daytime, sewage discharge information is identified using model B, which is used to identify the video; during the nighttime, sewage discharge information is identified based on the temperature threshold comparison of the thermal infrared images of the sewage outlets.

[0018] Fifthly, this application provides a computer-readable storage medium storing instructions that, when executed on a space-air-ground integrated river discharge outlet collaborative monitoring device, cause the space-air-ground integrated river discharge outlet collaborative monitoring device to perform the methods described in the first aspect and any possible implementation thereof.

[0019] In a sixth aspect, this application provides a computer program product containing instructions that, when the computer program product is run on a river discharge outlet collaborative monitoring device based on an integrated air-space-ground system, causes the river discharge outlet collaborative monitoring device based on an integrated air-space-ground system to perform the methods described in the first aspect and any possible implementation thereof.

[0020] This application provides a method and device for collaborative monitoring of river sewage outlets based on an integrated air-space-ground system. Through air-space-ground collaboration and dual-model linkage, it enables large-scale, high-precision, and all-weather monitoring of river sewage outlets. Remote sensing image preprocessing eliminates interference, providing accurate orthorectified data for subsequent monitoring; water body indices combined with deep learning extract water area boundaries, eliminating shadow interference and ensuring boundary accuracy; Model A, relying on random forests and measured data, addresses the problem of insufficient ground stations, achieving large-scale pollution inversion; UAV aerial photography of the first area, with dual cameras providing temperature and visual data; K-means segmentation of thermal infrared images combined with visible light verification accurately locates sewage outlets; Model B and day / night monitoring fill the nighttime gap, comprehensively improving monitoring efficiency and accuracy, and solving the technical problem of existing technologies being unable to simultaneously achieve large-scale, high-precision, and all-weather monitoring of river sewage outlets.

[0021] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0022] Figure 1 A system architecture diagram of a river sewage outlet collaborative monitoring system based on air-space-ground integration is provided for embodiments of this application; Figure 2A system architecture diagram of another river sewage outlet collaborative monitoring system based on air-space-ground integration provided in this application embodiment; Figure 3 A flowchart illustrating a collaborative monitoring method for river sewage outlets based on an integrated air-space-ground system, provided in an embodiment of this application; Figure 4 Remote sensing river water quality inversion and local magnification images provided in the embodiments of this application; Figure 5 This application provides a drone-based visible light aerial photograph of a sewage outlet as an embodiment of the invention. Figure 6 The thermal infrared image temperature-pixel count distribution map provided in the embodiments of this application; Figure 7 A partial thermal imaging view of a river outlet provided in an embodiment of this application; Figure 8 A flowchart illustrating another method for collaborative monitoring of river sewage outlets based on an integrated air-space-ground system, provided in this application embodiment; Figure 9 A flowchart illustrating another method for collaborative monitoring of river sewage outlets based on an integrated air-space-ground system, provided in this application embodiment; Figure 10 A schematic diagram of a collaborative monitoring device for river sewage outlets based on an integrated air-space-ground system, provided in an embodiment of this application; Figure 11 This is a schematic diagram of the hardware structure of a collaborative monitoring device for river sewage outlets based on an integrated air-space-ground system, provided in an embodiment of this application. Detailed Implementation

[0023] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0024] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0025] The integrated air-space-ground method for collaborative monitoring of river discharge outlets provided in this application can be applied to, for example... Figure 1 The integrated air-space-ground river discharge outlet collaborative monitoring system shown includes: remote sensing satellite 101, unmanned aerial vehicle 102, video monitoring device 103, and electronic equipment 104.

[0026] Among them, remote sensing satellite 101 is used to acquire remote sensing images of the area to be monitored; UAV 102 is used to acquire first thermal infrared images and first visible light images; video monitoring device 103 is used to acquire thermal infrared images and visible light images of the sewage outlet; electronic device 104 is used to preprocess the remote sensing images to obtain orthorectified remote sensing images; using remote sensing water index combined with deep learning model, the water area vector boundary of the area to be monitored is extracted from the orthorectified remote sensing images; using the water area vector boundary as a mask, the orthorectified remote sensing images are inverted through model A used to identify water pollution to obtain the first region; threshold segmentation is performed on the first thermal infrared image to determine the second region, and the location of the sewage outlet is verified and confirmed in combination with the first visible light image; the location of the sewage outlet is monitored day and night; the day and night monitoring process meets the following process: during the daytime, sewage information is identified through model B used to identify video; during the nighttime, sewage information is identified based on the temperature threshold comparison of the thermal infrared images of the sewage outlet.

[0027] As an example, such as Figure 2 As shown, the remote sensing satellite 101 first transmits the acquired remote sensing image of the area to be monitored to the electronic device 104; the electronic device 104 preprocesses the image, extracts the water area vector boundary, and trains model A. After obtaining the second area through model A inversion, it generates corresponding UAV flight path planning information and sends it to the UAV 102; the UAV 102 performs aerial photography of the second area according to the planned flight path, and transmits the acquired first thermal infrared image and first visible light image back to the electronic device 104; the electronic device 104 uses K-means clustering to threshold segment the first thermal infrared image, and combines it with the first visible light image to verify the accuracy of the data. The video monitoring device 103 transmits the collected thermal infrared and visible light images of the sewage outlet to the electronic device 104. The electronic device 104 uses areas with temperatures exceeding a set threshold in the thermal infrared image of the sewage outlet as pollution zone labels and trains the visible light image of the sewage outlet to obtain a model B for video recognition. During subsequent monitoring, the electronic device 104 uses model B during the day and compares the temperature thresholds based on the thermal infrared image of the sewage outlet at night to identify the sewage outlet location in real time. It can also obtain historical monitoring data from the historical data storage module to assist in the full-time tracking and accurate analysis of sewage information.

[0028] To address the technical problem that existing technologies cannot simultaneously achieve large-scale, high-precision, and all-weather monitoring of river sewage outlets, this application provides a collaborative monitoring method for river sewage outlets based on an integrated air-space-ground system. This method includes: acquiring remote sensing images of the river area to be monitored and preprocessing them to obtain orthorectified remote sensing images; extracting water body vector boundaries using remote sensing water body indices combined with deep learning; training model A using ground water quality measurement data and remote sensing features, and inverting polluted areas using the water body boundaries as masks; controlling a drone equipped with thermal infrared and visible light cameras to take aerial photographs of the first area; identifying suspected areas by K-means segmentation of the thermal infrared images and confirming sewage outlets using visible light; training model B using thermal infrared threshold annotation samples from the sewage outlet cameras; and finally, monitoring the sewage outlets day and night using model B and based on thermal infrared thresholds. Based on this, the method achieves large-scale screening, high-precision positioning, and all-weather monitoring of river sewage outlets simultaneously through air-space-ground collaboration and dual-model linkage.

[0029] Figure 3 A flowchart illustrating the integrated air-space-ground collaborative monitoring method for river sewage outlets provided in this application is shown below. Figure 3 As shown, the method includes: S301. Acquire remote sensing images of the area to be monitored, preprocess the remote sensing images to obtain orthophoto images.

[0030] The remote sensing images were acquired by remote sensing satellites.

[0031] In this embodiment of the application, the electronic device receives remote sensing images of the area to be monitored transmitted by remote sensing satellites, and sequentially performs radiometric calibration, geometric correction, orthorectification, fusion, stitching, cropping and cloud removal processing to obtain orthorectified remote sensing images.

[0032] Based on the above steps, preprocessing can eliminate sensor errors and environmental interference, and obtain high-precision orthophoto remote sensing images, providing reliable data for subsequent monitoring.

[0033] S302. Using remote sensing water index combined with a deep learning model, extract the water vector boundary of the area to be monitored from orthophoto remote sensing images.

[0034] Among them, the remote sensing water index is the normalized water index (NDWI).

[0035] In this embodiment, the electronic device calculates the NDWI index based on the green band and near-infrared band spectral values ​​of the orthorectified remote sensing image; sets an NDWI index threshold, performs preliminary water body extraction on the orthorectified remote sensing image, and obtains preliminary water body extraction vector patterns; performs post-processing on the vector patterns to remove small areas with areas smaller than a set area threshold; converts the post-processed vector patterns into raster data as a training sample set for a deep learning model (such as U-Net); trains the deep learning model using the training sample set, applies the trained deep learning model to the entire area of ​​the orthorectified remote sensing image, and performs accurate water body extraction; converts the extracted water body results of the entire area into a vector format to obtain the water body vector boundary of the area to be monitored.

[0036] It should be noted that this application does not specifically limit the type of deep learning model.

[0037] As an example, for orthophotos of a group of rivers in a city, an electronic device trains a U-Net model to extract the water area vector boundary for cases where the NDWI is greater than the NDWI exponential threshold.

[0038] Based on the above steps, the water area to be monitored can be accurately extracted, thus defining the scope of pollution monitoring.

[0039] S303. Train a random forest model based on the first sample set and feature set to obtain model A for identifying water pollution.

[0040] The first sample set consists of measured data from ground water quality monitoring stations in the first region, including pH, chemical oxygen demand, ammonia nitrogen, total nitrogen, and total phosphorus. The feature set consists of the band spectral values ​​of orthophoto remote sensing images and remote sensing water quality monitoring indices. The remote sensing water quality monitoring indices include the Normalized Difference Vegetation Index (NDVI), the Black and Odorous Water Index (BOI), the Normalized Difference Suspended Solids Index (NDSSI), the Normalized Difference Turbidity Index (NDTI), the Dual-Band Water Index (DBWI), and the Water Cleanliness Index (WCI).

[0041] In this embodiment, the electronic device cleans the first sample set, sets the core parameters of the random forest algorithm, including the number of independent decision trees to be constructed, uses the cleaned first sample set as the training label and the feature set as the input features, trains each independent decision tree separately, and generates the final comprehensive decision tree by averaging the decision tree output results, thus obtaining model A for identifying water pollution.

[0042] It should be noted that the selection of remote sensing water quality monitoring indices can be adjusted according to the focus of the verification area. Common water quality monitoring indices in remote sensing multispectral imagery include: Normalized Difference Vegetation Index (NDVI). Black and odorous water index Normalized suspended solids index Normalized turbidity index Dual-band water index Water cleanliness index ,in , , These represent the center wavelengths of the Blue, Red, and Green bands, respectively. NIR represents the spectral value of the near-infrared band, while Red, Green, and Blue represent the spectral values ​​of the red, green, and blue bands, respectively.

[0043] As an example, when training a random forest model, the number of decision trees is set to 100, and the depth of a single tree is set to 12.

[0044] Based on the above steps, the problem of insufficient ground stations can be solved, which can meet the need for rapid inversion of water pollution in large areas.

[0045] S304. Using the water area vector boundary as a mask, the orthophoto remote sensing image is inverted using model A, which is used to identify water pollution, to obtain the first region.

[0046] The first region is a key sub-region suspected of pollution after model A inversion.

[0047] In this embodiment, the electronic device uses the water area vector boundary extracted in S302 as a mask, aligns it with the orthophoto remote sensing image, crops and retains the river water area image within the mask, and removes non-water area data; inputs the cropped water area image into model A, identifies the polluted area based on the image band spectral values ​​and remote sensing water quality monitoring index, converts the identification result into vector format, and obtains the first region.

[0048] It should be noted that the masking operation eliminates non-water area interference, allowing the inversion to focus on the river channel.

[0049] As an example, such as Figure 4 As shown, the electronic device uses the water area vector boundary of the area to be monitored as a mask. After cropping the orthophoto remote sensing image of the area, the retained river water area image is input into model A. The first area obtained by inversion corresponds to the local magnified display of the remote sensing image, accurately focusing on the river sub-area suspected of pollution, clearly showing the screening and focusing process from the large area to be monitored to the small first area.

[0050] Based on the above steps, a wide-range to small-range focusing is achieved, defining a precise range for drone aerial photography.

[0051] S305. Threshold segmentation is performed on the first thermal infrared image to determine the second region, and the location of the sewage outlet is verified and confirmed in conjunction with the first visible light image.

[0052] The second area is a suspected sewage outlet area with abnormal temperature in the thermal infrared image. The first thermal infrared image and the first visible light image are thermal infrared and visible light images obtained by drone aerial photography of the first area.

[0053] In this embodiment, the electronic device plans the drone flight path according to the first area, controls the drone equipped with a thermal infrared camera and a visible light camera to take aerial photos of the first area, performs histogram statistical analysis on the temperature value of the first thermal infrared image, uses K-means clustering to perform threshold segmentation, locates the second area with abnormal temperature, and then retrieves the first visible light image at the same time and location as the first thermal infrared image. Combining its shape and texture information, the specific location of the sewage outlet in the second area is verified and confirmed.

[0054] It should be noted that visible light verification eliminates misjudgments caused by thermal infrared interference with non-sewage discharge heat.

[0055] As an example, electronic devices plan a zigzag flight path for a 15km² area, such as... Figure 5 As shown, the drone synchronously collects and transmits images. The drone's thermal infrared camera can be a DJI Zenmuse series unit. Videos are shot in MP4 format, and photos are taken in JPEG format. The drone's thermal infrared images are processed using PHOTOScan software. Figure 6 As shown, the temperature-pixel count distribution of the first thermal infrared image, after K-means clustering, exhibits two peak clusters. Temperature pixel distribution in non-sewage discharge areas such as environmental water bodies The temperature pixel distribution corresponding to the high-temperature water flow at the sewage outlet; the electronic device uses K-means clustering to threshold segment the image, and... The area corresponding to the temperature range was identified as the second region. Further examination using the first visible light image from the same time period revealed that this region exhibited texture and shape characteristics characteristic of pipe drainage, such as... Figure 7 As shown, the area within the box represents the final confirmed location of the sewage outlet.

[0056] Based on the above steps, the location of the sewage outlet can be accurately located, providing a target for real-time monitoring.

[0057] S306. Model B for video recognition is obtained by combining deep learning of visible light images of sewage outlets with threshold recognition of thermal infrared images of sewage outlets.

[0058] Among them, the thermal infrared image and the visible light image of the sewage outlet are both images collected by the sewage outlet video monitoring device.

[0059] In this embodiment, the electronic device sets a temperature threshold, receives thermal infrared images of the sewage outlet transmitted by the video monitoring device, and filters areas with temperatures greater than the set threshold as pollution area labeling samples. The pollution area labeling samples are spatially and temporally aligned with visible light images of the sewage outlet collected by cameras at the same time and location, so that the corresponding areas of the visible light images carry pollution area labeling information, and a training dataset for the deep learning model is constructed. The deep learning recognition model of the visible light images is trained using the training dataset. After the model training is completed, the visible light deep learning model is superimposed with the temperature threshold judgment logic of the sewage outlet thermal infrared images to form model B for video recognition.

[0060] Optionally, the temperature threshold satisfies ,in Here, x represents the temperature threshold, and x is the highest temperature in the thermal infrared image of the sewage outlet corresponding to the unpolluted water body at the same time the previous day at the current monitoring point. The temperature at the current time. The temperature was at the same time the previous day.

[0061] As an example, the video monitoring device at the sewage outlet continuously collects thermal infrared and visible light images of the outlet. The electronic device first sets a temperature threshold (current air temperature 28℃, air temperature at the same time the previous day 26℃, highest temperature of unpolluted water body the previous day 25℃, then the temperature threshold is 25.02℃), and filters out areas in the thermal infrared images of the sewage outlet with temperatures greater than 25.02℃ as labeled samples of polluted areas. The labeled samples are aligned with the visible light images of the sewage outlet at the same time and location to construct a training set. The YOLO deep learning model is trained using this training set. After the model is trained, the thermal infrared temperature threshold judgment logic is superimposed to obtain model B used for video recognition.

[0062] Based on the above steps, a recognition model adapted to the visual features of the sewage outlet is obtained.

[0063] S307. Monitor the location of sewage outlets day and night.

[0064] Among them, day and night monitoring refers to continuously monitoring the sewage outlet during the day and night using different methods at different times.

[0065] In this embodiment, during the daytime, sewage discharge information is identified in real time using Model B. The sewage discharge information includes the location information of the sewage outlet, abnormal temperature data of the discharged sewage body, water quality characteristic association information related to the sewage discharge behavior, and dynamic monitoring data of the sewage discharge behavior. During the nighttime, video monitoring devices deployed at the sewage outlet collect thermal infrared video of the sewage outlet through thermal infrared cameras. Electronic devices identify the thermal infrared video of the sewage outlet using a threshold segmentation method, and simultaneously obtain historical data to assist in verification, thereby identifying sewage discharge information in real time.

[0066] It should be noted that the multi-method monitoring fills the gap in nighttime visible light recognition, achieving full-time coverage.

[0067] As an example, during the day, Model B identifies intermittent sewage discharge from the outlet, while at night, it accurately captures the sewage that is continuously discharged from the outlet and has a temperature difference with the surrounding water.

[0068] Based on the above steps, uninterrupted 24-hour monitoring of sewage outlets can be achieved, ensuring the timeliness and completeness of monitoring.

[0069] Based on the above technical solutions, the problems of insufficient ground stations, limited coverage / accuracy of single methods, and lack of nighttime monitoring are solved by combining air-space-ground collaboration and model linkage. This achieves a closed loop from large-scale screening and high-precision positioning of river sewage outlets to real-time monitoring at all times, significantly improving the efficiency and accuracy of investigation.

[0070] In one possible approach, combining the above... Figure 3 ,like Figure 8 As shown, the specific process of threshold segmentation using K-means clustering in S306 above can be implemented through the following S801-S804: S801. Select multiple random sample values ​​from the pixel temperature values ​​of the first thermal infrared image as the initial cluster centroids.

[0071] The initial cluster centroid is the starting reference center for clustering and classification using the K-means algorithm.

[0072] In this embodiment of the application, the electronic device randomly selects a preset number of sample values ​​from all pixel temperature values ​​of the first thermal infrared image as the initial cluster centroid.

[0073] It should be noted that the preset number can be adjusted according to the temperature distribution characteristics of the image, usually 2-3 categories.

[0074] As an example, the electronic device randomly selects 23°C and 31°C as the initial centroid from images with pixel temperatures ranging from 19°C to 38°C.

[0075] Based on the above steps, the initial clustering benchmark is determined to ensure the orderly conduct of subsequent segmentation processes.

[0076] S802. Calculate the distance between each pixel temperature value and each initial centroid based on the Euclidean distance, and classify the pixel temperature value into the category of the nearest centroid.

[0077] Euclidean distance is a measure of the difference between pixel temperature value and centroid value.

[0078] In this embodiment of the application, the electronic device calculates the distance between the temperature value of each pixel in the first thermal infrared image and each initial centroid, and assigns the pixel temperature value to the category of the nearest centroid.

[0079] Optionally, the Euclidean distance calculation satisfies the following formula:

[0080] in, , Where n is the total number of pixels in the first thermal infrared image, K is the number of clusters, and m is the feature dimension. Let i be the temperature value of the i-th pixel in the first thermal infrared image. This represents the temperature value of the j-th initial cluster centroid in the K-means clustering algorithm. Let k be the feature value of the i-th pixel. Let be the k-th eigenvalue of the j-th centroid.

[0081] As an example, if the pixel temperature is 26℃, the initial centroid is 23℃ and 31℃, and m=1, then... , Electronic devices categorize it as belonging to the 23°C category.

[0082] Based on the above steps, a preliminary classification of pixel temperature is achieved.

[0083] S803. Calculate the average of the pixel temperature values ​​in each cluster category to obtain the updated centroid of that category.

[0084] The updated centroid is a value that reflects the temperature concentration trend of the cluster categories.

[0085] In this embodiment, the electronic device calculates the arithmetic mean of the temperature values ​​of all pixels within each cluster category to obtain the updated centroid of that category.

[0086] Optionally, the centroid update process satisfies the following formula:

[0087] in, , The total number of pixels contained in cluster category G. The updated centroid of class G.

[0088] As an example, category G contains pixels with temperatures of 21°C, 23°C, and 25°C. =3, then ℃, the electronic equipment will use 23℃ as the updated centroid.

[0089] Based on the above steps, centroids that fit the category features are obtained, thus improving clustering accuracy.

[0090] S804. Repeat S802 and S803 until the values ​​of all cluster centroids no longer change.

[0091] In this embodiment, the electronic device repeatedly performs distance calculation and category division in S802 and centroid update in S803 until all cluster centroid values ​​remain unchanged for two consecutive iterations.

[0092] It should be noted that the number of iterations is generally no more than 10, in order to balance efficiency and convergence.

[0093] As an example, after four iterations, the centroids of the two types stabilized at 23℃ and 30℃, and the electronic device stopped iterating.

[0094] Based on the above steps, clustering convergence is completed, and stable temperature category classification results are output.

[0095] Based on the above technical solution, the K-means clustering threshold segmentation process, combined with the Euclidean distance and centroid update formula, is used to achieve accurate calculation and obtain a stable temperature category classification. This provides a reliable basis for determining the second temperature anomaly region in the first thermal infrared image and ensures the accuracy of the initial location of the sewage outlet.

[0096] In one possible approach, combining the above... Figure 3 ,like Figure 9 As shown, after S307 monitors the location of the sewage outlet day and night, the method also includes S901: S901. Use the sewage discharge information and the monitoring results of model B as the second sample set to supplement the first sample set, and train and optimize model A.

[0097] In this embodiment of the application, the electronic device collects the sewage discharge information obtained by the day and night monitoring of S307 and the sewage discharge period and pollution type identified by model B to form a second sample set; the second sample set is added to the first sample set and input into the random forest algorithm for retraining to obtain the optimized model A.

[0098] It should be noted that this process forms a closed loop of monitoring and optimization, which continuously improves the accuracy of model A.

[0099] As an example, the first sample set of a certain watershed contains water quality data from 10 ground stations, supplemented by monitoring results from 20 sewage outlets (Model B), and then the data are combined to train Model A.

[0100] Based on the above technical solutions, Model A will be continuously optimized to enhance the accuracy of water pollution inversion in large areas, making large-scale upstream screening more precise.

[0101] The above primarily describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as the integrated air-space-ground river discharge outlet collaborative monitoring device, includes at least one of the hardware structures and software modules corresponding to each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0102] This application embodiment can divide the integrated air-space-ground river discharge outlet collaborative monitoring device into functional units according to the above method example. For example, each function can be divided into separate functional units, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or software. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0103] When using integrated units, Figure 10 A possible structural schematic diagram of the integrated air-space-ground river discharge outlet collaborative monitoring device (referred to as integrated air-space-ground river discharge outlet collaborative monitoring device 100) involved in the above embodiments is shown. The integrated air-space-ground river discharge outlet collaborative monitoring device 100 includes a processing unit 1001 and a communication unit 1002, and may also include a storage unit 1003. Figure 10 The schematic diagram shown can be used to illustrate the structure of the integrated air-space-ground river discharge outlet collaborative monitoring device involved in the above embodiments.

[0104] when Figure 10 The schematic diagram shown illustrates the structure of the integrated air-space-ground river discharge outlet collaborative monitoring device involved in the above embodiments. The processing unit 1001 is used to control and manage the operation of the integrated air-space-ground river discharge outlet collaborative monitoring device. The communication unit 1002 is used for the integrated air-space-ground river discharge outlet collaborative monitoring device to communicate with other devices. The storage unit 1003 is used to store the program code and data of the integrated air-space-ground river discharge outlet collaborative monitoring device.

[0105] For example, communication unit 1002 is used to acquire remote sensing images of the first area; Processing unit 1001 is used to preprocess remote sensing images to obtain orthorectified remote sensing images; using a remote sensing water index combined with a deep learning model, the water area vector boundary of the area to be monitored is extracted from the orthorectified remote sensing images; using the water area vector boundary as a mask, the orthorectified remote sensing images are inverted using model A, which is used to identify water pollution, to obtain a first region; threshold segmentation is performed on the first thermal infrared image to determine a second region, and the location of the sewage outlet is verified and confirmed in conjunction with the first visible light image; the location of the sewage outlet is monitored day and night; the day and night monitoring process satisfies the following process: during the daytime, sewage discharge information is identified using model B, which is used to identify video; during the nighttime, sewage discharge information is identified based on the temperature threshold comparison of the thermal infrared images of the sewage outlet.

[0106] In one possible implementation, the processing unit 1001 is further configured to extract the water body vector boundary of the monitored area from orthorectified remote sensing imagery by combining remote sensing water body indices with a deep learning model. This includes: calculating the NDWI index based on the green band and near-infrared band spectral values ​​of the orthorectified remote sensing imagery; setting an NDWI index threshold to perform preliminary water body extraction on the orthorectified remote sensing imagery, obtaining preliminary water body extraction vector patterns; post-processing the vector patterns to remove small areas smaller than a set area threshold; converting the post-processed vector patterns into raster data as a training sample set for the deep learning model; training the deep learning model using the training sample set; applying the trained deep learning model to the entire area of ​​the orthorectified remote sensing imagery for accurate water body extraction; and converting the extracted water body results for the entire area into a vector format to obtain the water body vector boundary.

[0107] In one possible implementation, model A is obtained by training a random forest model based on a first sample set and a feature set. This includes: cleaning the first sample set; setting the core parameters of the random forest algorithm; using the cleaned first sample set as training labels and the feature set as input features to train each independent decision tree separately; and generating the final comprehensive decision tree by averaging the decision tree output results to obtain model A for identifying water pollution.

[0108] In one possible implementation, the processing unit 1001 is further configured to use the water area vector boundary as a mask to invert the orthorectified remote sensing image through a model A used to identify water pollution, thereby obtaining a first region. This includes: using the water area vector boundary as a mask, aligning it with the orthorectified remote sensing image, cropping and retaining the river water area image within the mask, and removing non-water areas; inputting the cropped water area image into model A, identifying polluted areas based on the image band spectral values ​​and remote sensing water quality monitoring index, and converting the identification results into vector format to obtain the first region.

[0109] In one possible implementation, the communication unit 1002 is also used to acquire a first thermal infrared image and a first visible light image; In one possible implementation, the processing unit 1001 is further configured to perform threshold segmentation on the first thermal infrared image to determine the second region, and verify and confirm the location of the sewage outlet in conjunction with the first visible light image, including: performing histogram statistical analysis on the temperature values ​​of the first thermal infrared image, performing threshold segmentation using K-means clustering, and locating the second region with abnormal temperature; retrieving the first visible light image at the same time and location as the first thermal infrared image, and verifying and confirming the specific location of the sewage outlet within the second region in conjunction with its shape and texture information.

[0110] In one possible implementation, processing unit 1001 is further configured to perform threshold segmentation using K-means clustering, including: S1. Select multiple random sample values ​​from the pixel temperature values ​​of the first thermal infrared image as the initial cluster centroids; S2. Calculate the distance between each pixel temperature value and each initial centroid based on the Euclidean distance, and assign the pixel temperature value to the category of the nearest centroid; the distance satisfies the following formula:

[0111] in, , Where n is the total number of pixels in the first thermal infrared image, K is the number of clusters, and m is the feature dimension. Let i be the temperature value of the i-th pixel in the first thermal infrared image. This represents the temperature value of the j-th initial cluster centroid in the K-means clustering algorithm. Let k be the feature value of the i-th pixel. Let k be the eigenvalue of the j-th centroid; S3. Calculate the average temperature value of all pixels in each cluster to obtain the updated centroid of that cluster. The centroid update process satisfies the following formula:

[0112] in, , The total number of pixels contained in cluster category G. The updated centroid for category G; S4. Repeat S2 and S3 until the values ​​of all cluster centroids no longer change.

[0113] In one possible implementation, the communication unit 1002 is also used to acquire thermal infrared images and visible light images of the sewage outlet. In one possible implementation, model B is obtained by combining deep learning based on visible light images of the sewage outlet with threshold recognition based on thermal infrared images of the sewage outlet, including: setting a temperature threshold; the temperature threshold satisfying... ,in Here, x represents the temperature threshold, and x is the highest temperature in the thermal infrared image of the sewage outlet corresponding to the unpolluted water body at the same time the previous day at the current monitoring point. The temperature at the current time. The temperature was set for the same period the previous day. Frame-by-frame analysis of thermal infrared images captured by cameras at the sewage outlet locations was performed, marking areas with pixel temperature values ​​exceeding a set temperature threshold as polluted areas, thus creating polluted area labeling samples. These polluted area labeling samples were then spatially and temporally aligned with visible light images captured by cameras at the same time and location, ensuring that corresponding areas in the visible light images carried polluted area labeling information, thus constructing a training dataset for the deep learning model. The training dataset was used to train a deep learning recognition model for visible light images. After the model training was completed, the visible light deep learning model was superimposed with the temperature threshold judgment logic of the thermal infrared images to form Model B for video recognition.

[0114] In one possible implementation, the processing unit 1001 is further configured to, after monitoring the location of the sewage outlet day and night, supplement the first sample set with the sewage discharge information and the monitoring results of model B as a second sample set, and train and optimize model A.

[0115] In one possible implementation, the remote sensing water quality monitoring indices in the feature set include the Normalized Difference Vegetation Index (NDVI), the Black and Odorous Water Index (BOI), the Normalized Difference Suspended Solids Index (NDSSI), the Normalized Turbidity Index (NDTI), the Dual-Band Water Index (DBWI), and the Water Cleanliness Index (WCI).

[0116] The processing unit 1001 can be a processor or a controller, and the communication unit 1002 can be a communication interface, transceiver, transceiver circuit, transceiver device, etc. The term "communication interface" is a general term and may include one or more interfaces. The storage unit 1003 can be a memory. When the integrated air-space-ground river discharge outlet collaborative monitoring device 100 is a chip, the processing unit 1001 can be a processor or a controller, and the communication unit 1002 can be an input interface and / or an output interface, pins, or circuits, etc. The storage unit 1003 can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip (e.g., read-only memory (ROM), random access memory (RAM, etc.).

[0117] The communication unit can also be called a transceiver unit. The antenna and control circuit with transceiver functions in the integrated air-space-ground river discharge outlet collaborative monitoring device 100 can be considered as the communication unit 1002 of the integrated air-space-ground river discharge outlet collaborative monitoring device 100, and the processor with processing functions can be considered as the processing unit 1001 of the integrated air-space-ground river discharge outlet collaborative monitoring device 100. Optionally, the device in the communication unit 1002 that implements the receiving function can be considered as the communication unit, which is used to execute the receiving steps in the embodiments of this application. The communication unit can be a receiver, a receiver circuit, etc. The device in the communication unit 1002 that implements the transmitting function can be considered as the transmitting unit, which is used to execute the transmitting steps in the embodiments of this application. The transmitting unit can be a transmitter, a transmitter, a transmitting circuit, etc.

[0118] Figure 10 If the integrated units in the process are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. Storage media for storing computer software products include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0119] Figure 10 The units in the process can also be called modules; for example, a processing unit can be called a processing module.

[0120] This application also provides a hardware structure diagram of a river discharge outlet collaborative monitoring device based on integrated air-space-ground communication (referred to as integrated air-space-ground river discharge outlet collaborative monitoring device 110), see [link to diagram]. Figure 11 The integrated air-space-ground river discharge outlet collaborative monitoring device 110 includes a processor 1101, and optionally, a memory 1102 connected to the processor 1101.

[0121] In the first possible implementation, see Figure 11The integrated air-space-ground river discharge outlet collaborative monitoring device 110 also includes a transceiver 1103. The processor 1101, memory 1102, and transceiver 1103 are connected via a bus. The transceiver 1103 is used to communicate with other devices or communication networks. Optionally, the transceiver 1103 may include a transmitter and a receiver. The device in the transceiver 1103 that implements the receiving function can be considered as a receiver, which is used to perform the receiving steps in the embodiments of this application. The device in the transceiver 1103 that implements the transmitting function can be considered as a transmitter, which is used to perform the transmitting steps in the embodiments of this application.

[0122] Based on the first possible implementation method Figure 11 The schematic diagram shown can be used to illustrate the structure of the integrated air-space-ground river discharge outlet collaborative monitoring device involved in the above embodiments.

[0123] in, Figure 11 This can also be illustrated by the system chip in a space-air-ground integrated river discharge outlet collaborative monitoring device. In this case, the actions performed by the aforementioned space-air-ground integrated river discharge outlet collaborative monitoring device can be implemented by this system chip. The specific actions performed can be found above and will not be repeated here.

[0124] In implementation, each step of the method provided in this embodiment can be completed by integrated logic circuits in the processor hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0125] The processor in this application may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., which are various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor may be a separate semiconductor chip or integrated with other circuits into a single semiconductor chip. For example, it may be integrated with other circuits (such as encoding / decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a SoC (System-on-a-Chip), or it may be integrated as a built-in processor within an ASIC. The ASIC with the integrated processor may be packaged separately or together with other circuits. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0126] The memory in the embodiments of this application may include at least one of the following types: read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; or electrically erasable programmable-only memory (EEPROM). In some scenarios, the memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto.

[0127] This application also provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0128] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0129] This application also provides a chip including a processor and an interface circuit. The interface circuit is coupled to the processor. The processor is used to run computer programs or instructions to implement the above-described method. The interface circuit is used to communicate with other modules outside the chip.

[0130] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0131] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0132] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A collaborative monitoring method for river sewage outlets based on an integrated air-space-ground system, characterized in that, include: Acquire remote sensing images of the area to be monitored, and preprocess the remote sensing images to obtain orthophoto images; The water body vector boundary of the area to be monitored is extracted from the orthophoto image by combining remote sensing water index with a deep learning model. Using the water area vector boundary as a mask, the orthorectified remote sensing image is inverted using model A, which is used to identify water pollution, to obtain a first region; wherein, model A is obtained by training a random forest model based on a first sample set and a feature set; the first sample set is the measured data of ground water quality monitoring stations in the first region; the feature set is the band spectral values ​​and remote sensing water quality monitoring index of the orthorectified remote sensing image; Threshold segmentation is performed on the first thermal infrared image to determine the second region, and the location of the sewage outlet is verified and confirmed in conjunction with the first visible light image; the first thermal infrared image and the first visible light image are thermal infrared and visible light images obtained from aerial photography of the first region. The location of the sewage outlet is monitored day and night; the day and night monitoring process meets the following requirements: during the daytime, sewage information is identified by Model B, which is used to identify videos; during the nighttime, sewage information is identified based on the temperature threshold comparison of the sewage outlet's thermal infrared image; wherein, Model B is obtained by combining deep learning of the sewage outlet's visible light image with threshold identification of the sewage outlet's thermal infrared image.

2. The method according to claim 1, characterized in that, The method employs a remote sensing water index combined with a deep learning model to extract the water vector boundary of the area to be monitored from the orthophoto image, including: The NDWI index is calculated based on the green band and near-infrared band spectral values ​​of the orthophoto remote sensing image. Set an NDWI index threshold to perform preliminary water body extraction on orthophoto remote sensing images to obtain preliminary water body extraction vector patterns; perform post-processing on the vector patterns to remove small areas with an area smaller than a set area threshold. The post-processed vector graphics are converted into raster data, which is used as the training sample set for the deep learning model. A deep learning model is trained using a training sample set. The trained deep learning model is then applied to the entire area of ​​orthophoto remote sensing imagery to accurately extract water bodies. The extracted water body results are then converted into vector format to obtain the vector boundaries of the water bodies.

3. The method according to claim 1, characterized in that, The process of training a random forest model based on the first sample set and feature set to obtain model A includes: Perform data cleaning on the first sample set; Define the core parameters of the random forest algorithm; The first cleaned sample set is used as the training label, and the feature set is used as the input features to train each independent decision tree. The final comprehensive decision tree is generated by using the average decision tree output method, resulting in Model A for identifying water pollution.

4. The method according to claim 1, characterized in that, Using the water area vector boundary as a mask, the orthorectified remote sensing image is inverted using model A, which is used to identify water pollution, to obtain a first region, including: The water area vector boundary is used as a mask, aligned with the orthophoto image, and the river water area image within the mask is cropped and retained, while non-water areas are removed. The cropped water area image is input into model A. Based on the image band spectral values ​​and remote sensing water quality monitoring index, the polluted area is identified. The identification results are converted into vector format to obtain the first region.

5. The method according to claim 1, characterized in that, Threshold segmentation is performed on the first thermal infrared image to determine the second region. The location of the sewage outlet is then verified and confirmed in conjunction with the first visible light image, including: Histogram statistical analysis was performed on the temperature values ​​of the first thermal infrared image, and threshold segmentation was performed using K-means clustering to locate the second region of temperature anomaly. Retrieve a first visible light image from the same time period and location as the first thermal infrared image, and combine its shape and texture information to verify and confirm the specific location of the sewage outlet in the second area.

6. The method according to claim 5, characterized in that, The threshold segmentation using K-means clustering includes: S1. Select multiple random sample values ​​from the pixel temperature values ​​of the first thermal infrared image as the initial cluster centroids; S2. Calculate the distance between each pixel temperature value and each initial centroid based on the Euclidean distance, and assign the pixel temperature value to the category of the nearest centroid; the distance satisfies the following formula: in, , Where n is the total number of pixels in the first thermal infrared image, K is the number of clusters, and m is the feature dimension. Let i be the temperature value of the i-th pixel in the first thermal infrared image. This represents the temperature value of the j-th initial cluster centroid in the K-means clustering algorithm. Let k be the feature value of the i-th pixel. Let k be the eigenvalue of the j-th centroid; S3. Calculate the average temperature value of all pixels in each cluster to obtain the updated centroid of that cluster. The centroid update process satisfies the following formula: in, , The total number of pixels contained in cluster category G. The updated centroid for category G; S4. Repeat S2 and S3 until the values ​​of all cluster centroids no longer change.

7. The method according to claim 1, characterized in that, The model B obtained by combining deep learning based on visible light images of sewage outlets with threshold recognition of thermal infrared images of sewage outlets includes: Set a temperature threshold; the temperature threshold satisfies ,in Let x be the temperature threshold, and x be the highest temperature in the thermal infrared image of the sewage outlet corresponding to the unpolluted water body at the same time the previous day at the current monitoring point. The temperature at the current time. The temperature was at the same time the previous day. Frame-by-frame analysis of thermal infrared images captured by cameras at sewage outlet locations is performed, and areas with pixel temperature values ​​greater than a set temperature threshold are marked as polluted areas, forming polluted area marking samples. The polluted area labeled samples are spatially and temporally aligned with visible light images captured by cameras at the same time and location, so that the corresponding areas in the visible light images carry the polluted area labeling information, and a training dataset for the deep learning model is constructed. A deep learning recognition model for visible light images is trained using a training dataset. After the model is trained, the visible light deep learning model is superimposed with the temperature threshold judgment logic of thermal infrared images to form model B for video recognition.

8. The method according to claim 1, characterized in that, After monitoring the location of the sewage outlet day and night, the method further includes supplementing the first sample set with the sewage discharge information and the monitoring results of model B as a second sample set, and training and optimizing model A.

9. The method according to claim 1, characterized in that, The remote sensing water quality monitoring indices in the feature set include Normalized Difference Vegetation Index (NDVI), Black and Odorous Water Index (BOI), Normalized Difference Suspended Solids Index (NDSSI), Normalized Difference Turbidity Index (NDTI), Dual-Band Water Index (DBWI), and Water Cleanliness Index (WCI).

10. A collaborative monitoring device for river sewage outlets based on an integrated air-space-ground system, characterized in that, The device includes: a communication unit and a processing unit; The communication unit is used to acquire remote sensing images of the area to be monitored; acquire a first thermal infrared image and a first visible light image; acquire thermal infrared images and visible light images of the sewage outlet; The processing unit is used to preprocess the remote sensing image to obtain an orthorectified remote sensing image; extract the water area vector boundary of the area to be monitored from the orthorectified remote sensing image using a remote sensing water index combined with a deep learning model; use the water area vector boundary as a mask to invert the orthorectified remote sensing image using a model A for identifying water pollution to obtain a first region; perform threshold segmentation on the first thermal infrared image to determine a second region, and verify the location of the sewage outlet by combining it with the first visible light image; monitor the location of the sewage outlet day and night; the day and night monitoring process satisfies the following process: during the daytime, sewage discharge information is identified using a model B for identifying video; during the nighttime, sewage discharge information is identified based on the temperature threshold comparison of the thermal infrared image of the sewage outlet.

Images