A space-air-ground integrated river pollution outlet collaborative monitoring method and device

Abstract

The application provides a river sewage outlet cooperative monitoring method and device based on space-air-ground integration, relates to the technical field of remote sensing, and solves the technical problem that the prior art cannot simultaneously realize large-range, high-precision and full-period monitoring of river sewage outlets. The method specifically comprises the following steps: acquiring remote sensing image data of a first region, and obtaining orthographic remote sensing image data through preprocessing; extracting a water area vector boundary of the first region by using a remote sensing water body index combined with deep learning, training a model A for identifying water body pollution in combination with ground water quality data and remote sensing characteristics, and inversely deriving a second region; controlling a dual-camera unmanned aerial vehicle to take aerial photographs of the second region, and confirming a sewage outlet position by K-means clustering combined with visible light image data; taking a hot infrared over-threshold area of the sewage outlet camera as a sample, training a model B for identifying videos; and monitoring the sewage outlet day and night based on the model B and second infrared image data, and identifying sewage information in real time. The application is used for monitoring river sewage outlets.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of remote sensing, and in particular to a river sewage outlet cooperative monitoring method and device based on space-air-ground integration. BACKGROUND

[0002] The river sewage outlet is a key node for pollutants to enter the surface water body, and its monitoring is of great significance for protecting the ecological environment and ensuring human health. Traditional river sewage outlet monitoring relies on manual on-site checking and sampling analysis of pH value, chemical oxygen demand, ammonia nitrogen and other water quality indicators. With the development of technology, the existing technology CN114781537B discloses a method for constructing a sewage identification model based on high-resolution satellite remote sensing images to realize large-scale monitoring, and CN119313532A discloses a means for combining satellite remote sensing and unmanned aerial vehicle aerial survey to improve monitoring accuracy. Some schemes also attempt to introduce thermal infrared video monitoring to supplement the monitoring. However, the traditional manual method is slow and costly, and cannot meet the real-time and large-area monitoring requirements and is difficult to monitor night sewage. Although the combination of remote sensing and unmanned aerial vehicles can achieve large-scale and high-precision monitoring, it can only be carried out during the day, and pure video monitoring is also difficult to cover a large area. Therefore, the existing technology has the technical problem that it cannot simultaneously realize large-scale, high-precision and full-period monitoring of river sewage outlets. SUMMARY

[0003] The application provides a river sewage outlet cooperative monitoring method and device based on space-air-ground integration, which solves the technical problem that the existing technology cannot simultaneously realize large-scale, high-precision and full-period monitoring of river sewage outlets.

[0004] To achieve the above-mentioned purpose, the application adopts the following technical solutions:

[0005] In a first aspect, a remote sensing image of a to-be-monitored region is acquired, the remote sensing image is preprocessed to obtain an orthographic remote sensing image; a water area vector boundary of the to-be-monitored region is extracted from the orthographic remote sensing image by using a remote sensing water body index in combination with a deep learning model; the orthographic remote sensing image is inversed by using a model A for identifying water body pollution with the water area vector boundary as a mask to obtain a first region; the model A is obtained by training a random forest model based on a first sample set and a feature set; the first sample set is measured data of a ground water quality monitoring station in the first region; the feature set is a band spectral value of the orthographic remote sensing image and a remote sensing water quality monitoring index; a second region is determined by threshold segmentation on a first thermal infrared image, and a position of a pollution outlet is verified and confirmed in combination with a first visible light image; the first thermal infrared image and the first visible light image are thermal infrared and visible light images acquired by aerial photography on the first region; the position of the pollution outlet is monitored day and night; the day and night monitoring process satisfies the following processes: during a daytime period, pollution information is identified by using a model B for identifying videos; during a nighttime period, the pollution information is identified based on a temperature threshold comparison of a pollution outlet thermal infrared image; the model B is obtained by deep learning on a pollution outlet visible light image in combination with threshold identification on a pollution outlet thermal infrared image.

[0006] In combination with the first aspect, in a possible implementation manner, the water area vector boundary of the to-be-monitored region is extracted from the orthographic remote sensing image by using the remote sensing water body index in combination with the deep learning model, including: an NDWI index is calculated based on green band and near-infrared band spectral values of the orthographic remote sensing image; a threshold of the NDWI index is set, and water bodies are preliminarily extracted from the orthographic remote sensing image to obtain a preliminary water body extraction vector graph patch; the vector graph patch is post-processed to remove small regions with an area less than a set area threshold; the post-processed vector graph patch is converted into raster data as a training sample set of the deep learning model; the deep learning model is trained by using the training sample set, and the trained deep learning model is applied to the entire region of the orthographic remote sensing image to perform accurate water body extraction; the extracted water body result of the entire region is converted into a vector format to obtain the water area vector boundary.

[0007] In combination with the first aspect, in a possible implementation manner, the model A is obtained by training a random forest model based on the first sample set and the feature set, including: the first sample set is data cleaned; core parameters of the random forest algorithm are set; the cleaned first sample set is taken as a training label, and the feature set is taken as an input feature, and each independent decision tree is trained respectively; an average decision tree output result mode is used to generate a final comprehensive decision tree to obtain the model A for identifying water body pollution.

[0008] In conjunction with the first aspect mentioned above, in one possible implementation, using the water area vector boundary as a mask, the orthorectified remote sensing image is inverted using model A, which is used to identify water pollution, to obtain the first region. This includes: using the water area vector boundary as a mask, aligning it with the orthorectified remote sensing image, cropping and retaining the river water area image within the mask, and removing non-water areas; inputting the cropped water area image into model A, identifying polluted areas based on the image band spectral values ​​and remote sensing water quality monitoring index, and converting the identification results into vector format to obtain the first region.

[0009] In conjunction with the first aspect mentioned above, in one possible implementation, the first thermal infrared image is thresholded to determine the second region, and the location of the sewage outlet is verified and confirmed in conjunction with the first visible light image. This includes: performing histogram statistical analysis on the temperature values ​​of the first thermal infrared image, using K-means clustering for threshold segmentation, and locating the second region with abnormal temperature; retrieving the first visible light image from the same time period and location as the first thermal infrared image, and verifying and confirming the specific location of the sewage outlet within the second region in conjunction with its shape and texture information.

[0010] In conjunction with the first aspect mentioned above, one possible implementation involves using K-means clustering for threshold segmentation, including:

[0011] S1. Select multiple random sample values ​​from the pixel temperature values ​​of the first thermal infrared image as the initial cluster centroids;

[0012] S2. Calculate the distance between each pixel temperature value and each initial centroid based on the Euclidean distance, and assign the pixel temperature value to the category of the nearest centroid; the distance satisfies the following formula:

[0013]

[0014] in, , Where n is the total number of pixels in the first thermal infrared image, K is the number of clusters, and m is the feature dimension. Let i be the temperature value of the i-th pixel in the first thermal infrared image. This represents the temperature value of the j-th initial cluster centroid in the K-means clustering algorithm. Let k be the feature value of the i-th pixel. Let k be the eigenvalue of the j-th centroid;

[0015] S3. Calculate the average temperature value of all pixels in each cluster to obtain the updated centroid of that cluster. The centroid update process satisfies the following formula:

[0016]

[0017] in, , total number of pixels contained in the cluster category G, updated centroid of the category G;

[0018] S4, repeat S2 and S3 until the values of all cluster centroids no longer change.

[0019] In combination with the first aspect, in a possible implementation, the model B is obtained based on visible light image deep learning of the pollution outlet combined with threshold recognition of the pollution outlet thermal infrared image, and includes: setting a temperature threshold; the temperature threshold satisfies wherein the temperature threshold, x is the highest temperature in the thermal infrared image of the pollution outlet corresponding to the uncontaminated water body at the same period of the previous day at the current monitoring time point, the current time temperature, the temperature of the same period of the previous day; the thermal infrared image collected by the camera at the pollution outlet position is analyzed frame by frame, and the area with a pixel temperature value greater than the set temperature threshold is marked as a pollution area to form a pollution area marking sample; the pollution area marking sample and the visible light image collected by the camera at the same time and at the same position are spatially and temporally aligned, so that the corresponding area of the visible light image carries the pollution area marking information, and a training data set of the deep learning model is constructed; the deep learning recognition model of the visible light image is trained using the training data set, and after the model training is completed, the visible light deep learning model and the temperature threshold judgment logic of the thermal infrared image are superimposed to form the model B for identifying the video.

[0020] In combination with the first aspect, in a possible implementation, after the day and night monitoring of the pollution outlet position, the method further includes supplementing the pollution information and the monitoring result of the model B to the first sample set as a second sample set, and training and optimizing the model A.

[0021] In combination with the first aspect, in a possible implementation, the remote sensing water quality monitoring index in the feature set includes a normalized vegetation index NDVI, a black and odorous water body index BOI, a normalized suspended matter index NDSSI, a normalized turbidity index NDTI, a double-band water body index DBWI, and a water body cleaning index WCI.

[0022] In a second aspect, a river pollution outlet cooperative monitoring device based on space-air-ground integration is provided, comprising: a communication unit and a processing unit; the communication unit is configured to obtain remote sensing images of a region to be monitored; obtain a first thermal infrared image and a first visible light image; obtain a thermal infrared image and a visible light image of a pollution outlet; the processing unit is configured to preprocess the remote sensing images to obtain orthographic remote sensing images; extract a water area vector boundary of the region to be monitored from the orthographic remote sensing images by using a remote sensing water body index combined with a deep learning model; use the water area vector boundary as a mask, and perform inversion on the orthographic remote sensing images by using a model A for identifying water body pollution to obtain a first region; perform threshold segmentation on the first thermal infrared image to determine a second region, and verify and confirm the position of the pollution outlet in combination with the first visible light image; perform day and night monitoring on the position of the pollution outlet; the day and night monitoring process satisfies the following processes: during the daytime, identify pollution information by using a model B for identifying videos; during the night, identify pollution information based on temperature threshold comparison of the thermal infrared image of the pollution outlet.

[0023] In a third aspect, the present application provides a river pollution outlet cooperative monitoring device based on space-air-ground integration, comprising: a processor and a storage medium; the storage medium comprises instructions, and the processor is configured to run the instructions to implement the method described in the first aspect and any possible implementation manner of the first aspect. The river pollution outlet cooperative monitoring device based on space-air-ground integration can be an electronic device or a chip in an electronic device.

[0024] In a fourth aspect, the present application provides a river pollution outlet cooperative monitoring system based on space-air-ground integration, comprising: a remote sensing satellite, a unmanned aerial vehicle, a video monitoring device and an electronic device; wherein the remote sensing satellite is configured to obtain remote sensing images of a region to be monitored; the unmanned aerial vehicle is configured to obtain a first thermal infrared image and a first visible light image; the video monitoring device is configured to obtain a thermal infrared image and a visible light image of a pollution outlet; the electronic device is configured to preprocess the remote sensing images to obtain orthographic remote sensing images; extract a water area vector boundary of the region to be monitored from the orthographic remote sensing images by using a remote sensing water body index combined with a deep learning model; use the water area vector boundary as a mask, and perform inversion on the orthographic remote sensing images by using a model A for identifying water body pollution to obtain a first region; perform threshold segmentation on the first thermal infrared image to determine a second region, and verify and confirm the position of the pollution outlet in combination with the first visible light image; perform day and night monitoring on the position of the pollution outlet; the day and night monitoring process satisfies the following processes: during the daytime, identify pollution information by using a model B for identifying videos; during the night, identify pollution information based on temperature threshold comparison of the thermal infrared image of the pollution outlet.

[0025] In a fifth aspect, the present application provides a computer readable storage medium, which stores instructions. When the instructions are executed on the river sewage outlet collaborative monitoring device based on space-air-ground integration, the river sewage outlet collaborative monitoring device based on space-air-ground integration executes the method described in the first aspect and any possible implementation manner of the first aspect.

[0026] In a sixth aspect, the present application provides a computer program product containing instructions. When the computer program product is executed on the river sewage outlet collaborative monitoring device based on space-air-ground integration, the river sewage outlet collaborative monitoring device based on space-air-ground integration executes the method described in the first aspect and any possible implementation manner of the first aspect.

[0027] The present application provides a river sewage outlet collaborative monitoring method and device based on space-air-ground integration, which can realize large-range, high-precision and full-period monitoring of river sewage outlets through space-air-ground cooperation and double-model linkage. Remote sensing image preprocessing eliminates interference and provides accurate orthographic data for subsequent monitoring; water body index combined with deep learning extracts water area boundary, excludes shadow interference and ensures accurate boundary; model A relies on random forest and measured data to solve the problem of insufficient ground stations and realize large-range pollution inversion; the unmanned aerial vehicle takes aerial photography of the first area, and the double cameras provide temperature and visual data; K-means segmentation of thermal infrared images combined with visible light verification accurately locates the sewage outlet; model B fills in the night blank with day and night monitoring, comprehensively improves the monitoring efficiency and accuracy, and solves the technical problem that the prior art cannot simultaneously realize large-range, high-precision and full-period monitoring of river sewage outlets.

[0028] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in the present application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it can be understood that the description of a feature or beneficial effect means that the specific technical feature, technical solution or beneficial effect is included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in the specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the embodiments can be combined in any appropriate manner. Those skilled in the art will understand that the embodiments can be implemented without one or more specific technical features, technical solutions or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects can be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 A system architecture diagram of a river sewage outlet collaborative monitoring system based on space-air-ground integration provided by the embodiments of the present application is provided.

[0030] Figure 2 Another system architecture diagram of a river sewage outlet collaborative monitoring system based on space-air-ground integration provided by an embodiment of the present application;

[0031] Figure 3 A flowchart of a river sewage outlet collaborative monitoring method based on space-air-ground integration provided by an embodiment of the present application;

[0032] Figure 4 A remote sensing river water quality inversion and local zoomed-in image provided by an embodiment of the present application;

[0033] Figure 5 A visible light aerial photograph of a sewage outlet provided by an embodiment of the present application;

[0034] Figure 6 A thermal infrared image temperature-pixel number distribution diagram provided by an embodiment of the present application;

[0035] Figure 7 A thermal imaging river sewage outlet local image provided by an embodiment of the present application;

[0036] Figure 8 Another flowchart of a river sewage outlet collaborative monitoring method based on space-air-ground integration provided by an embodiment of the present application;

[0037] Figure 9 Another flowchart of a river sewage outlet collaborative monitoring method based on space-air-ground integration provided by an embodiment of the present application;

[0038] Figure 10 A structure diagram of a river sewage outlet collaborative monitoring device based on space-air-ground integration provided by an embodiment of the present application;

[0039] Figure 11 A hardware structure diagram of a river sewage outlet collaborative monitoring device based on space-air-ground integration provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] In the description of the present application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this document is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean that there are three cases of A alone, A and B together, and B alone. In addition, "at least one" means one or more, and "multiple" means two or more. "First", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. also do not necessarily mean different.

[0041] It should be noted that the words "exemplary" and "for example" are used herein to mean serving as an example, instance, or illustration. Any implementation or design scheme described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other implementations or design schemes. Rather, the goal is to present concepts in a concrete manner.

[0042] The method for monitoring a river pollution outlet based on space-air-ground integration provided by the embodiments of the present application can be applied to a system for monitoring a river pollution outlet based on space-air-ground integration as shown in the accompanying drawings. Figure 1 The system includes a remote sensing satellite 101, a UAV 102, a video monitoring device 103, and an electronic device 104.

[0043] The remote sensing satellite 101 is configured to acquire remote sensing images of a region to be monitored; the UAV 102 is configured to acquire first thermal infrared images and first visible light images; the video monitoring device 103 is configured to acquire thermal infrared images and visible light images of a pollution outlet; and the electronic device 104 is configured to preprocess the remote sensing images to obtain orthographic remote sensing images, extract a water area vector boundary of the region to be monitored from the orthographic remote sensing images by using a remote sensing water body index in combination with a deep learning model, perform inversion on the orthographic remote sensing images by using a model A for identifying water pollution with the water area vector boundary as a mask to obtain a first region, perform threshold segmentation on the first thermal infrared images to determine a second region, and verify and confirm a position of the pollution outlet in combination with the first visible light images; the electronic device 104 is further configured to perform day-and-night monitoring on the position of the pollution outlet; and the day-and-night monitoring process satisfies the following processes: during a daytime period, a model B for identifying videos is used to identify pollution information; and during a nighttime period, pollution information is identified based on a comparison of a temperature threshold of the thermal infrared images of the pollution outlet.

[0044] As an example, as shown in Figure 2As shown, the remote sensing satellite 101 first transmits the acquired remote sensing image of the to-be-monitored region to the electronic device 104; the electronic device 104 pre-processes the image, extracts the water area vector boundary, and trains the model A, obtains the second region through the inversion of the model A, generates the corresponding unmanned aerial vehicle flight route planning information and sends it to the unmanned aerial vehicle 102; the unmanned aerial vehicle 102 executes the aerial photography task on the second region according to the planned flight route, and transmits the collected first thermal infrared image and first visible light image to the electronic device 104; the electronic device 104 adopts the K-means clustering method to perform threshold segmentation on the first thermal infrared image, and verifies and confirms the location of the sewage outlet in combination with the first visible light image, at the same time, the video monitoring device 103 transmits the collected sewage outlet thermal infrared image and sewage outlet visible light image to the electronic device 104, the electronic device 104 marks the sample with the region with a temperature greater than a set threshold value in the sewage outlet thermal infrared image as a pollution area, and trains the model B for identifying the video based on the sewage outlet visible light image; in subsequent monitoring, the electronic device 104 identifies the sewage information in real time on the location of the sewage outlet through the model B during the day and the temperature threshold value comparison of the sewage outlet thermal infrared image at night, and can obtain historical monitoring data from the historical data storage module to assist in realizing the full-period tracking and accurate analysis of the sewage information.

[0045] To solve the technical problem that the prior art cannot simultaneously realize large-range, high-precision and full-period monitoring of river sewage outlets, the embodiments of the present application provide a river sewage outlet cooperative monitoring method based on space-ground integration, which comprises the following steps: acquiring a remote sensing image of a to-be-monitored river region and obtaining an orthographic remote sensing image through pre-processing, extracting a water area vector boundary through a remote sensing water body index in combination with deep learning, training a model A with ground water quality measurement data and remote sensing features and inverting a pollution region with the water area boundary as a mask, controlling an unmanned aerial vehicle carrying a thermal infrared and visible light camera to perform aerial photography on a first region, confirming a sewage outlet after K-means segmentation of a thermal infrared image to determine a suspected area in combination with visible light, training a model B with sample marking based on a thermal infrared threshold value of a sewage outlet camera, and finally monitoring the sewage outlet day and night through the model B during the day and the thermal infrared threshold value at night; based on this, the method simultaneously realizes large-range screening, high-precision positioning and full-period monitoring of river sewage outlets through space-ground cooperation and double-model linkage.

[0046] Figure 3 A flowchart of the river sewage outlet cooperative monitoring method based on space-ground integration provided by the embodiments of the present application is shown in Figure 3 , which comprises the following steps:

[0047] S301, acquiring a remote sensing image of a to-be-monitored region, pre-processing the remote sensing image, and obtaining an orthographic remote sensing image.

[0048] The remote sensing image is acquired by a remote sensing satellite.

[0049] In the embodiment of the present application, the electronic device receives the remote sensing image of the to-be-monitored area transmitted by the remote sensing satellite, and sequentially performs radiation calibration, geometric correction, orthographic correction, fusion, splicing, cropping and cloud removal processing to obtain the orthographic remote sensing image.

[0050] Based on the above steps, the preprocessing can eliminate sensor errors and environmental interference to obtain high-precision orthographic remote sensing images and provide reliable data for subsequent monitoring.

[0051] S302, the water area vector boundary of the to-be-monitored area is extracted from the orthographic remote sensing image by using a remote sensing water body index combined with a deep learning model.

[0052] The remote sensing water body index is a normalized water body index (NDWI).

[0053] In the embodiment of the present application, the electronic device calculates the NDWI index based on the green band and near-infrared band spectral values of the orthographic remote sensing image; sets the NDWI index threshold value to preliminarily extract the water body from the orthographic remote sensing image and obtain the preliminary water body extraction vector map patch; post-processes the vector map patch to remove small areas with an area less than a set area threshold value; converts the post-processed vector map patch into raster data as a training sample set of a deep learning model (such as U-Net); trains the deep learning model using the training sample set, applies the trained deep learning model to the entire area of the orthographic remote sensing image, and performs accurate water body extraction; and converts the extracted entire-area water body result into a vector format to obtain the water area vector boundary of the to-be-monitored area.

[0054] It should be noted that the specific type of the deep learning model is not limited in the present application.

[0055] As an example, the electronic device trains the U-Net model for the orthographic image of a certain urban river group, and extracts the water area vector boundary when the NDWI is greater than the NDWI index threshold value.

[0056] Based on the above steps, the accurate water body extraction of the to-be-monitored area is realized to delineate the range for pollution monitoring.

[0057] S303, a random forest model is trained based on the first sample set and the feature set to obtain a model A for identifying water body pollution.

[0058] The first sample set is the measured data of the ground water quality monitoring station in the first area, including pH value, chemical oxygen demand, ammonia nitrogen, total nitrogen and total phosphorus; the feature set is the band spectral value of the orthographic remote sensing image and the remote sensing water quality monitoring index; the remote sensing water quality monitoring index includes normalized vegetation index NDVI, black and odorous water body index BOI, normalized suspended matter index NDSSI, normalized turbidity index NDTI, double-band water body index DBWI and water body cleaning index WCI.

[0059] In the embodiment of the present application, the electronic device performs data cleaning on the first sample set, sets the core parameters of the random forest algorithm, including the number of independent decision trees to be constructed, takes the cleaned first sample set as the training label, takes the feature set as the input feature, trains each independent decision tree respectively, adopts the average decision tree output result mode, generates the final comprehensive decision tree, and obtains model A for identifying water pollution.

[0060] It should be noted that the selection of the remote sensing water quality monitoring index can be adjusted according to the focus of the verification area. Common water quality monitoring indexes of remote sensing multi-spectral images include: normalized vegetation index , black and odorous water body index , normalized suspended matter index , normalized turbidity index , dual-band water body index , water body cleaning index , wherein , , respectively represent the center wavelengths of the Blue, Red, and Green bands, NIR represents the near-infrared band spectrum value, and Red, Green, and Blue represent the red, green, and blue band spectrum values.

[0061] As an example, when the random forest model is trained, the number of decision trees is set to 100, and the depth of a single tree is set to 12.

[0062] Based on the above steps, the problem of insufficient ground stations is solved, and the demand for rapid inversion of large-area water pollution can be met.

[0063] S304, taking the water area vector boundary as a mask, performing inversion on the orthographic remote sensing image by using the model A for identifying water pollution to obtain a first area.

[0064] The first area is a key sub-area suspected of pollution after the model A inversion.

[0065] In the embodiment of the present application, the electronic device takes the water area vector boundary extracted in S302 as a mask, aligns with the orthographic remote sensing image, clips and retains the river water area image within the mask, and eliminates non-water area data; inputs the clipped water area image into the model A, identifies the pollution area based on the image band spectrum value and the remote sensing water quality monitoring index, converts the identification result into a vector format, and obtains the first area.

[0066] It should be noted that the mask operation excludes non-water area interference, so that the inversion focuses on the river.

[0067] As an example, as shown in Figure 4As shown, the electronic device uses the water area vector boundary of the area to be monitored as a mask. After cropping the orthophoto remote sensing image of the area, the retained river water area image is input into model A. The first area obtained by inversion corresponds to the local magnified display of the remote sensing image, accurately focusing on the river sub-area suspected of pollution, clearly showing the screening and focusing process from the large area to be monitored to the small first area.

[0068] Based on the above steps, a wide-range to small-range focusing is achieved, defining a precise range for drone aerial photography.

[0069] S305. Threshold segmentation is performed on the first thermal infrared image to determine the second region, and the location of the sewage outlet is verified and confirmed in conjunction with the first visible light image.

[0070] The second area is a suspected sewage outlet area with abnormal temperature in the thermal infrared image. The first thermal infrared image and the first visible light image are thermal infrared and visible light images obtained by drone aerial photography of the first area.

[0071] In this embodiment, the electronic device plans the drone flight path according to the first area, controls the drone equipped with a thermal infrared camera and a visible light camera to take aerial photos of the first area, performs histogram statistical analysis on the temperature value of the first thermal infrared image, uses K-means clustering to perform threshold segmentation, locates the second area with abnormal temperature, and then retrieves the first visible light image at the same time and location as the first thermal infrared image. Combining its shape and texture information, the specific location of the sewage outlet in the second area is verified and confirmed.

[0072] It should be noted that visible light verification eliminates misjudgments caused by thermal infrared interference with non-sewage discharge heat.

[0073] As an example, electronic devices plan a zigzag flight path for a 15km² area, such as... Figure 5 As shown, the drone synchronously collects and transmits images. The drone's thermal infrared camera can be a DJI Zenmuse series unit. Videos are shot in MP4 format, and photos are taken in JPEG format. The drone's thermal infrared images are processed using PHOTOScan software. Figure 6 As shown, the temperature-pixel count distribution of the first thermal infrared image, after K-means clustering, exhibits two peak clusters. Temperature pixel distribution in non-sewage discharge areas such as environmental water bodies The temperature pixel distribution corresponding to the high-temperature water flow at the sewage outlet; the electronic device uses K-means clustering to threshold segment the image, and... The area corresponding to the temperature range was identified as the second region. Further examination using the first visible light image from the same time period revealed that this region exhibited texture and shape characteristics characteristic of pipe drainage, such as... Figure 7 As shown, the area within the box represents the final confirmed location of the sewage outlet.

[0074] Based on the above steps, the precise positioning of the sewage outlet position provides a target for real-time monitoring.

[0075] S306, based on the sewage outlet visible light image depth learning combined with the sewage outlet thermal infrared image threshold recognition to obtain a model B for identifying the video.

[0076] Wherein, the sewage outlet thermal infrared image and the sewage outlet visible light image are images collected by the sewage outlet video monitoring device.

[0077] In the embodiment of the present application, the electronic device sets a temperature threshold, receives the sewage outlet thermal infrared image transmitted by the video monitoring device, and screens the area with a temperature greater than the set threshold as a pollution area label sample; the pollution area label sample and the sewage outlet visible light image collected by the camera at the same time and at the same position are aligned in space and time, so that the corresponding area of the visible light image carries the pollution area label information, and a training data set of the deep learning model is constructed; the deep learning identification model of the visible light image is trained using the training data set, and after the model training is completed, the visible light deep learning model and the temperature threshold judgment logic of the sewage outlet thermal infrared image are superimposed to form a model B for identifying the video.

[0078] Optionally, the temperature threshold satisfies , wherein is the temperature threshold, x is the highest temperature in the sewage outlet thermal infrared image corresponding to the unpolluted water body at the same time of the previous day at the current monitoring time point, is the current air temperature, is the air temperature at the same time of the previous day.

[0079] As an example, the video monitoring device of the sewage outlet continuously collects sewage outlet thermal infrared and visible light images, the electronic device first sets a temperature threshold (current air temperature 28℃, air temperature at the same time of the previous day 26℃, highest temperature of the unpolluted water body at the previous day 25℃, then the temperature threshold is 25.02℃), and screens out the area with a temperature greater than 25.02℃ in the sewage outlet thermal infrared image as a pollution area label sample; the label sample and the sewage outlet visible light image at the same time and at the same position are aligned to construct a training set, and the YOLO deep learning model is trained using the training set, and after the model training is completed, the thermal infrared temperature threshold judgment logic is superimposed to obtain a model B for identifying the video.

[0080] Based on the above steps, an identification model suitable for the visual features of the sewage outlet is obtained.

[0081] S307, day and night monitoring of the sewage outlet position.

[0082] Wherein, the day and night monitoring means continuously monitoring the sewage outlet in different time periods during the day and at night.

[0083] In this embodiment, during the daytime, sewage discharge information is identified in real time using Model B. The sewage discharge information includes the location information of the sewage outlet, abnormal temperature data of the discharged sewage body, water quality characteristic association information related to the sewage discharge behavior, and dynamic monitoring data of the sewage discharge behavior. During the nighttime, video monitoring devices deployed at the sewage outlet collect thermal infrared video of the sewage outlet through thermal infrared cameras. Electronic devices identify the thermal infrared video of the sewage outlet using a threshold segmentation method, and simultaneously obtain historical data to assist in verification, thereby identifying sewage discharge information in real time.

[0084] It should be noted that the multi-method monitoring fills the gap in nighttime visible light recognition, achieving full-time coverage.

[0085] As an example, during the day, Model B identifies intermittent sewage discharge from the outlet, while at night, it accurately captures the sewage that is continuously discharged from the outlet and has a temperature difference with the surrounding water.

[0086] Based on the above steps, uninterrupted 24-hour monitoring of sewage outlets can be achieved, ensuring the timeliness and completeness of monitoring.

[0087] Based on the above technical solutions, the problems of insufficient ground stations, limited coverage / accuracy of single methods, and lack of nighttime monitoring are solved by combining air-space-ground collaboration and model linkage. This achieves a closed loop from large-scale screening and high-precision positioning of river sewage outlets to real-time monitoring at all times, significantly improving the efficiency and accuracy of investigation.

[0088] In one possible approach, combining the above... Figure 3 ,like Figure 8 As shown, the specific process of threshold segmentation using K-means clustering in S306 above can be implemented through the following S801-S804:

[0089] S801. Select multiple random sample values ​​from the pixel temperature values ​​of the first thermal infrared image as the initial cluster centroids.

[0090] The initial cluster centroid is the starting reference center for clustering and classification using the K-means algorithm.

[0091] In this embodiment of the application, the electronic device randomly selects a preset number of sample values ​​from all pixel temperature values ​​of the first thermal infrared image as the initial cluster centroid.

[0092] It should be noted that the preset number can be adjusted according to the temperature distribution characteristics of the image, usually 2-3 categories.

[0093] As an example, the electronic device randomly selects 23°C and 31°C as the initial centroid from images with pixel temperatures ranging from 19°C to 38°C.

[0094] Based on the above steps, the clustering starting reference is determined to ensure the orderly development of the subsequent segmentation process.

[0095] S802, calculate the distance between each pixel temperature value and each initial centroid according to the Euclidean distance, and assign the pixel temperature value to the class to which the nearest centroid belongs.

[0096] The Euclidean distance is a measure of the difference between the pixel temperature value and the centroid value.

[0097] In the embodiment of the application, the electronic device calculates the distance between each pixel temperature value of the first thermal infrared image and each initial centroid, and assigns the pixel temperature value to the class to which the nearest centroid belongs.

[0098] Optionally, the Euclidean distance calculation satisfies the following formula:

[0099]

[0100] wherein, , n is the total number of pixels of the first thermal infrared image, K is the number of clustering categories, m is the feature dimension, is the temperature value of the i-th pixel in the first thermal infrared image, is the temperature value of the j-th initial clustering centroid in the K-means clustering algorithm, is the k-th feature value of the i-th pixel, is the k-th feature value of the j-th centroid.

[0101] As an example, the pixel temperature is 26℃, the initial centroid is 23℃ and 31℃, and m=1, then , The electronic device assigns it to the class to which 23℃ belongs.

[0102] Based on the above steps, the pixel temperature is preliminarily classified.

[0103] S803, calculate the average of all pixel temperature values in each clustering category to obtain the updated centroid of the category.

[0104] The updated centroid is a value reflecting the temperature trend of the clustering category.

[0105] In the embodiment of the application, the electronic device calculates the arithmetic mean of all pixel temperature values in each clustering category to obtain the updated centroid of the category.

[0106] Optionally, the centroid updating process satisfies the following formula:

[0107]

[0108] wherein, , total number of pixels contained in the cluster category G, the updated centroid of the category G.

[0109] As an example, the category G contains 21℃, 23℃, 25℃ pixels, = 3, then 23℃, the electronic device takes 23℃ as the updated centroid.

[0110] Based on the above steps, the centroid fitting the category feature is obtained, and the clustering accuracy is improved.

[0111] S804, repeat S802 and S803 until the values of all cluster centroids no longer change.

[0112] In the embodiments of the present application, the electronic device repeatedly performs the distance calculation and category division of S802 and the centroid update of S803 until the values of all cluster centroids do not change continuously for two iterations.

[0113] It should be noted that the number of iterations is generally not more than 10, balancing efficiency and convergence effect.

[0114] As an example, after 4 iterations, the two category centroids are stabilized at 23℃ and 30℃, and the electronic device stops iteration.

[0115] Based on the above steps, the clustering convergence is completed, and the stable temperature category division result is output.

[0116] Based on the above technical solution, through the K-means clustering threshold segmentation process, combined with the Euclidean distance and the centroid update formula, accurate calculation is realized, and stable temperature category division is obtained, which provides a reliable basis for determining the second region of temperature anomaly in the first thermal infrared image, and ensures the accuracy of the preliminary positioning of the pollution outlet.

[0117] In one possible way, combined with the above Figure 3 As shown in FIG. 7, after S307 day and night monitoring of the pollution outlet position, the method further includes S901: Figure 9

[0118] S901, taking the pollution information and the monitoring result of the model B as a second sample set, supplementing to the first sample set, training and optimizing the model A.

[0119] In the embodiments of the present application, the electronic device collects the pollution information obtained by S307 day and night monitoring and the pollution period, pollution type and the like identified by the model B, to form a second sample set; the second sample set is supplemented to the first sample set, and input into the random forest algorithm for retraining, to obtain the optimized model A.

[0120] It should be noted that this process forms a monitoring and optimization closed loop, so that the accuracy of the model A is continuously improved.​

[0121] As an example, a first sample set of a certain river basin contains 10 ground station water quality data, supplemented by 20 model B monitoring results of sewage outlets, and after merging, model A is trained.

[0122] Based on the above technical solutions, model A is continuously optimized to enhance the accuracy of large-area water pollution inversion and make upstream screening more accurate.

[0123] The above describes the solutions of the embodiments of the present application mainly from the perspective of device implementation. It can be understood that each device, for example, the river sewage outlet collaborative monitoring device based on space-air-ground integration, includes at least one of the corresponding hardware structure and software module for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the examples described in the embodiments disclosed herein, the present application can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software driven hardware depends on the specific application and design constraints of the technical solutions. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0124] The embodiments of the present application can divide the functional units of the river sewage outlet collaborative monitoring device based on space-air-ground integration according to the above method examples, for example, each functional unit can be divided according to each function, or two or more functions can be integrated in one processing unit. The integrated unit can be realized in the form of hardware or software functional unit. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical functional division. Actual implementation can have another division method.

[0125] In the case of using integrated units, Figure 10 A possible structure schematic diagram of the river sewage outlet collaborative monitoring device based on space-air-ground integration (denoted as river sewage outlet collaborative monitoring device based on space-air-ground integration 100) involved in the above embodiments is shown, which includes a processing unit 1001 and a communication unit 1002, and can also include a storage unit 1003. Figure 10 The structure schematic diagram shown can be used to illustrate the structure of the river sewage outlet collaborative monitoring device based on space-air-ground integration involved in the above embodiments.

[0126] When Figure 10The structure diagram shown is used to illustrate the structure of the space-ground integration based river sewage outlet collaborative monitoring device involved in the above embodiment. The processing unit 1001 is configured to control and manage the operation of the space-ground integration based river sewage outlet collaborative monitoring device. The communication unit 1002 is configured to communicate with other devices based on the space-ground integration based river sewage outlet collaborative monitoring device. The storage unit 1003 is configured to store the program code and data of the space-ground integration based river sewage outlet collaborative monitoring device.

[0127] For example, the communication unit 1002 is configured to obtain remote sensing images of the first region.

[0128] The processing unit 1001 is configured to pre-process the remote sensing images to obtain orthographic remote sensing images, extract water area vector boundaries of the monitoring region from the orthographic remote sensing images by using a remote sensing water body index in combination with a deep learning model, perform inversion on the orthographic remote sensing images by using a model A for identifying water body pollution to obtain the first region, perform threshold segmentation on the first thermal infrared image to determine a second region, and verify and confirm the position of the sewage outlet in combination with the first visible light image. The position of the sewage outlet is monitored day and night. The day and night monitoring process satisfies the following process: during the daytime, the model B for identifying video is used to identify sewage information; during the night, sewage information is identified based on temperature threshold comparison of the sewage outlet thermal infrared image.

[0129] In a possible implementation, the processing unit 1001 is further configured to extract water area vector boundaries of the monitoring region from the orthographic remote sensing images by using a remote sensing water body index in combination with a deep learning model, including: calculating an NDWI index based on the green band and near-infrared band spectral values of the orthographic remote sensing images; setting an NDWI index threshold to preliminarily extract water bodies from the orthographic remote sensing images to obtain a preliminary water body extraction vector graph patch; post-processing the vector graph patch to remove small regions with an area less than a set area threshold; converting the post-processed vector graph patch into raster data as a training sample set of the deep learning model; training the deep learning model by using the training sample set, applying the trained deep learning model to the entire region of the orthographic remote sensing images for accurate water body extraction; and converting the extracted water body results of the entire region into a vector format to obtain the water area vector boundaries.

[0130] In a possible implementation, the model A is obtained by training a random forest model based on the first sample set and the feature set, including: performing data cleaning on the first sample set; setting core parameters of the random forest algorithm; taking the cleaned first sample set as a training label and the feature set as input features to train each independent decision tree; and generating a final comprehensive decision tree by using an average decision tree output result mode to obtain the model A for identifying water body pollution.

[0131] In a possible implementation, the processing unit 1001 is further configured to perform inversion on the orthographic remote sensing image by using a model A for identifying water pollution, to obtain the first area, by using the water area vector boundary as a mask, including: aligning the water area vector boundary with the orthographic remote sensing image, and cutting to retain the river water area image within the mask, and eliminating non-water area regions; inputting the cut water area image into the model A, identifying the pollution area based on the image band spectrum value and the remote sensing water quality monitoring index, and converting the identification result into a vector format to obtain the first area.

[0132] In a possible implementation, the communication unit 1002 is further configured to obtain the first thermal infrared image and the first visible light image.

[0133] In a possible implementation, the processing unit 1001 is further configured to perform threshold segmentation on the first thermal infrared image to determine the second area, and verify and confirm the position of the pollution outlet in combination with the first visible light image, including: performing histogram statistical analysis on the temperature value of the first thermal infrared image, performing threshold segmentation by using the K-means clustering method, and locating the second area with temperature anomaly; calling the first visible light image of the same time period and the same position as the first thermal infrared image, verifying and confirming the specific position of the pollution outlet in the second area in combination with the shape and texture information thereof.

[0134] In a possible implementation, the processing unit 1001 is further configured to perform threshold segmentation by using the K-means clustering method, including:

[0135] S1, selecting a plurality of random sample values as initial clustering centroids from the pixel temperature values of the first thermal infrared image;

[0136] S2, calculating the distance between each pixel temperature value and each initial centroid according to the Euclidean distance, and classifying the pixel temperature value into the class to which the nearest centroid belongs; the distance satisfies the following formula:

[0137]

[0138] wherein, , n is the total number of pixels of the first thermal infrared image, K is the number of clustering categories, m is the feature dimension, is the temperature value of the i th pixel in the first thermal infrared image, is the temperature value of the j th initial clustering centroid in the K-means clustering algorithm, is the k th feature value of the i th pixel, is the k th feature value of the j th centroid;

[0139] S3, averaging all pixel temperature values in each clustering category to obtain the updated centroid of the category; the centroid updating process satisfies the following formula:

[0140]

[0141] wherein, , is the total number of pixels contained in the cluster class G, is the updated centroid of the class G;

[0142] S4, repeat S2 and S3 until the values of all cluster centroids no longer change.

[0143] In a possible implementation, the communication unit 1002 is further configured to acquire the thermal infrared image of the pollution outlet and the visible light image of the pollution outlet;

[0144] In a possible implementation, the model B is obtained based on the visible light image of the pollution outlet and the threshold recognition of the thermal infrared image of the pollution outlet, and the model B comprises: setting a temperature threshold; the temperature threshold satisfies wherein, is the temperature threshold, x is the highest temperature in the thermal infrared image of the pollution outlet corresponding to the uncontaminated water body at the same period of the previous day at the current monitoring time point, is the air temperature at the current time, is the air temperature at the same period of the previous day; the thermal infrared image collected by the camera at the pollution outlet position is analyzed frame by frame, and the region with a pixel temperature value greater than the set temperature threshold is marked as a contaminated region to form a contaminated region marking sample; the contaminated region marking sample and the visible light image collected by the camera at the same position and at the same time are spatially and temporally aligned, so that the corresponding region of the visible light image carries the contaminated region marking information, and a training data set of the deep learning model is constructed; the deep learning recognition model of the visible light image is trained by using the training data set, and after the model training is completed, the visible light deep learning model and the temperature threshold judgment logic of the thermal infrared image are superimposed to form the model B for identifying the video.

[0145] In a possible implementation, the processing unit 1001 is further configured to, after monitoring the pollution outlet position day and night, the method further comprises supplementing the pollution information and the monitoring result of the model B to the first sample set as a second sample set, and training and optimizing the model A.

[0146] In a possible implementation, the remote sensing water quality monitoring indexes in the feature set comprise a normalized vegetation index NDVI, a black and odorous water body index BOI, a normalized suspended matter index NDSSI, a normalized turbidity index NDTI, a dual-band water body index DBWI, and a water body cleaning index WCI.

[0147] The processing unit 1001 can be a processor or a controller, and the communication unit 1002 can be a communication interface, a transceiver, a transceiver, a transceiver circuit, a transceiver device, etc. The communication interface is collectively referred to, and can include one or more interfaces. The storage unit 1003 can be a memory. When the space-ground-ground integrated river sewage outlet cooperative monitoring device 100 is a chip, the processing unit 1001 can be a processor or a controller, and the communication unit 1002 can be an input interface and / or an output interface, a pin or a circuit, etc. The storage unit 1003 can be a storage unit (for example, a register, a cache, etc.) within the chip, or a storage unit (for example, a read-only memory (read-only memory, ROM), a random access memory (random access memory, RAM), etc.) located outside the chip.

[0148] The communication unit can also be referred to as a transceiver unit. The antenna and control circuit with transceiver function in the space-ground-ground integrated river sewage outlet cooperative monitoring device 100 can be regarded as the communication unit 1002 of the space-ground-ground integrated river sewage outlet cooperative monitoring device 100, and the processor with processing function can be regarded as the processing unit 1001 of the space-ground-ground integrated river sewage outlet cooperative monitoring device 100. Optionally, the device for realizing the receiving function in the communication unit 1002 can be regarded as a communication unit, and the communication unit is used to execute the receiving steps in the embodiments of the application. The communication unit can be a receiver, a receiver, a receiving circuit, etc. The device for realizing the sending function in the communication unit 1002 can be regarded as a sending unit, and the sending unit is used to execute the sending steps in the embodiments of the application. The sending unit can be a transmitter, a sender, a sending circuit, etc.

[0149] Figure 10 The integrated units in the above embodiments can be stored in a computer readable storage medium if they are realized in the form of software function modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the application can be embodied in the form of software products, and the computer software products are stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the application. The storage medium storing the computer software product includes a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0150] Figure 10 The units in the above embodiments can also be referred to as modules, for example, the processing unit can be referred to as a processing module.

[0151] The embodiment of the present application also provides a hardware structure schematic diagram of a river sewage outlet cooperative monitoring device based on space-aerospace-terrestrial integration (referred to as a river sewage outlet cooperative monitoring device 110 based on space-aerospace-terrestrial integration). Figure 11 The river sewage outlet cooperative monitoring device 110 based on space-aerospace-terrestrial integration comprises a processor 1101, and optionally further comprises a memory 1102 connected with the processor 1101.

[0152] In a first possible implementation, referring to Figure 11 The river sewage outlet cooperative monitoring device 110 based on space-aerospace-terrestrial integration further comprises a transceiver 1103. The processor 1101, the memory 1102 and the transceiver 1103 are connected through a bus. The transceiver 1103 is used for communicating with other devices or communication networks. Optionally, the transceiver 1103 can comprise a transmitter and a receiver. The device for realizing the receiving function in the transceiver 1103 can be regarded as a receiver, and the receiver is used for executing the receiving steps in the embodiment of the present application. The device for realizing the sending function in the transceiver 1103 can be regarded as a transmitter, and the transmitter is used for executing the sending steps in the embodiment of the present application.

[0153] Based on the first possible implementation, Figure 11 The structure schematic diagram shown can be used for illustrating the structure of the river sewage outlet cooperative monitoring device based on space-aerospace-terrestrial integration involved in the above embodiment.

[0154] Among them, Figure 11 The system chip in the river sewage outlet cooperative monitoring device based on space-aerospace-terrestrial integration can also be illustrated. In this case, the actions performed by the above river sewage outlet cooperative monitoring device based on space-aerospace-terrestrial integration can be realized by the system chip, and the specific actions performed can be referred to in the above, and will not be described here.

[0155] In the implementation process, each step in the method provided by the embodiment can be completed by the integrated logic circuit of hardware in the processor or the instruction in the form of software. The steps of the method disclosed in the embodiment of the present application can be directly embodied as the execution of the hardware processor, or the execution of the combination of the hardware and the software module in the processor.

[0156] The processor in the present application can include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, and the like, each of which is a computing device running software, and each of which can include one or more cores for executing software instructions to perform operations or processing. The processor can be a separate semiconductor chip, or can be integrated with other circuits as a semiconductor chip, for example, can be integrated with other circuits (such as coding and decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a SoC (system on chip), or can be integrated as a built-in processor in an ASIC. The ASIC integrated with the processor can be packaged separately or packaged together with other circuits. In addition to including cores for executing software instructions to perform operations or processing, the processor can further include necessary hardware accelerators, such as field programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits implementing special logic operations.

[0157] The memory in the embodiments of the present application can include at least one of the following types: read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, and can also be electrically erasable programmable read-only memory (EEPROM). In some scenarios, the memory can also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto.

[0158] The embodiments of the present application also provide a computer readable storage medium including instructions, which, when executed on a computer, cause the computer to perform any of the above methods.

[0159] The embodiments of the present application also provide a computer program product including instructions, which, when executed on a computer, cause the computer to perform any of the above methods.

[0160] The embodiment of the present application further provides a chip, which comprises a processor and an interface circuit, the interface circuit is coupled with the processor, the processor is used for running computer programs or instructions to realize the method described above, and the interface circuit is used for communicating with other modules outside the chip.

[0161] In the above embodiments, the implementation can be achieved by software, hardware, firmware or any combination thereof, entirely or partially. When implemented by software, the implementation can be achieved in the form of a computer program product, entirely or partially. The computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the present application is generated, entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or data storage device including one or more servers, data centers, etc. integrated with the medium. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)) and the like.

[0162] Although the present application is described herein in conjunction with various embodiments, other variations of the disclosed embodiments can be understood and implemented by those skilled in the art with reference to the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude a plurality. A single processor or other unit can implement several functions listed in the claims. Some measures described in mutually different dependent claims can be combined and produce good results.

[0163] Although the present application has been described in connection with certain specific features and embodiments thereof, it is to be understood that it is provided as an example to the best of the applicant's knowledge and that various modifications and combinations of the described features and embodiments are possible and are within the spirit and scope of the application. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense, and all such modifications and variations are considered within the scope of the present application as defined by the following claims and their equivalents. Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the claims and their equivalents, the present application can be practiced otherwise than as specifically described.

Claims

1. A space-air-ground integrated river pollution outlet cooperative monitoring method, characterized in that, The method comprises the following steps: acquiring remote sensing images of a monitoring area, and preprocessing the remote sensing images to obtain orthographic remote sensing images; extracting water area vector boundaries of the monitoring area from the orthographic remote sensing images by using a remote sensing water body index and a deep learning model; masking the water area vector boundaries, and inverting the orthographic remote sensing images by using a model A for identifying water body pollution to obtain a first area; the model A is obtained by training a random forest model based on a first sample set and a feature set; the first sample set is measured data of ground water quality monitoring stations in the first area; and the feature set is band spectral values and remote sensing water quality monitoring indexes of the orthographic remote sensing images; threshold segmentation is performed on a first thermal infrared image to determine a second area, and a position of a pollution outlet is verified and confirmed in combination with a first visible light image; the first thermal infrared image and the first visible light image are thermal infrared and visible light images obtained by aerial photography of the first area; day and night monitoring is performed on the position of the pollution outlet; the day and night monitoring process satisfies the following processes: during a daytime period, pollution information is identified by using a model B for identifying videos; during a nighttime period, pollution information is identified based on temperature threshold comparison of a pollution outlet thermal infrared image; the model B is obtained by deep learning based on a pollution outlet visible light image and threshold identification based on a pollution outlet thermal infrared image; the model B is obtained by deep learning based on a pollution outlet visible light image and threshold identification based on a pollution outlet thermal infrared image, which comprises the following steps: setting a temperature threshold; the temperature threshold satisfies wherein is the temperature threshold, x is the highest temperature in the thermal infrared image of the pollution outlet corresponding to the unpolluted water body at the same period of the previous day at the current monitoring time point, is the air temperature at the current time, is the air temperature at the same period of the previous day; frame-by-frame analysis is performed on thermal infrared images collected by a camera at the position of the pollution outlet, regions with pixel temperature values greater than a set temperature threshold are marked as pollution areas, and pollution area marking samples are formed; spatial and temporal alignment is performed on the pollution area marking samples and visible light images collected by the camera at the same position and in the same period, so that the corresponding regions of the visible light images carry pollution area marking information, and a training data set of a deep learning model is constructed; a deep learning identification model of the visible light image is trained by using the training data set, and after the model is trained, the visible light deep learning model and temperature threshold judgment logic of the thermal infrared image are superimposed to form the model B for identifying videos.

2. The method of claim 1, wherein, The method for extracting water area vector boundaries of the monitoring area from the orthographic remote sensing images by using a remote sensing water body index and a deep learning model comprises the following steps: NDWI indexes are calculated based on green band and near-infrared band spectral values of the orthographic remote sensing images; a threshold value of the NDWI indexes is set, water bodies are preliminarily extracted from the orthographic remote sensing images to obtain preliminary water body extraction vector patches, and post-processing is performed on the vector patches to remove small regions with areas less than a set area threshold value; the vector patches after the post-processing are converted into raster data as a training sample set of a deep learning model; the deep learning model is trained by using the training sample set, the trained deep learning model is applied to the entire area of the orthographic remote sensing images for accurate water body extraction, and the extracted water body results of the entire area are converted into a vector format to obtain water area vector boundaries.

3. The method of claim 1, wherein, The method for obtaining the model A by training a random forest model based on the first sample set and the feature set comprises the following steps: data cleaning is performed on the first sample set; core parameters of the random forest algorithm are set; The first sample set after cleaning is taken as a training label, and the feature set is taken as an input feature to train each independent decision tree respectively; An average decision tree output result mode is adopted to generate a final comprehensive decision tree, and a model A for identifying water pollution is obtained.

4. The method of claim 1, wherein, The water area vector boundary is taken as a mask, and the model A for identifying water pollution is used to inverse the orthographic remote sensing image to obtain a first area, including: The water area vector boundary is taken as a mask, and is aligned with the orthographic remote sensing image, and the river area image within the mask is retained by cutting, and the non-water area region is removed. The cut water area image is input into the model A, the pollution area is identified based on the image band spectrum value and the remote sensing water quality monitoring index, the identification result is converted into a vector format, and a first area is obtained.

5. The method of claim 1, wherein, The first thermal infrared image is threshold segmented to determine a second area, and the position of a pollution outlet is verified and confirmed in combination with the first visible light image, including: The temperature value of the first thermal infrared image is histogram statistically analyzed, the threshold is segmented by using a K-means clustering method, and the second area with temperature anomaly is located. The first visible light image at the same time period and the same position as the first thermal infrared image is called, and the specific position of the pollution outlet in the second area is verified and confirmed in combination with shape and texture information.

6. The method of claim 5, wherein, The threshold segmentation by using the K-means clustering method includes: S1, selecting a plurality of random sample values as initial clustering centroids from the pixel temperature value of the first thermal infrared image; S2, calculating the distance between each pixel temperature value and each initial centroid according to the Euclidean distance, and classifying the pixel temperature value into the class to which the nearest centroid belongs; the distance satisfies the following formula: wherein, , n is the total number of pixels of the first thermal infrared image, K is the number of clustering categories, m is the feature dimension, is the temperature value of the i-th pixel in the first thermal infrared image, is the temperature value of the j-th initial clustering centroid in the K-means clustering algorithm, is the k-dimensional feature value of the i-th pixel, is the k-dimensional feature value of the j-th centroid; S3, averaging all pixel temperature values in each clustering class to obtain the updated centroid of the class; the centroid updating process satisfies the following formula: wherein, , is the total number of pixels comprised by the cluster class G, is the updated centroid of the class G; S4, repeating S2 and S3 until the values of all clustering centroids no longer change.

7. The method of claim 1, wherein, After monitoring the position of the pollution outlet day and night, the method further includes supplementing the pollution information and the monitoring result of the model B to the first sample set to train and optimize the model A.

8. The method of claim 1, wherein, The remote sensing water quality monitoring index in the feature set includes a normalized vegetation index NDVI, a black and odorous water body index BOI, a normalized suspended matter index NDSSI, a normalized turbidity index NDTI, a double-band water body index DBWI, and a water body cleaning index WCI.

9. A space-air-ground integrated river pollution outlet cooperative monitoring device, characterized in that, The device includes a communication unit and a processing unit; The communication unit is configured to acquire remote sensing images of a region to be monitored, acquire a first thermal infrared image and a first visible light image, and acquire thermal infrared images and visible light images of pollution outlets; The processing unit is configured to train a model A for identifying water pollution by taking the remote sensing images as input features and taking a first sample set as training labels; The processing unit is configured to pre-process the remote sensing image to obtain an orthographic remote sensing image, extract a water area vector boundary of a monitoring area from the orthographic remote sensing image by using a remote sensing water body index in combination with a deep learning model, and obtain a first area by using a model A for identifying water body pollution to inverse the orthographic remote sensing image with the water area vector boundary as a mask. The model A is configured to perform threshold segmentation on a first thermal infrared image to determine a second area, and verify and confirm a position of a pollution outlet in combination with a first visible light image. The position of the pollution outlet is monitored day and night. The day and night monitoring process satisfies the following processes: during a daytime period, a model B for identifying video is used to identify pollution information; during a nighttime period, pollution information is identified based on a temperature threshold comparison of a pollution outlet thermal infrared image. The model B is obtained based on visible light image deep learning of the pollution outlet in combination with threshold identification of the pollution outlet thermal infrared image. The model B is obtained based on visible light image deep learning of the pollution outlet in combination with threshold identification of the pollution outlet thermal infrared image, including setting a temperature threshold. The temperature threshold satisfies wherein is the temperature threshold, x is a highest temperature in a pollution outlet thermal infrared image of a non-polluted water body at a same period of a previous day at a current monitoring time point, is a current time air temperature, is a previous day air temperature at the same period; a thermal infrared image collected by a camera at the position of the pollution outlet is analyzed frame by frame, a region with a pixel temperature value greater than the set temperature threshold is marked as a pollution area to form a pollution area marking sample; the pollution area marking sample is spatially and temporally aligned with a visible light image collected by the camera at the same period and at the same position, so that the corresponding region of the visible light image carries the pollution area marking information, and a training data set of a deep learning model is constructed; a deep learning identification model of the visible light image is trained by using the training data set, and after the model training is completed, the visible light deep learning model and the temperature threshold judgment logic of the thermal infrared image are superimposed to form the model B for identifying video.

Images