Water pollution source and sink detection method and intelligent equipment based on multi-modal remote sensing fusion

Abstract

The application discloses a water pollution source and sink detection method based on multi-modal remote sensing fusion, which comprises the following steps: obtaining a visible light remote sensing image of a water body near a river bank and a shoreline; inputting visible light remote sensing data and thermal infrared remote sensing data into a lightweight double-branch convolutional neural network to obtain a visible light-thermal infrared fusion feature map; obtaining preliminary position information of a pollution source through a YOLOv8 lightweight target detection network; obtaining spectral reflectance remote sensing data of a water body detection area closest to the geometric center coordinates of a suspected pollution source area; obtaining pollution concentration inversion results of the water body detection area; and outputting a final comprehensive detection result. The application also discloses intelligent equipment for water pollution source and sink detection based on multi-modal remote sensing fusion. The application realizes accurate detection of a pollution source near a water bank by using a visible light remote sensing image and a thermal infrared remote sensing image, realizes identification of a surface optical parameter of a water body near a pollution source and water pollution concentration inversion by using spectral reflectance remote sensing data, and finally fuses the two types of results.

Description

Technical Field

[0001] This invention belongs to the field of water environment monitoring technology, specifically relating to a method and intelligent equipment for detecting water pollution sources and sinks based on multimodal remote sensing fusion. It is applicable to the precise location of pollution sources such as sewage outlets, submerged pipes, and leaking sewage pipes in nearshore waters, as well as the identification of optical parameters on the surface of nearshore waters near pollution sources and the inversion of water pollution concentrations. This enables comprehensive detection of the location of pollution sources and the surrounding water quality, while providing integrated intelligent equipment support for detection. Background Technology

[0002] Currently, nearshore water pollution source detection and water quality monitoring mainly employ single-modal remote sensing methods and simple equipment, such as visible light remote sensing image recognition, thermal infrared remote sensing temperature anomaly detection, or point-based multispectral remote sensing water quality analysis. These devices are mostly single-function and lack integrated collaborative capabilities. Single-modal remote sensing monitoring has significant limitations: visible light remote sensing images are easily affected by lighting and weather, making it difficult to identify nearshore pollution sources such as hidden pipes concealed under vegetation or shadows; while thermal infrared remote sensing images can capture temperature anomalies in sewage discharge and assist in detecting nearshore pollution sources, they lack spatial detail and cannot simultaneously acquire information on the water quality surrounding the pollution source; point-based multispectral remote sensing data, although capable of analyzing water optical parameters and pollution concentrations, has limited spatial coverage, making it difficult to accurately locate nearshore pollution sources and unable to establish an effective correlation with the pollution source location, hindering comprehensive detection. Furthermore, existing intelligent equipment is mostly designed for fixed platforms, resulting in poor adaptability and inability to meet mobile monitoring needs; data processing and detection processes are separated, leading to low efficiency.

[0003] Multimodal remote sensing data fusion technology offers an effective solution to the aforementioned problems. By integrating complementary information from different modal remote sensing data, it can improve detection accuracy and anti-interference capabilities. However, existing multimodal fusion technologies are mostly designed for fixed platforms, making them unsuitable for the real-time and lightweight requirements of mobile platforms such as drones and mobile monitoring vehicles. Furthermore, most fusion technologies fail to clearly define the core uses of each modal remote sensing data, failing to achieve a precise division of labor between visible light-thermal infrared remote sensing for detecting pollution sources and point-based spectral remote sensing for inverting surrounding water quality. This results in poor fusion effects and an inability to meet the comprehensive detection needs of near-shore pollution source locations and surrounding water quality. In addition, existing intelligent equipment lacks an integrated design; each detection module operates independently, resulting in poor data coordination and an inability to achieve integrated operations of "data acquisition-processing-fusion-detection-output." Moreover, the equipment's adaptability is insufficient, making it difficult to meet the convenience and real-time requirements of mobile monitoring. For example, some technologies directly fuse the three types of remote sensing data at the decision level, failing to fully leverage the advantages of visible light and thermal infrared remote sensing in locating near-shore pollution sources, and failing to specifically utilize the role of point-based spectral remote sensing in identifying water body optical parameters and retrieving concentrations; some equipment only carries a single remote sensing detection device, with limited functionality, and cannot achieve simultaneous detection of pollution sources and water quality.

[0004] Therefore, there is an urgent need for a multimodal remote sensing data fusion technology that is compatible with mobile platforms, has a clear division of labor, a reasonable fusion strategy, and high detection accuracy, as well as integrated intelligent equipment. This technology should clarify the core uses of three types of remote sensing data: visible light-thermal infrared and point spectral data, to achieve precise location of near-shore pollution sources, identification of optical parameters of water bodies near pollution sources, and inversion of pollution concentration. At the same time, it should solve the problems of existing equipment having single functions, poor coordination, and insufficient adaptability, and provide reliable intelligent equipment support for the treatment of near-shore water pollution. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems in the existing technology by providing a method and intelligent equipment for detecting water pollution sources and sinks based on multimodal remote sensing fusion.

[0006] The above-mentioned objectives of the present invention are achieved by the following technical means:

[0007] A water pollution source and sink detection method based on multimodal remote sensing fusion includes the following steps:

[0008] Step 1: Obtain visible light remote sensing images of the water body and shoreline near the riverbank, and obtain thermal infrared remote sensing images of the surface of the water body near the riverbank. Perform resolution unification and spatial alignment on the visible light remote sensing images and thermal infrared remote sensing images.

[0009] Step 2: Input visible light remote sensing data and thermal infrared remote sensing data into a lightweight dual-branch convolutional neural network to extract primary features of visible light. and primary thermal infrared features Furthermore, max pooling is performed to obtain visible light pooling feature maps and thermal infrared pooling feature maps. Both the visible light pooling feature maps and thermal infrared pooling feature maps are then subjected to channel attention weighting and spatial attention weighting using the CBAM attention module, respectively, to obtain a weighted visible light pooling feature map. Weighted image of thermal infrared pooling features Visible pooling feature weighted map Weighted image of thermal infrared pooling features After channel stitching, feature fusion is performed through a 1×1 convolutional layer to obtain a visible light-thermal infrared fused feature map. Visible light-thermal infrared fusion feature map The data is input into the YOLOv8 lightweight object detection network to obtain preliminary location information of pollution sources, including suspected pollution source areas and the geometric center coordinates of the suspected pollution source areas. The maximum predicted probability of pollution source types in suspected pollution source areas and the confidence prediction value of suspected pollution source areas And further calculate the detection confidence level. ;

[0010] Step 3: Obtain the geometric center coordinates of the suspected pollution source area. Recent remote sensing data on spectral reflectance of the water body detection area was used to obtain pollution concentration inversion results for the water body detection area. The pollution concentration inversion results include pollutant type and pollutant concentration. Correlation coefficient Concentration inversion confidence level and pollutant concentration in the water body detection area Geometric center coordinates of connected contaminated areas exceeding the set concentration threshold ;

[0011] Step 4: Integrate the preliminary location information of the pollution source with the pollution concentration inversion results to output the final comprehensive detection results.

[0012] As described above, step 1, resolution unification, includes the following steps:

[0013] The resolution of thermal infrared remote sensing images is resampled to match that of visible light remote sensing images based on the following formula. ,

[0014] in, These are the pixel coordinates of a visible light remote sensing image. The pixel coordinates of a thermal infrared remote sensing image The fitted pixel values, , , and The coordinates of pixels in a thermal infrared remote sensing image are arranged sequentially. The coordinates of the four adjacent pixels, , , as well as For thermal infrared remote sensing images at pixel coordinates Pixel coordinates Pixel coordinates and pixel coordinates Pixel value at that location, and For interpolation weights, , .

[0015] As described above, step 1 involves the following steps for spatial alignment:

[0016] The Euclidean distances between feature points are obtained by traversing thermal infrared and visible light remote sensing images. Feature points with Euclidean distances less than or equal to... Pixels from thermal infrared remote sensing images and pixels from visible light remote sensing images are defined as non-collinear effective feature point pairs. For the first The pixel coordinates of feature points in a pair of non-collinear effective feature points in a visible light remote sensing image. For the first The pixel coordinates of feature points in a thermal infrared remote sensing image within a set of non-collinear effective feature point pairs. For matching distance threshold, As a reliability weight,

[0017] Constructing the affine transformation matrix :

[0018] ,

[0019] in, This is the horizontal scaling factor. This is the scaling factor in the vertical direction. The horizontal shear factor, The vertical shear factor. This represents the horizontal translation amount. The vertical translation is calculated as follows:

[0020] Initialize horizontal scaling factor Vertical scaling factor Horizontal shear factor Vertical shear factor Horizontal translation and vertical translation ,

[0021] The first Substituting the pixel coordinates of the feature points in the thermal infrared remote sensing image from the non-collinear effective feature point pairs into the following affine transformation formula, we obtain the corresponding fitted pixel coordinates. :

[0022] ,

[0023] To minimize the regularization constraint error function To determine the horizontal scaling factor, find the target value. Vertical scaling factor Horizontal shear factor Vertical shear factor Horizontal translation and vertical translation To further obtain the affine transformation matrix ,

[0024] The affine transformation matrix obtained by solving Spatial alignment of thermal infrared remote sensing images and visible light remote sensing images.

[0025] As mentioned above, reliability weight Based on the following formula:

[0026] ,

[0027] in, For the first The smaller neighborhood gray-level variance of feature points in visible light remote sensing images and feature points in thermal infrared remote sensing images in a set of non-collinear effective feature point pairs. It represents the maximum value of the neighborhood gray-level variance for all pairs of non-collinear valid feature points.

[0028] As described above, the regularization constraint error function Based on the following formula:

[0029] , ,

[0030] in, The regularization constraint error function is... Let be the error function. The regularization coefficient is . To find the function with the maximum value, This is the Euclidean distance operator. For matching distance threshold, This represents the number of pairs of valid non-collinear feature points.

[0031] As described above, step 2 involves probing the confidence level. Based on the following formula:

[0032] .

[0033] The pollution concentration inversion results in step 3, as described above, are obtained based on the following steps:

[0034] Based on the suspected pollution source area and the geometric center coordinates of the suspected pollution source area Identify the water body detection area 5m to 10m away from the nearest shoreline of the suspected pollution source area;

[0035] The spectral reflectance remote sensing data of the identified water body detection area are analyzed to extract the pollutant types and concentrations in the water body detection area. and water pollutant concentration Correlation coefficient with reflectance remote sensing data ;

[0036] Calculate the confidence level of concentration inversion ,

[0037] ,

[0038] in, For pollutant concentration thresholds, The function for calculating the minimum value.

[0039] Extract pollutant concentration in the water body detection area Geometric center coordinates of connected contaminated areas exceeding the set concentration threshold .

[0040] The final comprehensive detection results in step 4 above are obtained based on the following steps:

[0041] Step 4.1: Define the identification framework H = {First proposition h1 is "Pollution source exists and corresponding pollution concentration exists", second proposition h2 is "No pollution source exists", third proposition h3 is "Uncertain"};

[0042] Record the first piece of evidence The second piece of evidence corresponds to the preliminary location information of the pollution source in step 2. Based on the pollution concentration inversion results from step 3, calculate the first evidence. Second evidence Location matching degree :

[0043] ,

[0044] in, The coordinates of the geometric center of the suspected pollution source area determined in step 2. Coordinates of the geometric center of the connected contaminated area determined in step 3 Euclidean distance, This is the distance adjustment coefficient. For location reliability coefficient, ,

[0045] Step 4.2: Calculate the first piece of evidence The first proposition: trust level First evidence The second proposition: trust level and the first piece of evidence The third proposition of trust ;

[0046] Calculate the second evidence The first proposition: trust level Second evidence The second proposition: trust level and second evidence The third proposition of trust ;

[0047] ,

[0048] ,

[0049] in, The confidence level weighting coefficient is... , ; As a correction factor for the reliability of evidence, ,

[0050] Step 4.3: Calculate the overall confidence level of the first proposition. The overall confidence level of the second proposition And the overall confidence level of the third proposition. ,

[0051] ,

[0052] ,

[0053] ,

[0054] ,

[0055] in, The normalization constant is and To identify propositions in frame H, and The value range is 1 to 3, when At that time, it indicates the proposition of primary evidence. The proposition of second evidence There is no conflict.

[0056] Step 4.4, if and If the suspected pollution source area determined in step 2 is indeed a real pollution source, then the geometric center coordinates of the suspected pollution source area corresponding to the real pollution source will be output. Outputs the pollutant concentration in the water body detection area corresponding to the actual pollution source. Output the overall confidence level of the first proposition. ;

[0057] like and If so, it is determined that there is no pollution source in the suspected pollution source area identified in step 2;

[0058] like and If the area is not identified, the suspected pollution source area determined in step 2 is deemed uncertain.

[0059] Intelligent equipment for detecting water pollution sources and sinks based on multimodal remote sensing fusion, including a mobile platform as the carrier, also includes:

[0060] Data acquisition module: mounted on a mobile platform, used to acquire visible light remote sensing images of near-riverbank water bodies and shorelines, thermal infrared remote sensing images of the surface of near-riverbank water bodies, and spectral reflectance remote sensing data of near-riverbank water bodies, and to unify the resolution and spatially align the visible light remote sensing images and thermal infrared remote sensing images.

[0061] Data processing module: Integrated on the mobile platform, the data processing module inputs visible light remote sensing data and thermal infrared remote sensing data into a lightweight dual-branch convolutional neural network to extract primary features of visible light. and primary thermal infrared features Furthermore, max pooling is performed to obtain visible light pooling feature maps and thermal infrared pooling feature maps. Both the visible light pooling feature maps and thermal infrared pooling feature maps are then subjected to channel attention weighting and spatial attention weighting using the CBAM attention module, respectively, to obtain a weighted visible light pooling feature map. Weighted image of thermal infrared pooling features Visible pooling feature weighted map Weighted image of thermal infrared pooling features After channel stitching, feature fusion is performed through a 1×1 convolutional layer to obtain a visible light-thermal infrared fused feature map. Visible light-thermal infrared fusion feature map The data is input into the YOLOv8 lightweight object detection network to obtain preliminary location information of pollution sources, including suspected pollution source areas and the geometric center coordinates of the suspected pollution source areas. The maximum predicted probability of pollution source types in suspected pollution source areas and the confidence prediction value of suspected pollution source areas And further calculate the detection confidence level. ,

[0062] Optical parameter identification and concentration inversion unit: Integrated on the mobile platform, the optical parameter identification and concentration inversion unit is based on the geometric center coordinates of the suspected pollution source area. Obtain the geometric center coordinates of the suspected pollution source area. Recent remote sensing data on spectral reflectance of the water body detection area was used to obtain pollution concentration inversion results for the water body detection area. The pollution concentration inversion results include pollutant type and pollutant concentration. Correlation coefficient of pollutant types Concentration inversion confidence level and pollutant concentration in the water body detection area Geometric center coordinates of connected contaminated areas exceeding the set concentration threshold ,

[0063] Decision-level fusion unit: Integrated on a mobile platform, it fuses the preliminary location information of pollution sources with the pollution concentration inversion results, integrates the location of pollution sources with the surrounding water quality information, and outputs the final comprehensive detection results.

[0064] Compared with the prior art, the present invention has the following advantages:

[0065] This study clarifies the core applications of visible light remote sensing images, thermal infrared remote sensing images, and spectral reflectance remote sensing data. Through a layered fusion strategy, it utilizes visible light and thermal infrared remote sensing images to achieve precise detection of near-shore pollution sources, and uses spectral reflectance remote sensing data to identify optical parameters of near-shore water surfaces and retrieve water pollutant concentrations. Finally, by fusing the two types of results, it achieves precise, real-time, and comprehensive detection of the location of near-shore water pollution sources and the surrounding water quality. Simultaneously, it provides integrated intelligent equipment to address the problems of existing equipment's limited functionality, poor coordination, and insufficient adaptability. This enables integrated operation of data acquisition, processing, fusion, detection, and result output, adapting to the real-time monitoring needs of mobile platforms. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0067] Figure 2 This is a schematic diagram of the composition of the intelligent equipment of the present invention.

[0068] In the diagram: 1-Mobile platform; 2-Visible light camera; 3-Spot multispectral analyzer; 4-Thermal infrared camera. Detailed Implementation

[0069] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0070] Example 1:

[0071] Figure 1 This is a schematic diagram of the method flow of the present invention. The water pollution source and sink detection method based on multimodal remote sensing fusion includes the following steps:

[0072] Step 1: Obtain visible light remote sensing images of the water body and shoreline near the riverbank, and obtain thermal infrared remote sensing images of the surface of the water body near the riverbank. Perform resolution unification and spatial alignment on the visible light remote sensing images and the thermal infrared remote sensing images.

[0073] Step 1.1: Using the data acquisition module of the intelligent equipment, synchronous data is collected on the target near-riverbank water area. The data acquisition module includes a visible light camera and a thermal infrared camera. The visible light camera acquires visible light remote sensing images (spatial texture remote sensing images) of the near-riverbank water and shoreline, and the thermal infrared camera acquires thermal infrared remote sensing images (temperature distribution remote sensing images) of the near-riverbank water surface.

[0074] Step 1.2: Using the data processing module of the intelligent equipment, the visible light remote sensing images and thermal infrared remote sensing images are unified in resolution and spatially aligned to ensure the accuracy of subsequent fusion, detection, and inversion. This step includes three core processes, and the detailed process and key calculation formulas for each process are as follows:

[0075] Step 1.2.1: Unify the resolution of the thermal infrared remote sensing image to the resolution of the visible light remote sensing image to eliminate resolution differences and ensure spatial matching.

[0076] Bilinear interpolation is used to resample the resolution of thermal infrared remote sensing images to match that of visible light remote sensing images, eliminating position matching deviations caused by resolution differences and laying the foundation for subsequent feature extraction and spatial alignment.

[0077] Resolution resampling of thermal infrared remote sensing images is based on the following formula: ,

[0078] in, These are the pixel coordinates of a visible light remote sensing image. The pixel coordinates of a thermal infrared remote sensing image The fitted pixel values, , , and The coordinates of pixels in a thermal infrared remote sensing image are arranged sequentially. The coordinates of the four adjacent pixels, , , as well as For thermal infrared remote sensing images at pixel coordinates Pixel coordinates Pixel coordinates and pixel coordinates Pixel value at that location, and The interpolation weights are (0≤u, v≤1). , .

[0079] Step 1.2.2: Spatial alignment of the thermal infrared remote sensing image and the visible light remote sensing image.

[0080] Step 1.2.2.1: Use the pixel coordinates of the visible light remote sensing image as the unique reference coordinates for the entire process.

[0081] Step 1.2.2.2: Traverse the thermal infrared remote sensing image and the visible light remote sensing image to obtain the Euclidean distance between feature points, and select feature points whose Euclidean distance is less than or equal to... The pixels of the thermal infrared remote sensing image and the pixels of the visible light remote sensing image are used as feature points. The corresponding feature points of the thermal infrared remote sensing image and the visible light remote sensing image are defined as non-collinear valid feature point pairs. For the first The pixel coordinates of feature points in a pair of non-collinear effective feature points in a visible light remote sensing image. For the first The pixel coordinates of feature points in a thermal infrared remote sensing image within a set of non-collinear effective feature point pairs. =2.5 is the matching distance threshold. For reliability weights, the number of non-collinear valid feature point pairs N is greater than or equal to 3.

[0082] N pairs of non-collinear valid feature points serve as the basic samples for solving the affine transformation matrix. This invention adds a feature point reliability weight. .

[0083] ,

[0084] in, For the first The smaller neighborhood gray-level variance of feature points in visible light remote sensing images and feature points in thermal infrared remote sensing images in a set of non-collinear effective feature point pairs. This represents the maximum value of the neighborhood gray-level variance corresponding to all pairs of non-collinear valid feature points; after normalization calculation, The higher the contrast, the stronger the reliability of the feature points.

[0085] Constructing the affine transformation matrix :

[0086] ,

[0087] Wherein, the affine transformation matrix For a 3×3 fixed-format matrix:

[0088] ,

[0089] in, This is the horizontal scaling factor. This is the scaling factor in the vertical direction. The horizontal shear factor, The vertical shear factor. This represents the horizontal translation amount. The vertical translation is calculated as follows:

[0090] Step 1.2.2.3: Initialize the horizontal scaling factor Vertical scaling factor Horizontal shear factor Vertical shear factor Horizontal translation and vertical translation .

[0091] The first Substituting the pixel coordinates of the feature points in the thermal infrared remote sensing image from the non-collinear effective feature point pairs into the following affine transformation formula, we obtain the corresponding fitted pixel coordinates. :

[0092] .

[0093] Step 1.2.2.4: Minimize the regularization constraint error function. To determine the horizontal scaling factor, find the target value. Vertical scaling factor Horizontal shear factor Vertical shear factor Horizontal translation and vertical translation To further obtain the affine transformation matrix .

[0094] ,

[0095] in, This is the error function.

[0096] This invention introduces a new regularization constraint: a penalty term for outlier matching points is added to improve transformation robustness.

[0097] ,

[0098] in, The regularization constraint error function is... The regularization coefficient is . This is the Euclidean distance operator. For matching distance threshold, This is a function to find the maximum value.

[0099] Step 1.2.2.5: Obtain the affine transformation matrix by solving. Spatial alignment of thermal infrared remote sensing images and visible light remote sensing images.

[0100] Conventional multimodal remote sensing data spatial alignment uses only single feature point matching, without coordinate mapping matrix solving or error quantization constraints, resulting in severe cross-modal data coordinate misalignment (errors can reach ±5~10 pixels). This directly causes a disconnect between subsequent pollution source location and water quality concentration inversion location, making comprehensive detection impossible.

[0101] Through the above processing, the industry pain point of cross-modal data spatial misalignment was solved, and the spatial alignment error was reduced from the usual ±5~10 pixels to within ±2 pixels, improving the alignment accuracy by more than 60%.

[0102] Step 2: Input visible light remote sensing data and thermal infrared remote sensing data into a lightweight dual-branch convolutional neural network to extract primary features of visible light. and primary thermal infrared features Furthermore, max pooling is performed to obtain visible light pooling feature maps and thermal infrared pooling feature maps. Both the visible light pooling feature maps and thermal infrared pooling feature maps are then subjected to channel attention weighting and spatial attention weighting using the CBAM attention module, respectively, to obtain a weighted visible light pooling feature map. Weighted image of thermal infrared pooling features Visible pooling feature weighted map Weighted image of thermal infrared pooling features After channel stitching, feature fusion is performed through a 1×1 convolutional layer to obtain a visible light-thermal infrared fused feature map. Visible light-thermal infrared fusion feature map The data is input into the YOLOv8 lightweight object detection network to obtain preliminary location information of pollution sources, including suspected pollution source areas and the geometric center coordinates of the suspected pollution source areas. The maximum predicted probability of pollution source types in suspected pollution source areas and the confidence prediction value of suspected pollution source areas And further calculate the detection confidence level. .

[0103] This step achieves feature-level fusion of visible light remote sensing data and thermal infrared remote sensing data, fully leveraging the complementary advantages of both: the spatial detail advantage of visible light and the temperature anomaly advantage of thermal infrared. This enables precise detection of near-shore pollution sources and outputs preliminary location information of the pollution sources (including suspected pollution source areas and the geometric center coordinates of the suspected pollution source areas). The maximum predicted probability of pollution source types in suspected pollution source areas Confidence prediction value of suspected pollution source areas and detection confidence level Specifically, it includes:

[0104] Step 2.1: Under the unified visible light reference coordinates, extract the primary visible light features of the visible light remote sensing data using the first branch of a lightweight dual-branch convolutional neural network. The primary thermal infrared features of thermal infrared remote sensing data are extracted using the second branch of a lightweight dual-branch convolutional neural network. Preliminary core features were extracted from two types of remote sensing data: visible light and thermal infrared.

[0105] Step 2.2: Primary characteristics of visible light and primary thermal infrared features Max pooling is performed to obtain visible light pooling feature maps and thermal infrared pooling feature maps. The visible light pooling feature map is then sequentially weighted by channel attention and spatial attention using the CBAM attention module to obtain a weighted visible light pooling feature map. The thermal infrared pooling feature map is sequentially weighted by channel attention and spatial attention using the CBAM attention module to obtain the thermal infrared pooling feature map weighted. .

[0106] This invention introduces a CBAM attention module to process the extracted primary features of visible light. and primary thermal infrared features Dynamic weighting is performed separately to highlight key features related to near-shore pollution sources, such as the texture of sewage outlets in visible light and high-temperature anomalies in thermal infrared, while suppressing background interference features, such as shore vegetation and water surface ripples. The weighted visible light pooling feature weighting map and thermal infrared pooling feature weighting map are then output.

[0107] Conventional CBAM attention modules directly apply channel and spatial attention weights to the original feature map output by convolution without dimensionality reduction. This results in a large amount of redundant background information (such as shoreline vegetation and water ripples) in the original feature map, leading to an imbalance in attention weight allocation and interference with key features (sewage outlet texture, temperature anomalies), resulting in low accuracy in pollution source identification. This invention, however, performs max pooling before channel and spatial attention weights in the CBAM attention module to reduce dimensionality, thus avoiding the aforementioned problems.

[0108] Step 2.3: Weight the visible light pooling feature map Weighted image of thermal infrared pooling features After channel stitching, feature fusion is performed through a 1×1 convolutional layer to obtain a visible light-thermal infrared fused feature map. Visible light-thermal infrared fusion feature map The input is fed into the YOLOv8 lightweight object detection network, which outputs preliminary pollution source location information, including: the suspected pollution source area and the geometric center coordinates of the suspected pollution source area. The maximum predicted probability of pollution source types in suspected pollution source areas and the confidence prediction value of suspected pollution source areas And further calculate the detection confidence level. .

[0109] Step 2.3.1: Weight the visible light pooling feature map Weighted image of thermal infrared pooling features Perform channel stitching to obtain a stitched feature map. Describe the visible light pooling feature weighted map. Weighted image of thermal infrared pooling features The dimensions of each feature map are H×W×C, resulting in a stitched feature map. (Dimensions H×W×2C), where H, W, and C represent the height, width, and number of channels, respectively.

[0110] Step 2.3.2: Concatenate the feature maps using a convolutional layer with a 1×1 kernel. Feature fusion is performed to eliminate redundant information and obtain a visible light-thermal infrared fused feature map. (Dimensions H×W×C).

[0111] Step 2.3.3: Use the YOLOv8 lightweight target detection network to process the visible light-thermal infrared fusion feature map. Conduct near-shore pollution source detection and output preliminary pollution source location information, including: suspected pollution source area and geometric center coordinates of the suspected pollution source area. The maximum predicted probability of pollution source types in suspected pollution source areas and the confidence prediction value of suspected pollution source areas And further calculate the detection confidence level. .

[0112] .

[0113] Step 3: Using the optical parameter identification and concentration inversion unit of the intelligent equipment, based on the geometric center coordinates of the suspected pollution source area output in Step 2. Obtain the geometric center coordinates of the suspected pollution source area. Recent remote sensing data on spectral reflectance of the water body detection area was used to obtain pollution concentration inversion results for the water body detection area. The pollution concentration inversion results include pollutant type and pollutant concentration. Pollutant concentration Correlation coefficient with reflectance remote sensing data Concentration inversion confidence level and pollutant concentration in the water body detection area Geometric center coordinates of connected contaminated areas exceeding the set concentration threshold The specific process includes...

[0114] Step 3.1: Based on the suspected pollution source area and the geometric center coordinates of the suspected pollution source area output in Step 2. The detection area is located 5m to 10m away from the nearest shoreline to the suspected pollution source area.

[0115] Step 3.2: Analyze the spectral reflectance remote sensing data (which can be obtained by UAV) collected by a point-type multispectral instrument in the determined water body detection area to extract the pollutant type and concentration in the water body detection area. Correlation coefficient with reflectance remote sensing data For example, the Pearson correlation coefficient.

[0116] Preferably, the spectral reflectance remote sensing data in the 400-1000nm characteristic band is analyzed.

[0117] Step 3.3: Calculate the concentration inversion confidence level using the following formula. This is used for subsequent decision-level fusion.

[0118] ,

[0119] in, For pollutant concentration threshold, when hour, Take 1; when hour, Pick , The confidence level for concentration inversion ranges from 0 to 1. The closer the value is to 1, the more reliable the inversion result. This is a function for calculating the minimum value.

[0120] Step 3.4: Extract pollutant concentration in the water detection area. Geometric center coordinates of connected contaminated areas exceeding the set concentration threshold .

[0121] Step 4: Output of decision-level fusion and comprehensive detection results.

[0122] Through the decision-level fusion unit of intelligent equipment, and using DS evidence theory, the preliminary location information of the pollution source in step 2 and the pollution concentration inversion results in step 3 are fused at the decision level. This integrates the pollution source location and surrounding water quality information, and outputs the final comprehensive detection results. The specific process consists of four steps:

[0123] Step 4.1: Construct an identification framework, clarify the scope of the fusion decision-making proposition, and base it on the geometric center coordinates of the suspected pollution source area output in Step 2. Coordinates of the geometric center of the contaminated area connected in step 3 Calculate the location matching degree ,

[0124] process:

[0125] Define the identification framework H = {First proposition h1 is "Pollution source exists and corresponding pollution concentration exists", second proposition h2 is "No pollution source exists", and third proposition h3 is "Uncertain"}.

[0126] The preliminary location information of the pollution source in step 2 and the pollution concentration inversion result in step 3 are treated as two independent pieces of evidence. Let the first piece of evidence m1 correspond to the preliminary location information of the pollution source in step 2, and the second piece of evidence m2 correspond to the pollution concentration inversion result in step 3. The focus is on correlating the location matching degree between the first piece of evidence m1 and the second piece of evidence m2. Location matching degree This invention's unique design fills the gap in the lack of quantitative standards for the correlation between pollution sources and water quality locations, achieving a higher degree of location matching. The geometric center coordinates of the suspected pollution source area output from step 2 are used for quantification. Coordinates of the geometric center of the contaminated area connected in step 3 The degree of spatial overlap, with a value ranging from 0 to 1. The closer the value is to 1, the higher the degree of location matching between the two factors, and the stronger the correlation between the retrieved pollution concentration and the pollution source. This provides a core basis for subsequent decision-making and fusion. The core formula is as follows:

[0127] ,

[0128] in, The coordinates of the geometric center of the suspected pollution source area determined in step 2. Coordinates of the geometric center of the connected contaminated area determined in step 3 Euclidean distance, The distance adjustment coefficient (taken in this invention) =100m), used to adjust the effect of distance on matching degree; For the location reliability coefficient ( ), based on the location confidence of the pollution source The calculation yielded:

[0129] ,

[0130] Among them, the higher the location reliability, the higher the location reliability coefficient. The closer it is to 1, the better it can be used to avoid the problem of "inaccurate positioning but high matching degree".

[0131] Step 4.2: Calculate the confidence level of the first proposition of the first evidence m1. The credibility of the second proposition of the first piece of evidence m1 And the credibility of the third proposition of the first piece of evidence m1. .

[0132] Calculate the confidence level of the first proposition in the second evidence m2. The second proposition's credibility in the second piece of evidence m2 And the credibility of the third proposition of the second evidence m2. .

[0133] ,

[0134] ,

[0135] in, The confidence weighting coefficient (in this invention, it is taken as...) =0.9), used to adjust the effect of single-mode confidence, to ensure + , ; As a correction factor for the reliability of evidence, The newly added evidence reliability correction factor To balance the difference in confidence between the two types of evidence, when and The smaller the difference, The closer it is to 1, the better the ability to resolve evidence conflicts and the more stable the fusion decision-making process becomes.

[0136] Step 4.3: Evidence synthesis, eliminating evidence conflicts, and merging the two types of results.

[0137] Procedure: Using the Dempster-Shafer synthesis rule, the functions (m1, m2) of the two pieces of evidence are fused, and the normalization constant is calculated. Eliminating conflicting evidence yields the fused function m = m1⊕m2. Key calculation formula:

[0138] (1) Calculate the normalization constant Eliminate the conflict between the first and second pieces of evidence, and normalize the constant. The larger the value, the less conflict there is between the primary and secondary evidence.

[0139] ,

[0140] in, and To identify propositions in frame H, and The value range is 1 to 3, when At that time, it indicates the proposition of primary evidence. The proposition of second evidence There is a conflict, when At that time, it indicates the proposition of primary evidence. The proposition of second evidence There is no conflict.

[0141] (2) The fused function integrates the trust levels of the first and second pieces of evidence. :

[0142] ,

[0143] Where h represents any proposition in the recognition framework H, when At that time, calculate all of The sum, then divided by ,get ; hour, The calculation method is the same.

[0144] Expanded to:

[0145] ,

[0146] ,

[0147] ;

[0148] The overall confidence level of the first proposition. The overall confidence level of the second proposition. The overall confidence level of the third proposition.

[0149] Step 4.4, Result Decision and Output: Output the final detection results.

[0150] Select , , The maximum value is taken as the final detection result, and the final comprehensive detection result is output.

[0151] Decision-making rules:

[0152] (1) If and If the suspected pollution source area determined in step 2 is indeed a real pollution source, then the geometric center coordinates of the suspected pollution source area corresponding to the real pollution source will be output. Outputs the pollutant concentration in the water body detection area corresponding to the actual pollution source. Output the overall confidence level of the first proposition. This enables comprehensive detection of the location of pollution sources and the surrounding water quality.

[0153] (2) If and If the suspected pollution source area identified in step 2 is found to be non-polluting, then it is determined that there is no pollution source in that area.

[0154] (3) If and If the suspected pollution source area identified in step 2 is uncertain, it is necessary to re-collect data and conduct secondary detection using intelligent equipment.

[0155] Conventional multimodal fusion did not quantify the geometric center coordinates of the suspected pollution source area. Water and the geometric center coordinates of the connected contaminated area obtained by inversion The spatial correlation of water quality concentration inversion location directly splices and merges the two types of results, which can easily lead to false scenarios where "the location of the pollution source and the location of water quality inversion are misaligned, but are still judged as effectively merged", resulting in distorted decision results and extremely low comprehensive detection accuracy.

[0156] This invention addresses the industry pain point of false fusion, reducing cases of false fusion; it improves the accuracy of decision-level fusion results, achieving "precise location matching + reliable result fusion," making detection results more valuable and providing accurate data support for pollution control.

[0157] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0158] Example 2:

[0159] Intelligent equipment for detecting water pollution sources and sinks based on multimodal remote sensing fusion, using a mobile platform as a carrier, integrates multimodal remote sensing data acquisition, processing, fusion, detection, and result output. Figure 2 The diagram below shows the composition of the intelligent device of the present invention. The specific components are as follows:

[0160] Mobile platform: Selecting drones or mobile monitoring vehicles, it has flexible mobility and can realize large-area, all-round monitoring of water bodies near the riverbank, adapting to different terrains and monitoring scenarios.

[0161] Data acquisition module: mounted on a mobile platform, used to implement step 1 of embodiment 1, including a visible light camera, a thermal infrared camera, and a point multispectral analyzer. The three work synchronously to ensure the spatiotemporal consistency of multimodal remote sensing data. The visible light camera is used to acquire visible light remote sensing images (spatial texture remote sensing images) of the near-bank water body and shoreline. The thermal infrared camera is used to acquire thermal infrared remote sensing images (temperature distribution remote sensing images) of the near-bank water surface. The point multispectral analyzer is used to acquire spectral reflectance remote sensing data (400-1000nm characteristic band) of the near-bank water body. The visible light remote sensing images, thermal infrared remote sensing images, and spectral reflectance remote sensing data constitute multimodal remote sensing data. In one step, the resolution of the visible light remote sensing images and the thermal infrared remote sensing images are unified and spatially aligned.

[0162] Data processing module: Integrated on the mobile platform, used to implement step 2 of embodiment 1. In the data processing module: visible light remote sensing data and thermal infrared remote sensing data are input into a lightweight dual-branch convolutional neural network to extract primary features of visible light. and primary thermal infrared features Furthermore, max pooling is performed to obtain visible light pooling feature maps and thermal infrared pooling feature maps. Both the visible light pooling feature maps and thermal infrared pooling feature maps are then subjected to channel attention weighting and spatial attention weighting using the CBAM attention module, respectively, to obtain a weighted visible light pooling feature map. Weighted image of thermal infrared pooling features Visible pooling feature weighted map Weighted image of thermal infrared pooling features After channel stitching, feature fusion is performed through a 1×1 convolutional layer to obtain a visible light-thermal infrared fused feature map. Visible light-thermal infrared fusion feature map The data is input into the YOLOv8 lightweight object detection network to obtain preliminary location information of pollution sources, including suspected pollution source areas and the geometric center coordinates of the suspected pollution source areas. The maximum predicted probability of pollution source types in suspected pollution source areas and the confidence prediction value of suspected pollution source areas And further calculate the detection confidence level. .

[0163] Optical parameter identification and concentration inversion unit: integrated on a mobile platform, used to implement step 3 of embodiment 1. In the optical parameter identification and concentration inversion unit: based on the geometric center coordinates of the suspected pollution source area. Obtain the geometric center coordinates of the suspected pollution source area. Recent remote sensing data on spectral reflectance of the water body detection area was used to obtain pollution concentration inversion results for the water body detection area. The pollution concentration inversion results include pollutant type and pollutant concentration. Correlation coefficient of pollutant types Concentration inversion confidence level and pollutant concentration in the water body detection area Geometric center coordinates of connected contaminated areas exceeding the set concentration threshold .

[0164] Decision-level fusion unit: Integrated on the mobile platform, used to implement step 4 of embodiment 1, to perform decision-level fusion of preliminary pollution source location information and pollution concentration inversion results, integrate pollution source location and surrounding water quality information, and output the final comprehensive detection results.

[0165] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to replace them, without departing from the spirit of the invention or exceeding the scope defined by the invention.

Claims

1. A method for detecting water pollution sources and sinks based on multimodal remote sensing fusion, characterized in that, Includes the following steps: Step 1: Obtain visible light remote sensing images of the water body and shoreline near the riverbank, and obtain thermal infrared remote sensing images of the surface of the water body near the riverbank. Perform resolution unification and spatial alignment on the visible light remote sensing images and thermal infrared remote sensing images. Step 2: Input visible light remote sensing data and thermal infrared remote sensing data into a lightweight dual-branch convolutional neural network to extract primary features of visible light. and primary thermal infrared features Furthermore, max pooling was performed to obtain visible light pooling feature maps and thermal infrared pooling feature maps. Both the visible light pooling feature maps and the thermal infrared pooling feature maps were then subjected to channel attention weighting and spatial attention weighting using the CBAM attention module, respectively, to obtain a weighted visible light pooling feature map. Weighted image of thermal infrared pooling features Visible pooling feature weighted map Weighted image of thermal infrared pooling features After channel stitching, feature fusion is performed through a 1×1 convolutional layer to obtain a visible light-thermal infrared fused feature map. Visible light-thermal infrared fusion feature map The data is input into the YOLOv8 lightweight object detection network to obtain preliminary location information of pollution sources, including suspected pollution source areas and the geometric center coordinates of the suspected pollution source areas. The maximum predicted probability of pollution source types in suspected pollution source areas and the confidence prediction value of suspected pollution source areas And further calculate the detection confidence level. ; Step 3: Obtain the geometric center coordinates of the suspected pollution source area. Recent remote sensing data on spectral reflectance of the water body detection area was used to obtain pollution concentration inversion results for the water body detection area. The pollution concentration inversion results include pollutant type and pollutant concentration. Correlation coefficient Concentration inversion confidence level and pollutant concentration in the water detection area Geometric center coordinates of connected contaminated areas exceeding the set concentration threshold ; Step 4: Integrate the preliminary location information of the pollution source with the pollution concentration inversion results to output the final comprehensive detection results.

2. The water pollution source and sink detection method based on multimodal remote sensing fusion according to claim 1, characterized in that, The resolution unification process in step 1 includes the following steps: The resolution of thermal infrared remote sensing images is resampled to match that of visible light remote sensing images based on the following formula. ， in, These are the pixel coordinates of a visible light remote sensing image. The pixel coordinates of a thermal infrared remote sensing image The fitted pixel values, , , and The coordinates of pixels in a thermal infrared remote sensing image are arranged sequentially. The coordinates of the four adjacent pixels, , , as well as For thermal infrared remote sensing images at pixel coordinates Pixel coordinates Pixel coordinates and pixel coordinates Pixel value at that location, and For interpolation weights, , .

3. The water pollution source and sink detection method based on multimodal remote sensing fusion according to claim 2, characterized in that, The spatial alignment in step 1 includes the following steps: The Euclidean distances between feature points are obtained by traversing thermal infrared and visible light remote sensing images. Feature points with Euclidean distances less than or equal to... Pixels from thermal infrared remote sensing images and pixels from visible light remote sensing images are defined as non-collinear effective feature point pairs. For the first The pixel coordinates of feature points in a pair of non-collinear effective feature points in a visible light remote sensing image. For the first The pixel coordinates of feature points in a thermal infrared remote sensing image within a set of non-collinear effective feature point pairs. For matching distance threshold, For reliability weights, Constructing the affine transformation matrix : ， in, This is the horizontal scaling factor. This is the scaling factor in the vertical direction. The horizontal shear factor, The vertical shear factor. This is the horizontal translation amount. The vertical translation is calculated as follows: Initialize horizontal scaling factor Vertical scaling factor Horizontal shear factor Vertical shear factor Horizontal translation and vertical translation , The first Substituting the pixel coordinates of the feature points in the thermal infrared remote sensing image from the non-collinear effective feature point pairs into the following affine transformation formula, we obtain the corresponding fitted pixel coordinates. : ， To minimize the regularization constraint error function To determine the horizontal scaling factor, find the target value. Vertical scaling factor Horizontal shear factor Vertical shear factor Horizontal translation and vertical translation To further obtain the affine transformation matrix , The affine transformation matrix obtained by solving Spatial alignment of thermal infrared remote sensing images and visible light remote sensing images.

4. The water pollution source and sink detection method based on multimodal remote sensing fusion according to claim 3, characterized in that, The reliability weight Based on the following formula: ， in, For the first The smaller neighborhood gray-level variance of feature points in visible light remote sensing images and feature points in thermal infrared remote sensing images in a set of non-collinear effective feature point pairs. It represents the maximum value of the neighborhood gray-level variance for all pairs of non-collinear valid feature points.

5. The water pollution source and sink detection method based on multimodal remote sensing fusion according to claim 3, characterized in that, The regularization constraint error function Based on the following formula: ， ， in, The regularization constraint error function is... Let be the error function. The regularization coefficient is . To find the function with the maximum value, This is the Euclidean distance operator. For matching distance threshold, This represents the number of pairs of valid non-collinear feature points.

6. The water pollution source and sink detection method based on multimodal remote sensing fusion according to claim 3, characterized in that, In step 2, the confidence level is detected. Based on the following formula: 。 7. The water pollution source and sink detection method based on multimodal remote sensing fusion according to claim 3, characterized in that, The pollution concentration inversion results in step 3 are obtained based on the following steps: Based on the suspected pollution source area and the geometric center coordinates of the suspected pollution source area Identify the water body detection area 5m to 10m away from the nearest shoreline of the suspected pollution source area; The spectral reflectance remote sensing data of the identified water body detection area are analyzed to extract the pollutant types and concentrations in the water body detection area. and water pollutant concentration Correlation coefficient with reflectance remote sensing data ; Calculate the confidence level of concentration inversion , ， in, For pollutant concentration thresholds, The function for calculating the minimum value. Extract pollutant concentration in the water body detection area Geometric center coordinates of connected contaminated areas exceeding the set concentration threshold .

8. The water pollution source and sink detection method based on multimodal remote sensing fusion according to claim 7, characterized in that, The final comprehensive detection result in step 4 is obtained based on the following steps: Step 4.1: Define the identification framework H = {First proposition h1 is "Pollution source exists and corresponding pollution concentration exists", second proposition h2 is "No pollution source exists", third proposition h3 is "Uncertain"}; Record the first piece of evidence The second piece of evidence corresponds to the preliminary location information of the pollution source in step 2. Based on the pollution concentration inversion results from step 3, calculate the first evidence. Second evidence Location matching degree : ， in, The coordinates of the geometric center of the suspected pollution source area determined in step 2. The geometric center coordinates of the connected contaminated area determined in step 3 Euclidean distance, This is the distance adjustment coefficient. For location reliability coefficient, , Step 4.2: Calculate the first piece of evidence The first proposition: trust level First evidence The second proposition: trust level and the first piece of evidence The third proposition of trust ; Calculate the second evidence The first proposition: trust level Second evidence The second proposition: trust level and second evidence The third proposition of trust ; ， ， in, The confidence level weighting coefficient is... , ; As a correction factor for the reliability of evidence, , Step 4.3: Calculate the overall confidence level of the first proposition. The overall confidence level of the second proposition And the overall confidence level of the third proposition. , ， ， ， ， in, The normalization constant is and To identify propositions in frame H, and The value range is 1 to 3, when At that time, it indicates the proposition of primary evidence. The proposition of second evidence There is no conflict. Step 4.4, if and If the suspected pollution source area determined in step 2 is indeed a real pollution source, then the geometric center coordinates of the suspected pollution source area corresponding to the real pollution source will be output. Outputs the pollutant concentration in the water body detection area corresponding to the actual pollution source. Output the overall confidence level of the first proposition. ; like and If so, it is determined that there is no pollution source in the suspected pollution source area identified in step 2; like and If the area is not identified, the suspected pollution source area determined in step 2 is deemed uncertain.

9. Intelligent equipment for detecting water pollution sources and sinks based on multimodal remote sensing fusion, comprising a mobile platform as a carrier, characterized in that: Also includes: Data acquisition module: mounted on a mobile platform, used to acquire visible light remote sensing images of near-riverbank water bodies and shorelines, thermal infrared remote sensing images of the surface of near-riverbank water bodies, and spectral reflectance remote sensing data of near-riverbank water bodies, and to unify the resolution and spatially align the visible light remote sensing images and thermal infrared remote sensing images. Data processing module: Integrated on the mobile platform, the data processing module inputs visible light remote sensing data and thermal infrared remote sensing data into a lightweight dual-branch convolutional neural network to extract primary features of visible light. and primary thermal infrared features Furthermore, max pooling was performed to obtain visible light pooling feature maps and thermal infrared pooling feature maps. Both the visible light pooling feature maps and the thermal infrared pooling feature maps were then subjected to channel attention weighting and spatial attention weighting using the CBAM attention module, respectively, to obtain a weighted visible light pooling feature map. Weighted image of thermal infrared pooling features Visible pooling feature weighted map Weighted image of thermal infrared pooling features After channel stitching, feature fusion is performed through a 1×1 convolutional layer to obtain a visible light-thermal infrared fused feature map. Visible light-thermal infrared fusion feature map The data is input into the YOLOv8 lightweight object detection network to obtain preliminary location information of pollution sources, including suspected pollution source areas and the geometric center coordinates of the suspected pollution source areas. The maximum predicted probability of pollution source types in suspected pollution source areas and the confidence prediction value of suspected pollution source areas And further calculate the detection confidence level. , Optical parameter identification and concentration inversion unit: Integrated on the mobile platform, the optical parameter identification and concentration inversion unit is based on the geometric center coordinates of the suspected pollution source area. Obtain the geometric center coordinates of the suspected pollution source area. Recent remote sensing data on spectral reflectance of the water body detection area was used to obtain pollution concentration inversion results for the water body detection area. The pollution concentration inversion results include pollutant type and pollutant concentration. Correlation coefficient of pollutant types Concentration inversion confidence level and pollutant concentration in the water detection area Geometric center coordinates of connected contaminated areas exceeding the set concentration threshold , Decision-level fusion unit: Integrated on a mobile platform, it fuses preliminary location information of pollution sources with pollution concentration inversion results and outputs the final comprehensive detection results.

Images