Method and system for identifying aquatic algae and vegetation

Abstract

The present application relates to the technical field of water environment monitoring, and more particularly to a water body algae and vegetation identification method and system. The present application designs a water body algae and vegetation identification model, which comprises a visible light flow encoder, a multispectral flow encoder, an ASGA adaptive fusion module and an SCSC decoder; the visible light flow encoder, the multispectral flow encoder and the SCSC decoder replace the original 3*3 convolution with a multiscale snake convolution block, and the cross snake receptive field simultaneously captures the horizontal and vertical extension, so that the identification of the slender structure of the algae boundary is more accurate, the SCSC decoder also introduces multispectral features to increase the identification accuracy of submerged plants; the ASGA adaptive fusion module can adaptively generate fusion weights, thereby enhancing the fusion accuracy of the visible light feature map and the multispectral feature map. Through the design of the model and the loss function of the model, accurate identification of water body algae and vegetation identification is realized.

Description

Technical Field

[0001] This invention relates to the field of water environment monitoring technology, and in particular to a method and system for identifying algae and vegetation in aquatic bodies. Background Technology

[0002] Excessive algal blooms and degradation of aquatic vegetation are core issues in current water environment management. Cyanobacterial blooms continuously release highly toxic algal toxins, directly polluting drinking water sources and seriously threatening the drinking water safety and health of urban and rural residents. Aquatic vegetation, acting as a "purifier" of the aquatic ecosystem, directly reflects the state of ecological health through its distribution changes. Therefore, accurately identifying the spatial distribution and dynamic changes of aquatic algae and vegetation is a crucial prerequisite for water environment monitoring and management.

[0003] Traditional methods for identifying aquatic algae and vegetation mainly include manual sampling and single remote sensing identification methods. These primarily fall into two categories: manual field sampling and detection methods, and single remote sensing image identification methods. Both mainstream technologies have significant shortcomings, making it difficult to meet the demands of modern water environment monitoring for high precision, routine operation, and large-scale coverage. Manual sampling is the fundamental traditional method for aquatic ecological monitoring, relying mainly on personnel collecting water and vegetation samples by boat or wading, followed by laboratory physicochemical testing to determine algae concentration and identify vegetation species. This method can only obtain limited, discrete monitoring data from single points, with a very small coverage area, making it impossible to achieve simultaneous monitoring of large-area, comprehensive water bodies and failing to accurately reflect the overall spatial distribution pattern of algae and aquatic vegetation. Furthermore, subsequent laboratory testing procedures are cumbersome and time-consuming; routine chlorophyll a detection and vegetation species identification analyses can take 24-72 hours, and the testing process requires various consumables such as Nessler's reagent and high-performance liquid chromatography reagents, resulting in high overall monitoring costs. In addition, manual sampling is easily affected by accidental factors such as weather, water environment, and sampling site selection. The monitoring data is highly random and lacks representativeness, making it impossible to support accurate judgment of dynamic changes in the aquatic ecosystem.

[0004] To overcome the shortcomings of low efficiency and limited coverage of manual sampling methods, the industry has gradually introduced remote sensing identification technology for monitoring aquatic algae and aquatic vegetation. Early methods primarily relied on single remote sensing image identification schemes, but these schemes also have significant technical bottlenecks. Firstly, single visible light image identification is highly susceptible to external environmental interference. Environmental variables such as light intensity and water turbidity directly affect image quality, easily leading to misidentification of areas with high sediment reflectivity as algae clusters and shadowed areas as aquatic vegetation. Overall identification accuracy is generally below 80%, resulting in poor monitoring reliability. Secondly, single multispectral image identification schemes suffer from low spatial resolution, typically only 1-5 m / pixel, making it unable to identify small algae clusters smaller than 1 m², and difficult to capture localized micro-scale ecological anomalies in water bodies. Furthermore, the data preprocessing and analysis processes for this scheme are complex, and the equipment and computing power costs for standalone applications are high, resulting in insufficient cost-effectiveness and practicality.

[0005] To address the numerous drawbacks of traditional manual detection and single remote sensing identification technologies, existing research has gradually introduced deep learning algorithms and combined them with visible light imagery to construct intelligent identification models, achieving iterative upgrades in aquatic algae and vegetation identification technology. Compared to traditional monitoring methods, this intelligent solution eliminates the need for manual on-site sampling and cumbersome laboratory testing, significantly shortening the monitoring cycle and reducing labor and material costs. Simultaneously, leveraging the feature extraction capabilities of deep learning, it effectively improves the information utilization rate of image data and the degree of automation in identification, overcoming the technical limitations of traditional methods in large-scale, rapid monitoring. This provides a new technical approach for the routine monitoring of aquatic algae and vegetation.

[0006] However, the solution based on the intelligent recognition model relies on a single visible light data source for model training and recognition inference. The data dimension is limited. In complex water environments such as strong light refraction, turbid water, and interlacing light and shadow, the image features are highly interfering, and the model is prone to feature confusion, misjudgment, and missed judgment, resulting in poor environmental adaptability. Summary of the Invention

[0007] Therefore, the technical problem to be solved by the present invention is to overcome the shortcomings of intelligent recognition model-based schemes that rely solely on a single visible light data source for model training and recognition reasoning, resulting in a single data dimension and a tendency to produce misjudgments under complex lighting and water conditions.

[0008] To address the aforementioned technical problems, this invention provides a method for identifying algae and vegetation in aquatic bodies, comprising: Visible light and multispectral images of the target water body are acquired. The multispectral images are then registered with the visible light images to obtain the target multispectral images. Visible light images and target multispectral images are preprocessed. Based on the preprocessed target multispectral images, chlorophyll a concentration index and normalized vegetation index are calculated, and multispectral tensors are constructed. The preprocessed visible light image is input into the visible light flow encoder of the water body algae and vegetation recognition model, and the visible light feature map of each visible light flow coding layer is output; the visible light flow encoder is obtained by replacing the 3×3 convolution of each coding layer in the U-Net encoder with multi-scale serpentine convolution blocks. The multispectral tensor is input into the multispectral stream encoder of the aquatic algae and vegetation recognition model, and the multispectral feature map of each multispectral stream coding layer is output. The multispectral stream encoder replaces the first 3×3 convolution of each coding layer in the U-Net encoder with a multi-scale serpentine convolution block and the second 3×3 convolution with a 1×1 convolution. The multi-scale serpentine convolutional block includes three parallel branches: the first branch is a 3×3 convolution, the second branch is a 1×3 dilated convolution, and the third branch is a 3×1 dilated convolution. The input features of the multi-scale serpentine convolutional block are added element-wise to the outputs of the three parallel branches, and then passed through a batch normalization layer and a ReLU activation function to obtain the output features of the multi-scale serpentine convolutional block. Based on the visible light feature maps of each visible light stream coding layer and the multispectral feature maps of each multispectral stream coding layer, aquatic algae and vegetation in the target water body are identified.

[0009] Preferably, the method for calculating the chlorophyll a concentration index and the normalized vegetation index based on the preprocessed target multispectral image includes: Radiometric calibration is performed on the preprocessed target multispectral image to obtain the surface reflectance of each band in the preprocessed target multispectral image; The chlorophyll a concentration index and normalized vegetation index were calculated using the surface reflectance in the red and near-infrared bands after radiometric calibration.

[0010] Preferably, the method for identifying aquatic algae and vegetation in a target water body based on the visible light feature maps of each visible light stream coding layer and the multispectral feature maps of each multispectral stream coding layer includes: The visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are fused to obtain the l-th fused feature map; where l=1,2,… , l is the coding layer index, This represents the total number of layers in the encoder. Will The fused feature map is passed through a decoder to obtain the target features; the target features are then passed through a classifier to obtain the recognition results of aquatic algae and vegetation in the target water body.

[0011] Preferably, the method for fusing the visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer to obtain the l-th fused feature map includes: The visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are fused using the ASGA adaptive fusion module to obtain the l-th fused feature map, which includes: The visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are concatenated along the channel dimension, and then sequentially passed through a 1×1 convolution and a Sigmoid activation function to generate the l-th pixel-level weight map. Based on the l-th pixel-level weight map, the visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are fused to obtain the l-th initial fused feature map; The l-th initial fused feature map is processed by 3×3 convolution and 5×5 convolution respectively. The output feature maps of the two convolutions are added element-wise to the l-th initial fused feature map to obtain the l-th fused feature map.

[0012] Preferably, the method for fusing the visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer based on the l-th pixel-level weight map to obtain the l-th initial fused feature map includes: The l-th pixel-level weight map is multiplied element-wise with the visible light feature map of the l-th visible light flow coding layer to obtain the l-th weighted visible light feature map; The weight map, which is complementary to the weight map at the l-th pixel level, is multiplied element-wise with the multispectral feature map of the l-th multispectral stream coding layer to obtain the l-th weighted multispectral feature map. The l-th weighted visible light feature map and the l-th weighted multispectral feature map are added element by element to obtain the l-th initial fused feature map.

[0013] Preferably, the decoder is an SCSC decoder; Will Methods for obtaining target features from fused feature maps using an SCSC decoder include: Based on the multispectral feature map of the l-th multispectral stream coding layer, the l-th gated spectral feature is generated through a gate function; The feature map output from the Nn-th SCSC decoder layer is upsampled, then concatenated with the Nn-th fused feature map and the Nn-th gated spectral feature along the channel dimension. After processing by the Nn-1-th SCSC decoder layer, the feature map output by the Nn-1-th SCSC decoder layer is obtained. The SCSC decoder layer is obtained by replacing all 3×3 convolutions in the decoding layer of the U-Net decoder with multi-scale serpentine convolutional blocks. N is the total number of decoder layers, and n=0,1,…N-1. The feature map output from the first SCSC decoder layer is used as the target feature.

[0014] Preferably, the method for generating the l-th gated spectral feature based on the multispectral feature map of the l-th multispectral stream coding layer through a gate function includes: The multispectral feature map of the l-th multispectral stream coding layer is sequentially passed through a 1×1 convolution and a Sigmoid activation function to obtain the weights of the multispectral feature map of the l-th multispectral stream coding layer. The multispectral feature map of the l-th multispectral stream coding layer is multiplied element-wise with its weights to obtain the l-th gated spectral feature.

[0015] Preferably, the method for constructing the loss function of the aquatic algae and vegetation identification model includes: Construct a segmentation master loss based on Dice loss and class-weighted cross-entropy; The pixel prediction category labels output by the aquatic algae and vegetation recognition model are used as the grouping criteria. All pixels with the same prediction label are selected, and the feature vectors corresponding to each pixel in the multispectral feature map of the first multispectral stream coding layer are extracted to generate the feature vector set corresponding to each category label. The average value of all feature vectors corresponding to each category label is used as the feature center of that category label; Based on the Euclidean squared distance between the feature vectors within each category label and the feature centers of that category label, a spectral consistency loss is constructed. Based on segmentation principal loss and spectral consistency loss, a loss function is constructed for the identification model of algae and vegetation in water bodies.

[0016] Preferably, the loss function of the aquatic algae and vegetation identification model also includes boundary-aware loss, and the process of constructing the boundary-aware loss includes: Calculate the sum of the squared horizontal gradient and the squared vertical gradient of each pixel in the true value annotation map, find the square root of the sum, and use the smaller value between the square root and the value 1 as the boundary weight of the corresponding pixel. The cross-entropy loss of pixels at the same position is weighted by the boundary weights corresponding to each pixel, and the weighted cross-entropy loss of all pixels is accumulated to construct the boundary-aware loss.

[0017] The present invention also provides a system for identifying algae and vegetation in aquatic bodies, comprising: The data acquisition module is used to acquire visible light and multispectral images of the target water body. Based on the visible light image, the multispectral image is image registered to obtain the target multispectral image. The tensor construction module is used to preprocess visible light images and target multispectral images. Based on the preprocessed target multispectral images, the chlorophyll a concentration index and normalized vegetation index are calculated to construct multispectral tensors. The visible light feature extraction module is used to input the preprocessed visible light image into the visible light flow encoder and output the visible light feature map of each visible light flow coding layer; the visible light flow encoder is obtained by replacing the 3×3 convolution of each coding layer in the U-Net encoder with multi-scale serpentine convolution blocks. The multispectral feature extraction module is used to input the multispectral tensor into the multispectral stream encoder and output the multispectral feature map of each multispectral stream coding layer. The multispectral stream encoder replaces the first 3×3 convolution of each coding layer in the U-Net encoder with a multi-scale serpentine convolution block and the second 3×3 convolution with a 1×1 convolution. The multi-scale serpentine convolutional block includes three parallel branches: the first branch is a 3×3 convolution, the second branch is a 1×3 dilated convolution, and the third branch is a 3×1 dilated convolution. The input features of the multi-scale serpentine convolutional block are added element-wise to the outputs of the three parallel branches, and then passed through a batch normalization layer and a ReLU activation function to obtain the output features of the multi-scale serpentine convolutional block. The identification module is used to identify aquatic algae and vegetation in the target water body based on the visible light feature maps of each visible light stream coding layer and the multispectral feature maps of each multispectral stream coding layer.

[0018] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0019] This invention discloses a method and system for identifying algae and vegetation in aquatic bodies. It simultaneously incorporates dual-source data from visible light images and registered multispectral images. Through preprocessing, a multispectral tensor integrating chlorophyll a concentration index and normalized vegetation index is constructed. This achieves complementary fusion of visible light spatial texture features and multispectral aquatic biochemical features, effectively enriching the dimensions of the model input data and addressing the problems of limited information and insufficient feature representation capabilities from a single visible light data source. Furthermore, this invention optimizes and improves the convolutional structure of the U-Net encoder, replacing the traditional fixed convolutional structure with a multi-scale serpentine convolutional block. This convolutional block has three parallel convolutional branches: a standard 3×3 convolutional branch effectively captures the texture and blocky structure of the target's local central region; a 1×3 convolutional branch with a laterally extended receptive field structure, adapting to the extraction requirements of the lateral spread of cyanobacterial blooms; and a 3×1 convolutional branch with a vertically extended receptive field structure, accurately capturing the slender morphological features of the longitudinally spreading stems and leaves of submerged plants. Based on this, the convolutional block fuses the original input features with the output features of the three branches element by element, and combines batch normalization and RelU activation function to complete feature optimization. This not only preserves the original image features completely and alleviates the gradient vanishing problem during model training, but also breaks through the limitations of the traditional single 3×3 convolutional receptive field shape and poor adaptability. It can fully adapt to the irregular elongated distribution of algae and vegetation in water bodies, greatly improving the completeness and accuracy of target feature extraction under complex lighting and variable water body environments, and improving the recognition accuracy of algae and vegetation in water bodies. Attached Figure Description

[0020] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0021] Figure 1 This is a flowchart illustrating a method for identifying algae and vegetation in aquatic bodies according to the present invention.

[0022] Figure 2 This is a structural diagram of the aquatic algae and vegetation identification model.

[0023] Figure 3 This is a schematic diagram illustrating the difference between multi-scale serpentine convolution blocks and 3×3 convolution. Figure 3 (a) in the diagram is a schematic of the receptive field of a 3×3 convolution. Figure 3 (b) in the diagram is a schematic of the receptive field of a multi-scale serpentine convolution block. Figure 3 (c) in the diagram is a multi-scale serpentine convolution block structure diagram.

[0024] Figure 4 This is a schematic diagram of the ASGA adaptive fusion module structure.

[0025] Figure 5 This is a flowchart of the dataset construction process.

[0026] Figure 6 This is a diagram showing the structure of the loss function for the aquatic algae and vegetation identification model.

[0027] Figure 7 This is a comparison chart of the recognition results of single visible light U-Net, fixed weight fusion baseline and the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0029] In addition to visible light images, this invention also introduces multispectral images as an auxiliary verification source, which is suitable for large-area, high-precision monitoring of algae (such as cyanobacterial blooms) and aquatic vegetation (submerged, floating-leaved, and emergent plants) in water bodies such as lakes, rivers, and reservoirs.

[0030] Reference Figure 1 As shown, this embodiment provides a method for identifying algae and vegetation in aquatic bodies, including: Step S1: Acquire visible light and multispectral images of the target water body. Using the visible light image as a reference, perform image registration on the multispectral image to obtain the target multispectral image. In this embodiment, a drone is used to collect visible light and multispectral images of the target water body. After the drone imports the KM1 boundary of the target water body, it automatically analyzes the shape of the water body (e.g., a "parallel flight path along the river channel" is used for narrow rivers, and a "concentric circle flight path" is used for circular lakes). The optimal flight altitude is calculated to identify small areas of cyanobacteria clusters at an altitude of 50-100m with a resolution of 0.05-0.1m / pixel; and to identify large areas of aquatic vegetation at an altitude of 150m with a resolution of 0.2m / pixel, ensuring that no images are missed and that the overlap rate meets the standards. The visible light camera calibrates white balance using a standard color chart to avoid color deviation under different lighting conditions; the multispectral camera completes radiometric calibration by taking pictures with a gray card (20% / 50% / 80% reflectance) to provide a benchmark for subsequent reflectance calculations. By generating a route adapted to the water body, visible light and multispectral images are collected simultaneously to ensure complete image coverage and compliant resolution. The target water body in this embodiment is a nearly circular suburban lake with an area of ​​approximately 2 hectares. The flight parameters are shown in Table 1. Table 1 shows the UAV flight parameters for the target water body of the nearly circular suburban lake with an area of ​​approximately 2 hectares.

[0031] Table 1

[0032] The drone is equipped with both a visible light camera (RGB three-channel, Sony IMX586 sensor) and a multispectral camera (MicaSense RedEdge-MX, with five bands: blue 450nm, green 550nm, red 660nm, red edge 720nm, and near-infrared 840nm). The two cameras are synchronized via hardware triggering, with a time difference of less than 1 millisecond.

[0033] The significance of synchronous triggering: When the two cameras are shooting, the drone's position and attitude are completely consistent. Subsequent band registration only needs to eliminate the parallax introduced by the approximately 8-centimeter installation distance between the two cameras, without needing to handle flight position deviations, so the registration accuracy can reach sub-pixel level.

[0034] Step S2: Preprocess the visible light image and the target multispectral image. Based on the preprocessed target multispectral image, calculate the chlorophyll a concentration index and the normalized vegetation index, and construct the multispectral tensor. This invention performs preprocessing operations on the acquired raw visible light images. Specifically, the preprocessing operations include Gaussian filtering for noise reduction and geometric correction to remove noise interference and correct geometric distortion in the visible light images. Since the encoder structure of the aquatic algae and vegetation recognition model includes four 2x downsampling operations, requiring the width and height of the input image to be multiples of 16, this invention uniformly sets the single input image size for the aquatic algae and vegetation recognition model to 512×512 pixels.

[0035] To address the issue that the size of the preprocessed visible light image cannot match the standard 512×512 input size, a differentiated size adaptation strategy is adopted: When the resolution of the preprocessed visible light image is less than 512×512 pixels, zero-pixel padding is applied to the four edges of the visible light image to fill it to 512×512 pixels before inputting it into the aquatic algae and vegetation recognition model for recognition and inference; when the resolution of the preprocessed visible light image is greater than 512×512 pixels, a sliding window with a size of 512×512 pixels is used to segment and crop the entire visible light image into blocks, with a sliding window step size of 480 pixels, so that adjacent cropped sub-blocks form an overlapping area of ​​32 pixels in width. All cropped sub-blocks are input separately into the aquatic algae and vegetation recognition model to obtain the pixel-level prediction results corresponding to each sub-block.

[0036] For pixels in the 32-pixel overlap region between adjacent sub-blocks, the arithmetic mean of the Softmax classification probabilities obtained from two different window inferences is calculated, and the category corresponding to the maximum average probability is selected as the final classification result for that pixel. This processing method provides the window edge region with two independent prediction constraints, effectively compensating for the prediction bias caused by the incomplete receptive field of pixels at the sliding window edge, completely eliminating the stitching gaps generated after large image block prediction stitching, and ensuring the integrity and accuracy of the water algae and vegetation recognition results.

[0037] In this embodiment, specifically, the method for calculating the chlorophyll a concentration index and the normalized vegetation index based on the preprocessed target multispectral image includes: Radiometric calibration is performed on the preprocessed target multispectral image to obtain the surface reflectance of each band in the preprocessed target multispectral image; This invention utilizes a gray card calibration method (three standard gray cards with reflectivities of 5% / 25% / 50%), taking photos once before and after each flight. The average DN value for each band is calculated for the central 80% region of the gray cards, and the calibration coefficients are fitted using the least squares method. and The formula for calculating the surface reflectance of each band in the preprocessed multispectral image of the target is as follows: , in, coordinates The surface reflectance obtained by solving the radiometric calibration of a pixel. Multispectral image coordinates The original grayscale count value of the sensor at that location, These are the calibration slope coefficients corresponding to a single band. These are the calibration intercept coefficients corresponding to a single band. Represents the x-axis, Represents the vertical axis.

[0038] The coefficients for each spectral band are solved independently. When the relative deviation of the coefficient k obtained from the two calibrations at takeoff and landing exceeds 3%, the calibration coefficients for the entire flight path are corrected by linear interpolation. Finally, the reflectance of ground objects with a value range between 0 and 1 is obtained by conversion using the calibration formula.

[0039] The chlorophyll a concentration index was calculated using the surface reflectance in the red and near-infrared bands after radiometric calibration. Normalized Difference Vegetation Index (NDVI), the formula is: , , in, Near-infrared surface reflectance, Red light band surface reflectance; cyanobacteria zone The value ranges from 0.50 to 1.20, in the green algae region. The value ranges from 0.15 to 0.50, in the Qingshui District. The NDVI values ​​range from -0.10 to 0.10, the NDVI values ​​for submerged plants range from 0.20 to 0.50, the NDVI values ​​for cyanobacteria range from 0.05 to 0.25, and the NDVI values ​​for open water surfaces are less than 0.

[0040] In this embodiment, the multispectral tensor includes seven channels, as shown in Table 2. Table 2 is a schematic diagram of the multispectral tensor structure.

[0041] Table 2

[0042] Step S3: Input the multispectral tensor (7 channels) and the preprocessed visible light image (3 channels) into the aquatic algae and vegetation recognition model, and output the recognition results of aquatic algae (such as cyanobacteria and green algae) and vegetation (such as emergent plants and floating-leaved plants) (the category label of each pixel), so as to realize the high-precision and numerical representation of the spatial distribution of algae and aquatic vegetation in the water. Because visible light images and multispectral images differ in spatial resolution and spectral information, appropriate processing methods must be used during data fusion to ensure optimal fusion.

[0043] The preprocessed visible light image is input into the visible light flow encoder of the water body algae and vegetation recognition model, and the visible light feature map of each visible light flow coding layer is output; the visible light flow encoder is obtained by replacing the 3×3 convolution of each coding layer in the U-Net encoder with multi-scale serpentine convolution blocks. The multispectral tensor is input into the multispectral stream encoder of the aquatic algae and vegetation recognition model, and the multispectral feature map of each multispectral stream coding layer is output. The multispectral stream encoder replaces the first 3×3 convolution of each coding layer in the U-Net encoder with a multi-scale serpentine convolution block and the second 3×3 convolution with a 1×1 convolution. Among them, such as Figure 3 As shown, Figure 3 This diagram illustrates the difference between multi-scale serpentine convolution blocks and 3×3 convolutions. Figure 3 (a) in the diagram is a schematic of the receptive field of a 3×3 convolution. Figure 3 (b) in the diagram is a schematic diagram of the receptive field of a multi-scale serpentine convolution block. Figure 3 (c) in the diagram is a multi-scale serpentine convolution block structure diagram.

[0044] The multi-scale serpentine convolutional block consists of three parallel branches: the first branch (branch A) is a 3×3 convolution, the second branch (branch B) is a 1×3 dilated convolution, and the third branch (branch C) is a 3×1 dilated convolution. The input features of the multi-scale serpentine convolutional block are added element-wise to the outputs of the three parallel branches, and then passed through a batch normalization layer and a ReLU activation function to obtain the output features of the multi-scale serpentine convolutional block. The receptive field of a standard Conv3×3 is a 3×3 square, which can only see a localized blocky area of ​​9 pixels in the center. The distribution characteristics of aquatic algae do not match this shape: cyanobacterial blooms spread along the water surface in long, thin strips, and the stems and leaves of submerged plants are also long and thin. A square receptive field easily misses these horizontally or vertically extending structures.

[0045] As shown in Table 3, Table 3 is a schematic diagram of the multi-scale serpentine convolution block structure.

[0046] Table 3

[0047] The meaning of dilation=2: For Conv1×3, dilation=2 means that the 3 sampling points of the convolution kernel are spaced 2 pixels apart, and the actual coverage area is 1×5 (spanning 5 columns from left to right). However, the middle two columns are not included in the calculation, resulting in a jump in the receptive field. In this way, a smaller convolution kernel (1×3, with only 3 parameters) can cover a larger area (1×5), and the computational cost is less than using Conv1×5 directly.

[0048] As shown in Table 4, Table 4 shows the structural differences between the first visible light flow coding layer of the visible light flow encoder and the corresponding coding layer of the original U-Net.

[0049] Table 4

[0050] Based on the visible light feature maps of each visible light stream coding layer and the multispectral feature maps of each multispectral stream coding layer, aquatic algae and vegetation in the target water body are identified, including: The visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are fused to obtain the l-th fused feature map; where l=1,2,… , l is the coding layer index, This represents the total number of layers in the encoder. In this embodiment, optionally, the visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are weighted and fused to obtain the l-th fused feature map.

[0051] like Figure 4 As shown, Figure 4This is a schematic diagram of the ASGA adaptive fusion module. The ASGA adaptive fusion module generates a weight α for each pixel to determine whether the position should rely more on visible light or multispectral light.

[0052] In this embodiment, preferably, the method for fusing the visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer to obtain the l-th fused feature map includes: The visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are fused using the ASGA adaptive fusion module to obtain the l-th fused feature map, which includes: The visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are concatenated along the channel dimension, and then sequentially passed through 1×1 convolution and Sigmoid activation function to generate the l-th pixel-level weight map α. The visible light feature map and the multispectral feature map are concatenated along the channel dimension, doubling the number of channels. This is then compressed into a single channel by a 1×1 convolution (outputting only 1 channel, including bias, without batch normalization or ReLU activation). Finally, a sigmoid function is used to map the result to the range 0 and 1, yielding a pixel-level weight map α. The formula is as follows:

[0053] in, For the l-th pixel-level weight, It is the Sigmoid activation function. For 1×1 convolution, This is the visible light feature map of the l-th visible light flow coding layer. This is the multispectral feature map of the l-th multispectral stream coding layer. Each position value is between (0,1); the closer to 0, the more the position depends on multispectral light, and the closer to 1, the more it depends on visible light; each scale l=1,2,3,4 has a set of independent parameters, which are automatically determined by the training data.

[0054] After training convergence Measured distribution: Cyanobacteria-dense areas ( Approximately 0.86) Approximately 0.12 (88% multispectral contribution), revetment area (CIchla approximately 0.02). Approximately 0.90 (90% visible light contribution), open water surface Approximately 0.50 (balanced utilization of both modes).

[0055] Based on the l-th pixel-level weight map, the visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are fused to obtain the l-th initial fused feature map; The l-th initial fused feature map The output feature maps of the two convolutions are added element-wise to the l-th initial fused feature map using 3×3 and 5×5 convolutions respectively, to obtain the l-th fused feature map. The formula is: , in, It is a 3×3 convolution (padding=1). The output channel is the same as the input channel, and no activation function is added after the residuals are stacked to maintain a smooth gradient.

[0056] The methods for obtaining the l-th initial fused feature map include: The l-th pixel-level weight map is multiplied element-wise with the visible light feature map of the l-th visible light flow coding layer to obtain the l-th weighted visible light feature map; The weight map, which is complementary to the weight map at the l-th pixel level, is multiplied element-wise with the multispectral feature map of the l-th multispectral stream coding layer to obtain the l-th weighted multispectral feature map. The l-th weighted visible light feature map and the l-th weighted multispectral feature map are added element by element to obtain the l-th initial fused feature map.

[0057] The l-th initial fusion feature map The calculation formula is: , Will The fused feature map is passed through a decoder to obtain the target features; the target features are then passed through a classifier to obtain the recognition results of aquatic algae and vegetation in the target water body.

[0058] In this embodiment, optionally, The fused feature maps are processed by the U-Net decoder to obtain the target features.

[0059] In this embodiment, preferably, the following is used: Methods for obtaining target features from fused feature maps using an SCSC decoder include: Based on the multispectral feature map of the l-th multispectral stream coding layer, the l-th gated spectral feature is generated through a gate function, including: The multispectral feature map of the l-th multispectral stream coding layer is sequentially passed through a 1×1 convolution and a Sigmoid activation function to obtain the weights of the multispectral feature map of the l-th multispectral stream coding layer. The multispectral feature map of the l-th multispectral stream coding layer is multiplied element-wise with its weights to obtain the l-th gated spectral feature.

[0060] The l-th gated spectral feature The formula is: .

[0061] The Sigmoid function generates weights independently for each channel, suppressing irrelevant spectral channels (approaching 0) and retaining channels with discriminative value.

[0062] The feature map output from the Nn-th SCSC decoder layer is upsampled, then concatenated with the Nn-th fused feature map and the Nn-th gated spectral feature along the channel dimension. After processing by the Nn-1-th SCSC decoder layer, the feature map output by the Nn-1-th SCSC decoder layer is obtained. The SCSC decoder layer is obtained by replacing all 3×3 convolutions in the decoding layer of the U-Net decoder with multi-scale serpentine convolutional blocks. N is the total number of decoder layers, and n=0,1,…N-1. The feature map output from the first SCSC decoder layer is used as the target feature.

[0063] The SCSC decoder performs four upsampling operations from the deep layer (32×32) to the shallow layer (512×512), doubling the spatial size each time, and filtering out irrelevant channels through a gating function.

[0064] As shown in Table 5, Table 5 is a schematic diagram of the SCSC decoder and classifier.

[0065] Table 5

[0066] The convolutional dimensionality reduction layer of the SCSC decoder was also replaced with MS-SnakeConv. The snake-shaped receptive field also helps to recover the slender structure of the algae boundary during the decoding stage. Finally, a 512×512×4 probability map is output. The category corresponding to the highest probability is taken as the recognition result. If the original image has been cropped by a sliding window, the results of each window are stitched back to the original image size. The probability of the overlapping area is averaged and then the category with the highest probability is taken.

[0067] like Figure 2 As shown, Figure 2 This is a structural diagram of the aquatic algae and vegetation identification model.

[0068] Includes: visible light flow encoder, multispectral flow encoder, ASGA adaptive fusion module and SCSC decoder; In this embodiment, 500 pairs of photos (pairs 001 to 500) were collected to train the aquatic algae and vegetation recognition model. Each pair contains one visible light image and one set of 5-band multispectral images, totaling 1000 raw files.

[0069] like Figure 5 As shown, Figure 5Build a flowchart for the dataset.

[0070] The visible light images were annotated at the pixel level, as shown in Table 6. Table 6 shows the pixel-level annotation features for different categories.

[0071] Table 6

[0072] Blue-green algae and green algae are similar in color; their boundary regions are referenced from the same multispectral image. value, An index greater than 0.50 is prioritized for identification as cyanobacteria. The boundary between the revetment and submerged plants is determined by the NDWI index; an NDWI greater than 0 indicates open water, which is classified as category D. Additionally, 20 sampling points were set up according to the national standard sampling point distribution distance.

[0073] The dataset is divided into three stratified sampling sets: 400 pairs for training, 50 pairs for validation, and 50 pairs for test. The pixel ratios of the four classes are kept consistent across all subsets. The test set is archived throughout the process and used only once during the final evaluation.

[0074] The two cameras are installed about 8 centimeters apart, which corresponds to an offset of about 0.2 meters or about 10 pixels on the ground at a flight altitude of 40 meters.

[0075] Registration steps: Using visible light images as a reference, feature points in the multispectral red band (660nm) are extracted and matched with corresponding point pairs in the visible light image, with approximately 80 to 120 pairs per image.

[0076] The affine transformation matrix H is estimated using the RANSAC algorithm to eliminate the interference of incorrect matches.

[0077] Transformation matrix H was applied to all five bands of the multispectral array, and bilinear interpolation resampling was used to ensure complete pixel alignment between the two images. After registration, the cross-correlation coefficient r = 0.97, and the pixel offset was reduced from 8.3 pixels to 0.4 pixels.

[0078] The visible light label mask is projected onto the multispectral coordinate system using the same transformation matrix H (nearest neighbor interpolation, keeping integer labels unblurred), so the multispectral image does not need to be labeled separately.

[0079] The training set was augmented to include 3200 pairs: Background type D accounted for 62%, while submerged plants type C accounted for only 8%, a difference of nearly 8 times. To address the imbalance, 8 types of enhancements were applied to 400 pairs in the training set, generating 8 pairs for each pair, for a total of 3200 pairs. All enhancements applied the same transformation simultaneously to visible light, multispectral, and labeled mask.

[0080] As shown in Table 7, Table 7 illustrates the strategies for data augmentation of the training set.

[0081] Table 7

[0082] In this embodiment, the method for preprocessing visible light images includes: Gaussian filtering and geometric correction were performed on the visible light image. Specifically, Gaussian filtering was applied to each of the RGB channels, with a kernel size of 5×5 and a standard deviation σ=0.8. The reason for choosing σ=0.8 is that σ=0.5 is insufficient for denoising (signal-to-noise ratio only improves by 1 dB), and σ=1.5 results in 18% edge loss. σ=0.8 represents the optimal balance between the two, improving the signal-to-noise ratio by approximately 3 dB while minimizing edge loss by less than 5%.

[0083] Construct a Gaussian filter kernel with a size of 5×5. The weights at each position within the kernel are calculated using the following formula: , in, coordinates within the Gaussian filter kernel The weight, , These are the horizontal and vertical relative offset coordinates of the Gaussian filter kernel. , It is an exponential function with the natural constant as its base. Standard deviation and This is the traversal variable used to iterate through all coordinate positions within the 5×5 Gaussian filter kernel. After normalization, the sum of each weight is 1, and the filtering does not change the average brightness of the image.

[0084] Geometric correction: Nine 3×3 grid points (GCPs) were deployed around the lake. RTK-GPS coordinate accuracy was set to no more than 2 cm. The RPC model was fitted using the least squares method, with residuals required to be no more than 0.8 pixels. All 500 photos were resampled using bilinear interpolation and uniformly projected onto the WGS84 coordinate system. After correction, the stitching offset between adjacent photos was reduced from 4.2 pixels to 0.7 pixels.

[0085] The visible light flow encoder is initialized with pre-trained U-Net weights. Since the ImageNet dataset was used during pre-training, the visible light images are normalized according to ImageNet statistics. The normalization formula is: , in, For single-channel raw pixel values, This represents the mean of the corresponding channels in the ImageNet dataset. The standard deviation of the corresponding channel in the ImageNet dataset. This is the standardized channel value range.

[0086] The red channel has a mean of 0.485 and a standard deviation of 0.229, the green channel has a mean of 0.456 and a standard deviation of 0.224, and the blue channel has a mean of 0.406 and a standard deviation of 0.225; the standardized range is approximately -2.1 to 2.6.

[0087] like Figure 6 As shown, Figure 6 This is a diagram showing the structure of the loss function for the aquatic algae and vegetation identification model.

[0088] In this embodiment, preferably, the method for constructing the loss function (MC-Loss multi-constraint joint loss function) of the aquatic algae and vegetation identification model includes: For a single batch of training samples, based on Dice loss Category-weighted cross-entropy Construct the segmented principal loss To address class imbalance; Class D background accounts for 62%, while Class C submerged plants only account for 8%, a difference of nearly 8 times. A standard loss function can cause the network to become lazy, classifying all pixels as background. Composed of Dice loss plus class-weighted cross-entropy This forces the internet to seriously study rare categories.

[0089] Dice loss The calculation formula is: , The measured weights of this invention are as follows: cyanobacteria have a category weight of approximately 6.9, green algae have a category weight of approximately 10.4, submerged plants have a category weight of approximately 15.6, and background has a category weight of approximately 2.0; these weights are recalculated and dynamically updated before each batch.

[0090] Category-weighted cross-entropy The calculation formula is: , , in, For the first The category weights of each category This represents the total number of pixels in the current training batch of images. The total number of categories, For the first in the current batch The number of pixels corresponding to each category Predict the category probability for the pixel in the i-th row and j-th column. Let be the truth label of the pixel in the i-th row and j-th column. For the pixel in row i and column j, the corresponding pixel is... The predicted probabilities of each category, For the pixel in row i and column j, the corresponding pixel is... Truth labels for each category; Dice loss For rare species, add 1e-6 to prevent division by zero; The odds were approximately 0.821 in the first round and approximately 0.168 in the 100th round, showing a continuous decline.

[0091] The pixel prediction category labels output by the aquatic algae and vegetation recognition model after inference of a single batch of training samples are used as the grouping basis. All pixels with the same prediction label are selected, and the feature vectors corresponding to each pixel in the multispectral feature map (shape 256×256×32) of the first multispectral stream coding layer are extracted to generate the feature vector set corresponding to each category label. If a certain category has no corresponding prediction pixel in the training samples of this batch, the category is discarded. The average value of all feature vectors corresponding to each category label is used as the feature center of that category label; A spectral consistency loss is constructed by summing the mean Euclidean squared distances between the feature vectors within each category label and the feature centers of that category label. This technology was used to resolve the confusion caused by cyanobacteria, reducing the confusion rate from 18% to approximately 10%. , in, Indicates category, For a set of categories, For the current batch, the predicted rank is [number]. Total number of pixels in the class Indicates the first Within the category label, the first 1 eigenvector Indicates the first Feature centers of each category label.

[0092] Based on the weighted fusion of segmentation principal loss, spectral consistency loss, and boundary perception loss, a total loss function is constructed for the water algae and vegetation identification model.

[0093] The process of constructing the boundary-aware loss includes: Take the ground truth map corresponding to a single batch of training samples, calculate the sum of the squared horizontal gradient and the squared vertical gradient of each pixel in the ground truth map, find the square root of the sum, and use the smaller value between the square root and the value 1 as the boundary weight of the corresponding pixel. The boundary weights corresponding to each pixel are used to weight the cross-entropy loss of pixels at the same position. The weighted cross-entropy loss of all pixels is accumulated to construct the boundary-aware loss. This reduced the boundary misjudgment rate from 12% to 5%, a decrease of 58%.

[0094] The formula for the total loss function of the aquatic algae and vegetation identification model is: , in, For the total loss function, The weighting coefficients for segmenting the main loss, , For the first The weighting coefficients of the training spectral consistency loss. For the first The weighting coefficients of the boundary-aware loss during round training. The odds are 0.300 in round 0, 0.222 in round 30, and 0.111 in round 100; the odds are higher in the early stages and lower in the later stages. The odds are 0 in round 0, 0.126 in round 30, and 0.188 in round 100; small in the early stages and large in the later stages.

[0095] In the early stages of training, classification is not yet stable. Forcibly increasing the boundary penalty will amplify the gradients of incorrect predictions, introducing training instability. Therefore, in the first 30 rounds... First, establish a solid foundation for spectral discrimination; then gradually increase the intensity after 30 rounds. The focus shifted to refining the boundaries.

[0096] As shown in Table 8, Table 8 illustrates the parameter values ​​and effects for different training rounds.

[0097]

[0098] Framework: PyTorch 2.1; Hardware: NVIDIA RTX 3090 24GB; Training time: approximately 16 hours (slightly longer than v8, MS-SnakeConv has slightly more parameters). The training configuration for the aquatic algae and vegetation recognition model is as follows: The training setup for this aquatic algae and vegetation recognition model is as follows: Batch input contains four sets of samples, each set accompanied by a 4×512×512×3 visible light image and a 4×512×512×7 multispectral image. The two types of data are input into their respective branch encoders. The model uses the Adam optimizer, with hyperparameters configured as follows: β1=0.9 (exponential decay rate of the first moment estimate, used to control the moving average of the gradient mean), β2=0.999 (exponential decay rate of the second moment estimate, used to control the moving average of the squared gradient), ε=1e-8 (numerical stability constant, used to prevent the denominator from being zero during parameter updates). The learning rate is differentiated: the initial learning rate for the visible light branch (V-Stream) and decoder is set to 0.001, while the learning rate for the multispectral branch (S-Stream) and ASGA is set to... The initial learning rate for the module and the SCSC module is 0.0005. A periodic decay strategy is used, with the learning rate decreasing to 0.5 times its original value every 10 training epochs. By the 100th epoch, the learning rate is approximately 9.77 × 10⁻⁻⁶. 5 The weight initialization schemes are differentiated. V-Stream reuses pre-trained U-Net weights for initialization, while S-Stream, ASGA, SCSC, and MS-SnakeConv use Kaiming uniform distribution for random initialization of new network parameters. Early stopping is enabled during training; training terminates when the overall classification accuracy (OA) on the validation set shows no improvement for 15 consecutive rounds. The model optimization goal is to achieve a validation set OA of 92%. Additional regularization constraints are also added. Used to adjust the weight coefficients of the ASGA module This regular expression is for each layer of the network. The squared differences between the parameters and 0.5 are summed and a regularization coefficient of 0.01 is set to constrain the weight distribution and prevent the model training process from being overly biased towards a single image modality.

[0099] Large-scale outbreaks of algae such as cyanobacteria and green algae are called algal blooms, which can severely damage aquatic ecosystems. Timely and accurate identification of algal distribution is a prerequisite for controlling algal blooms. Currently, there are three core challenges in using drones to take photos and then identify the algae.

[0100] Challenge 1: Blue-green algae and green algae are too similar in color to be distinguishable in ordinary photos. In RGB photos, both appear as green hues, resulting in a misclassification rate as high as 12% for ordinary convolutional networks.

[0101] Challenge 2: The blending ratio of the two photos cannot be perfectly aligned. Figure 1 For example, multispectral imaging can distinguish spectral differences, but areas with dense cyanobacteria mainly rely on spectral signals, while areas along revetments mainly rely on texture signals, and the ability to integrate at fixed ratios is poor.

[0102] Challenge 3: Ordinary convolutional imaging cannot detect the banded distribution pattern of algae. Cyanobacterial blooms spread along the water surface in the form of long, thin bands. Ordinary 3×3 square receptive fields can only see localized blocky areas, easily missing these long, thin, spreading structures.

[0103] Challenge 4: Boundary regions are easily confused with similar targets. The color of cyanobacterial patches changes gradually at the edges, and ordinary loss functions do not focus on the boundaries; the spectral features of cyanobacteria are close together in the feature space, making them easy to confuse.

[0104] This invention addresses these four challenges by designing a water algae and vegetation identification model and its loss function (MC-loss function). The overall accuracy is improved from 85% to 92%, and the boundary misjudgment rate is reduced from 12% to 5%. Specific improvements are shown in Table 9, which presents the improvements of this invention, the technical problems it solves, and its beneficial effects.

[0105] Table 9

[0106] U-Net is a semantic segmentation network that uses an encoder-decoder structure. It passes features from each layer of the encoder to the decoder via skip connections. This framework is widely used in the field of medical image segmentation. The aquatic algae and vegetation recognition model (AS-MFNet) of this invention references the encoder-decoder approach in its overall framework, but makes substantial modifications to four key parts, forming an independent new network. The specific differences are shown in Table 10, which highlights the differences between this invention and U-Net.

[0107] Table 10

[0108] Multi-scale serpentine convolutional blocks replace square receptive fields with cross-shaped serpentine receptive fields, extending 4 pixels horizontally and vertically to specifically capture the strip-like distribution patterns of algae. This improves the ability to recognize elongated boundary structures, and together with MC-Loss, reduces the boundary misclassification rate from 12% to 5%.

[0109] The ASGA adaptive fusion module requires no manual parameter tuning; the network automatically learns the optimal fusion ratio for each pixel. The multispectral contribution in densely populated cyanobacteria areas automatically reaches 88%, and the visible light contribution in revetment areas automatically reaches 90%. OA (Optimal Aspect Ratio) is improved from 88% with fixed-weight fusion to 92%, an increase of 4 percentage points.

[0110] The SCSC decoder incorporates fresh multispectral features into each of its four decoding layers to prevent spectral information from being diluted by four upsampling operations. It is most effective for submerged plants (Category C), improving user accuracy from 79.8% to 89.5%, an improvement of nearly 10 percentage points.

[0111] MC-loss loss function This reduced the confusion rate of blue-green algae from 18% to approximately 10%. The boundary misjudgment rate was reduced from 12% to 5% (a reduction of 58%); dynamic weight scheduling ensured training stability, establishing a spectral foundation in the early stage and refining the boundary in the later stage.

[0112] This invention is based on the U-Net encoder-decoder approach and features a completely redesigned network for multimodal remote sensing identification of water bodies, encompassing five aspects: convolutional blocks, encoding structure, fusion mechanism, decoder, and loss function. The weights of the visible light flow (V-Stream) encoder are initialized using pre-trained U-Net weights to accelerate convergence. The multispectral flow (S-Stream) encoder, ASGA adaptive fusion module, SCSC decoder, and MC-loss loss function are all trained from scratch without borrowing any existing pre-trained parameters.

[0113] This invention follows a progressive process: aerial data acquisition → visible light preprocessing → multispectral feature calculation → multimodal deep learning fusion → results application. First, the operational flight path is planned using KML boundary files, and raw visible and multispectral images are acquired by the UAV at an altitude of 20-120m and 80% image overlap. Second, Gaussian filtering for noise reduction, RPC geometric correction, and sample data enhancement are sequentially applied to the visible light data. Finally, band registration and radiometric calibration preprocessing are performed on the multispectral data, and calculations are performed. Chlorophyll index and NDVI normalized vegetation index were used to construct a 7-channel multispectral feature tensor. Then, the AS-MFNet fusion model was adopted, relying on dual-stream coding, ASGA feature fusion and SCSC decoding structure, and MC-loss multi-classification loss function to complete pixel-level intelligent classification and generate pixel classification map. Finally, the output of SHP spatial distribution vector map, area statistics Excel table and PDF accuracy acceptance report were generated. The overall classification accuracy of the whole algorithm is OA≥92% and Kappa coefficient≥0.90. The boundary misjudgment rate after correction is controlled at 5%. Compared with the single visible light model, the classification accuracy is improved by 7%, forming a complete and practical aquatic algae / vegetation fine interpretation technology link.

[0114] This invention deploys 50 field sampling points in water bodies (at least 12 for each category), collects water samples on-site for testing to confirm the true category, and compares the results with the prediction results of the test set to calculate the accuracy index, as shown in Table 11. Table 11 is a schematic diagram of the accuracy evaluation of this invention, the fixed-weight fusion baseline, and the single visible light U-Net. The "fixed-weight fusion baseline" refers to a control model that fuses visible light features and multispectral features according to a preset fixed ratio (α=0.5, i.e., the two features are added with equal weight). Its network structure is completely consistent with this invention, except that the ASGA module is replaced with a fixed-weight addition operation to verify the contribution of the ASGA adaptive fusion mechanism to the improvement of recognition accuracy.

[0115] Table 11

[0116] like Figure 7 As shown, Figure 7 This is a comparison chart of the recognition results of single visible light U-Net, fixed weight fusion baseline, and the present invention. Figure 7 (a) shows a comparison of the overall accuracy of a single visible light U-Net, a fixed-weight fusion baseline, and the present invention. Figure 7 (b) in the figure is a comparison of the boundary misjudgment rate of single visible light U-Net, fixed weight fusion baseline and the present invention.

[0117] As shown in Table 12, Table 12 is a schematic diagram comparing the accuracy of single visible light U-Net, fixed weight fusion baseline and the present invention.

[0118] Table 12

[0119] This invention comprises three designs: First, an ASGA adaptive feature fusion module. Compared to a fixed-weight fusion scheme (overall accuracy OA=88%), the introduction of this module improves the overall OA of the model to 92%, contributing a 4 percentage point accuracy gain. This module adaptively allocates visible light and multispectral modal fusion weights pixel by pixel, prioritizing multispectral features for cyanobacteria regions (corresponding to 88% accuracy) and visible light features for revetment regions (corresponding to 90% accuracy), replacing the traditional fixed-ratio fusion mode. Second, an SCSC feature supplement decoder. Addressing the shortcoming of only 79.8% accuracy in single visible light recognition of submerged plants (Category C), this decoder continuously supplements fresh spectral features during the four upsampling processes, avoiding the loss of key spectral information such as NDVI during the decoding stage, thereby improving the classification accuracy of submerged plants to 89.5%, and the single-category accuracy to improve by 9.7 percentage points. Third, an MC-Loss composite loss function. This loss is composed of... and Collaborative composition, By constraining the spatial clustering of similar characteristics, the confusion rate of cyanobacteria was reduced from 18% to 10%. Applying double the penalty to boundary pixels reduced the boundary misclassification rate from 12% to 5%, with reductions of 44% and 58% respectively. At the same time, the model training process was kept stable by relying on the dynamic weight scheduling strategy of the loss term.

[0120] This model can output four types of standardized results: First, a SHP vector distribution map in the WGS84 (EPSG:4326) coordinate system, which vectorizes the pixel-level recognition results to obtain independent vector layers for cyanobacteria, green algae, and submerged plants, supporting direct loading and overlay in GIS software such as ArcGIS and QGIS; Second, an Excel area statistics report, which, based on a 2 cm ground resolution (one pixel corresponds to 4 cm²), calculates the actual area by multiplying the number of pixels by the area of ​​a single pixel, automatically calculating the actual hectare area of ​​cyanobacteria, green algae, and submerged plants, the proportion of each type of water body, and the cyanobacteria outbreak rate; Third, a PDF format accuracy assessment report, which includes a classification confusion matrix, overall accuracy (OA), user accuracy and producer accuracy for each category, Kappa coefficient, and ASGA module weights. The heat map; fourth, the time-series line graph of cyanobacteria area. The model inference process is repeatedly executed on the monthly remote sensing data from May to October 2026 to generate the time-series change curve of cyanobacteria area, so as to realize the dynamic monitoring of the entire process of algal bloom occurrence, expansion and decline.

[0121] Compared with traditional manual sampling and identification methods, this invention significantly reduces manpower, time, and material costs, and avoids the randomness of single-point sampling. Compared with identification methods based on a single remote sensing data source, it improves the reliability of identification through "high-resolution visible light identification + multispectral feature verification," and can quickly obtain the spatiotemporal dynamic changes of algae and aquatic vegetation in large-area water bodies. This provides scientific basis and technical support for water environment monitoring (such as early warning of cyanobacterial blooms), aquatic ecological protection, and water resource management, and helps to promote smart water environment governance and sustainable development in cities.

[0122] This second embodiment provides a system for identifying algae and vegetation in aquatic bodies, including: The data acquisition module is used to acquire visible light and multispectral images of the target water body. Based on the visible light image, the multispectral image is image registered to obtain the target multispectral image. The tensor construction module is used to preprocess visible light images and target multispectral images. Based on the preprocessed target multispectral images, the chlorophyll a concentration index and normalized vegetation index are calculated to construct multispectral tensors. The visible light feature extraction module is used to input the preprocessed visible light image into the visible light flow encoder and output the visible light feature map of each visible light flow coding layer; the visible light flow encoder is obtained by replacing the 3×3 convolution of each coding layer in the U-Net encoder with multi-scale serpentine convolution blocks. The multispectral feature extraction module is used to input the multispectral tensor into the multispectral stream encoder and output the multispectral feature map of each multispectral stream coding layer. The multispectral stream encoder replaces the first 3×3 convolution of each coding layer in the U-Net encoder with a multi-scale serpentine convolution block and the second 3×3 convolution with a 1×1 convolution. The multi-scale serpentine convolutional block includes three parallel branches: the first branch is a 3×3 convolution, the second branch is a 1×3 dilated convolution, and the third branch is a 3×1 dilated convolution. The input features of the multi-scale serpentine convolutional block are added element-wise to the outputs of the three parallel branches, and then passed through a batch normalization layer and a ReLU activation function to obtain the output features of the multi-scale serpentine convolutional block. The identification module is used to identify aquatic algae and vegetation in the target water body based on the visible light feature maps of each visible light stream coding layer and the multispectral feature maps of each multispectral stream coding layer.

[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for identifying algae and vegetation in aquatic bodies, characterized in that, include: Visible light and multispectral images of the target water body are acquired. The multispectral images are then registered with the visible light images to obtain the target multispectral images. Visible light images and target multispectral images are preprocessed. Based on the preprocessed target multispectral images, chlorophyll a concentration index and normalized vegetation index are calculated, and multispectral tensors are constructed. The preprocessed visible light image is input into the visible light flow encoder of the water body algae and vegetation recognition model, and the visible light feature map of each visible light flow coding layer is output; the visible light flow encoder is obtained by replacing the 3×3 convolution of each coding layer in the U-Net encoder with multi-scale serpentine convolution blocks. The multispectral tensor is input into the multispectral stream encoder of the aquatic algae and vegetation recognition model, and the multispectral feature map of each multispectral stream coding layer is output. The multispectral stream encoder replaces the first 3×3 convolution of each coding layer in the U-Net encoder with a multi-scale serpentine convolution block and the second 3×3 convolution with a 1×1 convolution. The multi-scale serpentine convolutional block includes three parallel branches: the first branch is a 3×3 convolution, the second branch is a 1×3 dilated convolution, and the third branch is a 3×1 dilated convolution. The input features of the multi-scale serpentine convolutional block are added element-wise to the outputs of the three parallel branches, and then passed through a batch normalization layer and a ReLU activation function to obtain the output features of the multi-scale serpentine convolutional block. Based on the visible light feature maps of each visible light stream coding layer and the multispectral feature maps of each multispectral stream coding layer, aquatic algae and vegetation in the target water body are identified.

2. The method for identifying aquatic algae and vegetation according to claim 1, characterized in that, Methods for calculating chlorophyll a concentration index and normalized vegetation index based on preprocessed target multispectral images include: Radiometric calibration is performed on the preprocessed target multispectral image to obtain the surface reflectance of each band in the preprocessed target multispectral image; The chlorophyll a concentration index and normalized vegetation index were calculated using the surface reflectance in the red and near-infrared bands after radiometric calibration.

3. The method for identifying aquatic algae and vegetation according to claim 1, characterized in that, Methods for identifying aquatic algae and vegetation in target water bodies based on visible light feature maps of each visible light stream coding layer and multispectral feature maps of each multispectral stream coding layer include: The visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are fused to obtain the l-th fused feature map; where l=1,2,… , l is the coding layer index, This represents the total number of layers in the encoder. Will The fused feature map is passed through a decoder to obtain the target features; the target features are then passed through a classifier to obtain the recognition results of aquatic algae and vegetation in the target water body.

4. The method for identifying aquatic algae and vegetation according to claim 3, characterized in that, The method for fusing the visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer to obtain the l-th fused feature map includes: The visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are fused using the ASGA adaptive fusion module to obtain the l-th fused feature map, which includes: The visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are concatenated along the channel dimension, and then sequentially passed through a 1×1 convolution and a Sigmoid activation function to generate the l-th pixel-level weight map. Based on the l-th pixel-level weight map, the visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer are fused to obtain the l-th initial fused feature map; The l-th initial fused feature map is processed by 3×3 convolution and 5×5 convolution respectively. The output feature maps of the two convolutions are added element-wise to the l-th initial fused feature map to obtain the l-th fused feature map.

5. The method for identifying aquatic algae and vegetation according to claim 4, characterized in that, The method for fusing the visible light feature map of the l-th visible light stream coding layer and the multispectral feature map of the l-th multispectral stream coding layer based on the l-th pixel-level weight map to obtain the l-th initial fused feature map includes: The l-th pixel-level weight map is multiplied element-wise with the visible light feature map of the l-th visible light flow coding layer to obtain the l-th weighted visible light feature map; The weight map, which is complementary to the weight map at the l-th pixel level, is multiplied element-wise with the multispectral feature map of the l-th multispectral stream coding layer to obtain the l-th weighted multispectral feature map. The l-th weighted visible light feature map and the l-th weighted multispectral feature map are added element by element to obtain the l-th initial fused feature map.

6. The method for identifying aquatic algae and vegetation according to claim 3, characterized in that, The decoder is an SCSC decoder; Will Methods for obtaining target features from fused feature maps using an SCSC decoder include: Based on the multispectral feature map of the l-th multispectral stream coding layer, the l-th gated spectral feature is generated through a gate function; The feature map output from the Nn-th SCSC decoder layer is upsampled, then concatenated with the Nn-th fused feature map and the Nn-th gated spectral feature along the channel dimension. After processing by the Nn-1-th SCSC decoder layer, the feature map output by the Nn-1-th SCSC decoder layer is obtained. The SCSC decoder layer is obtained by replacing all 3×3 convolutions in the decoding layer of the U-Net decoder with multi-scale serpentine convolutional blocks. N is the total number of decoder layers, and n=0,1,…N-1. The feature map output from the first SCSC decoder layer is used as the target feature.

7. The method for identifying aquatic algae and vegetation according to claim 6, characterized in that, The method for generating the l-th gated spectral features based on the multispectral feature map of the l-th multispectral stream coding layer through a gate function includes: The multispectral feature map of the l-th multispectral stream coding layer is sequentially passed through a 1×1 convolution and a Sigmoid activation function to obtain the weights of the multispectral feature map of the l-th multispectral stream coding layer. The multispectral feature map of the l-th multispectral stream coding layer is multiplied element-wise with its weights to obtain the l-th gated spectral feature.

8. The method for identifying aquatic algae and vegetation according to claim 1, characterized in that, Methods for constructing the loss function of aquatic algae and vegetation identification models include: Construct a segmentation master loss based on Dice loss and class-weighted cross-entropy; The pixel prediction category labels output by the aquatic algae and vegetation recognition model are used as the grouping criteria. All pixels with the same prediction label are selected, and the feature vectors corresponding to each pixel in the multispectral feature map of the first multispectral stream coding layer are extracted to generate the feature vector set corresponding to each category label. The average value of all feature vectors corresponding to each category label is used as the feature center of that category label; Based on the Euclidean squared distance between the feature vectors within each category label and the feature centers of that category label, a spectral consistency loss is constructed. Based on segmentation principal loss and spectral consistency loss, a loss function is constructed for the identification model of algae and vegetation in water bodies.

9. The method for identifying aquatic algae and vegetation according to claim 8, characterized in that, The loss function of the aquatic algae and vegetation identification model also includes boundary-aware loss, and the construction process of boundary-aware loss includes: Calculate the sum of the squared horizontal gradient and the squared vertical gradient of each pixel in the true value annotation map, find the square root of the sum, and use the smaller value between the square root and the value 1 as the boundary weight of the corresponding pixel. The cross-entropy loss of pixels at the same position is weighted by the boundary weights corresponding to each pixel, and the weighted cross-entropy loss of all pixels is accumulated to construct the boundary-aware loss.

10. A system for identifying algae and vegetation in aquatic bodies, characterized in that, include: The data acquisition module is used to acquire visible light and multispectral images of the target water body. Based on the visible light image, the multispectral image is image registered to obtain the target multispectral image. The tensor construction module is used to preprocess visible light images and target multispectral images. Based on the preprocessed target multispectral images, the chlorophyll a concentration index and normalized vegetation index are calculated to construct multispectral tensors. The visible light feature extraction module is used to input the preprocessed visible light image into the visible light flow encoder and output the visible light feature map of each visible light flow coding layer; the visible light flow encoder is obtained by replacing the 3×3 convolution of each coding layer in the U-Net encoder with multi-scale serpentine convolution blocks. The multispectral feature extraction module is used to input the multispectral tensor into the multispectral stream encoder and output the multispectral feature map of each multispectral stream coding layer. The multispectral stream encoder replaces the first 3×3 convolution of each coding layer in the U-Net encoder with a multi-scale serpentine convolution block and the second 3×3 convolution with a 1×1 convolution. The multi-scale serpentine convolutional block includes three parallel branches: the first branch is a 3×3 convolution, the second branch is a 1×3 dilated convolution, and the third branch is a 3×1 dilated convolution. The input features of the multi-scale serpentine convolutional block are added element-wise to the outputs of the three parallel branches, and then passed through a batch normalization layer and a ReLU activation function to obtain the output features of the multi-scale serpentine convolutional block. The identification module is used to identify aquatic algae and vegetation in the target water body based on the visible light feature maps of each visible light stream coding layer and the multispectral feature maps of each multispectral stream coding layer.

Images