Wetland vegetation intelligent monitoring method and system based on multi-dimensional data fusion

Abstract

The invention provides a wetland vegetation intelligent monitoring method and system based on multi-dimensional data fusion, and the method comprises the following steps: collecting a water area unmanned plane image of a wetland water area through a monitoring unmanned plane, and obtaining a water area remote sensing image of the wetland water area through a remote sensing satellite; the water area unmanned aerial vehicle image and the water area remote sensing image are fused into a water area fusion image, the submerged vegetation distribution condition is recognized, and a submerged vegetation monitoring route is planned; the underwater monitoring robot is controlled to continuously detect a wetland water area, and a submerged vegetation standard image and a submerged vegetation multispectral image are collected at the same time; performing image enhancement on the submerged vegetation standard image through a submerged vegetation identification model, and identifying and marking target submerged vegetation from the submerged vegetation enhanced image; selecting a target multispectral region containing target submerged vegetation from the submerged vegetation multispectral image; and vegetation feature data of the target submerged vegetation are calculated based on the target multispectral region. According to the invention, related data of submerged vegetation can be continuously and accurately monitored.

Description

Technical Field

[0001] The present invention belongs to the technical field of data fusion processing, and in particular relates to a wetland vegetation intelligent monitoring method and system based on multi-dimensional data fusion. Background Art

[0002] As one of the most productive ecosystems on Earth, wetlands play a vital role in maintaining biodiversity, regulating climate, and purifying water quality. As a key component of wetland ecosystems, submerged vegetation plays an important role in maintaining water quality, providing biological habitats, and regulating nutrient cycles in water bodies. However, in recent years, due to the impact of climate change and human activities, submerged vegetation in many wetlands is facing serious threats. In this context, accurate monitoring and assessment of the status of submerged vegetation has become a top priority in wetland protection and management. Traditional submerged vegetation monitoring methods mainly rely on manual sampling and simple underwater photography techniques. Although these methods can provide certain data support, they often have problems such as limited sampling range, low accuracy, and difficulty in large-scale continuous monitoring. Summary of the invention

[0003] The present invention provides a wetland vegetation intelligent monitoring method and system based on multi-dimensional data fusion, so as to solve the problem that it is difficult to continuously and accurately perform data monitoring on submerged vegetation in wetlands.

[0004] In a first aspect, the present invention provides a wetland vegetation intelligent monitoring method based on multi-dimensional data fusion, which is applied to a wetland vegetation integrated monitoring system, wherein the wetland vegetation integrated monitoring system includes a monitoring drone and an underwater monitoring robot deployed underwater in a wetland water area, and the method includes the following steps:

[0005] The monitoring drone is used to collect drone images of the wetland water area, and the remote sensing satellite is used to obtain remote sensing images of the wetland water area;

[0006] The water area UAV image and the water area remote sensing image are fused into a water area fusion image by a data fusion method, the submerged vegetation distribution is identified from the water area fusion image, and a submerged vegetation monitoring route of the underwater monitoring robot is planned;

[0007] Controlling the underwater monitoring robot to continuously detect the wetland water area according to the submerged vegetation monitoring route, and simultaneously collecting submerged vegetation standard images and submerged vegetation multispectral images of different types of submerged vegetation in the wetland water area;

[0008] Construct a submerged vegetation recognition model based on convolutional neural network;

[0009] Performing image enhancement on the submerged vegetation standard image by using the submerged vegetation recognition model to obtain a submerged vegetation enhanced image, and identifying and marking target submerged vegetation from the submerged vegetation enhanced image;

[0010] According to the target submerged vegetation marked in the submerged vegetation enhanced image, selecting a target multispectral region containing the target submerged vegetation in the submerged vegetation multispectral image of the same timestamp;

[0011] The vegetation characteristic data of the target submerged vegetation is calculated based on the target multi-spectral region.

[0012] Optionally, the step of fusing the water area UAV image and the water area remote sensing image into a water area fusion image by a data fusion method, identifying the distribution of submerged vegetation from the water area fusion image, and planning the submerged vegetation monitoring route of the underwater monitoring robot comprises the following steps:

[0013] Preprocessing the water area UAV image and the water area remote sensing image, and fusing the preprocessed water area UAV image and the water area remote sensing image into a water area fused image;

[0014] Extracting vegetation index features of the wetland water area from the water area fusion image;

[0015] Based on the vegetation index characteristics and using a pre-trained water vegetation classification model, the water vegetation distribution characteristics of the wetland water area are identified, and the water vegetation classification model is constructed based on a random forest classifier;

[0016] Based on the submerged vegetation distribution characteristics in the water area vegetation distribution characteristics, a submerged vegetation monitoring route of the underwater monitoring robot is generated by adopting a path planning algorithm.

[0017] Optionally, the preprocessing of the water area UAV image and the water area remote sensing image, and fusing the preprocessed water area UAV image and the water area remote sensing image into a water area fused image comprises the following steps:

[0018] Performing atmospheric correction and cloud removal processing on the water area remote sensing image;

[0019] Geometrically correcting the water area UAV image to be spatially aligned with the water area remote sensing image;

[0020] Projecting the water area UAV image and the water area remote sensing image into a unified standard coordinate system, and using a dynamic time warping algorithm to complete the time alignment of the water area UAV image and the water area remote sensing image;

[0021] The water area UAV image and the water area remote sensing image are fused into a water area fused image by adopting an orthogonalization method.

[0022] Optionally, the vegetation index feature includes one or more of a normalized difference vegetation index, a normalized difference water index, an enhanced vegetation index, a modified normalized difference vegetation index and a water vegetation index.

[0023] Optionally, the wetland vegetation integrated monitoring system further includes ground monitoring equipment deployed around the wetland water area, and the submerged vegetation monitoring route of the underwater monitoring robot is generated based on the submerged vegetation distribution characteristics in the water area vegetation distribution characteristics and using a path planning algorithm, including the following steps:

[0024] Determining a plurality of submerged vegetation distribution areas in the water area fusion image based on the submerged vegetation distribution characteristics in the water area vegetation distribution characteristics;

[0025] According to the area of ​​the submerged vegetation distribution area, a number of monitoring points are generated in each submerged vegetation distribution area by using a random sampling method;

[0026] Acquiring water level information of all monitoring points in the wetland water area through the ground monitoring equipment;

[0027] Based on all the monitoring points and the corresponding water level information, an optimal path algorithm is used to generate a submerged vegetation monitoring route for the underwater monitoring robot.

[0028] Optionally, the constructing of the submerged vegetation recognition model based on the convolutional neural network comprises the following steps:

[0029] Construct underwater image enhancement module based on U-NET network;

[0030] Construct the initial submerged vegetation recognition module based on the YOLOv5 network;

[0031] Using a preset submerged vegetation image data set to complete the training of the initial submerged vegetation recognition module to obtain a submerged vegetation recognition module;

[0032] The output end of the underwater image enhancement module is connected to the input end of the submerged vegetation recognition module to obtain a submerged vegetation recognition model.

[0033] Optionally, the underwater image enhancement module includes an image feature input layer, a multidimensional attention mechanism layer, a multidimensional convolution layer and an image feature output layer connected in sequence, the multidimensional attention mechanism layer includes an image channel attention mechanism layer, an image space attention mechanism layer and a refined feature output layer connected in sequence, the multidimensional convolution layer includes a feature map transformation layer, a multi-kernel convolution layer and a convolution feature fusion layer connected in sequence, and the loss function of the underwater image enhancement module includes any one or more of a smooth L1 loss function, a perceptual loss function and a multi-scale structural similarity loss function.

[0034] Optionally, the initial submerged vegetation identification module includes a feature input layer, a feature extraction layer, a feature sampling layer and a result output layer connected in sequence, the feature extraction layer includes a downsampling layer, a core feature extraction layer, a residual structure layer and a feature map fusion layer, the feature sampling layer includes a variant convolution layer, a feature amplification layer, a feature splicing layer, the residual structure layer and the multi-dimensional attention mechanism layer, and the loss function of the initial submerged vegetation identification module includes any one or more of a cross entropy loss function, a mean square error loss function and a mean absolute error loss function.

[0035] Optionally, the vegetation characteristic data includes the submerged vegetation coverage and the total biomass of the target submerged vegetation.

[0036] In the second aspect, the present invention also provides a wetland vegetation intelligent monitoring system based on multidimensional data fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the wetland vegetation intelligent monitoring method based on multidimensional data fusion as described in the first aspect is implemented.

[0037] The beneficial effects of the present invention are:

[0038] The present invention realizes all-round three-dimensional monitoring of wetland waters by combining a variety of advanced equipment such as drones, remote sensing satellites and underwater robots, overcoming the limitation that traditional methods are difficult to simultaneously monitor both above-water and underwater vegetation. Secondly, the data fusion technology is used to organically combine drone images and remote sensing images, which not only improves the accuracy and coverage of the data, but also provides a scientific basis for the path planning of the underwater robot, greatly improving the monitoring efficiency. Next, the underwater robot is used for real-time detection and image acquisition, and a submerged vegetation recognition model is constructed based on a convolutional neural network combined with image enhancement technology. The vegetation images collected by the underwater robot are identified and classified through the submerged vegetation recognition model, which greatly improves the accuracy and automation of submerged vegetation recognition. Finally, multispectral image analysis is used to obtain rich feature data of the target submerged vegetation to be monitored. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of a wetland vegetation intelligent monitoring method based on multi-dimensional data fusion in one embodiment of the present application.

[0040] Figure 2 This is a schematic diagram of the standard architecture of the U-NET network in one of the implementation modes of the present application.

[0041] Figure 3 This is a schematic diagram of the structure of an underwater image enhancement module in one embodiment of the present application.

[0042] Figure 4 This is a schematic diagram of the structure of the image channel attention mechanism layer in one implementation of the present application.

[0043] Figure 5 This is a schematic diagram of the structure of the image space attention mechanism layer in one of the implementation modes of the present application.

[0044] Figure 6 This is a schematic diagram of the structure of a submerged vegetation identification module in one embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0046] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0047] The present invention discloses a wetland vegetation intelligent monitoring method based on multi-dimensional data fusion, which is applied to a wetland vegetation comprehensive monitoring system. The wetland vegetation comprehensive monitoring system is an advanced ecological monitoring solution, which is composed of three core components: monitoring drones, ground monitoring equipment, and underwater monitoring robots. When combined with remote sensing satellites, it can form an integrated wetland monitoring mode of "sky-air-ground-water", realizing all-round and multi-dimensional monitoring of wetland ecosystems. Through this multi-dimensional and multi-level monitoring method, the wetland vegetation comprehensive monitoring system can comprehensively and accurately evaluate the health status of wetland ecosystems, providing a scientific basis for wetland protection and management.

[0048] The monitoring drone is the aerial monitoring unit of the system, which is mainly composed of the body, power system, flight control system, navigation system, image acquisition system and data transmission system. The body is made of lightweight composite materials to ensure flight stability and endurance. The power system usually uses an electric motor with a high-efficiency lithium battery to provide lasting flight power. The flight control system includes an attitude sensor, accelerometer and gyroscope to adjust the flight attitude in real time to ensure flight stability. The navigation system combines GPS, inertial navigation and visual positioning technology to achieve accurate route planning and autonomous flight. The image acquisition system is equipped with a high-resolution visible light camera and a multispectral camera to capture detailed surface information and vegetation features. The data transmission system uses high-bandwidth wireless communication technology to transmit the collected image data to the ground control station in real time. Through the collaborative work of these systems, the drone can fly autonomously along the preset route and collect high-quality wetland water image data.

[0049] The ground monitoring equipment is the land monitoring unit of the system, which is mainly composed of a sensor array, a data acquisition unit, an energy supply system and a communication module. The sensor array includes water level sensors, temperature sensors, humidity sensors, pH sensors and light intensity sensors, etc., which comprehensively monitor the environmental parameters of the wetland. The data acquisition unit uses a low-power microprocessor, which is responsible for the collection, preliminary processing and storage of sensor data. The energy supply system combines solar panels and long-lasting batteries to ensure the long-term stable operation of the equipment. The communication module uses low-power wide area network technology (such as LoRa or NB-IoT) to realize remote transmission of data. Through the cooperation of these components, the ground monitoring equipment can continuously and uninterruptedly collect environmental data around the wetland and provide important auxiliary information for vegetation monitoring.

[0050] The underwater monitoring robot is the underwater monitoring unit of the system, which is mainly composed of a waterproof shell, a propulsion system, an attitude control system, a navigation system, an image acquisition system, a water quality sensor and a communication system. The waterproof shell is made of high-strength pressure-resistant materials to ensure the safety of the internal electronic equipment. The propulsion system uses multiple vector thrusters to achieve flexible three-dimensional movement. The attitude control system includes a buoyancy adjustment device and a counterweight to ensure underwater stability. The navigation system combines sonar, inertial navigation and visual positioning technology to achieve accurate underwater positioning and path planning. The image acquisition system is equipped with an underwater high-definition camera and multi-spectral imaging equipment, which can obtain clear images of submerged vegetation under different water depths and lighting conditions. The water quality sensor is used to collect parameters such as water temperature, dissolved oxygen, and turbidity. The communication system uses acoustic communication technology to realize data exchange with the surface control station. Through the collaborative work of these systems, the underwater monitoring robot can autonomously detect wetland waters according to the preset route and collect high-quality submerged vegetation images and water quality data.

[0051] Figure 1 FIG. 1 is a flow chart of a wetland vegetation intelligent monitoring method based on multi-dimensional data fusion in one embodiment. Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the above method may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps. Figure 1 As shown, the wetland vegetation intelligent monitoring method based on multi-dimensional data fusion disclosed in the present invention specifically includes the following steps:

[0052] S101. Collect drone images of wetland waters through monitoring drones, and obtain remote sensing images of wetland waters through remote sensing satellites.

[0053] Among them, drone images of water areas are collected by monitoring drones. These drones are equipped with high-resolution cameras and can capture detailed images of wetland waters while flying at low altitudes. Drones usually fly along preset routes to ensure coverage of the entire monitoring area. During the flight, drones will automatically take images according to pre-set time intervals or spatial intervals. These images will contain information about the water surface, underwater vegetation, and the surrounding environment. The advantage of drone images is their high spatial resolution and flexible acquisition time, which can provide real-time and detailed images of wetland waters.

[0054] At the same time, remote sensing images of water areas are obtained through remote sensing satellites. Remote sensing satellites are usually equipped with multispectral or hyperspectral sensors, which can capture information about wetland waters on a larger scale. The acquisition frequency of satellite images may be low, but they can provide a wider perspective and richer spectral information. Satellite remote sensing images usually include multiple spectral channels such as visible light bands, near-infrared bands, and short-wave infrared bands. The information in these different bands is crucial for subsequent vegetation analysis. The combined use of the two image acquisition methods can give full play to their respective advantages. UAV images provide local details with high spatial resolution, while satellite remote sensing images provide large-scale overall overviews and rich spectral information.

[0055] S102. The water area UAV image and the water area remote sensing image are fused into a water area fusion image by a data fusion method, the submerged vegetation distribution is identified from the water area fusion image, and the submerged vegetation monitoring route of the underwater monitoring robot is planned.

[0056] Among them, after obtaining the water area UAV image and water area remote sensing image, the next step is to fuse the two images into a water area fusion image through data fusion method. The purpose of data fusion is to make full use of the advantages of data from different sources and improve the integrity and accuracy of information. The fusion process usually includes several key steps: first, image registration to ensure that the two images correspond in space and time; then feature extraction to extract key information from the two images; and finally, the application of fusion algorithms, such as principal component analysis (PCA), wavelet transform or deep learning methods.

[0057] Specific fusion methods may involve pixel-level fusion, feature-level fusion, or decision-level fusion. For example, using the Brovey transform method, high spatial resolution drone images can be combined with multi-spectral satellite images to improve the spatial and spectral resolution of the fused image. The fusion formula can be expressed as:

[0058] F i =(DN i / (R+G+B))*PAN

[0059] Among them, F i is the pixel value of the ith band after fusion, DN i It is the pixel value of the i-th band of the original multispectral image, R, G, B are the pixel values ​​of the red, green, and blue bands respectively, and PAN is the pixel value of the panchromatic band (high resolution).

[0060] The fused water fusion image retains the high spatial resolution details of the drone image and contains the rich spectral information of the satellite remote sensing image. Next, the distribution of submerged vegetation is identified from the water fusion image. This step usually involves image classification or target detection technology. Supervised learning methods such as support vector machines (SVM) or random forest algorithms can be used to train a classifier that can distinguish between water bodies, submerged vegetation and other landforms. In the classification process, vegetation indices such as the normalized difference vegetation index (NDVI) and the normalized difference water index (NDWI) may be used as features.

[0061] For example, the calculation formula for NDVI is:

[0062] NDVI=(NIR-RED) / (NIR+RED)

[0063] Among them, NIR is the reflectance of the near-infrared band, and RED is the reflectance of the red light band. The higher the NDVI value, the more luxuriant the vegetation coverage.

[0064] Next, the underwater monitoring robot's submerged vegetation monitoring route is planned based on the identified submerged vegetation distribution. Path planning needs to consider multiple factors, such as the distribution density of submerged vegetation, water depth, obstacles, etc. Heuristic algorithms such as A* algorithm or genetic algorithm can be used to generate the optimal path. The goal of path planning is to ensure that the monitoring robot can efficiently cover all important submerged vegetation areas while minimizing energy consumption and time costs.

[0065] For example, when using the A* algorithm, you can define a heuristic function h(n) to estimate the cost from the current node n to the target node:

[0066] h(n)=D*(dx+dy)

[0067] Where D is a weight coefficient, dx and dy are the distances from the current node to the target node in the x and y directions respectively.

[0068] S103. Control the underwater monitoring robot to continuously detect the wetland water area according to the submerged vegetation monitoring route, and simultaneously collect submerged vegetation standard images and submerged vegetation multispectral images of different types of submerged vegetation in the wetland water area.

[0069] Among them, underwater monitoring robots are usually equipped with a variety of sensors and imaging devices, which can autonomously navigate underwater and collect various data. When performing monitoring tasks, the robot needs to accurately control its movement to ensure that it can follow the predetermined route while avoiding possible obstacles. In order to achieve precise control, underwater monitoring robots usually use a composite navigation system, combining an inertial navigation system (INS), an acoustic positioning system (such as a long baseline acoustic positioning system LBL or an ultra-short baseline acoustic positioning system USBL), and a pressure depth gauge. The robot's motion control system processes the data from these sensors in real time and adjusts the output of the thrusters to maintain the predetermined route and depth.

[0070] During the detection process, the underwater monitoring robot simultaneously collects standard images and multispectral images of different types of submerged vegetation in the wetland waters. Standard images are usually taken in the visible light range using a high-resolution optical camera, which can provide detailed morphological information of submerged vegetation. These images are essential for subsequent vegetation identification and classification. Multispectral images are collected using a multispectral camera, which can simultaneously record spectral information of multiple different bands. Typical multispectral cameras include blue light, green light, red light, red edge and near infrared bands. The collection of multispectral images requires special attention to the absorption and scattering characteristics of water bodies for light of different wavelengths. For example, red light attenuates faster in water than blue and green light, so in deep water areas it may be necessary to adjust camera parameters or use special filters to optimize image quality.

[0071] In actual operation, the robot may need to dynamically adjust its imaging parameters according to different depths and water quality conditions. For example, in turbid water bodies, it may be necessary to shorten the exposure time and increase the light source intensity; in clear water bodies, longer exposure times can be used to capture more details. In this way, the underwater monitoring robot can systematically collect high-quality image data of various types of submerged vegetation in wetland waters. These data not only contain the morphological characteristics of vegetation, but also contain rich spectral information, providing comprehensive data support for subsequent vegetation identification, classification, and ecological parameter estimation.

[0072] S104. Construct a submerged vegetation recognition model based on convolutional neural network.

[0073] S105. Perform image enhancement on the submerged vegetation standard image using a submerged vegetation recognition model to obtain a submerged vegetation enhanced image, and identify and mark the target submerged vegetation from the submerged vegetation enhanced image.

[0074] S106. According to the target submerged vegetation marked in the submerged vegetation enhanced image, a target multispectral region including the target submerged vegetation is selected in the submerged vegetation multispectral image with the same time stamp.

[0075] Among them, according to the target submerged vegetation marked in the submerged vegetation enhanced image, selecting the target multispectral area containing the target submerged vegetation in the submerged vegetation multispectral image of the same timestamp is a complex process involving image registration, feature matching and multispectral data processing. The purpose of this step is to associate the identified and marked submerged vegetation area with the corresponding multispectral data to obtain richer spectral information for more in-depth analysis. First, it is necessary to ensure that the submerged vegetation enhanced image and the multispectral image have the same timestamp. This is usually achieved through synchronous acquisition or post-data matching. Temporal consistency is critical to ensure that the two images reflect the same vegetation state, especially in rapidly changing underwater environments.

[0076] Next, image registration is required to ensure that the enhanced image and the multispectral image accurately correspond in space. The registration process usually includes the following steps:

[0077] Extract salient feature points from two images. Common methods include SIFT (Scale Invariant Feature Transform) or SURF (Speeded Up Robust Features) algorithms. These algorithms can identify key points in the image and generate feature vectors that describe these points.

[0078] Match the feature points in the two images. You can use a nearest neighbor matching algorithm, such as kd-tree or FLANN (Fast Nearest Neighbor Search Library). The matching process can be expressed as: d = ||f1-f2||, f1 and f2 are the descriptors of the feature points in the two images, and d is the Euclidean distance between them.

[0079] Based on the matched feature point pairs, the geometric transformation relationship between the two images is estimated. The RANSAC (Random Sample Consensus) algorithm is usually used to eliminate false matches and estimate the transformation matrix. The transformation matrix T can be expressed as: [x′, y′, 1] = T*[x, y, 1], where (x, y) and (x', y') are the coordinates of the corresponding points in the source image and the target image respectively.

[0080] Using the estimated transformation matrix, the multispectral image is resampled to the same coordinate system as the enhanced image. This typically involves an interpolation algorithm such as bilinear interpolation or cubic spline interpolation.

[0081] After completing the registration, the next step is to locate and extract the submerged vegetation areas in the multispectral image that correspond to the markers in the enhanced image. Specifically, the marked areas in the enhanced image (usually binary masks) are first converted to the coordinate system of the registered multispectral image. Then, based on the converted markers, the corresponding regional data is extracted from each band of the multispectral image. The extracted multiband data is then organized into a multidimensional array, with each pixel corresponding to a spectral vector.

[0082] S107. Obtain vegetation characteristic data of the target submerged vegetation based on the target multispectral region calculation.

[0083] Among them, the purpose of this step is to extract meaningful ecological parameters from multispectral data, especially submerged vegetation cover and total submerged vegetation biomass. These parameters are crucial for assessing the health of wetland ecosystems and monitoring their dynamic changes.

[0084] Submerged vegetation coverage refers to the proportion of the area covered by vegetation to the total area. The calculation process is as follows:

[0085] Use multispectral data to calculate vegetation indices. For underwater environments, commonly used indices include:

[0086] a) Modified Normalized Difference Vegetation Index (MNDVI):

[0087] MNDVI=(R NIR -R Green MNDVI=(R NIR -R Green ) / (R NIR +R Green )

[0088] Among them, R_NIR is the reflectivity of the near-infrared band, and R_Green is the reflectivity of the green light band.

[0089] b) Submerged Vegetation Index (SVI):

[0090] SVI=(R Green -R Red ) / (R Green +R Red )

[0091] Among them, R_Red is the reflectivity of the red light band.

[0092] Based on the calculated vegetation index, a threshold value is set to distinguish between vegetated and non-vegetated areas.

[0093] Finally, the ratio of vegetation pixels to the total number of pixels is calculated.

[0094] Next, the total biomass of submerged vegetation needs to be estimated. Biomass is not only related to the vegetation coverage area, but also closely related to the height, density and physiological state of the vegetation. The estimation process usually includes the following steps:

[0095] The minimum sampling number was determined by calculation, and the sampling method was set to obtain the average biomass per unit area of ​​submerged plants in the wetland waters3. The average biomass per unit area was multiplied by the total cover of submerged plants to obtain the total biomass of submerged plants in the wetland waters.

[0096] In one embodiment, a water area UAV image and a water area remote sensing image are fused into a water area fusion image by a data fusion method, and the submerged vegetation distribution is identified from the water area fusion image and a submerged vegetation monitoring route of an underwater monitoring robot is planned, including the following steps:

[0097] Preprocessing water area UAV images and water area remote sensing images, and fusing the preprocessed water area UAV images and water area remote sensing images into a water area fusion image;

[0098] Extract the vegetation index characteristics of wetland waters from the fused waters image;

[0099] Based on the vegetation index characteristics and using the pre-trained water vegetation classification model, the water vegetation distribution characteristics of the wetland water area are identified. The water vegetation classification model is built based on the random forest classifier;

[0100] Based on the distribution characteristics of submerged vegetation in the water area vegetation distribution characteristics, a path planning algorithm is used to generate the submerged vegetation monitoring route of the underwater monitoring robot.

[0101] In this embodiment, geometric correction and radiation correction are required for water drone images. Geometric correction aims to eliminate image deformation caused by factors such as drone flight altitude and attitude changes, and usually adopts control point matching and polynomial transformation methods. For example, an affine transformation can be used: [x′, y′, 1] = [a, b, c; d, e, f; 0, 0, 1] * [x, y, 1], where (x, y) is the original coordinate, (x', y') is the corrected coordinate, and a to f are transformation parameters. Radiation correction is used to eliminate the influence of factors such as atmosphere and sun angle. Commonly used methods include dark pixel method and empirical linear method.

[0102] For water remote sensing images, in addition to geometric correction and radiation correction, atmospheric correction is also required to eliminate the effects of atmospheric scattering and absorption. Common atmospheric correction methods include FLAASH and 6S models. Next, the two images need to be registered to ensure that they correspond precisely in space. The registration process usually includes feature point extraction (such as SIFT algorithm), feature matching, and geometric transformation estimation. For example, the RANSAC algorithm can be used to estimate the transformation matrix and remove incorrect matching points. After preprocessing, image fusion is performed. Common fusion methods include IHS transformation, principal component analysis (PCA), and wavelet transformation. Taking the IHS transformation method as an example, the RGB space of the drone image is first converted to the IHS space. Then the 1 component is replaced with the high-resolution panchromatic band of the remote sensing image, and the IHS inverse transformation is performed to obtain the fusion result. This fusion method can retain the high spatial resolution details of the drone image while fusing the spectral information of the remote sensing image. The spectral response differences of different sensors also need to be considered in the fusion process, and spectral matching or histogram matching may be required.

[0103] Extracting the vegetation index characteristics of wetland waters from the fused waters image is a key step, involving the calculation and analysis of multiple spectral indices. Vegetation index is a mathematical combination based on the reflectance of different bands, which can effectively reflect the physiological characteristics and growth status of vegetation. First, the reflectance data of each spectral band needs to be extracted from the fused image. For typical multispectral images, this usually includes blue, green, red and near-infrared bands. The spectral response function of the sensor and the atmospheric influence need to be considered in the extraction process, and radiometric calibration and atmospheric correction may be required. Next, a series of vegetation indices are calculated. Vegetation index characteristics include one or more of the normalized difference vegetation index, normalized difference water index, enhanced vegetation index, modified normalized difference vegetation index and water vegetation index. The most commonly used is the normalized difference vegetation index (NDVI): NDVI = (NIR-RED) / (NIR + RED), where NIR and RED are the reflectances of the near-infrared and red bands, respectively. NDVI is sensitive to vegetation coverage and biomass, with a range of -1 to 1. Usually, the NDVI in vegetated areas is greater than 0.2. For aquatic environments, the Normalized Difference Water Index (NDWI) can also be calculated: NDWI = (GREEN-NIR) / (GREEN+NIR) to distinguish between water bodies and land. In addition, the Enhanced Vegetation Index (EVI): EVI = 2.5*(NIR-RED) / (NIR+6*RED-7.5*BLUE+1) can reduce soil background and atmospheric effects and perform better in densely vegetated areas. For submerged vegetation, the Submerged Vegetation Index (SVI) can be used: SVI = (GREEN-RED) / (GREEN+RED), which is more sensitive to underwater vegetation. When calculating these indices, it is necessary to pay attention to handling outliers and division by zero errors. Thresholds can be set or conditional statements can be used. After the calculation is completed, a spatial distribution map of the vegetation index can be generated, with different colors used to represent changes in the index value. In order to extract richer features, texture features such as contrast, correlation and other statistics of the gray-level co-occurrence matrix (GLCM) can also be calculated.

[0104] Based on the vegetation index characteristics and using the pre-trained water vegetation classification model, the water vegetation distribution characteristics of the wetland water area are identified. The water vegetation classification model used in this embodiment is built based on the random forest classifier, which is a powerful integrated learning method. The random forest consists of multiple decision trees, each tree is independently trained and predicted, and the final result is determined by the vote of all trees. The training process of the model is as follows: First, a large amount of wetland vegetation sample data of known categories is collected, including various vegetation indices (such as NDVI, NDWI, EVI, SVI, etc.) as features. Then, the bootstrap method is used to randomly select samples from the training set, and for each decision tree, a feature subset is randomly selected when the node is split. This randomness helps to reduce overfitting and improve the generalization ability of the model. The growth process of the decision tree uses Gini impurity or information gain as the splitting criterion. For example, the calculation formula of Gini impurity is: Gini = 1-∑(pi 2 ), where p i is the probability of each category. When applying the trained model for classification, the water fusion image is first divided into grids, and the vegetation index features are calculated for each grid. Then, these features are input into the random forest model, and the model outputs the probability that each grid belongs to each vegetation type. The final classification result usually uses the category corresponding to the highest probability. Through the above process, a detailed vegetation distribution map of the wetland waters can be obtained, including the spatial distribution of different types of aquatic vegetation (such as submerged vegetation, floating leaf vegetation, and emergent vegetation).

[0105] Next, it is necessary to extract the spatial distribution information of submerged vegetation from the distribution characteristics of water vegetation. This usually involves spatial clustering and boundary extraction technology. The DBSCAN (density-based spatial clustering) algorithm can be used to identify the clustering areas of submerged vegetation. The core idea of ​​the algorithm is to find density-connected areas. The two key parameters of DBSCAN are ε (neighborhood radius) and MinPts (minimum number of points). For example, for each point p, if there are at least MinPts points in its ε neighborhood, p is considered to be a core point. By connecting all density-reachable core points, a cluster of submerged vegetation can be formed. The α-shape algorithm can be used for boundary extraction, which can more accurately describe the shape of submerged vegetation patches. The principle of the α-shape algorithm is to "carve" a point set with a disk with a radius of α, and the choice of α value will affect the fineness of the boundary. After extracting the boundary, the continuous spatial representation needs to be discretized into a series of sampling points, which will serve as potential target points for the underwater monitoring robot. Finally, the path planning algorithm is used to generate the optimal path for the underwater monitoring robot.

[0106] In one embodiment, preprocessing a water area UAV image and a water area remote sensing image, and fusing the preprocessed water area UAV image and water area remote sensing image into a water area fused image comprises the following steps:

[0107] Perform atmospheric correction and cloud removal on water remote sensing images;

[0108] Geometrically correct the water area UAV image to align it with the water area remote sensing image;

[0109] Project the water area UAV images and water area remote sensing images into a unified standard coordinate system, and use the dynamic time warping algorithm to complete the time alignment of the water area UAV images and water area remote sensing images;

[0110] The orthogonalization method is used to fuse water area UAV images and water area remote sensing images into water area fusion images.

[0111] In this embodiment, atmospheric correction and cloud removal processing of water remote sensing images are key steps in remote sensing image preprocessing, which aims to eliminate the influence of the atmosphere on spectral information and remove cloud cover to obtain more accurate surface reflectance data. Atmospheric correction processing first needs to determine the atmospheric model, usually using MODTRAN (MODerate resolution atmospheric TRANsmission) or 6S (Second Simulation of Satellite Signal in the Solar Spectrum) model. These models take into account the scattering and absorption effects of gas molecules, aerosols and water vapor in the atmosphere on electromagnetic waves. The core of atmospheric correction is to solve the radiation transfer equation: L = Lp + (Eg * Tg + Ed) * ρ / (π * (1-S * ρ)), where L is the radiation brightness received by the sensor, Lp is the atmospheric path radiation, Eg is the direct irradiance of the surface, Tg is the atmospheric transmittance, Ed is the atmospheric scattered irradiance, ρ is the surface reflectivity, and S is the albedo of the large balloon surface. By inverting this equation, the true surface reflectivity can be extracted from the radiance observed by satellite.

[0112] Next, the water UAV image needs to be geometrically corrected to align with the water remote sensing image in space. The purpose of this step is to eliminate the geometric deformation caused by factors such as flight altitude, attitude change and lens distortion in the UAV image, so that it can accurately correspond to the remote sensing image in spatial position. First, the distortion model of the UAV image needs to be determined, which usually includes radial distortion and tangential distortion. The radial distortion can be represented by a polynomial model:

[0113] x′=x(1+k1r 2 +k2r 4 +k3r 6 )

[0114] y′=y(1+k1r 2 +k2r 4 +k3r6 )

[0115] Where (x, y) is the ideal coordinate, (x', y') is the actual coordinate, r is the distance to the center of the image, k1, k2, k3 are the distortion coefficients. The tangential distortion model is:

[0116] x"=x′+[2p1xy+p2(r 2 +2x 2 )]

[0117] y"=y′+[p1(r 2 +2y 2 )+2p2xy]

[0118] Where p1 and p2 are tangential distortion parameters. After obtaining these parameters through camera calibration, the UAV image can be subjected to preliminary distortion correction. Next, it is necessary to establish a correspondence between the UAV image and the remote sensing image. This is usually achieved by matching control points, and feature point detection and matching algorithms such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) can be used. The core idea of ​​the SIFT algorithm is to detect local extreme points in different scale spaces and calculate the direction histogram of these key points as descriptors. For example, the SIFT descriptor is a 128-dimensional vector representing the gradient histogram of 8 directions in a 4x4 grid around the key point. The matching process uses the nearest neighbor algorithm, and a kd tree is usually used to accelerate the search. Through this series of processing, the UAV image will be accurately aligned with the remote sensing image in space, providing a basis for subsequent data fusion and analysis. The corrected image will show consistent geographic features and spatial relationships with the remote sensing image, greatly improving the comparability and complementarity of the two data sources.

[0119] Project the water area UAV images and water area remote sensing images into a unified standard coordinate system. The spatial projection transformation needs to consider the coordinate system of the original data and the target standard coordinate system. Commonly used standard coordinate systems include WGS84, UTM, etc. The core of projection transformation is coordinate transformation. In actual operation, open source libraries such as GDAL (Geospatial Data Abstraction Library) can be used to perform these complex coordinate transformations. After the projection transformation, resampling is required to ensure that the two images have the same spatial resolution and grid alignment. When the acquisition time of the UAV image and the remote sensing image is not exactly the same, the Dynamic Time Warping (DTW) algorithm is needed to solve this problem. The core idea of ​​DTW is to find the best alignment between two time series so that the distance between them is minimized. Assuming there are two time series X = (x1,...,xn) and Y = (y1,...,ym), the steps of the DTW algorithm are as follows:

[0120] 1) Construct a distance matrix D, where D(i,j) represents the distance between xi and yj (usually using Euclidean distance).

[0121] 2) Construct the cumulative distance matrix C, where C(i,j) represents the minimum cumulative distance from (1,1) to (i,j). The recursive formula is: C(i,j)=min(C(i-1,j-1),C(i-1,j),C(i,j-1))+D(i,j).

[0122] 3) Start from C(n,m) and trace back to find the optimal alignment path. In remote sensing image processing, the time series of each pixel can be used as input, and the DTW algorithm can be used to find the best time correspondence between the UAV image and the remote sensing image.

[0123] In order to process large-scale image data, a block processing strategy can be adopted to divide the image into small blocks and execute the DTW algorithm in parallel.

[0124] The core idea of ​​the orthogonal fusion method is to decompose high spatial resolution UAV images and high spectral resolution remote sensing images into orthogonal components, and then reconstruct the fused image through reasonable combination. This method can effectively retain the key information of the two images while minimizing information loss and distortion. A common implementation of orthogonal fusion is based on principal component analysis (PCA). The steps of PCA orthogonal fusion are as follows:

[0125] 1) Perform principal component analysis on the multispectral remote sensing image to obtain the principal component images PC1, PC2, ..., PCn. The mathematical expression of PCA is: Y = PX, where X is the original multispectral data, P is the eigenvector matrix, and Y is the principal component.

[0126] 2) Perform histogram matching of the high spatial resolution drone image (usually the panchromatic band) with the first principal component PC1. The goal of histogram matching is to make the statistical characteristics of the drone image (such as mean and standard deviation) consistent with PC1. This can be achieved using the cumulative distribution function (CDF): s = T(r) = (cdf(PC1))^(-1)(cdf(PAN)), where r is the original drone image pixel value and s is the matched value.

[0127] 3) Replace PC1 with the matched drone image to obtain a new principal component set PC1', PC2,..., PCn. 4) Perform an inverse PCA transformation to convert the new principal component set back to the original spectral space: X' = P^(-1)Y', where Y' is the new principal component set containing PC1', and X' is the fused high-resolution multispectral image. The advantage of this method is that it can maintain the spectral characteristics of the original multispectral image while significantly improving the spatial resolution.

[0128] Through this complex and comprehensive orthogonal fusion method, the final generated water fusion image will have both high spatial resolution and rich spectral information. This fused image can clearly show details such as water body boundaries, underwater terrain, and aquatic vegetation distribution, while retaining the spectral characteristics of different types of water bodies and vegetation. This provides a high-quality data foundation for subsequent water environment analysis, vegetation monitoring, and ecological assessment. The fused image can be used to accurately identify and classify aquatic vegetation types, assess vegetation coverage and biomass, monitor water quality changes, and even detect underwater targets.

[0129] In one embodiment, the wetland vegetation integrated monitoring system further includes ground monitoring equipment deployed around the wetland water area, and based on the submerged vegetation distribution characteristics in the water area vegetation distribution characteristics, generating a submerged vegetation monitoring route of the underwater monitoring robot using a path planning algorithm includes the following steps:

[0130] Determine several submerged vegetation distribution areas in the water area fusion image based on the submerged vegetation distribution characteristics in the water area vegetation distribution characteristics;

[0131] According to the regional area of ​​submerged vegetation distribution, a number of monitoring points were generated by random sampling method in each submerged vegetation distribution area;

[0132] Obtain water level information at all monitoring points in the wetland waters through ground monitoring equipment;

[0133] Based on all monitoring points and the corresponding water level information, the optimal path algorithm is used to generate the submerged vegetation monitoring route of the underwater monitoring robot.

[0134] In this embodiment, it is first necessary to extract the characteristic information of submerged vegetation from the fused image of the water area. This usually involves the calculation and analysis of multiple spectral indices, such as the Normalized Difference Vegetation Index (NDVI), the Submerged Vegetation Index (SVI), etc. SVI is particularly sensitive to underwater vegetation and can effectively distinguish underwater vegetation from other water features. Next, the SVI image is processed using an image segmentation algorithm to identify and divide the submerged vegetation distribution area. Commonly used segmentation methods include threshold segmentation, regional growth, and watershed algorithms. Taking threshold segmentation as an example, the Otsu method can be used to automatically determine the optimal threshold and divide the image into submerged vegetation areas and non-submerged vegetation areas. Finally, the connected region analysis algorithm is used to mark each independent submerged vegetation distribution area. This process can not only determine the spatial distribution of submerged vegetation, but also provide information such as the area and shape of each distribution area, providing an important basis for subsequent sampling and monitoring.

[0135] Next, it is necessary to determine the number of sampling points based on the area of ​​each submerged vegetation distribution area. A commonly used method is to use area proportional sampling, that is, the number of sampling points is proportional to the area of ​​the region: n_i = (A_i / A_total)*N_total, where n_i is the number of sampling points in the i-th region, A_i is the area of ​​the i-th region, A_total is the total area of ​​all regions, and N_total is the predetermined total number of sampling points. This method ensures that more sampling points are obtained in large areas, improving the representativeness of the overall sampling. Next, random sampling is performed within each area. The simplest method is uniform random sampling, which generates uniformly distributed random numbers within the x and y coordinates of the region:

[0136] x=x_min+(x_max-x_min)*rand(),y=y_min+(y_max-y_min)*rand()

[0137] Where rand() is a random number between 0 and 1. To avoid too much concentration of sampling points, stratified random sampling or systematic random sampling can be used. Stratified random sampling first divides the area into grids and then randomly selects points within each grid. Systematic random sampling evenly arranges grid points in the area and randomly offsets them around each grid point.

[0138] Next, the water level information of all monitoring points in the wetland waters is obtained through ground monitoring equipment. The ground monitoring equipment is equipped with a pressure water level gauge, an ultrasonic water level gauge or a radar water level gauge. Taking the pressure water level gauge as an example, the principle is to calculate the water level by measuring the pressure of the water column: h = P / (ρg), where h is the water level, P is the measured pressure, ρ is the density of water, and g is the acceleration of gravity. In order to improve the measurement accuracy, atmospheric pressure compensation is usually required. In actual operation, the GPS positioning system can be used to accurately locate the position of each monitoring point. For monitoring points that are difficult to reach directly, it is possible to consider using a remote-controlled vessel or drone equipped with a portable water level gauge for measurement.

[0139] The monitoring points and water level information are converted into a weighted graph model, where the nodes represent the monitoring points and the weights of the edges can take into account factors such as the distance between the points, the water level difference, and the energy consumption. Next, the classic optimal path algorithm, such as the Dijkstra algorithm or the A* algorithm, can be applied to find the best path. Taking the Dijkstra algorithm as an example, its core idea is to start from the starting point and continuously update the shortest distance to each point until all points are covered. In order to adapt to the particularity of the underwater environment, the algorithm can be modified, such as considering the turning radius limit and obstacle avoidance requirements of the underwater robot. In addition, considering the need to cover all monitoring points, this is actually a traveling salesman problem (TSP). Heuristic algorithms such as the ant colony algorithm (ACO) or the genetic algorithm (GA) can be used to solve it. Through multiple iterations, an optimal monitoring route that balances the path length, water level change, and energy consumption can be obtained. Finally, the generated path needs to be smoothed to adapt to the motion characteristics of the underwater robot.

[0140] In one embodiment, constructing a submerged vegetation recognition model based on a convolutional neural network includes the following steps:

[0141] Construct underwater image enhancement module based on U-NET network;

[0142] Construct the initial submerged vegetation recognition module based on the YOLOv5 network;

[0143] Using a preset submerged vegetation image data set to complete the training of an initial submerged vegetation recognition module, and obtain a submerged vegetation recognition module;

[0144] The output end of the underwater image enhancement module is connected to the input end of the submerged vegetation recognition module to obtain a submerged vegetation recognition model.

[0145] In this embodiment, the underwater image enhancement module is a complex structure built based on the U-NET network. Figure 2 , Figure 2It is the basic structure of U-NET network. The design of underwater image enhancement module fully considers the impact of underwater environment on image quality, such as light scattering, color distortion, contrast reduction and other issues, and realizes image enhancement through multi-level processing. Specifically, refer to Figure 3 ,The underwater image enhancement module consists of several key components, including the image feature input layer, ,multidimensional attention mechanism layer, multidimensional convolutional layer and image feature output layer. ,These components are connected in a specific order to form an end-to-end image ,processing flow.

[0146] like Figure 3 As shown in the figure, first, the image feature input layer is responsible for receiving the original underwater image data. This layer contains some preprocessing steps, such as normalization or preliminary feature extraction, to prepare for subsequent processing. Next is the multi-dimensional attention mechanism layer, which is one of the core parts of this module. The multi-dimensional attention mechanism layer is further subdivided into three sub-layers: image channel attention mechanism layer, image space attention mechanism layer, and refined feature output layer. This multi-dimensional attention mechanism design allows the network to focus on important information in the image from different perspectives.

[0147] The image channel attention mechanism layer mainly focuses on different color channels or feature maps of the image. The structure of the image channel attention mechanism layer is as follows: Figure 4 As shown in the figure. In an underwater environment, light of different wavelengths decays at different rates, which means that the information in some color channels may be more important or reliable than that in other channels. By introducing the channel attention mechanism, the network can learn to automatically adjust the attention to different channels, thereby better handling the color distortion problem in underwater environments. The image space attention mechanism layer focuses on the spatial information of the image. The structure of the image space attention mechanism layer is as follows: Figure 5 As shown in the figure. In underwater images, different areas may be affected by different degrees of degradation. For example, objects closer to the camera may retain more details, while objects farther away may be blurrier. The spatial attention mechanism enables the network to assign different attention weights to different areas of the image, so that different areas can be processed more targeted during the enhancement process. The refined feature output layer is the last step of the attention mechanism processing, which contains some additional convolutions or other operations to further refine and integrate the features obtained from the first two attention layers. The output of the multi-dimensional attention mechanism layer is then passed to the multi-dimensional convolution layer.

[0148] The multi-dimensional convolution layer is another key component, which includes the feature map transformation layer, the multi-core convolution layer, and the convolution feature fusion layer. This series of convolution operations aims to extract multi-scale and multi-level feature information from the image. The feature map transformation layer may use 1x1 convolution or other methods to adjust the dimension or shape of the feature map. This step can be regarded as a feature reorganization or dimensionality reduction operation, which helps to reduce the computational complexity and highlight important features. The multi-core convolution layer is the core of the multi-dimensional convolution, which uses convolution kernels of different sizes or types to process the image. Convolution kernels of different sizes can capture image features of different scales. Small convolution kernels are suitable for extracting local details, while large convolution kernels can capture contextual information of a larger range. This multi-scale feature extraction is particularly important for processing underwater images, because objects in underwater environments may present multiple scales and forms. The task of the convolution feature fusion layer is to effectively combine the features of different scales extracted by the multi-core convolution layer. It involves the concatenation of feature maps, weighted summation, or more complex fusion methods. By fusing features of different scales, the network can obtain a more comprehensive and richer image representation, which is beneficial to subsequent image enhancement tasks.

[0149] After being processed by the multi-dimensional convolution layer, the feature information is finally output through the image feature output layer. This layer contains some post-processing steps, such as deconvolution operations to restore the image size, or some detail enhancement operations to ensure that the output enhanced image retains the main content of the original image while having higher quality and richer details.

[0150] The choice of loss function is also a key factor in the training process of the underwater image enhancement module. According to the description, the module can use one or more of the smooth L1 loss function, the perceptual loss function, and the multi-scale structural similarity loss function. These loss functions have their own characteristics, and they can work together to optimize the enhancement effect from different angles. The smooth L1 loss function is a variant of the L1 loss, which uses a square function instead of an absolute value function near the origin. This design makes the loss function smoother when dealing with small errors, which is conducive to the stable training of the network. In the image enhancement task, the smooth L1 loss can help the network better handle pixel-level differences, especially when dealing with subtle brightness and color changes. The main purpose of the perceptual loss function is to keep the high-level semantic features of the enhanced image consistent with the original image. This is particularly important for underwater image enhancement, because it is necessary to improve the image quality without changing the essential features of the objects in the image. Perceptual loss can help the network learn to retain the content and structural information of the image during the enhancement process. The multi-scale structural similarity loss function is an extension of the structural similarity index (SSIM). It evaluates the structural similarity of images at multiple scales, which can ensure that the enhanced image maintains a similar structure to the original image at different scales. The combination of these loss functions can comprehensively guide network learning. The smooth L1 loss focuses on accurate reconstruction at the pixel level, the perceptual loss ensures the retention of semantic information, and the multi-scale structural similarity loss helps maintain the structural integrity of the image at all scales. Through this multi-objective optimization strategy, the underwater image enhancement module can learn an enhancement method that can both improve image quality and maintain the essential characteristics of the image.

[0151] The underwater image enhancement module systematically processes the input underwater image through a multi-dimensional attention mechanism and multi-dimensional convolution. The attention mechanism enables the network to adaptively focus on the most important and relevant features in the image, both in color channels and spatial positions. This selective attention helps the network to more effectively handle problems such as uneven illumination and color distortion in underwater environments. At the same time, multi-dimensional convolution is able to capture and process multi-scale information in the image by using convolution kernels of different scales. This is very important for enhancing objects and details of different scales in underwater images, because objects in underwater environments may present different clarity and detail levels depending on the distance. In addition, the design of the underwater image enhancement module also takes into account the special needs of underwater image enhancement. Underwater images are usually affected by light scattering, absorption, and suspended particles, resulting in image blur, low contrast, color distortion, and other problems. The multi-dimensional attention mechanism can help the network identify and highlight image regions or features that are less affected by these factors, while the multi-dimensional convolution can reconstruct and enhance the affected areas by extracting multi-scale features.

[0152] Reference Figure 6,The initial submerged vegetation recognition module is a complex structure built on the YOLOv5 network, which aims to achieve accurate recognition and classification of submerged vegetation in underwater environments. The module consists of multiple layers, including feature input layer, feature extraction layer, feature sampling layer and result output layer, which are connected in sequence to form a complete end-to-end recognition system. The feature input layer is the starting point of the entire module and is responsible for receiving the image data processed by the underwater image enhancement module. This layer may contain some preliminary preprocessing operations, such as image resizing, data normalization, etc., to ensure the consistency and applicability of the input data. The feature extraction layer is one of the core parts of the module, which consists of a downsampling layer, a core feature extraction layer, a residual structure layer, and a feature map fusion layer. The downsampling layer reduces the computational complexity by reducing the image resolution while retaining key features. This is usually achieved through a convolution operation with a stride of 2 or a maximum pooling. The core feature extraction layer may contain multiple convolutional layers to extract various features of the image, such as edges, textures, shapes, etc. These features are crucial for identifying different types of submerged vegetation. The residual structure layer introduces skip connections, allowing information to flow more directly in the network, helping to alleviate the gradient vanishing problem in deep networks and improve the efficiency of feature extraction. The feature map fusion layer is responsible for integrating features extracted at different levels to form a richer and more comprehensive feature representation.

[0153] The feature sampling layer is another key component, including variant convolutional layers, feature amplification layers, feature concatenation layers, residual structure layers, and multi-dimensional attention mechanism layers. The variant convolutional layer may use techniques such as dilated convolution or deformable convolution, which can increase the receptive field and capture a wider range of contextual information, which is particularly useful for identifying submerged vegetation of different sizes and shapes. The feature amplification layer may use deconvolution or upsampling techniques to restore the spatial resolution of the feature map, which is important for accurately locating the location of vegetation. The feature concatenation layer concatenates feature maps of different scales, combining low-level detail information and high-level semantic information. The role of the residual structure layer here is similar to that in the feature extraction layer, which is used to optimize information flow and gradient propagation. The multi-dimensional attention mechanism layer introduces the same attention mechanism as the underwater image enhancement module, allowing the network to better focus on key areas and features in the image, which is particularly important for processing complex underwater scenes because submerged vegetation may have a high degree of similarity with the background environment. The result output layer is the last part of the module, responsible for converting the extracted and processed features into the final recognition results.

[0154] During the training process, the initial submerged vegetation recognition module uses a variety of loss functions to optimize model performance. The cross entropy loss function is mainly used to optimize category prediction, which can effectively measure the difference between the predicted category and the true category. The mean square error loss function and the mean absolute error loss function may be used to optimize bounding box prediction. The former is more sensitive to large errors, while the latter treats all errors equally. The combination of these loss functions can comprehensively guide network learning, focusing on both the accuracy of category classification and the accuracy of target positioning.

[0155] The specific process of vegetation recognition and classification can be divided into several main steps: First, the enhanced underwater image enters the network through the feature input layer. Then, the feature extraction layer starts to work, extracting multi-scale and multi-level features from the image through a series of downsampling, convolution and residual connection operations. In this process, the network gradually learns from low-level edge and texture features to high-level semantic features. Then, the feature sampling layer further processes the extracted features. The variant convolution layer may be used to capture irregularly shaped vegetation features, and feature amplification and splicing operations help to restore and integrate information at different scales. The multi-dimensional attention mechanism plays an important role in this stage, helping the network focus on the most relevant features, which is particularly important for distinguishing similar vegetation types. After these processes, the network generates a series of feature maps that contain rich spatial and semantic information. Finally, the result output layer uses these feature maps for final prediction. For each predefined anchor box, the network predicts whether it contains the target (submerged vegetation), and if so, predicts its precise location (adjusted by the bounding box) and category. At the same time, the network also outputs a confidence score to indicate the reliability of the prediction. In the post-processing stage, the non-maximum suppression (NMS) algorithm is usually applied to eliminate overlapping detection results and obtain the final recognition result. The whole process is end-to-end, and the network continuously adjusts its parameters through the back-propagation algorithm and the selected loss function to minimize the difference between the predicted results and the true labels. This design enables the initial submerged vegetation recognition module to effectively handle various challenges in underwater environments, such as light changes, water disturbances, vegetation occlusion, etc., thereby achieving accurate recognition and classification of submerged vegetation. By combining with the underwater image enhancement module, the system can achieve efficient vegetation monitoring and ecological assessment in complex underwater environments.

[0156] The present invention also discloses a wetland vegetation intelligent monitoring system based on multidimensional data fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the wetland vegetation intelligent monitoring method based on multidimensional data fusion as described in any one of the above embodiments is implemented.

[0157] Among them, the processor can adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc., and this application does not impose any restrictions on this.

[0158] Among them, the memory can be an internal storage unit of a computer device, such as a hard disk or memory of a computer device, or an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD) or a flash memory card (FC) equipped on the computer device, etc., and the memory can also be a combination of an internal storage unit and an external storage device of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or is to be output, and this application does not impose any restrictions on this.

[0159] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0160] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A wetland vegetation intelligent monitoring method based on multidimensional data fusion, characterized in that: Applied to a wetland vegetation integrated monitoring system, the wetland vegetation integrated monitoring system includes a monitoring drone and an underwater monitoring robot deployed underwater in a wetland water area, and the method includes the following steps: The monitoring drone is used to collect drone images of the wetland water area, and the remote sensing satellite is used to obtain remote sensing images of the wetland water area; The water area UAV image and the water area remote sensing image are fused into a water area fusion image by a data fusion method, the submerged vegetation distribution is identified from the water area fusion image, and a submerged vegetation monitoring route of the underwater monitoring robot is planned; Controlling the underwater monitoring robot to continuously detect the wetland water area according to the submerged vegetation monitoring route, and simultaneously collecting submerged vegetation standard images and submerged vegetation multispectral images of different types of submerged vegetation in the wetland water area; Construct a submerged vegetation recognition model based on convolutional neural network; Performing image enhancement on the submerged vegetation standard image by using the submerged vegetation recognition model to obtain a submerged vegetation enhanced image, and identifying and marking target submerged vegetation from the submerged vegetation enhanced image; According to the target submerged vegetation marked in the submerged vegetation enhanced image, selecting a target multispectral region containing the target submerged vegetation in the submerged vegetation multispectral image of the same timestamp; The vegetation characteristic data of the target submerged vegetation is calculated based on the target multi-spectral region.

2. The wetland vegetation intelligent monitoring method based on multidimensional data fusion according to claim 1 is characterized in that: The method of fusing the water area UAV image and the water area remote sensing image into a water area fusion image by a data fusion method, identifying the distribution of submerged vegetation from the water area fusion image and planning the submerged vegetation monitoring route of the underwater monitoring robot comprises the following steps: Preprocessing the water area UAV image and the water area remote sensing image, and fusing the preprocessed water area UAV image and the water area remote sensing image into a water area fused image; Extracting vegetation index features of the wetland water area from the water area fusion image; Based on the vegetation index characteristics and using a pre-trained water vegetation classification model, the water vegetation distribution characteristics of the wetland water area are identified, and the water vegetation classification model is constructed based on a random forest classifier; Based on the submerged vegetation distribution characteristics in the water area vegetation distribution characteristics, a submerged vegetation monitoring route of the underwater monitoring robot is generated by adopting a path planning algorithm.

3. The wetland vegetation intelligent monitoring method based on multidimensional data fusion according to claim 2 is characterized in that: The preprocessing of the water area UAV image and the water area remote sensing image, and fusing the preprocessed water area UAV image and the water area remote sensing image into a water area fused image comprises the following steps: Performing atmospheric correction and cloud removal processing on the water area remote sensing image; Geometrically correcting the water area UAV image to be spatially aligned with the water area remote sensing image; Projecting the water area UAV image and the water area remote sensing image into a unified standard coordinate system, and using a dynamic time warping algorithm to complete the time alignment of the water area UAV image and the water area remote sensing image; The water area UAV image and the water area remote sensing image are fused into a water area fused image by adopting an orthogonalization method.

4. The wetland vegetation intelligent monitoring method based on multidimensional data fusion according to claim 2 is characterized in that: The vegetation index feature includes one or more of a normalized difference vegetation index, a normalized difference water index, an enhanced vegetation index, a modified normalized difference vegetation index and a water vegetation index.

5. The wetland vegetation intelligent monitoring method based on multidimensional data fusion according to claim 4 is characterized in that: The wetland vegetation integrated monitoring system further includes ground monitoring equipment deployed around the wetland water area. The submerged vegetation monitoring route of the underwater monitoring robot is generated based on the submerged vegetation distribution characteristics in the water area vegetation distribution characteristics and using a path planning algorithm, including the following steps: Determining a plurality of submerged vegetation distribution areas in the water area fusion image based on the submerged vegetation distribution characteristics in the water area vegetation distribution characteristics; According to the area of ​​the submerged vegetation distribution area, a number of monitoring points are generated in each submerged vegetation distribution area by using a random sampling method; Acquiring water level information of all monitoring points in the wetland water area through the ground monitoring equipment; Based on all the monitoring points and the corresponding water level information, an optimal path algorithm is used to generate a submerged vegetation monitoring route for the underwater monitoring robot.

6. The wetland vegetation intelligent monitoring method based on multidimensional data fusion according to claim 1 is characterized in that: The submerged vegetation recognition model based on the convolutional neural network comprises the following steps: Construct underwater image enhancement module based on U-NET network; Construct the initial submerged vegetation recognition module based on the YOLOv5 network; Using a preset submerged vegetation image data set to complete the training of the initial submerged vegetation recognition module to obtain a submerged vegetation recognition module; The output end of the underwater image enhancement module is connected to the input end of the submerged vegetation recognition module to obtain a submerged vegetation recognition model.

7. The wetland vegetation intelligent monitoring method based on multi-dimensional data fusion according to claim 6 is characterized in that: The underwater image enhancement module includes an image feature input layer, a multidimensional attention mechanism layer, a multidimensional convolution layer and an image feature output layer connected in sequence, the multidimensional attention mechanism layer includes an image channel attention mechanism layer, an image space attention mechanism layer and a refined feature output layer connected in sequence, the multidimensional convolution layer includes a feature map transformation layer, a multi-kernel convolution layer and a convolution feature fusion layer connected in sequence, and the loss function of the underwater image enhancement module includes any one or more of a smooth L1 loss function, a perceptual loss function and a multi-scale structural similarity loss function.

8. The wetland vegetation intelligent monitoring method based on multi-dimensional data fusion according to claim 7 is characterized in that: The initial submerged vegetation recognition module includes a feature input layer, a feature extraction layer, a feature sampling layer and a result output layer connected in sequence, the feature extraction layer includes a downsampling layer, a core feature extraction layer, a residual structure layer and a feature map fusion layer, the feature sampling layer includes a variant convolution layer, a feature amplification layer, a feature splicing layer, the residual structure layer and the multi-dimensional attention mechanism layer, and the loss function of the initial submerged vegetation recognition module includes any one or more of a cross entropy loss function, a mean square error loss function and a mean absolute error loss function.

9. The wetland vegetation intelligent monitoring method based on multi-dimensional data fusion according to claim 1 is characterized in that: The vegetation characteristic data includes the submerged vegetation coverage and the total biomass of the submerged vegetation of the target submerged vegetation.

10. An intelligent wetland vegetation monitoring system based on multi-dimensional data fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the intelligent wetland vegetation monitoring method based on multidimensional data fusion as described in any one of claims 1 to 9 is implemented.

Images