A multi-scale collaborative algal bloom assessment method integrating underwater video

Abstract

The present invention relates to a multi-scale collaborative algal bloom assessment method that integrates underwater videos and belongs to the field of image recognition. The method comprises the following steps: S1: acquisition of underwater, surface, and aerial images; S2: acquisition of water quality data and meteorological data; S3: alignment of data in time and space; S4: binary mask segmentation of the image using image segmentation technology; S5: vectorization of the binary mask of the image and concatenation with the water quality data and meteorological data as input; S6: prediction using an improved Transformer model; S7: fusion of the binary mask of the image using a weighted summation method to obtain the binary mask of the algal bloom area and the fused algal bloom coverage ratio of the monitored area. The method of the present invention can realize the calculation and prediction of the binary mask of the algal bloom area and the fused algal bloom coverage ratio of the monitored area at multiple moments, which is conducive to the dynamic and accurate assessment of the algal bloom phenomenon.

Description

Technical Field

[0001] The present invention relates to a multi-scale collaborative algal bloom assessment method integrating underwater videos, belongs to the field of image recognition, and particularly relates to a multi-scale collaborative algal bloom assessment method integrating underwater videos. Background Art

[0002] Algal blooms, also known as water blooms, are large-scale algal blooms caused by the abnormal growth of certain algae species. These blooms often lead to degraded water quality, deterioration of the aquatic ecosystem, and even the death of fish and other aquatic organisms, significantly impacting the aesthetics and functionality of the water. With global climate change and the impact of human activities, algal blooms are becoming increasingly frequent, becoming a global environmental concern. Therefore, accurately and rapidly assessing the distribution and severity of algal blooms is crucial for water management and emergency response.

[0003] Traditional algal bloom assessments rely primarily on field sampling and satellite remote sensing. While field sampling can obtain accurate water quality data, including algal species and abundance, it is time-consuming and labor-intensive, and cannot achieve continuous monitoring over large areas. Satellite remote sensing technology can cover wider areas and infer algal bloom distribution by analyzing reflectance spectra. However, this method is subject to time delays, and factors such as data update frequency and cloud cover limit its ability to monitor in real time. Furthermore, these traditional technologies often struggle to achieve real-time monitoring and rapid response, failing to meet urgent environmental protection and water management needs.

[0004] To overcome these limitations, researchers have begun exploring new methods for real-time algal bloom assessment that integrate multiple data sources. Specifically, image processing techniques are being used to identify and measure algal bloom density and distribution. This image data, combined with data from other sensors (such as dissolved oxygen, temperature, and light intensity), can be analyzed using machine learning models to predict algal bloom trends and outbreak points. The implementation of these methods requires a robust data processing platform and efficient algorithm optimization to ensure real-time data processing and analysis.

[0005] However, integrating these methods for real-time algal bloom assessment still faces numerous challenges. Effectively integrating and synchronizing disparate data sources requires the development of new data fusion techniques and models to improve assessment accuracy and efficiency. Furthermore, underwater video technology, by capturing direct images of underwater algae, can more comprehensively reflect the growth conditions and environmental factors of algal blooms, and can be effectively utilized to achieve accurate bloom assessments. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the above-mentioned existing technologies and provide a multi-scale collaborative algal bloom assessment method that integrates underwater videos. It intends to align and fuse multi-scale images and monitoring data of underwater, surface and high altitude in time and space, and use interpolation methods to supplement missing data. A preliminary judgment is made through image segmentation methods, and then time series features are constructed. A Transformer model is established to correct the segmented images to achieve accurate identification of algal bloom phenomena and their development.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A multi-scale collaborative algal bloom assessment method integrating underwater video includes the following steps:

[0009] S1: Take underwater images, surface images, and aerial images of the monitoring area according to the sampling time to obtain an image set;

[0010] S2: Collect water quality data and meteorological data of the monitoring area according to the sampling time;

[0011] S3: Align the image collection, water quality data, and meteorological data of the monitoring area in time and space;

[0012] S4: Using image segmentation technology to perform binary mask segmentation of the algal bloom area on the aligned image set one by one to obtain a binary mask for each image;

[0013] S5: The binary mask of each image is vectorized and concatenated with the water quality data and meteorological data as the input of the Transformer model at the corresponding time;

[0014] S6: Use the Transformer model to predict the binary mask of each image at the next moment;

[0015] S7: The binary mask of each image is fused using a weighted summation method to obtain the binary mask of the algal bloom area and the fused algal bloom coverage ratio of the monitoring area at the corresponding moment.

[0016] Furthermore, the underwater image described in step S1 is a high-definition image parallel to the water surface taken by an underwater infrared camera, and the surface image is taken by a fixed high-resolution camera or a radar installed on a drone; the high-altitude image is a satellite optical remote sensing image.

[0017] Preferably, for the high-altitude image, an interpolation method can be used to complete the data of areas blocked by clouds.

[0018] Furthermore, the water quality data described in step S2 is data measured by a temperature sensor, a dissolved oxygen sensor, a pH sensor, and a nitrogen and phosphorus concentration sensor, including: water temperature (°C), pH (dimensionless), dissolved oxygen (mg / L), conductivity (μS / cm), turbidity (NTU), permanganate index (mg / L), ammonia nitrogen (mg / L), total phosphorus (mg / L), total nitrogen (mg / L), chlorophyll a (ug / L); the meteorological data includes: air temperature (°C), precipitation (mm), wind speed (m / s), relative humidity (%), and air pressure (Pa).

[0019] Furthermore, the step S3 is specifically as follows:

[0020] S301: Using a timestamp filter, select all images and data of the monitoring area corresponding to time T from the image set, water quality data, and meteorological data as images and data to be analyzed;

[0021] S302: Mapping the image to be analyzed and the data to be analyzed to a unified coordinate system using a geographic coordinate system, and adjusting the image to be analyzed to the same resolution using a bilinear interpolation method or a nearest neighbor interpolation method.

[0022] Furthermore, the step S4 is specifically as follows:

[0023] S401: Build a MobileNet or EfficientNet model;

[0024] S402: Input the images to be analyzed one by one, extract the regions of interest, and output a binary mask of the algal bloom, where 1 represents the algal bloom area and 0 represents the background;

[0025] S403: Calculate the ratio of the algal bloom area to the entire image to be analyzed, which is the corresponding algal bloom coverage ratio.

[0026] Furthermore, before the step S4, the image to be analyzed needs to be enhanced.

[0027] Preferably, the enhancement process for underwater images is as follows: (1) decomposing the red, green and blue channels of the underwater image,

[0028] Adopting Multi-Scale Retinex with Color Restoration (MSRCR) algorithm,

[0029] Enhance the contrast of the image; (2) For the enhanced image, use fully convolutional GAN ​​to achieve image detail restoration and color correction.

[0030] Furthermore, the Transformer model described in step S5 is composed of a Transformer network with a softmax layer as the output layer and a binary mask segmentation function connected in series with it, and the output value is 0 or 1; wherein the binary mask segmentation function Among them, w is the weight coefficient to be trained, and x is the output of the softmax layer.

[0031] Preferably, the binary mask of the image in step S5 is vectorized by sampling the binary mask of the image in a sliding window manner, stretching each sliding window into a vector, and concat-joining the vectors of the sliding windows in order.

[0032] Furthermore, the step S7 is specifically as follows:

[0033] S701: Use the weighted summation method to fuse the binary masks of each image at time t to obtain the binary mask of the algal bloom area in the monitoring area at time t. Among them, ω i is the binary mask X of the i-th image i (t) the corresponding weight;

[0034] S702: Count the number of pixels with a value of 1 in the binary mask X(t) of the algal bloom area in the monitoring area at time t, and the fused algal bloom coverage ratio is the ratio of the number of pixels with a value of 1 to the total number of pixels.

[0035] Preferably, the weight ω in step S701 is i Set by experience from the expert knowledge base.

[0036] The beneficial effects of the present invention are: providing a multi-scale collaborative algal bloom assessment method that integrates underwater videos, using image segmentation technology to perform binary segmentation on underwater images, surface images, and aerial images of the monitoring area, and then aligning them with water quality data and meteorological data in time and space dimensions, and using the Transformer model to realize the calculation and prediction of the binary mask of the algal bloom area and the fused algal bloom coverage ratio in the monitoring area at multiple moments, which is conducive to the dynamic and accurate assessment of the algal bloom phenomenon. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the purpose and technical solution of the present invention more clear, the present invention provides the following drawings for illustration:

[0038] Figure 1 This is a flow chart of a multi-scale collaborative algal bloom assessment method integrating underwater video according to the present invention;

[0039] Figure 2 This is a diagram of the Transformer model architecture in Example 1 of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose and technical solution of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0041] Example 1: In order to achieve real-time monitoring and dynamic assessment and early warning of the blue-green algae bloom phenomenon in the waters of Dianchi Lake, there are high-definition high-altitude images taken by existing satellite optical remote sensing, multiple water surface images taken by radar installed by high-resolution cameras and drones flying in the air covering paths, and underwater images parallel to the water surface taken by infrared cameras by unmanned boats traveling on the water surface covering paths; for this scenario, the present invention provides "a multi-scale collaborative algae bloom assessment method integrating underwater videos."

[0042] Combine Figure 1 , which includes the following steps:

[0043] S1: Take underwater images, surface images, and aerial images of the monitoring area according to the sampling time to obtain an image set.

[0044] S2: Collect water quality data and meteorological data of the monitoring area according to the sampling time.

[0045] The water quality data are data measured by temperature sensors, dissolved oxygen sensors, pH sensors, and nitrogen and phosphorus concentration sensors, including: water temperature (°C), pH (dimensionless), dissolved oxygen (mg / L), conductivity (μS / cm), turbidity (NTU), permanganate index (mg / L), ammonia nitrogen (mg / L), total phosphorus (mg / L), total nitrogen (mg / L), chlorophyll a (ug / L); the meteorological data include: air temperature (°C), precipitation (mm), wind speed (m / s), relative humidity (%), and air pressure (Pa).

[0046] S3: Align the image collection, water quality data and meteorological data of the monitoring area in time and space, and perform image enhancement processing.

[0047] Specifically:

[0048] S301: Using a timestamp filter, select all images and data of the monitoring area corresponding to time T from the image set, water quality data, and meteorological data as images and data to be analyzed;

[0049] S302: Mapping the image to be analyzed and the data to be analyzed to a unified coordinate system using a geographic coordinate system, and adjusting the image to be analyzed to the same resolution using a bilinear interpolation method or a nearest neighbor interpolation method.

[0050] The specific enhancement processing for underwater images is as follows: (1) decomposing the red, green and blue channels of the underwater image, and using a multi-scale retinal enhancement algorithm with color restoration to enhance the contrast of the image; (2) using a fully convolutional GAN ​​to restore image details and correct color for the enhanced image.

[0051] S4: Use image segmentation technology to perform binary mask segmentation of the algal bloom area on the aligned image set one by one to obtain a binary mask for each image. Specifically:

[0052] S401: Build a MobileNet or EfficientNet model;

[0053] S402: Input the images to be analyzed one by one, extract the regions of interest, and output a binary mask of the algal bloom, where 1 represents the algal bloom area and 0 represents the background;

[0054] S403: Calculate the ratio of the algal bloom area to the entire image to be analyzed, which is the corresponding algal bloom coverage ratio.

[0055] S5: The binary mask of each image is vectorized and concatenated with the water quality data and meteorological data as the input of the Transformer model at the corresponding time.

[0056] Combine Figure 2 The Transformer model is composed of an embedding layer, a multi-head attention mechanism, a linear layer, an Add&Norm layer, a feedforward neural network layer, an Add&Norm layer, a pooling layer, a Softmax layer, and a binary mask segmentation function. The input is an image binary mask vector that is embedded with weights in the embedding layer and then concatenated with water quality data and meteorological data. The output value of the model is 0 or 1. The binary mask segmentation function is Among them, w is the weight coefficient to be trained, and x is the output of the softmax layer.

[0057] The binary mask of the image is vectorized by sampling the binary mask of the image with a 4*4 sliding window, stretching each sliding window into a 16-dimensional vector, and concat-joining the vectors of the sliding windows one by one in order from left to right and from top to bottom.

[0058] S6: Use the Transformer model to predict the binary mask of each image at the next moment.

[0059] S7: Use the weighted summation method to fuse the binary masks of each image to obtain the binary mask of the algal bloom area and the fused algal bloom coverage ratio of the monitoring area at the corresponding moment. Specifically:

[0060] S701: Use the weighted summation method to fuse the binary masks of each image at time t to obtain the binary mask of the algal bloom area in the monitoring area at time t. Among them, ω i is the binary mask X of the i-th image i (t) the corresponding weight;

[0061] S702: Count the number of pixels with a value of 1 in the binary mask X(t) of the algal bloom area in the monitoring area at time t, and the fused algal bloom coverage ratio is the ratio N of the number of pixels with a value of 1 to the total number of pixels. algae .

[0062] The weight ω i Set by experience from the expert knowledge base.

[0063] S8: Calculate the coverage area A of the algal bloom based on the image resolution R algae :

[0064] A algae =N algae ×R

[0065] Among them, R is the actual area represented by each pixel.

[0066] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A multi-scale collaborative algal bloom assessment method integrating underwater video, characterized by: The steps include: S1: Take underwater images, surface images, and aerial images of the monitoring area according to the sampling time to obtain an image set; S2: Collect water quality data and meteorological data of the monitoring area according to the sampling time; S3: Align the image collection, water quality data, and meteorological data of the monitoring area in time and space; S4: Using image segmentation technology to perform binary mask segmentation of the algal bloom area on the aligned image set one by one to obtain a binary mask for each image; S5: The binary mask of each image is vectorized and concatenated with the water quality data and meteorological data as the input of the Transformer model at the corresponding time; S6: Use the Transformer model to predict the binary mask of each image at the next moment; S7: using a weighted summation method to fuse the binary masks of each image, and obtain the binary mask of the algal bloom area and the fused algal bloom coverage ratio of the monitoring area at the corresponding moment; The underwater image in step S1 is a high-definition image parallel to the water surface taken by an underwater infrared camera, and the surface image is taken by a fixed high-resolution camera or a radar installed on a drone; the high-altitude image is a satellite optical remote sensing image; The water quality data described in step S2 is data measured by a temperature sensor, a dissolved oxygen sensor, a pH sensor, and nitrogen and phosphorus concentration sensors, including: water temperature (°C), pH (dimensionless), dissolved oxygen (mg / L), conductivity (μS / cm), turbidity (NTU), permanganate index (mg / L), ammonia nitrogen (mg / L), total phosphorus (mg / L), total nitrogen (mg / L), and chlorophyll a (ug / L); the meteorological data includes: temperature (°C), precipitation (mm), wind speed (m / s), relative humidity (%), and air pressure (Pa); The step S3 is specifically as follows: S301: Using a timestamp filter, select all images and data of the monitoring area corresponding to time T from the image set, water quality data, and meteorological data as images and data to be analyzed; S302: Mapping the image to be analyzed and the data to be analyzed to a unified coordinate system using a geographic coordinate system, and adjusting the image to be analyzed to the same resolution using a bilinear interpolation method or a nearest neighbor interpolation method; The step S4 is specifically as follows: S401: Build a MobileNet or EfficientNet model; S402: Input the images to be analyzed one by one, extract the regions of interest, and output a binary mask of the algal bloom, where 1 represents the algal bloom area and 0 represents the background; S403: Calculate the ratio of the algal bloom area to the entire image to be analyzed, which is the corresponding algal bloom coverage ratio; The Transformer model described in step S5 is composed of a Transformer network with a softmax layer as the output layer and a binary mask segmentation function connected in series with it, and the output value is 0 or 1; wherein the binary mask segmentation function Among them, w is the weight coefficient to be trained, and x is the output of the softmax layer; The step S7 is specifically as follows: S701: Use the weighted summation method to fuse the binary masks of each image at time t to obtain the binary mask of the algal bloom area in the monitoring area at time t. Among them, ω i is the binary mask X of the i-th image i (t) the corresponding weight; S702: Count the number of pixels with a value of 1 in the binary mask X(t) of the algal bloom area in the monitoring area at time t, and the fused algal bloom coverage ratio is the ratio of the number of pixels with a value of 1 to the total number of pixels.

2. The multi-scale collaborative algal bloom assessment method based on underwater video fusion according to claim 1 is characterized in that: For the high-altitude images, interpolation can be used to complete the data in areas blocked by clouds.

3. The multi-scale collaborative algal bloom assessment method based on underwater video fusion according to claim 1 is characterized in that: Before the step S4, the image to be analyzed needs to be enhanced.

4. The multi-scale collaborative algal bloom assessment method based on underwater video fusion according to claim 3 is characterized in that: The specific enhancement processing for underwater images is as follows: (1) the red, green and blue channels of the underwater image are decomposed, and the Multi-Scale Retinex with Color Restoration (MSRCR) algorithm is used to enhance the contrast of the image; (2) for the enhanced image, the fully convolutional GAN ​​is used to restore the image details and correct the color.

5. The multi-scale collaborative algal bloom assessment method integrating underwater video according to claim 1 is characterized in that: The binary mask of the image described in step S5 is vectorized by sampling the binary mask of the image in a sliding window manner, stretching each sliding window into a vector, and concat-joining the vectors of the sliding windows in order.

6. The multi-scale collaborative algal bloom assessment method based on underwater video fusion according to claim 1, characterized in that: The weight ω described in step S701 i Set by experience from the expert knowledge base.

Images