Blue-green algae detection method based on multi-task deep learning and adaptive background removal

Abstract

The invention is applicable to the field of blue-green algae detection, and particularly provides a blue-green algae detection method based on multi-task deep learning and adaptive background removal, in the detection method, a MAGAN model is constructed, local features are extracted through a graph convolutional network, global context perception is enhanced by using a self-attention mechanism, and the detection accuracy is improved. And finally, the foreground and background distinguishing is optimized through a background removing module, so that the self-adaptive background removing task is realized. Then, a multi-task deep learning framework is adopted; the targets of the multiple tasks are learned in the same model at the same time, so that the model can share knowledge from the multiple tasks; the blue-green algae detection task and the background removal task share part of features and are cooperatively optimized, background removal is achieved, support is provided for blue-green algae recognition, the blue-green algae recognition provides reference information for background removal, the cooperative effect between the tasks is enhanced, the model can more accurately distinguish blue-green algae from other water substances, and false detection and missing detection are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection by machine vision, and in particular to a cyanobacteria detection method based on multi-task deep learning and adaptive background removal. Background Art

[0002] The occurrence of cyanobacteria blooms poses a major threat to water quality, ecological environment, and even human health. Although traditional cyanobacteria detection methods, including microscopy, chemical analysis, and sensor technology, can provide relatively accurate detection results, they all have disadvantages such as long sampling cycles, large detection area limitations, and cumbersome operations. Therefore, the use of remote sensing images and image processing technology for automated cyanobacteria detection has become a hot topic of extensive research in recent years. The spectral reflectance characteristics of cyanobacteria in remote sensing data (such as chlorophyll a concentration, etc.) are unique, but the identification of cyanobacteria is challenging due to the presence of interferences such as algae, phytoplankton, and suspended matter in water bodies. In order to solve these problems, researchers have made significant progress by using intelligent algorithms such as deep learning, combined with image analysis, target detection, and background removal technologies.

[0003] Existing deep learning cyanobacteria identification methods can be roughly divided into two categories: The first category: those based on image recognition mainly focus on the target detection, classification and segmentation of cyanobacteria through water images. The disadvantages of the methods based on image recognition are: on the one hand, different water environments and types of cyanobacteria may limit the generalization ability of the model; in new environments or when facing unprecedented types of cyanobacteria, the recognition accuracy of the model may decrease; on the other hand, due to the presence of various substances in water bodies, such as phytoplankton, suspended matter, etc., their spectral characteristics may be similar to those of cyanobacteria, which makes the existing single target detection methods prone to false detection or missed detection, especially when the spectral differences between cyanobacteria and other biological substances are not obvious.

[0004] The second category: remote sensing data-based methods use satellite, multispectral or hyperspectral remote sensing images to monitor and predict the dynamics of cyanobacteria on a large scale. The disadvantages of methods based on remote sensing data are: on the one hand, remote sensing data are usually high-dimensional and large-capacity, and processing and analyzing these data requires professional techniques and tools; in addition, the acquisition and processing of remote sensing data may be affected by factors such as satellite orbits and cloud cover, resulting in incompleteness and inaccuracy of the data; on the other hand, the spatial resolution of satellite images is limited, and the periodicity of image acquisition limits the timely monitoring of changes in cyanobacteria. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a cyanobacteria detection method based on multi-task deep learning and adaptive background removal, aiming to solve the technical problems that the existing methods are insufficient in processing background interference and have poor adaptability in complex water environments, and that traditional methods such as optical sensor-based detection and simple image processing algorithms are difficult to cope with the dynamic changes of the water environment, resulting in insufficient detection accuracy and robustness.

[0006] To achieve the above object, the present invention provides the following technical solutions.

[0007] An embodiment of the present invention provides a cyanobacteria detection method based on multi-task deep learning and adaptive background removal, and the cyanobacteria detection method comprises the following steps: Build a detection model, which includes a multimodal adaptive graph attention network MAGAN and a multi-task deep learning framework; In the multimodal adaptive graph attention network MAGAN, the local features of the image are extracted through the graph convolutional network GCN, and the attention level of each area in the image is adaptively adjusted and enhanced in combination with the graph attention mechanism to obtain the output features after background removal; The shared convolutional layer is used to extract common features from the features output after background removal to generate feature maps; in the multi-task deep learning framework, the convolutional layer and the fully connected layer are used to classify or segment cyanobacteria, generate detection results, determine whether the image contains cyanobacteria, and locate the cyanobacteria area; and the image background is reconstructed through the convolutional layer and the fully connected layer to generate an image after background removal; a multi-task loss function is designed, combining the cyanobacteria detection loss and the background removal task loss, and performing joint optimization through weighted combination; Train and optimize the model parameters, take the water surface image as the input of the model, and output the detection results of cyanobacteria in the water surface image.

[0008] Furthermore, the step of extracting local features of the image through the graph convolutional network GCN includes: For the input image, the superpixel partitioning algorithm SLIC is used to divide the image into multiple regions with similar features, and the vector F is obtained through multimodal feature extraction. i , F i =[C i ,T i ,S i ,L i ], C i represents the color feature vector, T i represents the texture feature vector, S i represents the shape feature vector, L i Represents the illumination feature vector.

[0009] Furthermore, in combination with the graph attention mechanism, the attention level of each region in the enhanced image is adaptively adjusted to obtain the features output after background removal, including: Use the graph attention mechanism to adaptively weight image features and calculate the attention weights between nodes to dynamically adjust the attention level of each area in the image; The attention weight will perform weighted summation on the node features to obtain the updated node features; The features of each node are updated by weighted summing with the features of neighboring nodes in the graph convolution operation; By calculating the contrast loss between the foreground and background, the difference between regions is enhanced and the background is removed to obtain the output features.

[0010] Furthermore, in the graph attention mechanism, the model calculates an attention weight α for each node ij , attention weight α ij Indicates the correlation between a node and other nodes; The calculation formula of the attention mechanism is as follows: ; Where: Q i is the query vector of node i, K j is the key vector of node j, k represents the set of neighbor nodes of node i, α ij is the attention weight of node i to node j; W Q represents the weight matrix of the query vector Q; T represents the transpose operation; Represents the key vector of node k; by calculating To measure the similarity between node i and node j, a nonlinear transformation is introduced through the LeakyReLU activation function; the attention coefficient is normalized by softmax so that the sum of the attention weights of each node is 1; Finally, the feature of node i The following weighted aggregation process will be used to obtain: , where h j is the feature of node j.

[0011] Furthermore, in the graph convolutional network GCN, the feature vector of each superpixel region In the graph convolution layer, the pixel features of its neighborhood are updated to obtain ; The update of the graph convolutional layer is as follows: ; in, is the feature representation of node i at layer l; N(i) represents the set of neighbor nodes of node i; A ijis the adjacency matrix element between nodes i and j, reflecting their similarity; W （l） is the learning weight matrix of the graph convolutional layer; σ is the ReLu activation function; c ij represents the normalization coefficient, where , where deg(i) represents the degree of node i; deg(j) represents the degree of node j.

[0012] Furthermore, the step of strengthening the difference between regions by calculating the contrast loss between the foreground and the background and removing the background to obtain the output features includes: After being processed by graph convolution and self-attention mechanism, the obtained node features contain foreground information and background information ; A background removal module is introduced to differentiate the extracted features. This module uses the contrast loss L contrastiv To distinguish the difference between cyanobacteria and background, it is expressed as follows: ; in, and Represent the features of the foreground and background areas respectively, and N is the number of samples; The output layer separates the foreground and background, and uses a fully connected layer to map each pixel or region feature of the image to the category label of the background and foreground, expressed as: i =softmax(W out h i ); where h i represents the feature vector of node i, y i represents the classification result of node i, and the classification structure is foreground or background; W out is the weight matrix of the fully connected layer.

[0013] Furthermore, the step of extracting common features from the features output after background removal through a shared convolutional layer to generate a feature map includes: The features output after background removal are input into a shared convolutional layer F shared , which is expressed as follows: F shared (x) = ReLU(Conv(H MAGAN )); Among them, F shared The extracted features are universal visual features, which are applicable to all subsequent tasks. The nonlinearity is enhanced by ReLU activation function. Each convolutional layer extracts features at different levels of the image. MAGAN Represents the node feature matrix obtained after processing the input image.

[0014] Furthermore, the step of using the convolutional layer and the fully connected layer to classify or segment the cyanobacteria, generate a detection result, determine whether the image contains cyanobacteria, and locate the cyanobacteria area includes: After all convolutional features are extracted, the feature maps are flattened and passed to the fully connected layer for the final cyanobacteria segmentation; Using the fully connected layer F algae_fc Segment the cyanobacteria, determine whether the image contains cyanobacteria, and accurately locate the cyanobacteria area, as shown below: F algae (x) = ReLU(F algae_fc (flatten(F shared (x)))); F algae_fc (x)=W i ·x+b i ; Where i is the number of fully connected layers, and when i is 1, 2, or 3, the corresponding [W1, W2, W3] and [b1, b2, b3] are the weight matrix and bias term of the corresponding fully connected layer respectively; The output layer is a Sigmoid layer, which is used for segmentation tasks and outputs the probability of whether each pixel belongs to the cyanobacteria area.

[0015] Furthermore, the step of reconstructing the image background through the convolution layer and the fully connected layer to generate the image after removing the background includes: Transposed convolution is used to upsample the feature map and reconstruct the background; among them, the input feature map F in The dimensions are H′×W′×C in , upsample it to H×W×C by transposed convolution out , the transposed convolution operation is expressed as follows: F out =DeConv(F in ,K deconv ,stride,padding); where: F in The size is H'×W'×C in Input feature map of K deconv is a convolution kernel of a transposed convolution of size 3×3, which is used to learn the spatial relationship in the image; stride represents the stride; padding represents padding; F out The size is H×W×C out The output feature map of .

[0016] Furthermore, the step of designing a multi-task loss function, combining the cyanobacteria detection loss and the background removal task loss, and performing joint optimization through weighted combination includes: A multi-task loss function is designed, which combines the losses of cyanobacteria detection and background removal tasks and performs joint optimization through weighted combination. algae Using cross entropy loss, expressed as: ; in, Indicates the actual algae value, is the predicted algae value; Background removal loss L bg Using mean square error MSE loss, it is expressed as follows: ; Among them, y bgijk Represents the corresponding real pixel value in the input image; y bg adjustedijk Represents the pixel value of the image after background removal; H and W are the height and width of the output image, respectively, and C is the number of channels of the output image; the total loss L total It is expressed as follows: L total =λ algae L algae +λ bg L bg ; Among them, λ algae and λ bg Both represent weight coefficients, which are used to balance the losses of the cyanobacteria detection task and the background removal task.

[0017] Compared with the prior art, the cyanobacteria detection method based on multi-task deep learning and adaptive background removal of the present invention has the following beneficial effects: First, the graph convolutional network of the MAGAN model architecture of the present invention can process the spatial information of the image in the graph structure, convert the water body image into a graph structure, and extract local features by learning the relationship between the nodes in the graph. This enables the model to have stronger spatial perception ability when processing the complex morphology of cyanobacteria and its relationship with the background; Second, the self-attention mechanism of the MAGAN model architecture of the present invention further enhances the model's ability to accurately distinguish between cyanobacteria and background in complex environments by paying attention to the global context information of each region in the image, which can effectively suppress background noise and improve the recognition accuracy of cyanobacteria; Third, the present invention also adopts an adaptive background removal mechanism, which enables the model to dynamically adjust the background processing method according to the characteristics of different water environments, ensuring that the effect of background removal does not affect the accuracy of cyanobacteria detection. Through this adaptive adjustment, the background removal module can effectively avoid the problem of excessive or insufficient removal of background interference in complex scenes.

[0018] Fourth, the present invention combines a multi-task deep learning framework so that the cyanobacteria detection task and the background removal task can share some features and be optimized collaboratively, thereby further improving the overall performance. This multi-task collaborative optimization mechanism ensures effective information transfer and complementarity between the cyanobacteria detection task and the background removal task, enhances the adaptability and robustness of the model, and enables the method to show superior detection accuracy and reliability in complex dynamic aquatic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0020] Figure 1 It is an architecture diagram of a multi-task deep learning framework in the cyanobacteria detection method based on multi-task deep learning and adaptive background removal of the present invention; Figure 2 for Figure 1 The structural block diagram of the shared convolutional layer in ; Figure 3 This is an architecture diagram of the multimodal adaptive graph attention network MAGAN in the cyanobacteria detection method based on multi-task deep learning and adaptive background removal of the present invention; Figure 4 The overall architecture diagram of the cyanobacteria detection method based on multi-task deep learning and adaptive background removal of the present invention; Figure 5 This is a flow chart of the implementation of the cyanobacteria detection method based on multi-task deep learning and adaptive background removal of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0022] In view of the problems of insufficient background interference processing ability and poor adaptability of existing cyanobacteria detection methods in complex water environments, the present invention proposes a cyanobacteria detection method based on multi-task deep learning and adaptive background removal. In this method, first, a MAGAN model is constructed to extract local features through a graph convolutional network, and the global context perception is enhanced by using a self-attention mechanism. Finally, the distinction between foreground and background is optimized by a background removal module, thereby realizing the adaptive background removal task; then, a multi-task deep learning framework is adopted. By learning the objectives of multiple tasks simultaneously in the same model, the model can share knowledge from multiple tasks; the cyanobacteria detection task and the background removal task share some features and are optimized collaboratively, the background removal provides support for cyanobacteria recognition, and the cyanobacteria recognition provides reference information for background removal, enhancing the synergy between tasks, and the model can more accurately distinguish cyanobacteria from other water substances, reduce false detection and missed detection, and at the same time, the adaptive background removal strategy enables the model to be adjusted according to the characteristics of different water environments, effectively respond to detection tasks under various water conditions, and provide more flexible and reliable solutions.

[0023] The specific implementation of the cyanobacteria detection method based on multi-task deep learning and adaptive background removal of the present invention is described in detail below in conjunction with specific embodiments.

[0024] Please refer to Figure 1-Figure 5 In one embodiment of the present invention, a cyanobacteria detection method based on multi-task deep learning and adaptive background removal is provided, and the cyanobacteria detection method comprises the following steps: S101. Build a detection model, which includes a multimodal adaptive graph attention network MAGAN and a multi-task deep learning framework; S102. In the multimodal adaptive graph attention network MAGAN, the local features of the image are extracted by the graph convolutional network GCN, and the attention level of each area in the image is adaptively adjusted and enhanced in combination with the graph attention mechanism to obtain the output features after background removal; S103, extracting common features from the features output after background removal through a shared convolution layer to generate a feature map; in a multi-task deep learning framework, using convolution layers and fully connected layers to classify or segment cyanobacteria, generate detection results, determine whether the image contains cyanobacteria, and locate the cyanobacteria area; and reconstructing the image background through convolution layers and fully connected layers to generate an image after background removal; designing a multi-task loss function, combining cyanobacteria detection loss and background removal task loss, and performing joint optimization through weighted combination; S104, training and optimizing model parameters, taking the water surface image as the input of the model, and outputting the detection result of cyanobacteria in the water surface image.

[0025] Before step S101, the present invention further includes the steps of data collection and preprocessing; specifically, the original water surface image is collected, and the original image x is subjected to data enhancement by methods such as rotation, flipping, and scaling to increase the diversity of training data. The image is normalized to enhance the contrast and clarity of the image, and enhanced data x′ is obtained; In one implementation, the present invention collects water surface original images through drones, satellite remote sensing or water body monitoring equipment to collect water surface images, and checks whether the format of the input image is a supported format (such as JPEG, PNG, etc.). If not, the format is converted. The image is adjusted to a fixed size (such as 256x256 pixels) to ensure consistency in subsequent processing; Furthermore, in the process of preprocessing the original image, the original image x is convolved using a Gaussian filter and a median filter to remove Gaussian noise and salt and pepper noise in the image; the image is histogram equalized to enhance the image contrast; and the image pixel values ​​are normalized to the range of [0,1] to obtain enhanced data x′ for subsequent processing.

[0026] For further information, please refer to Figure 1 and Figure 2 In step S102 of the present invention, the step of extracting local features of an image by using a graph convolutional network (GCN) includes: For the input image x′, the superpixel partitioning algorithm SLIC is used to divide the image into multiple regions with similar features, and the vector F is obtained by multimodal feature extraction. i , F i =[C i ,T i ,S i ,L i ], C i represents the color feature vector, T i represents the texture feature vector, S i represents the shape feature vector, L i Represents the illumination feature vector.

[0027] Furthermore, in combination with the graph attention mechanism, the attention level of each region in the enhanced image is adaptively adjusted to obtain the features output after background removal, including: Use the graph attention mechanism to adaptively weight image features and calculate the attention weight α between nodes ij To dynamically adjust the level of attention for each area in the image; Attention weight α ij The node features will be weighted and summed to obtain the updated node features ; Characteristics of each node In the graph convolution operation, it is updated by weighted summing with the features of the neighboring nodes, and we get ; The background removal module calculates the contrast loss L between the foreground and background. contrastive Strengthen the differences between regions and remove the background to obtain the output feature H MAGAN ; Among them, for the input layer of the multimodal adaptive graph attention network MAGAN, the image x′ is first segmented using superpixel SLIC (Simple Linear Iterative Clustering) to generate superpixel regions with regular shapes and uniform sizes by clustering pixel points. Each superpixel usually contains a relatively consistent area, which can reduce redundant information in subsequent calculations. For each superpixel area, the present invention extracts color, texture, shape, and illumination features and converts them into vectors F i , F i =[C i ,T i ,S i ,L i ], C i represents the color feature vector, T i represents the texture feature vector, S i represents the shape feature vector, L i Represents the illumination feature vector.

[0028] In color feature C i In one implementation, the color feature represents the color information of each superpixel region. For all pixels in each superpixel region, the RGB average color value of the pixels in the region is calculated as the color feature of the region. The color feature C i It is expressed as: , where I(p) is the color value of pixel p, P i is the set of pixels in superpixel region i, N i is the number of pixels in the superpixel region, C i is the color feature vector of superpixel region i; for color feature C i Quantized as C i =[R i ,G i ,B i ].

[0029] In the texture feature T i In one implementation, the gray-level co-occurrence matrix (GLCM) extracts the texture features (contrast, energy, homogeneity, entropy) of the image by counting the co-occurrence relationship of pixel pairs with different gray values ​​in the image, and converts the image from RGB (color image) to grayscale image; where: I gray=0.2989×R+0.5870×G+0.1140×B, where R, G, and B are the pixel values ​​of the red, green, and blue channels, respectively; assuming that the direction we choose is "horizontal", that is, the horizontal distance between two adjacent pixels is 1, a gray-level co-occurrence matrix P(i, j, d, θ) is generated, where i and j represent the grayscale levels, d is the distance between pixel pairs (such as 1 pixel, 2 pixels), and θ is the direction of the pixel pair (such as 0°, 45°, 90°, 135°); To measure the contrast of image texture, we calculate the difference between matrix elements, which is expressed as: ; Energy is used to measure the uniformity of texture in an image and reflects the smoothness of the image. It is expressed as: ; Homogeneity is used to measure the smoothness of an image and reflects the consistency of the texture, expressed as: ; Entropy is used to measure the complexity or amount of information in image texture and is expressed as: ; Among them, the texture feature vector T i It is expressed as: T i =[Contrast i ,Energy i ,Homogeneity i ,Entropy i ].

[0030] Furthermore, in one implementation of the shape feature, the edges in the image are connected into a closed contour using a contour extraction method (findContours in OpenCV). At this point, the contour information of each superpixel region is extracted. The area of ​​the contour can help distinguish regions of different sizes. As part of the shape feature, the area of ​​the contour is expressed as follows: A i =Area(C i ); where C i represents the contour of the i-th superpixel region; By calculating the bounding box of the outline, the size and shape of the area can be obtained, as shown below: i =[x min ,y min ,x max ,y max ]; Perimeter can represent the edge length of the contour, expressed as follows: P i=Perimeter(C i ); Furthermore, the geometric invariance characteristics of the contour, such as rotation, scaling and mirror invariance, are captured by Hu moments; Hu moments are usually calculated by contour points and are expressed as follows: S i =[Hu1,Hu2,...,Hu7]; Through the above method, we can extract multiple geometric features (such as area, perimeter, shape descriptor, etc.) from the contour of the superpixel region and combine them into a feature vector S i , expressed as: S i =[A i ,P i ,BoundingBox i ,Hu i ].

[0031] In one implementation of the illumination feature, the brightness information of each superpixel region is extracted by converting the RGB image into a grayscale image; Gray value L i As the illumination feature of the superpixel area, it is expressed as follows: ; Among them, gray(p) is the color value of pixel p, P i is the set of pixels in the superpixel region, N i is the number of pixels within the superpixel region.

[0032] The graph convolutional network of the MAGAN model architecture of the present invention can process the spatial information of the image in the graph structure, convert the water body image into a graph structure, and extract local features by learning the relationship between the nodes in the graph. This enables the model to have stronger spatial perception ability when dealing with the complex morphology of cyanobacteria and its relationship with the background.

[0033] Furthermore, in the graph attention mechanism provided by the embodiment of the present invention, the model calculates an attention weight α for each node. ij , attention weight α ij Represents the correlation between a node and other nodes; the calculation formula of the attention mechanism is as follows: ; Among them, Q i is the query vector of node i, K j is the key vector of node j, k represents the set of neighbor nodes of node i, α ij is the attention weight of node i to node j; W Q represents the weight matrix of the query vector Q; T represents the transpose operation; Represents the key vector of node k, by calculating To measure the similarity between node i and node j, a nonlinear transformation is introduced through the LeakyReLU activation function; the attention coefficient is normalized by softmax so that the sum of the attention weights of each node is 1; Specifically, the query vector Q i is the transformation of the feature representation of node i, which is obtained by combining the feature of node i with a weight matrix W Q Multiply the query vector Q i It is used to match the features of other nodes in the self-attention mechanism to determine the degree of attention of node i to other nodes, where Q i =W Q h i ,h i is the original feature of node i, W Q is the learned weight matrix, usually a d×d′ matrix, where d is the dimension of the original feature and d′ is the dimension of the query vector; Key vector K j is the transformation of the feature representation of node j, which is used together with the query vector to calculate the similarity between nodes. In the self-attention mechanism, node i will be i The key vector K of node j j Matching is performed to calculate the degree of attention, expressed as: K j =W k h j ; The weight matrix is ​​the two matrices W Q and W k are learned parameters used to map the features of nodes to the space of query vectors and key vectors. Through these weight matrices, the model can learn how to calculate attention between different nodes, thereby dynamically adjusting the focus area according to the context. Q ∈R d×d′ , W k ∈R d×d′ .

[0034] Further, the query and key calculation: For each node i, first calculate its query vector Q i and the key vector K of each neighbor node j j . Similarity calculation: Then, calculate the query vector Q i and the key vector K j Similarity between , which is usually done by taking the dot product of the vectors; Finally, the feature of node i The following weighted aggregation process will be used to obtain: ; Among them, h jis the characteristic of node j. This aggregation process helps the model focus more on important neighboring nodes, thereby enhancing the focus on the cyanobacteria area.

[0035] Furthermore, in the graph convolutional network GCN, the feature vector of each superpixel region In the graph convolution layer, the pixel features of its neighborhood are updated to obtain ; The update of the graph convolution layer is shown in the formula: ;in, is the feature representation of node i at layer l; N(i) represents the set of neighbor nodes of node i; A ij is the adjacency matrix element between nodes i and j, reflecting their similarity; W (l) is the learning weight matrix of the graph convolutional layer; σ is the ReLu activation function; c ij represents the normalization coefficient, , deg(i) represents the degree of node i; deg(j) represents the degree of node j.

[0036] Furthermore, in an embodiment of the present invention, the step of strengthening the difference between regions by calculating the contrast loss between the foreground and the background and removing the background to obtain the output features includes: After being processed by graph convolution and self-attention mechanism, the obtained node features contain foreground information and background information ; In order to perform adaptive background removal, the present invention introduces a background removal module to perform differential processing on the extracted features. The module uses the contrast loss L contrastiv To distinguish the difference between cyanobacteria and the background, it is convenient to better distinguish the foreground information from the background information, as shown below: ; in, and Represent the features of the foreground and background areas respectively, and N is the number of samples; The loss function L contrastiv Aims to enhance the model's ability to distinguish between the foreground and background by maximizing the difference between the features; Represents the square of the Euclidean distance between foreground and background feature vectors; The output layer separates the foreground and background, and uses a fully connected layer to map each pixel or region feature of the image to the category label of the background and foreground, expressed as: i =softmax(W out h i ); where h i represents the feature vector of node i, y irepresents the classification result of node i, and the classification structure is foreground or background; W out is the weight matrix of the fully connected layer.

[0037] Eigenvector h i Feature dimension: In image processing tasks, nodes usually correspond to pixels or superpixel regions in the image, and the feature vector h i The dimension is determined by the network architecture design, and usually includes multiple feature channels such as color, texture, and shape. For example, color features may include the values ​​of the three channels of red, green, and blue, texture features may include texture information extracted by filters, and shape features may include edge detection results, etc. Information fusion: In MAGAN, the node feature h i It is obtained by fusing information from multiple modalities (such as color, texture, shape, etc.). This fusion enables the feature vector of each node to fully represent its attributes and contextual information in the image.

[0038] The present invention adopts an adaptive background removal mechanism, which enables the model to dynamically adjust the background processing method according to the characteristics of different water environments, ensuring that the effect of background removal does not affect the accuracy of cyanobacteria detection. Through this adaptive adjustment, the background removal module can effectively avoid the problem of excessive or insufficient removal of background interference in complex scenes.

[0039] For further information, please refer to Figure 1 and Figure 2 ,In the embodiment of the present invention, the key of multi-task learning is to share features, allowing the model to be collaboratively optimized in multiple related tasks; Set the output feature of the MAGAN model to be H MAGAN , which is a matrix containing image region features. Typically, the output features of MAGAN will be fed into a shared feature layer, and then the output of this layer is passed to different task-specific output layers; specifically: The feature H output after background removal MAGAN Input to a shared convolutional layer F shared , expressed as follows: F shared (x) = ReLU(Conv(H MAGAN )), where H MAGAN represents the node feature matrix obtained after processing the input image, F sharedThe extracted features are general visual features, which are applicable to all subsequent tasks. The nonlinearity is enhanced by the ReLU activation function. Each convolution layer extracts features at different levels of the image, from low-level features (edges, textures) to high-level features (cyanobacteria shape and structure). The 3×3 convolution kernel is used, and the standard convolution kernel size can effectively capture local features in the image. As the convolution layer goes deeper, the number of channels gradually increases from 64 to 512, which can extract more complex features. Maximum pooling: pooling operation is applied after each convolution layer, using a 2×2 pooling window with a stride of 2, which further reduces the size of the feature map and reduces the amount of calculation.

[0040] Output feature H MAGAN is the node feature matrix obtained after processing the input image. These features represent the semantic information of each region in the image and can be used for subsequent tasks such as cyanobacteria detection and background removal. Specifically, H MAGAN is an N×D matrix, where N is the number of regions in the image, i.e., nodes, and D is the feature dimension of each node. These features capture the spatial and semantic relationships between image regions, providing rich information for multi-task learning; MAGAN When input into the subsequent multi-task deep learning framework, common features can be further extracted through shared convolutional layers and then passed to each task-specific branch (such as cyanobacteria detection and background removal) to achieve collaborative optimization.

[0041] Furthermore, the step of using the convolutional layer and the fully connected layer to classify or segment the cyanobacteria, generate a detection result, determine whether the image contains cyanobacteria, and locate the cyanobacteria area includes: After extracting all the convolutional features, the feature maps are flattened and passed to the fully connected layer for the final cyanobacteria segmentation; Using the fully connected layer F algae_fc Segment the cyanobacteria, determine whether the image contains cyanobacteria, and accurately locate the cyanobacteria area, as shown below: F algae (x) = ReLU(F algae_fc (flatten(F shared (x)))); F algae_fc (x)=W i ·x+b i ; Where i is the number of fully connected layers, and when i is 1, 2, or 3, the corresponding [W1, W2, W3] and [b1, b2, b3] are the weight matrix and bias term of the corresponding fully connected layer respectively; Among them, F algae Represents the overall function of the cyanobacteria detection branch, which is responsible for extracting features from the input image and classifying or segmenting cyanobacteria; specifically: Falgae (x) represents the processing process of the cyanobacteria detection branch on the input image x, including feature extraction, flattening, fully connected layer processing and activation function application, and finally generates the cyanobacteria detection result; F algae_fc It is the fully connected layer function in the cyanobacteria detection branch, which is responsible for converting the flattened feature map into the final output result; Furthermore, the output layer is a Sigmoid layer, which is used for segmentation tasks and outputs the probability y of whether each pixel belongs to the cyanobacteria area. algae predi .

[0042] The self-attention mechanism of the MAGAN model architecture of the present invention further enhances the model's ability to accurately distinguish between cyanobacteria and background in complex environments by paying attention to the global contextual information of each area in the image, which can effectively suppress background noise and improve the recognition accuracy of cyanobacteria.

[0043] Furthermore, in an embodiment of the present invention, the step of reconstructing the image background through a convolutional layer and a fully connected layer to generate an image after removing the background includes: The background removal task aims to remove the background from the image and retain the cyanobacteria area. To this end, the present invention uses transposed convolution to upsample the feature map and reconstruct the background; wherein the input feature map F in The dimensions are H′×W′×C in , upsample it to H×W×C by transposed convolution out , the transposed convolution operation is expressed as follows: F out =DeConv(F in ,K deconv ,stride,padding); Among them: F in is the input feature map, with size H′×W′×C in ; K deconv is the convolution kernel of the transposed convolution, with a size of 3×3. This convolution kernel learns the spatial relationship in the image; stride is the step size, which determines the size of the step in each convolution, usually set to 2 or 4 to achieve upsampling; padding is padding, which determines how the edges are handled during the convolution operation; F out It is the output feature map, with size H×W×C out It has a higher spatial resolution; ultimately, is the image output of the background subtraction task.

[0044] The present invention combines a multi-task deep learning framework, so that the cyanobacteria detection task and the background removal task can share some features and be optimized collaboratively, thereby further improving the overall performance. This multi-task collaborative optimization mechanism ensures effective information transfer and complementarity between the cyanobacteria detection task and the background removal task, enhances the adaptability and robustness of the model, and enables the method to show superior detection accuracy and reliability in complex dynamic water environments.

[0045] Furthermore, the step of designing a multi-task loss function, combining the cyanobacteria detection loss and the background removal task loss, and performing joint optimization through weighted combination includes: Design a multi-task loss function L, combine the losses of cyanobacteria detection and background removal tasks, and perform joint optimization through weighted combination; cyanobacteria detection loss L algae The cross entropy loss is used, which is expressed as follows: ; in, Indicates the actual algae value, is the predicted algae value; Background removal loss L bg Using mean square error MSE loss, it is expressed as follows: ; Among them, y bgijk Represents the corresponding real pixel value in the input image; y bg adjustedijk Represents the pixel value of the image after background removal; H and W are the height and width of the output image, respectively, and C is the number of channels of the output image; the total loss L total It is expressed as follows: L total =λ algae L algae +λ bg L bg ; Among them, λ algae and λ bg Both represent weight coefficients, which are used to balance the losses of the cyanobacteria detection task and the background removal task.

[0046] Furthermore, in step S104 provided by the present invention, during the process of training and optimizing the detection model, the parameters of the model are first initialized, the learning rate η is 0.001, the first-order momentum decay rate β1 is 0.9, the second-order momentum decay rate β2 is 0.999, and the value ε is 10 -8 ; The optimizer is combined with the model's parameters α, W (l) , W out , W conv , W fc , b fc ,λ algae ,λ bgBind; M={F shared ,F algae ,F bg ,α};R={W (l) ,W out ,W conv ,W fc ,b fc ,λ algae ,λ bg}; Among them, W out represents the weight matrix of the fully connected layer; W conv Represents the convolution kernel weight of the shared convolution layer; W fc represents the weight of the fully connected layer of the cyanobacteria detection branch; b fc represents the bias of the fully connected layer of the cyanobacteria detection branch; λ algae , bg Represents the weight parameter in the loss function; F shared represents the extracted general visual features; F algae represents the fully connected layer used for cyanobacteria classification or segmentation; F bg Represents the background removal result, which can be the predicted label of the background area or the image reconstruction output.

[0047] Training using the Adam optimizer: , which means that in the Adam optimizer, the total L total The process of updating the model parameters θ; ; Indicates that in the Adam optimizer, the contrast loss L is used contrastive The process of updating the model parameters θ; ; Indicates that in the Adam optimizer, the blue algae detection loss L is used algae The process of updating the model parameters θ; Among them, θ represents the model parameters and η represents the learning rate; Training loop: Optimize model parameters through training data, taking multi-task deep learning as an example: ; ; ; ; ; exist middle, Representation: Common feature map F output by the shared feature layer shared (x) is input to the cyanobacteria segmentation branch F algae, this branch decodes the features through a fully connected layer and outputs the probability that each pixel belongs to the cyanobacteria area; Representation: Prediction results of cyanobacteria segmentation branches is the actual algae value, is the predicted algae value, and the cross entropy loss L is calculated. algae , used to optimize the classification ability of the model; exist middle, Representation: The output features of the cyanobacteria segmentation branch are passed to the background removal branch, and upsampled by transposed convolution to generate an image after background removal. ,This process aims to reconstruct an image that retains only the cyanobacteria regions; Means: The mean square error (MSE) loss L is calculated between the image after background removal and the real background removal result (i.e., the clean foreground image) bg , used to optimize the accuracy of background removal; exist Specifically, it means to perform the following operations: Input: current parameter θ, total loss L total , learning rate η; Calculate the gradient: Calculate the total loss L through back propagation total The gradient of θ; Update parameters: adjust θ according to the Adam formula to update it in the direction of reducing loss; Output: Updated parameters θ for the next round of training.

[0048] Specifically, in an embodiment of the present invention, the convolution kernel weights and biases of the shared convolution layer are updated, and the update is performed according to the rules of the gradient and Adam optimizer; the weights and biases of the fully connected layer of the cyanobacteria detection branch are updated, and the parameters are updated by applying the Adam optimizer; the transposed convolution kernel weights and biases of the background removal branch are updated by calculating the gradient and applying the Adam update step; the weight parameter λ in the loss function is updated, and it is adjusted according to the rules of the gradient and Adam optimizer; Input the preprocessed image x, and perform adaptive background removal through the Multimodal Adaptive Graph Attention Network (MAGAN) to obtain the features after background removal. The features after background removal are input into the multi-task deep learning framework, and the shared convolution layer extracts common features to generate feature maps. The cyanobacteria detection branch uses the convolution layer and the fully connected layer to classify or segment the cyanobacteria and generate the detection results, which are expressed as: ; The background removal branch reconstructs the image background through convolutional layers and fully connected layers to generate an image after background removal, which is expressed as: ; Combining the cyanobacteria detection loss and the background removal task loss, a multi-task loss function is obtained through weighted combination: .

[0049] Repeat the above process of forward propagation, loss calculation, back propagation and parameter update until the performance of the model on the validation set reaches the expected level or meets certain convergence conditions.

[0050] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. A cyanobacteria detection method based on multi-task deep learning and adaptive background removal, characterized in that: The cyanobacteria detection method comprises the following steps: Build a detection model, which includes a multimodal adaptive graph attention network MAGAN and a multi-task deep learning framework; In the multimodal adaptive graph attention network MAGAN, the local features of the image are extracted through the graph convolutional network GCN, and the attention level of each area in the image is adaptively adjusted and enhanced in combination with the graph attention mechanism to obtain the features output after background removal; the common features of the features output after background removal are extracted through the shared convolution layer to generate a feature map; In the multi-task deep learning framework, convolutional layers and fully connected layers are used to classify or segment cyanobacteria, generate detection results, determine whether the image contains cyanobacteria, and locate the cyanobacteria area; and reconstruct the image background through convolutional layers and fully connected layers to generate an image with the background removed; Design a multi-task loss function, combine the cyanobacteria detection loss and the background removal task loss, and perform joint optimization through weighted combination; Train and optimize the model parameters, take the water surface image as the input of the model, and output the detection results of cyanobacteria in the water surface image.

2. The method for detecting cyanobacteria based on multi-task deep learning and adaptive background removal according to claim 1, characterized in that: The steps of extracting local features of an image through a graph convolutional network GCN include: For the input image, the superpixel partitioning algorithm SLIC is used to divide the image into multiple regions with similar features, and the vector F is obtained through multimodal feature extraction. i , F i =[C i ,T i ,S i ,L i ]; Among them, C i represents the color feature vector, T i represents the texture feature vector, S i represents the shape feature vector, L i Represents the illumination feature vector.

3. The method for detecting cyanobacteria based on multi-task deep learning and adaptive background removal according to claim 2, characterized in that: Combined with the graph attention mechanism, the steps of adaptively adjusting the attention level of each area in the enhanced image and obtaining the output features after background removal include: Use the graph attention mechanism to adaptively weight image features and calculate the attention weights between nodes to dynamically adjust the attention level of each area in the image; The attention weight will perform weighted summation on the node features to obtain the updated node features; The features of each node are updated by weighted summing with the features of neighboring nodes in the graph convolution operation; By calculating the contrast loss between the foreground and background, the difference between regions is enhanced and the background is removed to obtain the output features.

4. The method for detecting cyanobacteria based on multi-task deep learning and adaptive background removal according to claim 3, characterized in that: In the graph attention mechanism, the model calculates an attention weight α for each node ij , attention weight α ij Indicates the correlation between a node and other nodes; Among them, the calculation formula of the attention mechanism is expressed as follows: ； Where: Q i is the query vector of node i, K j is the key vector of node j, k represents the set of neighbor nodes of node i, α ij is the attention weight of node i to node j; W Q represents the weight matrix of the query vector Q; T represents the transpose operation; represents the key vector of node k; By calculation To measure the similarity between node i and node j, a nonlinear transformation is introduced through the LeakyReLU activation function; the attention coefficient is normalized by softmax so that the sum of the attention weights of each node is 1; the feature of node i The following weighted aggregation process will be used to obtain: , where h j is the feature of node j.

5. The method for detecting cyanobacteria based on multi-task deep learning and adaptive background removal according to claim 4, characterized in that: In the graph convolutional network GCN, the feature vector of each superpixel region is In the graph convolution layer, the pixel features of its neighborhood are updated to obtain ; The update of the graph convolutional layer is expressed as: ； in: is the feature representation of node i at layer l; N(i) represents the set of neighbor nodes of node i; A ij is the adjacency matrix element between node i and node j, reflecting their similarity; W （l） is the learning weight matrix of the graph convolutional layer; σ is the ReLu activation function; c ij represents the normalization coefficient, where: , where deg(i) represents the degree of node i; deg(j) represents the degree of node j.

6. The method for detecting cyanobacteria based on multi-task deep learning and adaptive background removal according to claim 5, characterized in that: The steps of strengthening the difference between regions by calculating the contrast loss between the foreground and the background and removing the background to obtain the output features include: After being processed by graph convolution and self-attention mechanism, the obtained node features contain foreground information and background information ; A background removal module is introduced to differentiate the extracted features. The background removal module uses the contrast loss L contrastive To distinguish the difference between cyanobacteria and background, it is expressed as follows: ； in, and Represent the features of the foreground and background areas respectively, and N is the number of samples; The output layer separates the foreground and background, and uses a fully connected layer to map each pixel or region feature of the image to the category label of the background and foreground, expressed as: i =softmax(W out h i ), where h i represents the feature vector of node i; y i represents the classification result of node i, and the classification structure is foreground or background; W out is the weight matrix of the fully connected layer.

7. The method for detecting cyanobacteria based on multi-task deep learning and adaptive background removal according to claim 6, characterized in that: The steps of extracting common features from the features output after background removal through a shared convolutional layer and generating a feature map include: The features output after background removal are input into a shared convolutional layer F shared , which is expressed as follows: F shared (x)=ReLU(Conv(H MAGAN )); Among them, F shared What is extracted are common visual features; H MAGAN Represents the node feature matrix obtained after processing the input image; Enhance nonlinearity through ReLU activation function; Each convolutional layer extracts features at different levels of the image.

8. The method for detecting cyanobacteria based on multi-task deep learning and adaptive background removal according to claim 7, characterized in that: The steps of using the convolutional layer and the fully connected layer to classify or segment the cyanobacteria, generate the detection results, determine whether the image contains cyanobacteria, and locate the cyanobacteria area include: After all convolutional features are extracted, the feature maps are flattened and passed to the fully connected layer for the final cyanobacteria segmentation. algae_fc Segment the cyanobacteria, determine whether the image contains cyanobacteria, and accurately locate the cyanobacteria area, as shown below: F algae (x)=ReLU(F algae_fc (flatten(F shared (x)))); F algae_fc (x)=W i ·x+b i ; Where i is the number of fully connected layers, and when i is 1, 2, or 3, the corresponding [W1, W2, W3] and [b1, b2, b3] are the weight matrix and bias term of the corresponding fully connected layer respectively; The output layer is a Sigmoid layer, which is used for segmentation tasks and outputs the probability of whether each pixel belongs to the cyanobacteria area.

9. The method for detecting cyanobacteria based on multi-task deep learning and adaptive background removal according to claim 8, characterized in that: The steps of reconstructing the image background through the convolution layer and the fully connected layer to generate the image after removing the background include: Transposed convolution is used to upsample the feature map and reconstruct the background; among them, the input feature map F in The dimensions are H′×W′×C in , upsample it to H×W×C by transposed convolution out , the transposed convolution operation is expressed as follows: F out =DeConv(F in ,K deconv ,stride,padding); Among them: F in The size is H'×W'×C in Input feature map of K deconv is a convolution kernel of a transposed convolution of size 3×3, which is used to learn the spatial relationship in the image; stride represents the stride; padding represents padding; F out The size is H×W×C out The output feature map of .

10. The method for detecting cyanobacteria based on multi-task deep learning and adaptive background removal according to claim 9, characterized in that: The steps of designing a multi-task loss function, combining the cyanobacteria detection loss and the background removal task loss, and performing joint optimization through weighted combination include: A multi-task loss function is designed, which combines the losses of cyanobacteria detection and background removal tasks and performs joint optimization through weighted combination. algae Using cross entropy loss, expressed as: ; in, Indicates the actual algae value, is the predicted algae value; Background removal loss L bg Using mean square error MSE loss, it is expressed as follows: ； Among them, y bgijk Represents the corresponding real pixel value in the input image; y bg adjustedijk Represents the pixel value of the image after background removal; H and W are the height and width of the output image, respectively, and C is the number of channels of the output image; the total loss L total It is expressed as follows: L total =λ algae L algae +λ bg L bg ; Among them, λ algae and λ bg Both represent weight coefficients, which are used to balance the losses of the cyanobacteria detection task and the background removal task.

Images