A method for constructing an activated sludge microscopic image segmentation model

Abstract

The application provides an activated sludge microscopic image segmentation model construction method, relates to a sewage treatment intelligent detection method, and comprises the following steps: collecting an activated sludge image, creating an activated sludge microscopic image data set, constructing an activated sludge microscopic image segmentation model based on a mixed-Transformer encoder and a pyramid pooling multi-layer semantic flow alignment decoder, training the activated sludge microscopic image segmentation model by using a mixed loss function, deploying and reasoning the segmentation model. The method provided by the application can more accurately identify the boundary information of flocs and filamentous bacteria in the activated sludge microscopic image, reduces the edge contour blur problem caused by artifacts and the local detail information loss problem caused in the feature extraction process of the neural network, is helpful for accurately segmenting the activated sludge microscopic image, more effectively monitors and prevents the activated sludge expansion and other phenomena, and provides more important information for downstream applications.

Description

Technical Field

[0001] This invention relates to an intelligent detection method for wastewater treatment, and in particular to a method for constructing a microscopic image segmentation model of activated sludge. Background Technology

[0002] The microscopic morphology and characteristics of microorganisms in phase-contrast micrographs of activated sludge directly reflect the settling performance of activated sludge in wastewater treatment plants. The current characterization of flocs and filamentous bacteria is crucial for monitoring and preventing sludge bulking. However, in activated sludge micrographs, flocs of varying densities grow on slender filamentous skeletons, exhibiting heterogeneity and complex structures. This can lead to the loss of detailed information during feature extraction by deep neural networks. Furthermore, inherent artifacts exist when observing flocs and filamentous bacteria using phase-contrast microscopy, blurring their edge contours and affecting the identification of edge details.

[0003] Traditional activated sludge segmentation methods mainly include threshold-based segmentation, edge-based segmentation, and region-based segmentation. Using the Otsu thresholding algorithm to segment activated sludge phase-contrast microscopic images can lead to over-segmentation of flocs, affecting the final segmentation result. Watershed-based segmentation is not only computationally intensive, but the selection of the threshold and the number of pixel grayscale values ​​also influence the segmentation result. Therefore, traditional image segmentation methods are sensitive to parameters, requiring manual adjustment, and struggle to achieve accurate segmentation results for flocs and filamentous bacteria in complex environments. With the development of deep learning technology, computer vision algorithms using deep convolutional neural networks as the main method have achieved significantly improved performance, offering higher segmentation accuracy compared to traditional methods. Summary of the Invention

[0004] The purpose of this invention is to provide a method for constructing a segmentation model for activated sludge microscopic images. This invention uses a hybrid-Transformer encoder proposed by Segformer to extract multi-scale features, and a pyramid pooling multi-layer semantic flow alignment module as the decoder. Finally, the model introduces skip connections to fuse global and local features, achieving more accurate segmentation. To obtain a smaller number of difficult-to-segment samples, a hybrid loss function is constructed to backpropagate and update the model parameters, effectively improving the segmentation accuracy of activated sludge microscopic images and solving the technical bottleneck of wastewater detection.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for constructing a segmentation model of activated sludge microscopic images, comprising the construction of an automatic segmentation system for activated sludge samples, which includes a server and a client, wherein the server and client are interconnected; the client is used to acquire activated sludge images in real time and transmit them to the server for model inference, and obtain the segmentation result from the server's model inference; the preparation steps are as follows:

[0007] S1, Acquire images of activated sludge;

[0008] S2, Create a dataset of activated sludge microscopic images;

[0009] S3, construct an activated sludge microscopic image segmentation model based on a hybrid-Transformer encoder and a pyramid pooling multi-layer semantic flow aligned decoder;

[0010] S4, Use a hybrid loss function to train the activated sludge microscopic image segmentation model;

[0011] S5, segmentation model deployment and inference.

[0012] The method for constructing a microscopic image segmentation model of activated sludge, wherein the step S1 of acquiring activated sludge images includes the following steps:

[0013] S11, Collect activated sludge samples from the wastewater treatment plant;

[0014] S12 uses a phase contrast microscope, an industrial digital camera, and an image acquisition system. The image acquisition software scans the sample on the slide from the upper left to the lower right.

[0015] The method for constructing a microscopic image segmentation model of activated sludge, wherein step S2 of constructing the activated sludge segmentation model includes the following steps:

[0016] S21, Select a typical activated sludge image from the acquired activated sludge images;

[0017] S22, the selected activated sludge image is labeled to obtain a label image of the same size as the original image. The label image includes the background, flocs and filamentous bacteria.

[0018] The method for constructing a microscopic image segmentation model of activated sludge, wherein step S3 of constructing the activated sludge segmentation model includes the following steps:

[0019] S31, the hybrid-Transformer proposed in Segformer is selected as the encoder. The encoder consists of four Transformer blocks, each of which comprises an efficient self-attention module, a hybrid feedforward neural network, and an overlapping patch merging module. The feature maps after the four Transformer blocks are respectively the resolution of the original image. Multi-scale features can be extracted from activated sludge images;

[0020] S32, the highest-level feature extracted from the multi-scale features by the encoder, is processed by a pyramid pooling model (PPM) to extract higher-level semantic information. The deepest features contain a larger receptive field, which can extract multi-level features; this provides more feature information for the subsequent multi-layer semantic flow alignment module; in the multi-layer semantic flow alignment module, each feature map goes through a 1×1 convolutional layer to unify the channel depth; the high-level feature map and the low-level feature map of two adjacent feature maps are denoted as F. H F L ;F H After bilinear interpolation and upsampling to F L Same size, facilitating subsequent assembly; F L The result after 1×1 convolution and F L The upsampled results are concatenated along the channel dimension to incorporate features of different scales. Then, a 3×3 convolutional layer is used to extract the fused features, yielding the final semantic flow field. This semantic flow field records the positional information between the two feature maps. This process generates the semantic flow field and F... H Alignment is performed to obtain the output feature map; this allows the image to retain global features while adding more local information, achieving a fine-grained upsampling operation.

[0021] S33 introduces skip connections, enabling the feature map obtained after PPMSFAM to be compared with the four feature maps F of different scales generated in the encoder. i Splicing together F at different scales i Each feature map passes through a linear layer, which is composed of a multilayer perceptron, and then is upsampled to make the feature map size consistent with the size of the feature map obtained after PPMSFAM, which facilitates subsequent stitching.

[0022] The method for constructing a microscopic image segmentation model of activated sludge, wherein step S4 of training the microscopic image segmentation model of activated sludge using a hybrid loss function includes the following steps:

[0023] S41, In order to obtain a small number of difficult-to-distinguish samples, define the model's mixture loss function:

[0024] L(p t ) Fl =-(1-p t ) γ log(p t )

[0025]

[0026] Mixloss=ω1L(p t ) Fl +ω2L LS

[0027] Wherein, -log(p t ) is the initial cross-entropy loss function, (1-p t ) γ Δ is the simple / hard sample adjustment factor, γ is the focusing parameter, m(c) is the pixel error vector, c is the class, |C| is the total number of classes, and Δ Jc It is the Jaccard loss function. ω1 and ω2 are the surrogate loss functions of Jaccard loss, and the weights corresponding to focal loss and Lovász-Softmax loss are respectively.

[0028] S42, use the defined hybrid loss function to train the activated sludge segmentation model and obtain the model parameters.

[0029] The method for constructing a segmentation model of activated sludge microscopic images, wherein the segmentation model deployment and inference step S5 includes the following steps:

[0030] S51, Deploy the segmentation model;

[0031] S52 uses real-time acquired activated sludge images to infer the segmentation model and obtain the final segmentation result.

[0032] The advantages and effects of this invention are:

[0033] This invention achieves more accurate segmentation results. During model training, a hybrid-Transformer model proposed by Segformer is used as the encoder to extract multi-scale features of flocs and filamentous bacteria from activated sludge microscopic images. The decoder employs a pyramid pooling multi-level semantic flow alignment module, feeding the highest-level features extracted from the encoder into the pyramid pooling model (PPM) to obtain higher-level semantic information. Subsequently, the extracted higher-level semantic information is fed into the multi-level semantic flow alignment module, adding more local feature information while retaining more global feature information, achieving a finer upsampling operation. Skip connections are introduced to fuse global and local feature information, resulting in finer edge information for flocs and filamentous bacteria. To obtain data from a smaller number of difficult-to-separate samples, a hybrid loss function is used to backpropagate and update parameters of the overall model, improving the model's segmentation accuracy and obtaining the final segmentation result. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating a method for constructing a microscopic image segmentation model of activated sludge according to an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram illustrating the structure and jump connection details of a segmentation model based on a pyramid pooling multi-layer semantic flow alignment module according to an embodiment of the present invention.

[0036] Figure 3 This is an example diagram showing the segmentation results of the present invention on real-time acquired activated sludge images;

[0037] Figure 4 This is a framework diagram of an automatic activated sludge sample segmentation system according to an embodiment of the present invention. Detailed Implementation

[0038] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings.

[0039] Figure 1 This is a flowchart illustrating a method for constructing a segmentation model based on activated sludge microscopic images, as exemplified by this invention. Figure 1 As shown, the segmentation model construction method includes:

[0040] S1, Acquire images of activated sludge. S1 includes the following steps:

[0041] S11, Phase contrast microscopic image acquisition process for activated sludge: Activated sludge samples are collected from the outlet of the aeration tank of the sewage treatment plant at a distance of 50cm from the liquid surface. Each sample is 500mL and the time interval between sample collection and image acquisition shall not exceed 3 hours.

[0042] S12, take 10 μL of sample on a glass slide, cover it with a 24 mm × 24 mm coverslip, and use an optical microscope, an industrial digital camera and image acquisition software (ToupView) to scan the sludge sample on the glass slide in a zigzag path from the upper left to the lower right.

[0043] S2, Create a dataset of activated sludge microscopic images. S2 includes the following steps:

[0044] S21. Select 323 typical activated sludge images from the collected activated sludge images as the segmentation dataset.

[0045] S22. Label the original activated sludge images using the data labeling tool LabelMe to label the 323 activated sludge images, resulting in label images of the same size as the original images. The label images contain three categories: background, flocs, and filamentous bacteria.

[0046] S3. An activated sludge microscopic image segmentation model (hereinafter referred to as Pyramid Pooling Multilayer Semantic FlowAlign Module, i.e. PPMSFAM segmentation model) is constructed based on a hybrid-Transformer encoder and a pyramid pooling multilayer semantic flow alignment decoder. The introduction of skip connections can fuse local feature information and global feature information.

[0047] Specifically, this segmentation model addresses the morphological characteristics of flocs and filamentous bacteria in activated sludge microscopic images, as well as the artifacts and blurred edge contours that occur when observing flocs and filamentous bacteria using phase contrast microscopy. It achieves finer edge segmentation of flocs and filamentous bacteria, enabling a more precise upsampling operation. For example... Figure 2 As shown, the segmentation model mainly consists of two parts: an encoder and a decoder. S3 includes the following steps:

[0048] S31, the hybrid-Transformer proposed in Segformer is selected as the encoder. The encoder consists of four Transformer blocks, each of which comprises an efficient self-attention module, a hybrid feedforward neural network, and an overlapping patch merging module. The feature maps after the four Transformer blocks are respectively the resolution of the original image. Multi-scale features can be extracted from activated sludge images;

[0049] S32 represents the highest-level feature extracted from the multi-scale features by the encoder. This highest-level feature is then processed by a pyramid pooling model (PPM) to extract even higher-level semantic information. The deepest features contain a larger receptive field, allowing for the extraction of multi-level features. This provides more feature information for the subsequent multi-layer semantic flow alignment module. In the multi-layer semantic flow alignment module, each feature map passes through a 1×1 convolutional layer to unify the channel depth. The higher-level and lower-level feature maps of two adjacent feature maps are denoted as Fhigher-level and Flower-level, respectively. H F L F H After bilinear interpolation and upsampling to F L Same size, facilitating subsequent assembly. F L The result after 1×1 convolution and F L The upsampled results are concatenated along the channel dimension to incorporate features of different scales. Then, a 3×3 convolutional layer is used to extract the fused features, yielding the final semantic flow field. This semantic flow field records the positional information between the two feature maps. This will generate the semantic flow field and F... H Alignment is performed to obtain the output feature map. This allows the image to retain global features while adding more local information, achieving a fine-grained upsampling operation;

[0050] S33 introduces skip connections, enabling the feature map obtained after PPMSFAM to be compared with the four feature maps F of different scales generated in the encoder. i Then proceed with the assembly. Specific details are as follows: Figure 2 As shown, F at four different scales i It needs to go through a linear layer and upsampling to make F at different scales i Consistent sizes facilitate subsequent stitching, allowing local and global features to be further integrated for more refined segmentation results.

[0051] S4, train the activated sludge microscopic image segmentation model using a hybrid loss function. S4 includes the following steps:

[0052] S41, the activated sludge microscopic image dataset is divided into two parts: a training set and a test set;

[0053] S42, in order to obtain a small number of difficult-to-distinguish samples, a constructed hybrid loss function was used to back-optimize the model parameters;

[0054] Alternatively, the hybrid loss function can be constructed and its formula is as follows:

[0055] Focal loss is employed. This loss function dynamically transforms between easily classifiable and difficult-to-classify samples during training, which is more helpful in learning the features of difficult-to-classify samples. The formula is shown below:

[0056] L(p t ) Fl =-(1-p t ) γ log(p t )

[0057] Wherein, -log(p t ) is the initial cross-entropy loss function, (1-p t ) γ γ is the adjustment factor for easy / difficult samples, while γ is the focusing parameter.

[0058] To further extract detailed information about the smaller targets, a Lovász-Softmax loss was added. This loss function is a direct optimization of the neural network mIoU loss based on the Lovasz extension of the submodule loss, and it is derived from the Jaccard loss function Δ. Jc Lovász extension The sum and error vector m(c) constitute the expression, defined as follows:

[0059]

[0060] Where C is the total number of categories, c is the class to which the pixel belongs, m(c) is the pixel error vector, and Δ Jc The Jaccard loss function is used. This is a proxy loss function for Jaccard loss.

[0061] Specifically, the Jaccard loss function Δ Jc The definition is as follows:

[0062]

[0063]

[0064] Among them, M c For the set of incorrectly predicted pixels of category c; y * For true labels, This is the predicted result.

[0065] The definition of is:

[0066]

[0067]

[0068] g i (m)=Δ({π1,...,π i})-Δ({π1,...,π i-1})

[0069] Where m i (c) is the error vector of pixel i, f i (c) is the probability distribution of pixel i belonging to category c calculated using SoftMax; g i (m) is a Lovász extension The derivative of π is the component of m arranged in descending order.

[0070] The overall model is optimized using a hybrid loss function composed of the two loss functions mentioned above. The combined formula is shown below:

[0071] Mixloss=ω1L(p t ) Fl +ω2L LS

[0072] Where ω1 and ω2 are the weights corresponding to focal loss and Lovász-Softmax loss, respectively.

[0073] S5, segmentation model deployment and inference. S5 includes the following steps:

[0074] S51, Deploy the segmentation model;

[0075] S52, real-time acquired activated sludge images are used to test the segmentation model, and the final segmentation results are obtained, such as... Figure 3 As shown, the first row is a partial activated sludge image selected during the acquisition of activated sludge images, the second row is the label image corresponding to the original image (where black is the background, gray is flocs, and white is filamentous bacteria), and the third row is the segmentation result image obtained by the method constructed in this invention.

[0076] Specifically, to verify the segmentation performance of the PPMSFAM-based segmentation model on flocculents and filamentous bacteria, precision, recall, intersection-over-union (IoU), and mean intersection-over-union (mIoU) were used as evaluation metrics. Precision is the ratio of correctly predicted pixels to the total number of pixels in a given class; Recall is the probability that a positive sample is predicted as positive; IoU is the ratio of the intersection to the union of predicted and true values ​​for a given class; and mIoU is the average IoU across all classes. The evaluation metrics are shown in Table 1.

[0077] Table 1. Segmentation Indicators for Different Modules

[0078]

[0079] The test results of the model were obtained as follows: the original Segformer model had an IoU of 0.8232 for flocs, 0.5131 for filaments, and a mIoU of 0.7668. After introducing Mixloss, the segmentation accuracy of flocs and filaments improved slightly, with the IoU for flocs reaching 0.8245 (an improvement of 0.13%), and the IoU for filaments reaching 0.5153 (an improvement of 0.22%), resulting in an overall mIoU improvement of 0.09%. After introducing the PPMSFAM module, the segmentation accuracy was significantly improved, with the IoU for flocs reaching 0.8296 (an improvement of 0.5%), and the IoU for filaments reaching 0.5323 (an improvement of 1.7%), resulting in an overall mIoU improvement of 0.79%. This indicates that the PPMSFAM module, as a decoder, can segment more accurately, helping to compensate for details lost during encoding and downsampling, and also aiding in the segmentation of blurred outlines in filaments and flocs. After introducing the skip connection module, the IoU of flocculents reached 0.8315, an improvement of 0.19%, and the IoU of filamentous bacteria reached 0.5376, an improvement of 0.53%, with an overall improvement of 0.25% in mIoU. This demonstrates that skip connections can fuse global semantic information and local feature information at different scales, which is beneficial for recovering fine-grained feature information of flocculents and filamentous bacteria.

[0080] Figure 4 An automated activated sludge sample segmentation system according to an embodiment of the present invention is shown. Figure 4 As shown, the automatic activated sludge sample segmentation system includes a server and a client, wherein the server and the client are connected. The server uses the above-mentioned method to train the model; the client is used to collect activated sludge images in real time and upload them to the server for model inference, and obtain the segmentation results from the server model inference.

[0081] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-described technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for constructing a microscopic image segmentation model of activated sludge, characterized in that, The method includes the construction of an automatic activated sludge sample segmentation system, which consists of a server and a client, wherein the server and client are interconnected; the client is used to acquire activated sludge images in real time and transmit them to the server for model inference, and obtain the segmentation result from the server's model inference; the preparation steps are as follows: S1, Acquire images of activated sludge; S2, Create a dataset of activated sludge microscopic images; S3, construct an activated sludge microscopic image segmentation model based on a hybrid-Transformer encoder and a pyramid pooling multi-layer semantic flow aligned decoder; S4, Use a hybrid loss function to train the activated sludge microscopic image segmentation model; S5, segmentation model deployment and inference; Step S3 of constructing the activated sludge segmentation model includes the following steps: S31, the hybrid-Transformer proposed in Segformer is selected as the encoder. The encoder consists of four Transformer blocks, each of which comprises an efficient self-attention module, a hybrid feedforward neural network, and an overlapping patch merging module. The feature maps after the four Transformer blocks are respectively the resolution of the original image. Multi-scale features of activated sludge images were extracted; S32 represents the highest-level feature extracted from the multi-scale features by the encoder. This highest-level feature is then processed by a pyramid pooling model to extract even higher-level semantic information. The deepest features contain a larger receptive field, enabling the extraction of multi-level features. This provides more feature information for the subsequent multi-layer semantic flow alignment module. In this module, to unify the channel depth, each feature map passes through a 1×1 convolutional layer. The higher-level and lower-level feature maps in two adjacent feature maps are denoted as S32 and S32, respectively. , ; Upsampled to the value after bilinear interpolation The same size makes subsequent splicing easier; The result after 1×1 convolution and The upsampled results are concatenated along the channel dimension to incorporate features at different scales. Then, a 3×3 convolutional layer is used to extract the fused features, yielding the final semantic flow field. This semantic flow field records the positional information between the two feature maps. This process generates the semantic flow field and... Alignment is performed to obtain the output feature map; this adds more local information to the image while preserving global features, achieving a fine-grained upsampling operation. S33 introduces skip connections, enabling the feature map obtained after PPMSFAM to be integrated with the four feature maps of different scales generated in the encoder. To splice together, different scales Each feature map passes through a linear layer, which is composed of a multilayer perceptron, and then is upsampled to make the feature map size consistent with the size of the feature map obtained after PPMSFAM, which facilitates subsequent stitching.

2. The method for constructing a microscopic image segmentation model of activated sludge according to claim 1, characterized in that, Step S1, which involves acquiring images of activated sludge, includes the following steps: S11, Collect activated sludge samples from the wastewater treatment plant; S12 uses a phase contrast microscope, an industrial digital camera, and an image acquisition system. The image acquisition software scans the sample on the slide from the upper left to the lower right.

3. The method for constructing a microscopic image segmentation model of activated sludge according to claim 1, characterized in that, Step S2, which involves creating a dataset of activated sludge microscopic images, includes the following steps: S21, Select a typical activated sludge image from the acquired activated sludge images; S22, the selected activated sludge image is labeled to obtain a label image of the same size as the original image. The label image includes the background, flocs and filamentous bacteria.

4. The method for constructing a microscopic image segmentation model of activated sludge according to claim 1, characterized in that, Step S4, which involves training the activated sludge microscopic image segmentation model using a hybrid loss function, includes the following steps: S41, In order to obtain a small number of difficult-to-distinguish samples, define the model's mixture loss function: ； in, Let the initial cross-entropy loss function be... The easy / difficult sample adjustment factor, while γ is the focusing parameter. It is the pixel error vector. It is the class to which it belongs. It is the total number of categories. It is the Jaccard loss function. It is a surrogate loss function of Jaccard loss. , These are the weights corresponding to focal loss and Lovász-Softmax loss, respectively. S42, use the defined hybrid loss function to train the activated sludge segmentation model and obtain the model parameters.

5. The method for constructing a microscopic image segmentation model of activated sludge according to claim 1, characterized in that, The segmentation model deployment and inference step S5 includes the following steps: S51, Deploy the segmentation model; S52 uses real-time acquired activated sludge images to infer the segmentation model and obtain the final segmentation result.

Images