In-situ membrane pollution type recognition method and system based on convolutional neural network, computer readable storage medium and membrane module

Abstract

The application provides an in-situ membrane pollution type identification method and system based on a convolutional neural network, a computer readable storage medium and a membrane module, and belongs to the technical field of membrane pollution control. The method uses in-situ membrane images of a membrane surface collected in real time by a low-cost in-situ digital microscope as input, and outputs the probability of the corresponding membrane pollution type based on a pre-trained convolutional neural network, thereby realizing rapid discrimination of inorganic pollution, organic pollution and biological pollution, providing theoretical support for real-time formulation and optimization of membrane distillation chemical cleaning strategies, and being suitable for real-time, non-invasive intelligent identification of membrane pollution types in a membrane process, and providing a decision basis for formulating a targeted chemical cleaning strategy.

Description

Technical Field

[0001] This application belongs to the field of membrane fouling control technology, and specifically relates to an in-situ membrane fouling type identification method, system, computer-readable storage medium, and membrane module based on convolutional neural networks. Background Technology

[0002] Membrane fouling is a key limiting factor in the application of membrane technology. When the flux recovery efficiency of physical cleaning decreases after long-term operation, chemical cleaning of fouled membranes becomes necessary. However, different cleaning agents are selected for inorganic, biological, and organic fouling. Targeted selection of cleaning agents based on the type of membrane fouling can improve cleaning effectiveness and prevent unnecessary damage to the membrane from chemical cleaning. Therefore, obtaining information on the dominant pollutant and quickly determining the type of fouling is of great significance.

[0003] Currently, methods for obtaining information on primary contaminants on membrane surfaces can be categorized into invasive and non-invasive types. Invasive observation methods mainly include SEM, EDS, and AFM, which can obtain membrane contamination information such as the microscopic morphology and elemental composition of the contaminant layer. However, these methods require shutdown and membrane removal, representing typical post-contamination analysis, making it difficult to obtain contamination information in real time. In contrast, in-situ microscopy / camera observation, as a non-invasive characterization method, is currently widely used for obtaining real-time membrane contamination information. However, it is difficult to determine the type of contaminants from a large amount of real-time membrane surface contamination information based on human experience, necessitating a method that can quickly respond and determine the contaminant type. Summary of the Invention

[0004] Based on the aforementioned technical needs, this application provides an in-situ membrane fouling type identification method based on convolutional neural networks to solve the technical problem that non-invasive characterization methods in the prior art are difficult to accurately determine the type of membrane fouling.

[0005] This application also provides an in-situ membrane contamination type identification system based on convolutional neural networks.

[0006] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the aforementioned in-situ membrane contamination type identification method based on a convolutional neural network.

[0007] This application also provides a membrane module including the above-mentioned in-situ membrane fouling type identification system based on convolutional neural networks, in order to solve the technical problem in the prior art that membrane chemical cleaning strategies are affected by membrane fouling type and are difficult to define accurately.

[0008] To achieve the above objectives, the technical solution of this application is as follows: A method for identifying in-situ membrane contamination types based on convolutional neural networks includes the following steps: S10 Acquire in-situ membrane image; S20 performs standardization processing on the in-situ membrane image to obtain a standard image block of membrane fouling; S30 Based on a pre-trained convolutional neural network model, membrane pollution features are extracted from the standard image patch of membrane pollution to obtain membrane pollution category response values; S40 Determine the membrane fouling category based on the membrane fouling category response value; the membrane fouling category includes at least one of inorganic fouling, organic fouling, and biological fouling.

[0009] Preferably, step S20, "standardizing the in-situ membrane image to obtain a membrane fouling standard image block," includes: The in-situ membrane image is randomly cropped into a group of image blocks with pixel values ​​smaller than the preprocessed in-situ membrane image to obtain a first-level image block; The pixel values ​​of the first-level image block are normalized to the range of [0,1] to obtain a normalized image block; The normalized image block is subjected to illumination correction and contrast stretching to obtain the membrane fouling standard image block.

[0010] Preferably, in step S30, "extracting membrane pollution features from the standard image patch based on a pre-trained convolutional neural network model to obtain membrane pollution category response values" includes: Based on a pre-trained convolutional neural network model, membrane pollution features are extracted from the standard image patches of membrane pollution to obtain image patch-level category response values; The image block-level category response values ​​are integrated to obtain image-level membrane fouling category response values.

[0011] Preferably, an average pooling strategy is used to integrate the image patch-level category response values. The average pooling strategy is expressed as follows:

[0012] Among them, R c S represents the image-level membrane fouling category response value of the entire image to pollutant of category c. c,j represents the image patch level category response value of the j-th image patch to the c-th pollutant, and N represents the total number of image patches obtained from the division of the entire image.

[0013] Preferably, the pre-trained convolutional neural network model includes a feature extraction part and a classification output part; the feature extraction part consists of multiple convolutional layers, activation functions, and pooling layers; the multiple convolutional layers slide the convolutional kernels on the input image patch to extract edge, color, texture, spots, crystals, and local morphological features in the membrane pollution image; the pooling layer is used to reduce the feature map size and computational complexity while retaining key discriminative features; the classification output part inputs the high-level membrane pollution features into the fully connected layer and converts them into membrane pollution category response values.

[0014] Preferably, the pre-trained convolutional neural network model is constructed based on a ResNet18 network pre-trained on the ImageNet dataset; the ResNet18 network includes multi-layer convolutional structures and residual connection structures.

[0015] As a preferred approach, the ResNet18 convolutional neural network model is trained using a full-network fine-tuning strategy.

[0016] An in-situ membrane contamination type identification system based on convolutional neural networks, comprising: The image acquisition module is used to acquire in-situ membrane images; A standardization processing module is used to standardize the in-situ membrane image to obtain a standard image block of membrane fouling. The feature extraction module is used to extract membrane pollution features from the standard image patch based on a pre-trained convolutional neural network model to obtain membrane pollution category response values. The membrane fouling category determination module is used to determine the membrane fouling category based on the membrane fouling category response value; the membrane fouling category includes at least one of inorganic fouling, organic fouling and biological fouling.

[0017] A computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the in-situ membrane contamination type identification method based on a convolutional neural network as described above.

[0018] A membrane assembly including the in-situ membrane fouling type identification system based on a convolutional neural network as described above.

[0019] Compared with the prior art, this application has at least the following beneficial effects: This invention provides an in-situ membrane fouling type identification method based on convolutional neural networks, enabling rapid differentiation of inorganic, organic, and biological fouling. The method uses in-situ membrane images acquired in real-time by a low-cost in-situ digital microscope as input, and outputs the probability of the corresponding membrane fouling type based on a pre-trained convolutional neural network. The proposed identification method provides theoretical support for the real-time formulation and optimization of membrane distillation chemical cleaning strategies. It is applicable to real-time, non-invasive intelligent identification of membrane fouling types in membrane processes, providing a decision-making basis for developing targeted chemical cleaning strategies. By effectively identifying membrane fouling types, the method allows for targeted chemical cleaning of fouled membranes, improving cleaning efficiency and mitigating the impact of ineffective chemical cleaning on membrane material lifespan, providing an effective technical solution for intelligent cleaning of membranes in membrane distillation and other processes. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a convolutional neural network model architecture in one embodiment of the present invention.

[0021] Figure 2 This is a comparison of in-situ microscope images and SEM images of three types of membrane fouling in one embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of image preprocessing and image block cropping in one embodiment of the present invention.

[0023] Figure 4 This is a comparison chart of accuracy and loss curves under different fine-tuning strategies in one embodiment of the present invention.

[0024] Figure 5 This is a comparison chart of classification accuracy under different image patch integration strategies in one embodiment of the present invention. Detailed Implementation

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The technical solutions of this application will be further described below with reference to the accompanying drawings of the embodiments, and this application is not limited to the following specific implementation methods.

[0026] It should be understood that the same or similar reference numerals in the accompanying drawings of the embodiments correspond to the same or similar components. In the description of this application, it should be understood that if terms such as "upper," "lower," "inner," "outer," "left," "right," "front," "rear," "top," and "bottom" indicate directions or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the structure or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms describing positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limitations on this patent. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0027] The following is combined Figure 1 The present application will be further described in detail with reference to specific embodiments.

[0028] In one specific embodiment of the present invention, please refer to Figures 1 to 5 A method for identifying in-situ membrane contamination types based on convolutional neural networks includes the following steps: S10 Acquire in-situ membrane image; S20 performs standardization processing on the in-situ membrane image to obtain a standard image block of membrane fouling; S30 Based on a pre-trained convolutional neural network model, membrane pollution features are extracted from the standard image patch of membrane pollution to obtain membrane pollution category response values; S40 Determine the membrane fouling category based on the membrane fouling category response value; the membrane fouling category includes at least one of inorganic fouling, organic fouling, and biological fouling.

[0029] The in-situ membrane fouling type identification method based on convolutional neural networks provided in this invention uses in-situ membrane images acquired in real time by a low-cost in-situ digital microscope as input. Based on a pre-trained convolutional neural network, it outputs the probability of the corresponding membrane fouling type, achieving rapid identification of inorganic, organic, and biological fouling. This in-situ membrane fouling type identification method based on convolutional neural networks provides theoretical support for the real-time formulation and optimization of membrane distillation chemical cleaning strategies. It is applicable to real-time, non-invasive intelligent identification of membrane fouling types in membrane processes, providing a decision-making basis for formulating targeted chemical cleaning strategies. By effectively identifying membrane fouling types, the method can perform targeted chemical cleaning of fouled membranes, improving cleaning efficiency and mitigating the impact of ineffective chemical cleaning on membrane material lifespan, providing an effective technical solution for intelligent cleaning of membranes in membrane distillation and other processes.

[0030] In one feasible implementation, in-situ digital microscopy is used to acquire in-situ membrane images in a direct-contact membrane distillation apparatus, which may be contaminated with inorganic, organic, and biological contamination. For example, a transparent optical window is provided on the membrane module, and in-situ membrane images are acquired in real time using a digital microscope.

[0031] Preferably, step S20, "standardizing the in-situ membrane image to obtain a membrane fouling standard image block," includes: The in-situ membrane image is randomly cropped into a group of image blocks with pixel values ​​smaller than the preprocessed in-situ membrane image to obtain a first-level image block; The pixel values ​​of the first-level image block are normalized to the range of [0,1] to obtain a normalized image block; The normalized image block is subjected to illumination correction and contrast stretching to obtain the membrane fouling standard image block.

[0032] Specifically, the original membrane fouling image undergoes standardization processing, which includes random cropping, normalization, flat-field correction, and contrast stretching. Through this standardization process, the original digital microscopy image is converted into standardized image patches suitable for input to a convolutional neural network, while simultaneously enhancing the local morphology and texture differences of membrane fouling and improving the model's ability to extract features related to membrane fouling types.

[0033] For example, the in-situ membrane image is randomly cropped into a set of 224×224 pixel first image patches, with no overlap between the patches. This patch cropping decomposes the entire in-situ membrane image into multiple local regions, allowing the model to learn the local contamination morphology, texture distribution, and grayscale variation features at different locations on the membrane surface, avoiding the weakening of local contamination features when using the entire image as input. These first image patches serve as the basic input units for the convolutional neural network, enabling the model to perform feature extraction and category determination at a uniform size.

[0034] Furthermore, the pixel values ​​of the cropped first image patch are normalized to the range [0,1]. Normalization can unify the pixel value scale of different first image patches, reduce the impact of differences in image brightness and numerical range on model training, and improve the stability of the model training process.

[0035] Furthermore, a flat-field correction method is employed to correct for non-uniform background illumination in the image. Since digital microscopy imaging may encounter issues such as inconsistent brightness between the central and edge regions, localized shadows, and uneven lens illumination, flat-field correction can reduce the interference of non-uniform background illumination on the extraction of membrane fouling morphology features, allowing the model to focus more on the morphology and texture features of the fouling itself.

[0036] Furthermore, contrast stretching is performed based on the image's grayscale distribution to enhance detail information and grayscale differences between different regions. Contrast stretching can amplify the grayscale difference between the membrane contamination area and the background area, making local features such as inorganic crystals, organic contamination spots, and biocontamination layers more prominent, which is beneficial for convolutional neural networks to extract clearer contamination type discrimination information.

[0037] Preferably, in step S30, "extracting membrane pollution features from the standard image patch based on a pre-trained convolutional neural network model to obtain membrane pollution category response values" includes: Based on a pre-trained convolutional neural network model, membrane pollution features are extracted from the standard image patches of membrane pollution to obtain image patch-level category response values; The image block-level category response values ​​are integrated to obtain image-level membrane fouling category response values.

[0038] Specifically, in some recommended implementations, a convolutional neural network model based on transfer learning is used to extract and classify membrane contamination features from standardized image patches. For example, the convolutional neural network model takes a 224×224 pixel standard image patch as input and outputs three types of contamination: inorganic contamination, organic contamination, and biological contamination.

[0039] Recommendedly, the pre-trained convolutional neural network model includes a feature extraction part and a classification output part. The feature extraction part consists of multiple convolutional layers, activation functions, and pooling layers. The convolutional layers slide convolutional kernels across the input image patches to extract edge, color, texture, spots, crystals, and local morphological features from the membrane pollution image; different convolutional kernels can learn different types of image features and form multi-channel feature maps. Pooling layers are used to reduce the feature map size and computational complexity while retaining key discriminative features, improving the model's adaptability to local positional changes. The classification output part inputs the high-level membrane pollution features into a fully connected layer and converts them into category response values ​​for three types of pollutants: inorganic pollution, organic pollution, and biological pollution.

[0040] Preferably, the pre-trained convolutional neural network model is built based on a ResNet18 network pre-trained on the ImageNet dataset. Since the number of membrane pollution image samples is limited, directly training a deep convolutional neural network from scratch can easily lead to insufficient training or inadequate generalization ability. Employing transfer learning methods can utilize the general visual features already learned by the pre-trained network, including edge, texture, color, and local morphological features, thereby improving the training efficiency and recognition stability of the membrane pollution classification model.

[0041] Furthermore, the ResNet18 network includes multi-layer convolutional structures and residual connection structures. Residual connections can alleviate gradient decay and feature degradation problems during deep network training, enabling the network to maintain good training stability when extracting deep morphological features of membrane pollution. By transferring the ResNet18 pre-trained model to the membrane pollution image classification task, the model's ability to recognize differentiated features such as inorganic crystals, organic spots, and biological pollution layers can be enhanced.

[0042] Preferably, a full-network fine-tuning strategy is used to train the ResNet18 convolutional neural network model, that is, to unfreeze all layer parameters of the backbone network and jointly train the feature extraction backbone network and the classification layer on a membrane pollution image dataset. Compared with the linear detection strategy of training only the classification layer, full-network fine-tuning enables the pre-trained backbone network to further adapt to the task-specific features of membrane pollution images, thereby improving the model's ability to classify membrane pollution types.

[0043] Since the same in-situ membrane image is cropped into multiple standard image patches, the convolutional neural network first outputs the response value or probability value of each standard image patch for inorganic, organic, and biological contamination. To obtain the final contamination type of the entire in-situ membrane image, it is necessary to integrate the classification responses of multiple standard image patches in the same image.

[0044] As a recommended implementation, an average pooling strategy is used to integrate the response values ​​of each standard image patch to obtain the image-level membrane contamination category response value of the entire image for each contamination type. The average pooling strategy can integrate the information of all image patches in the same image, reducing the adverse effects on the overall image classification result caused by the indistinct contamination features, uneven illumination, edge region interference, or local misjudgment of a single local image patch.

[0045] Specifically, the average pooling strategy is expressed as follows:

[0046] Among them, R c S represents the image-level membrane fouling category response value of the entire image to pollutant of category c. c,j represents the image patch level category response value of the j-th image patch to the c-th pollutant, and N represents the total number of image patches obtained from the division of the entire image.

[0047] Furthermore, the image-level membrane fouling category response values ​​are converted into probability distributions for each membrane fouling type using a Softmax activation function, outputting the probability values ​​of the entire image belonging to inorganic, organic, and biological fouling. For any input image, if its inorganic fouling probability value is the highest, the image is likely to be classified as inorganic fouling; if its organic fouling probability value is the highest, the image is likely to be classified as organic fouling; and if its biological fouling probability value is the highest, the image is likely to be classified as biological fouling.

[0048] Furthermore, by using ResNet18 as the backbone network, training the model with a full-network fine-tuning strategy, and integrating the response values ​​of multiple image patches using an average pooling strategy, stable identification of three types of membrane fouling—inorganic, organic, and biological fouling—is achieved, providing a basis for membrane fouling type judgment and subsequent cleaning strategy selection.

[0049] An in-situ membrane contamination type identification system based on convolutional neural networks, comprising: The image acquisition module is used to acquire in-situ membrane images; A standardization processing module is used to standardize the in-situ membrane image to obtain a standard image block of membrane fouling. The feature extraction module is used to extract membrane pollution features from the standard image patch based on a pre-trained convolutional neural network model to obtain membrane pollution category response values. The membrane fouling category determination module is used to determine the membrane fouling category based on the membrane fouling category response value; the membrane fouling category includes at least one of inorganic fouling, organic fouling and biological fouling.

[0050] The working process and working principle of each unit and module in the above-mentioned in-situ membrane contamination type identification system based on convolutional neural networks are similar to those of the in-situ membrane contamination type identification method based on convolutional neural networks described above, and will not be repeated here.

[0051] A computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the in-situ membrane contamination type identification method based on a convolutional neural network as described above.

[0052] A membrane assembly including the in-situ membrane fouling type identification system based on a convolutional neural network as described above.

[0053] The following combination Figures 2-5 This paper elaborates on the training process, working process, and technical advantages of the pre-trained convolutional neural network model mentioned in the embodiments of the present invention. Constructing a membrane fouling image dataset

[0054] A transparent optical window is set on the membrane module of the membrane distillation device, so that the digital microscope can focus on the membrane surface through the transparent optical window, thereby acquiring images of the fouling morphology of the membrane surface in real time during the membrane distillation process.

[0055] The membrane distillation apparatus is preferably a direct-contact membrane distillation apparatus. The membrane module is made of an optically transparent material, and a digital microscope is positioned outside the transparent optical window of the membrane module to acquire in-situ images of contamination on the membrane surface.

[0056] To construct image datasets of different types of membrane fouling, feed solutions were configured to induce inorganic fouling, organic fouling, and biofouling, respectively. The inorganic fouling feed solution was used to form salt crystal deposition on the membrane surface; the organic fouling feed solution was used to form an organic fouling layer on the membrane surface; and the biofouling feed solution was used to form a biofouling layer on the membrane surface composed of microorganisms and their extracellular polymers.

[0057] Specifically, the inorganic contaminant feed solution is prepared using deionized water, in which NaHCO3, Na2SO4, and CaCl2 are dissolved at predetermined concentrations. Preferably, the NaHCO3 concentration is 0.25 g / L, the Na2SO4 concentration is 2.13 g / L, and the CaCl2 concentration is 1.83 g / L, thereby forming a mixed salt solution containing calcium salts, sulfates, and carbonates. This feed solution can promote the formation of inorganic deposits such as CaSO4 and CaCO3 on the membrane surface during membrane distillation concentration, thereby inducing the formation of inorganic contaminants.

[0058] The organic pollutant feed solution is prepared using humic acid. Preferably, 25 mg of humic acid is weighed and dissolved in 100 mL of a 0.01 g / L NaOH solution to ensure thorough dispersion. After the humic acid is uniformly dissolved, it is diluted with deionized water to prepare a 50 mg / L humic acid solution. This feed solution can adsorb, enrich, and form an organic pollutant layer on the membrane surface during membrane distillation, thereby inducing the formation of organic pollution.

[0059] The biofouling feed solution is prepared using a microbial floc suspension. Preferably, samples are taken from an aerobic granular sludge reactor, retaining the microbial flocs or granular sludge flocs, and diluted with tap water to prepare a microbial floc suspension with a turbidity of 5 NTU. To prevent the microbial flocs from settling at the bottom of the feed tank, a stirring device is installed during the feed solution operation to continuously stir the solution, maintaining a relatively uniform suspension of the microbial flocs. This feed solution can induce the adhesion and accumulation of microbial flocs and their extracellular polymers on the membrane surface during membrane distillation, thereby forming biofouling.

[0060] In the three types of membrane fouling experiments described above, the initial volume of each feed solution was 500 mL. In-situ digital micrographs were obtained under three different membrane fouling conditions by running experiments with inorganic fouling, organic fouling, and biological fouling.

[0061] like Figure 2 As shown, different types of membrane fouling exhibit distinctly different morphological and textural features in in-situ digital microscopy images. Inorganic fouling typically presents as obvious crystalline, needle-like, or platy deposits; organic fouling usually shows darker colors, spot-like patterns, or localized continuous coverage; and biofouling typically shows localized aggregation, biofilm coverage, or relatively smooth fouling layers. SEM images further validate that digital microscopy images can reflect the main morphological differences among different fouling types. Therefore, in-situ digital microscopy images can serve as input data for membrane fouling type identification models.

[0062] During membrane distillation, in-situ morphological photographs of the membrane surface under inorganic, organic, and biological contamination were acquired using a digital microscope. After acquisition, the images were manually screened and labeled as inorganic, organic, and biological contamination, respectively.

[0063] In this embodiment, the collected and screened membrane fouling images include 270 images of inorganic fouling, 271 images of organic fouling, and 273 images of biological fouling. These images are randomly divided into a training set, a validation set, and a test set in a 70:15:15 ratio. The training set is used for learning the parameters of the convolutional neural network model, the validation set is used for model selection and parameter adjustment during training, and the test set is used to evaluate the final model's ability to identify fouling types. Membrane fouling image preprocessing

[0064] To convert the raw membrane contamination images into standardized data suitable for input to a convolutional neural network model, preprocessing is performed on the membrane contamination images. This preprocessing includes random cropping, normalization, flattening correction, and contrast stretching.

[0065] like Figure 3 As shown, each original membrane contamination image is randomly cropped into six 224×224 pixel image blocks. By dividing the entire image into multiple local image blocks, the convolutional neural network can more fully learn the local contamination morphology, texture distribution, and grayscale variation characteristics of different regions on the membrane surface.

[0066] Secondly, the cropped image blocks are normalized to scale the pixel values ​​to the range of [0,1] to reduce the impact of pixel value scale differences between different images on model training and improve the stability of model training.

[0067] Next, flat-field correction is performed on the normalized image patches. Since there may be problems such as uneven illumination, inconsistent background brightness, local shadows, or brightness attenuation at the lens edge during digital microscopy, flat-field correction is used to correct non-uniform background illumination in the image and reduce the interference of lighting conditions on the extraction of membrane fouling features.

[0068] Finally, the image blocks are contrast-stretched based on the image grayscale distribution to enhance the grayscale difference between the membrane fouling area and the background area, highlighting detailed features such as inorganic crystals, organic fouling spots, and biofouling layers.

[0069] After the above preprocessing, standardized membrane fouling image patch data is obtained and used as input to the convolutional neural network model. A convolutional neural network model based on transfer learning is used to extract and classify image patch contamination features.

[0070] A convolutional neural network model for membrane contamination classification based on transfer learning was constructed. The model takes a 224×224 pixel membrane contamination image patch as input and outputs three contamination types: inorganic contamination, organic contamination, and biological contamination.

[0071] Since the number of membrane pollution image samples is relatively limited, this embodiment uses transfer learning to build the model. Specifically, a convolutional neural network pre-trained on the ImageNet dataset is used as the initial model. The general visual features learned by the pre-trained model are utilized to improve the training efficiency and classification stability of the membrane pollution image classification model. Backbone Network Comparison and Determination

[0072] This embodiment selects MobileNetV2 and DenseNet201 pre-trained skeletons and compares their pollution feature extraction capabilities with ResNet18. All three skeleton models are trained and tested under the same dataset partitioning and the same linear probing training strategy. The test results are shown in Table 1: Table 1 Test results of three skeleton models

[0073] The results in the table show that while MobileNetV2 can identify more organic pollution samples, its precision for organic pollution is low, and its recall for biological pollution is also low, making it prone to misclassifying some biological pollution as organic pollution. Its ability to distinguish between organic and biological pollution is insufficient. DenseNet201 has a slightly higher overall accuracy, but its precision for organic pollution and recall for biological pollution are both low, indicating a significant class bias between organic and biological pollution. ResNet18 shows relatively similar recall, precision, and F1 scores for the three types of pollutants, with more balanced classification results across different categories, and is able to establish a more stable class distinction boundary.

[0074] Considering the overall accuracy, the balance of identification of the three types of pollutants, and the model complexity, ResNet18 is the optimal backbone network for membrane fouling type identification. Comparison and Determination of Fine-Tuning Strategies

[0075] After determining ResNet18 as the backbone network, the classification performance of the model using the full-network fine-tuning strategy was further compared with other fine-tuning strategies. These other fine-tuning strategies included linear probing and full-network fine-tuning after linear probing.

[0076] The linear probing strategy involves freezing the parameters of the ResNet18 backbone network and training only the final fully connected classification layer. This strategy trains fewer parameters, but the pre-trained backbone network cannot adequately adapt to the task-specific features in membrane-contaminated images.

[0077] The full-network fine-tuning strategy involves unfreezing all parameters of the ResNet18 backbone network and training both the backbone network and the classification layer on a membrane pollution image dataset. This strategy enables the model to learn membrane pollution features such as inorganic crystals, organic spots, and biofilm coverings, building upon pre-trained general visual features.

[0078] The linear probing followed by full-network fine-tuning strategy involves first freezing the backbone network to train the classification layer, and then unfreezing the entire network for joint training. This strategy can establish an initial classification boundary before further adjusting the parameters of the entire network, but the training process is significantly affected by the allocation of the two-stage training cycles.

[0079] When using the linear probing strategy, freezing only the backbone network and training the classification head resulted in a test set accuracy of 82.64%, indicating that training only the classification layer is insufficient for the pre-trained model to fully adapt to the membrane contamination image classification task. When using the full-network fine-tuning strategy, unfreezing both the backbone network and the classification head and performing overall training improved the test set accuracy to 93.11%, a significant improvement compared to the linear probing strategy. This demonstrates that full-network fine-tuning can further learn the task-specific features in membrane contamination images.

[0080] Furthermore, when using the LP-FT(5-20) strategy, which involves 5 training epochs of linear probing followed by 20 training epochs of full-network fine-tuning, the model's test set accuracy is 93.25%, slightly higher than the full-network fine-tuning strategy. When using the LP-FT(10-15) strategy, which involves 10 training epochs of linear probing followed by 15 training epochs of full-network fine-tuning, the model's test set accuracy is 92.29%. Although the accuracy of LP-FT(5-20) is slightly higher, combined with... Figure 4As shown in the accuracy and loss curves under different fine-tuning strategies, the convergence process of the LP-FT two-stage training strategy lags behind that of the full-network fine-tuning strategy, and may produce more obvious class bias between different contamination types. Although the accuracy of the full-network fine-tuning strategy is slightly lower than that of LP-FT (5-20), the training and validation losses decrease more significantly, the model convergence process is more stable, the classification results of different contamination types are more balanced, and there are fewer misclassifications.

[0081] Therefore, considering the overall accuracy of the test set, the convergence of the loss curve, and the classification balance of the three types of pollutants, this embodiment determines that full network fine-tuning is the best training method.

[0082] The membrane pollution image patch classification model described above, which uses ResNet18 as the backbone network and employs a full-network fine-tuning strategy, can output the response values ​​of the three types of pollutants—inorganic, organic, and biological—for any input image patch. Image patch response value integration and membrane fouling type classification results output

[0083] Since a single original image of membrane contamination is cropped into multiple image patches, the convolutional neural network initially obtains contamination type response values ​​at the patch level. To obtain the final classification result for the entire membrane contamination image, it is necessary to integrate the response values ​​from multiple image patches corresponding to the same original image.

[0084] This embodiment compares two image patch response value integration strategies: average pooling, max pooling, and logarithmic summation exponential pooling.

[0085] Max pooling selects the highest category response value among multiple image patches as the classification criterion for the entire image. This method can highlight areas with strong local responses, but when the highest-response image patch is misclassified, the entire image is prone to misclassification. Using max pooling, the test set accuracy was 95.87%, with two images showing organic pollution and three showing biological pollution being misclassified, indicating its sensitivity to locally misclassified image patches.

[0086] Average pooling involves averaging the category response values ​​of all image patches within the same image to obtain the overall response value of the entire image for each type of membrane contamination. When using average pooling, the test set accuracy reached 99.13%, with only one misclassified biological contamination image. This result demonstrates that average pooling can integrate the response values ​​of all image patches within the same image, reducing the impact of a single misclassified image patch on the final classification result. Therefore, it is more suitable for membrane contamination images where the contamination area is unevenly distributed and local features vary significantly.

[0087] The logarithmic summation exponential pooling strategy falls between max pooling and average pooling. Using this strategy, the test set accuracy was 98.35%, with one misclassified organic pollution image and one misclassified biological pollution image. This strategy represents an improvement over max pooling, but it can still be affected by local anomalies.

[0088] like Figure 5 As shown, the classification accuracy of the max pooling strategy is lower than that of average pooling and logarithmic summation exponential pooling; the average pooling strategy achieves the best results among the three integration strategies. Therefore, the average pooling strategy is the best method for integrating image patch response values.

[0089] The category response values ​​are converted into probability distributions for each membrane fouling type using the Softmax activation function. For any input image, the model outputs the probability value of whether it belongs to inorganic, organic, or biological fouling, and uses the category with the highest probability value as the preliminary classification result for that image patch.

[0090] In this embodiment, ResNet18 is used as the backbone network, the model is trained using a full-network fine-tuning strategy, and the response values ​​of six image patches are integrated using an average pooling strategy. The final test set accuracy reaches 99.13%. This result shows that the method of the present invention can achieve stable identification of three types of membrane pollution: inorganic pollution, organic pollution, and biological pollution.

[0091] Finally, the system outputs a classification result for the type of contamination corresponding to the entire membrane contamination image, namely inorganic contamination, organic contamination, or biological contamination. This classification result can be further used for selecting cleaning strategies during membrane distillation operations.

[0092] Obviously, the above embodiments of this application are merely examples for clearly illustrating this application, and are not intended to limit the implementation of this application. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for identifying in-situ membrane contamination types based on convolutional neural networks, characterized in that, Includes the following steps: S10 Acquire in-situ membrane image; S20 performs standardization processing on the in-situ membrane image to obtain a standard image block of membrane fouling; S30 Based on a pre-trained convolutional neural network model, membrane pollution features are extracted from the standard image patch of membrane pollution to obtain membrane pollution category response values; S40 Determine the membrane fouling category based on the membrane fouling category response value; the membrane fouling category includes at least one of inorganic fouling, organic fouling, and biological fouling.

2. The in-situ membrane contamination type identification method based on convolutional neural networks as described in claim 1, characterized in that, In step S20, "standardizing the in-situ membrane image to obtain a standard image block of membrane fouling" includes: The in-situ membrane image is randomly cropped into a group of image blocks with pixel values ​​smaller than the preprocessed in-situ membrane image to obtain a first-level image block; The pixel values ​​of the first-level image block are normalized to the range of [0,1] to obtain a normalized image block; The normalized image block is subjected to illumination correction and contrast stretching to obtain the membrane fouling standard image block.

3. The in-situ membrane contamination type identification method based on convolutional neural networks as described in claim 1, characterized in that, In step S30, "extracting membrane pollution features from the standard image patch based on a pre-trained convolutional neural network model to obtain membrane pollution category response values" includes: Based on a pre-trained convolutional neural network model, membrane pollution features are extracted from the standard image patches of membrane pollution to obtain image patch-level category response values; The image block-level category response values ​​are integrated to obtain image-level membrane fouling category response values.

4. The in-situ membrane fouling type identification method based on convolutional neural networks as described in claim 3, characterized in that, The image patch-level category response values ​​are integrated using an average pooling strategy, which is expressed as follows: Among them, R c S represents the image-level membrane fouling category response value of the entire image to pollutant of category c. c,j represents the image patch level category response value of the j-th image patch to the c-th pollutant, and N represents the total number of image patches obtained from the division of the entire image.

5. The in-situ membrane contamination type identification method based on convolutional neural networks as described in claim 1 or 3, characterized in that, The pre-trained convolutional neural network model includes a feature extraction part and a classification output part; The feature extraction part consists of multiple convolutional layers, activation functions, and pooling layers; the multiple convolutional layers slide on the input image patch through convolutional kernels to extract edge, color, texture, spots, crystals, and local morphological features in the membrane contamination image; Pooling layers are used to reduce feature map size and computational complexity while preserving key discriminative features; The classification output section inputs the high-level membrane fouling characteristics into the fully connected layer and converts them into membrane fouling category response values.

6. The in-situ membrane contamination type identification method based on convolutional neural networks as described in claim 5, characterized in that, The pre-trained convolutional neural network model is built on a ResNet18 network pre-trained on the ImageNet dataset; the ResNet18 network includes multi-layer convolutional structures and residual connection structures.

7. The in-situ membrane contamination type identification method based on convolutional neural networks as described in claim 6, characterized in that, A ResNet18 convolutional neural network model was trained using a full-network fine-tuning strategy.

8. An in-situ membrane contamination type identification system based on convolutional neural networks, characterized in that, include: The image acquisition module is used to acquire in-situ membrane images; A standardization processing module is used to standardize the in-situ membrane image to obtain a standard image block of membrane fouling. The feature extraction module is used to extract membrane pollution features from the standard image patch based on a pre-trained convolutional neural network model to obtain membrane pollution category response values. The membrane fouling category determination module is used to determine the membrane fouling category based on the membrane fouling category response value; the membrane fouling category includes at least one of inorganic fouling, organic fouling and biological fouling.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it can implement the in-situ membrane contamination type identification method based on any one of claims 1-7.

10. A membrane module, characterized in that, Including the in-situ membrane contamination type identification system based on convolutional neural networks as described in claim 8.

Images