Microorganism image recognition method, device and equipment based on deep learning, medium and product

Abstract

The invention discloses a microorganism image recognition method, device and equipment based on deep learning, a medium and a product, and relates to the field of microorganism detection. The method comprises the following steps: acquiring images of various microorganisms in the activated sludge, and labeling the types of the microorganisms in the images; carrying out data enhancement on the images, and forming an activated sludge microorganism image data set together with the marked types of microorganisms in the images; training a Faster RCNN network by using the activated sludge microorganism image data set to obtain a microorganism recognition model; and inputting the image of the to-be-detected microorganism into the microorganism identification model, and identifying the category of the to-be-detected microorganism. According to the method, the microorganisms in the activated sludge can be efficiently and accurately detected and classified.

Description

Technical Field

[0001] The present application relates to the field of microbial detection, and in particular to a method, device, equipment, medium and product for microbial image recognition based on deep learning. Background Art

[0002] Currently, the activated sludge process is the core technology for sewage treatment, and its efficiency is directly related to the effluent quality. Studies have shown that the richness and activity of microbial communities in activated sludge are key indicators for evaluating the effectiveness of sewage treatment. Therefore, accurately understanding the types and quantities of these microorganisms is crucial for optimizing the treatment process.

[0003] The traditional method of microbial detection is for inspectors to manually identify microorganisms through microscopic examination. This process is not only inefficient but also highly dependent on the professional skills of inspectors. However, with the leap forward in artificial intelligence technology, especially the progress in the field of deep learning, automatic detection of microbial images has become possible, showing unprecedented potential.

[0004] However, there are still some problems with the identification of microorganisms in activated sludge: first, the actual operating environment of sewage treatment plants is usually complex, and the image background is often mixed with various interference factors such as algae and suspended matter, which makes the quality of images collected on-site difficult to match laboratory standards, thus affecting the recognition accuracy; second, the current public microbial image data sets are scarce in resources, and the debris in the background is unevenly distributed, which limits the amount of effective data for model training and raises questions about the accuracy of the model. Summary of the invention

[0005] The purpose of this application is to provide a deep learning-based microbial image recognition method, device, equipment, medium and product, which can efficiently and accurately detect and classify microorganisms in activated sludge.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In the first aspect, the present application provides a microbial image recognition method based on deep learning, comprising: collecting images of multiple microorganisms in activated sludge, and marking the categories of the microorganisms in the images; performing data enhancement on the images, and forming an activated sludge microbial image dataset together with the marked categories of the microorganisms in the images; using the activated sludge microbial image dataset to train a Faster RCNN network to obtain a microbial recognition model; inputting the image of the microorganism to be detected into the microbial recognition model to identify the category of the microorganism to be detected.

[0008] In the second aspect, the present application provides a microbial image recognition device based on deep learning, including: an acquisition module, a data enhancement module, a training module and an identification module. The acquisition module is used to acquire images of various microorganisms in activated sludge and annotate the categories of the microorganisms in the images; the data enhancement module is used to perform data enhancement on the images and form an activated sludge microbial image dataset together with the annotated categories of the microorganisms in the images; the training module is used to train the Faster RCNN network using the activated sludge microbial image dataset to obtain a microbial recognition model; the identification module is used to input the image of the microorganism to be detected into the microbial recognition model to identify the category of the microorganism to be detected.

[0009] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the deep learning-based microbial image recognition methods described above.

[0010] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the deep learning-based microbial image recognition methods described above.

[0011] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the deep learning-based microbial image recognition methods described above.

[0012] According to the specific embodiments provided in this application, this application has the following technical effects:

[0013] The present application provides a method, device, equipment, medium and product for microbial image recognition based on deep learning, which performs data enhancement on the collected images, expands the effective data volume of Faster RCNN network training, and improves the training accuracy of Faster RCNN network; the Faster RCNN network model is used to identify microorganisms in activated sludge, which can more accurately detect the morphological characteristics of microorganisms, and is not affected by various interference factors such as algae and suspended matter mixed in the image background, thereby realizing efficient and accurate detection and classification of microorganisms in activated sludge. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 A schematic diagram of a process of a microbial image recognition method based on deep learning provided in one embodiment of the present application;

[0016] Figure 2 A schematic diagram of the Faster RCNN network structure provided in another embodiment of the present application;

[0017] Figure 3 A schematic diagram of the results of identifying microorganisms by a microorganism identification model provided in another embodiment of the present application;

[0018] Figure 4 A schematic diagram of the working principle of a microbial image recognition method based on deep learning provided in one embodiment of the present application;

[0019] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0022] In an exemplary embodiment, Figure 1 As shown, a method for microbial image recognition based on deep learning is provided, comprising the following steps 101 to 104. Among them:

[0023] Step 101: Collect images of various microorganisms in activated sludge, and label the categories of the microorganisms in the images.

[0024] Step 102: performing data enhancement on the image, and forming an activated sludge microorganism image dataset together with the labeled categories of the microorganisms in the image.

[0025] Step 103: Use the activated sludge microorganism image dataset to train a Faster RCNN network to obtain a microorganism recognition model.

[0026] Step 104: Input the image of the microorganism to be detected into the microorganism recognition model to identify the type of the microorganism to be detected.

[0027] By implementing the above steps 101 to 104, data enhancement is performed on the collected images, the amount of effective data for Faster RCNN network training is expanded, and the training accuracy of Faster RCNN network is improved; the Faster RCNN network model is used to identify microorganisms in activated sludge, which can more accurately detect the morphological characteristics of microorganisms, and is not affected by various interference factors such as algae and suspended matter mixed in the image background, thereby achieving efficient and accurate detection and classification of microorganisms in activated sludge.

[0028] In another exemplary embodiment of the present application, water samples are collected from a sewage treatment plant, and images of various microorganisms in activated sludge are obtained using a microscope, including eleven activated sludge microorganisms, including bellworms, chambered insects, rotifers, suckerworms, Euplotes, whipworms, doorworms, branchworms, scale insects, bristleworms, and boxworms. The LabelImg annotation tool is used to annotate the images of various microorganisms in activated sludge with microbial categories.

[0029] In another exemplary embodiment of the present application, after the categories of the microorganisms in the image are labeled, the image is normalized and saved.

[0030] In another exemplary embodiment of the present application, the specific method of performing data enhancement on the image in the above step 2 is: performing data enhancement on the image by means of cutout, brightness adjustment, noise addition, rotation, cropping, translation and mirroring. Then, an activated sludge microorganism image dataset is established based on the data after data enhancement.

[0031] In another exemplary embodiment of the present application, Figure 2 As shown, the Faster RCNN network in the above step 103 includes: a convolutional layer, a region proposal network, a region of interest pooling layer and a classification regression layer; the convolutional layer is used to extract the features of the image and obtain a feature map; the region proposal network is used to identify multiple regions of interest in the feature map; the region of interest pooling layer is used to take the feature map and multiple regions of interest as input, integrate the feature map and multiple regions of interest, and output a regional feature map; the classification regression layer is used to classify the regional feature map through softmax, output the category of the microorganism, and correct the position of the microorganism through border regression.

[0032] The Faster RCNN network consists of Conv layers (convolutional layers), Region Proposal Network (RPN for short), ROIPooling (region of interest pooling layer) and Classification and Regression (classification and regression layer).

[0033] a. Convolutional layer: The input is an image, and the output is the features of an image, i.e., feature map. The feature map of the image is extracted through a set of conv+relu+pooling layers for subsequent RPN networks and fully connected layers.

[0034] b. Region Proposal Network: The input is the feature map extracted by the convolutional layer, and the output is multiple regions of interest (ROI). Each output ROI is specifically represented by a probability value (used to determine whether the anchor is the foreground or background) and four coordinate values. The probability value represents the probability that there is an object in the ROI. This probability is obtained by performing a binary classification of each region through softmax; the coordinate value is the predicted position of the object. During training, this coordinate will be used to regress with the actual coordinate to make the predicted object position more accurate during testing.

[0035] c. Region of interest pooling layer: This layer takes the region of interest output by the RPN network and the feature map output by the Conv layers as input, combines the two to obtain a fixed-size regional feature map (proposal featuremap) and outputs it to the subsequent fully connected network for classification.

[0036] d. Classification and regression layer: The input is the proposal feature map obtained in the previous layer, and the output is the category of the object in the region of interest and the exact location of the object in the image. This layer classifies the image through softmax and corrects the exact location of the object through bounding box regression.

[0037] The Faster RCNN algorithm is used to identify microorganisms in activated sludge, and data enhancement is performed through random cropping and horizontal flipping to prevent overfitting. Using ResNet 50 as the backbone can accelerate the training process and improve model performance. It achieves a good balance between accuracy and speed. Introducing Faster RCNN into the field of microbial image recognition can significantly improve the efficiency and accuracy of microbial detection and identification.

[0038] In another exemplary embodiment of the present application, the process of training the Faster RCNN network in step 103 above can be replaced by the following steps 201 to 202:

[0039] Step 201: Divide the activated sludge microorganism image dataset into a training set, a validation set, and a test set in a ratio of 6:2:2.

[0040] Step 202: The Faster RCNN network is trained, verified and tested in sequence using the training set, the validation set and the test set to obtain a microorganism recognition model.

[0041] After training, the best model from the training set is used to test the validation set images.

[0042] In another exemplary embodiment of the present application, when the Faster RCNN network after verification is tested using a test set, the evaluation indicators used include: precision, recall rate and F1 score.

[0043] Precision, recall and F1-Score are used as evaluation indicators of the model. The formulas are shown in (1)-(3):

[0044] Precision=TP / (TP+FP) (1)

[0045] Recall=TP / (TP+FN) (2)

[0046] F1-Score=2*(Precision*Recall) / (Precision+Recall) (3)

[0047] Among them, TP stands for True Positives, which means that the sample is classified as a positive sample and the assignment is correct. TN stands for True Negatives, which means that the sample is classified as a negative sample and the assignment is correct. FP stands for False Positives, which means that the sample is classified as a positive sample but the assignment is wrong. FN stands for False Negatives, which means that the sample is classified as a negative sample but the assignment is wrong. Precision stands for accuracy, which indicates the proportion of the number of correctly assigned positive samples to the total number of assigned positive samples. Recall stands for recall rate, which indicates the proportion of the number of correctly assigned positive samples to the total number of positive samples. F1-Score, also known as F1 score, is the harmonic mean of precision and recall rate. Therefore, using the FasterRCNN algorithm to identify microbial images, its evaluation indicators Precision is 80%, mAP is 93.12%, Recall is 94.3%, and F1-Score is 84.6%, which can basically achieve accurate recognition of microbial images.

[0048] Figure 3 The process of Faster RCNN identifying microorganisms in activated sludge during the test is shown. Figure 3 Part (a) is an image of microorganisms in activated sludge to be detected; Figure 3 Part (b) is the image of microorganism identification in activated sludge.

[0049] Figure 4 This is a working principle diagram of the deep learning-based microbial image recognition method of this application. This application combines computer vision, deep learning and other methods to recognize microbial images during sewage treatment, adopts the FasterRCNN deep learning algorithm, and designs a microbial image recognition process, which can obtain the status of microorganisms in a timely and accurate manner, greatly improve the intelligence level of the sewage treatment process, and provide more refined data support for water quality monitoring, fault diagnosis and process optimization.

[0050] Based on the same inventive concept, the embodiment of the present application also provides a deep learning-based microbial image recognition device for implementing the deep learning-based microbial image recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more deep learning-based microbial image recognition device embodiments provided below can refer to the limitations of the deep learning-based microbial image recognition method above, and will not be repeated here.

[0051] In an exemplary embodiment, a deep learning-based microbial image recognition device is provided, comprising: an acquisition module, a data enhancement module, a training module, and a recognition module.

[0052] The acquisition module is used to acquire images of various microorganisms in activated sludge and mark the categories of the microorganisms in the images; the data enhancement module is used to perform data enhancement on the images and form an activated sludge microorganism image data set together with the marked categories of the microorganisms in the images; the training module is used to train the Faster RCNN network using the activated sludge microorganism image data set to obtain a microorganism recognition model; the recognition module is used to input the image of the microorganism to be detected into the microorganism recognition model to identify the category of the microorganism to be detected.

[0053] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the category of microorganisms to be detected. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a microorganism image recognition method based on deep learning is implemented.

[0054] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0055] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0056] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0057] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0058] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0059] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0060] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A microbial image recognition method based on deep learning, characterized in that: include: Collecting images of multiple microorganisms in activated sludge and marking the categories of the microorganisms in the images; Performing data enhancement on the image and forming an activated sludge microorganism image dataset together with the marked categories of microorganisms in the image; Using the activated sludge microorganism image dataset to train a Faster RCNN network to obtain a microorganism recognition model; The image of the microorganism to be detected is input into the microorganism recognition model to identify the type of the microorganism to be detected.

2. The microbial image recognition method based on deep learning according to claim 1, characterized in that: Performing data enhancement on the image, specifically including: Data enhancement is performed on the image by means of cutout, brightness adjustment, noise addition, rotation, cropping, translation and mirroring.

3. The microbial image recognition method based on deep learning according to claim 1, characterized in that: The Faster RCNN network includes: a convolutional layer, a region proposal network, a region of interest pooling layer and a classification regression layer; The convolution layer is used to extract features of the image and obtain a feature map; The region proposal network is used to identify multiple regions of interest in the feature map; The region of interest pooling layer is used to take the feature map and multiple regions of interest as input, integrate the feature map and multiple regions of interest, and output a regional feature map; The classification and regression layer is used to classify the regional feature map through softmax, output the category of the microorganism, and correct the position of the microorganism through bounding box regression.

4. The microbial image recognition method based on deep learning according to claim 1, characterized in that: The Faster RCNN network is trained using the activated sludge microbial image dataset to obtain a microbial recognition model, which specifically includes: The activated sludge microorganism image dataset is divided into a training set, a validation set and a test set in a ratio of 6:2:2; The Faster RCNN network was trained, verified and tested in sequence using the training set, validation set and test set to obtain a microbial recognition model.

5. The microbial image recognition method based on deep learning according to claim 4, characterized in that: When using the test set to test the verified Faster RCNN network, the evaluation indicators used include: precision, recall, and F1 score.

6. The microbial image recognition method based on deep learning according to claim 1, characterized in that: Collect images of various microorganisms in activated sludge and annotate the categories of the microorganisms in the images, and then include: The image is normalized and saved.

7. A microbial image recognition device based on deep learning, characterized in that: The microorganism image recognition device based on deep learning includes: A collection module, used for collecting images of various microorganisms in activated sludge and marking the categories of the microorganisms in the images; A data enhancement module, used for performing data enhancement on the image and forming an activated sludge microorganism image dataset together with the marked categories of microorganisms in the image; A training module, used to train a Faster RCNN network using the activated sludge microorganism image dataset to obtain a microorganism recognition model; The recognition module is used to input the image of the microorganism to be detected into the microorganism recognition model to identify the type of the microorganism to be detected.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the deep learning-based microbial image recognition method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the deep learning-based microbial image recognition method described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the deep learning-based microbial image recognition method described in any one of claims 1 to 6 is implemented.

Images