Bacterial colony classification method and device based on attention convolutional network

Abstract

The invention relates to the technical field of bacterial colony classification, in particular to a bacterial colony classification method and device based on an attention convolutional network and a computer readable medium, and the method comprises the steps: obtaining common bacterial colony images in food; labeling and enhancing the bacterial colony image to obtain an enhanced image; dividing the enhanced image into a training set, a test set and a verification set; training and verifying the attention convolutional network according to the training set, the test set and the verification set; and classifying bacterial colonies to be detected by adopting the attention convolutional network which passes the verification. The fusion space attention module, the self-attention module and the channel attention module are embedded into the Unet network. The technical problem of low bacterial colony classification accuracy in the prior art can be at least solved.

Description

Technical Field

[0001] This application relates to the technical field of colony classification, and particularly to a colony classification method and device based on an attention convolutional network. Background Art

[0002] Colony classification is of great significance in the fields of microbiology, food safety, pharmaceuticals, etc. It can help researchers understand the characteristics and behaviors of different bacterial species, evaluate the quantity and density of bacteria in samples, thereby judging the quality and safety of foods, water sources, etc., or evaluating the inhibitory effect of drugs on pathogenic bacteria, etc.

[0003] However, the inventors have found that the existing technologies are time-consuming and laborious when identifying colonies, and when identifying colonies based on images, due to the overlapping and adhering morphologies of colonies, it becomes difficult to accurately classify and count colonies. Summary of the Invention

[0004] An object of this application is to provide a colony classification method and device based on an attention convolutional network, at least to solve the problem of insufficient accuracy in colony classification in the existing technologies.

[0005] To achieve the above object, some embodiments of this application provide the following aspects:

[0006] In a first aspect, some embodiments of this application provide a colony classification method based on an attention convolutional network, including:

[0007] Obtain colony images common in foods;

[0008] Perform annotation and enhancement processing on the colony images to obtain enhanced images;

[0009] Divide the enhanced images into a training set, a test set, and a validation set;

[0010] Train the attention convolutional network according to the training set and the test set;

[0011] Validate the trained attention convolutional network according to the validation set;

[0012] Use the attention convolutional network passed in validation to classify the colonies to be detected;

[0013] Among them, the construction of the attention convolutional network includes the following steps:

[0014] Construct a spatial attention module, a self-attention module, and a channel attention module, and connect them sequentially to form an enhanced attention module; embed the enhanced attention module into the Unet network to obtain the attention convolutional network for replacing the 3*3 convolutional layer in the expansion structure of the Unet network; the enhanced attention module is used to process the input features by the spatial attention module to obtain feature map A, feature map A is processed by the self-attention module to obtain feature map B, and the feature map is processed by the channel attention module to obtain feature map C, and use feature map C as the output.

[0015] In a second aspect, some embodiments of the present application further provide a colony classification device based on an attention convolutional network. The classification device includes:

[0016] An acquisition module for acquiring images of common colonies in food;

[0017] A preprocessing module for performing annotation and enhancement processing on the colony image to obtain an enhanced image;

[0018] A division module for dividing the enhanced image into a training set, a test set, and a validation set;

[0019] A training module for training the attention convolutional network according to the training set and the test set; validating the trained attention convolutional network according to the validation set;

[0020] A classification module for classifying the colonies to be detected by using the attention convolutional network that has passed the validation.

[0021] In a third aspect, some embodiments of the present application further provide a computer-readable medium, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the method as described above.

[0022] Compared with the prior art, in the solution provided by the embodiments of the present application, for the colony classification method of the attention convolutional network proposed in the present application, a colony image common in food is obtained; the colony image is labeled and enhanced to obtain an enhanced image; the enhanced image is divided into a training set and a test set; the attention convolutional network is trained according to the training set and the test set; the trained attention convolutional network is used to classify the colony to be detected. An attention convolutional network is constructed, including: constructing a spatial attention module, a self-attention module, and a channel attention module, and connecting them in sequence to form an enhanced attention module; embedding the enhanced attention module into the Unet network to obtain the attention convolutional network for replacing the 3*3 convolutional layer in the expansion structure of the Unet network. For the attention convolutional network constructed by the fused enhanced attention module, in the present application, through the attention convolutional network with an attention module, the key features in the input data can be more effectively focused on and the focus of attention can be flexibly adjusted, thereby improving the accuracy of colony classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is an architecture diagram of the enhanced attention module of the attention convolutional network provided by the embodiment of the present application;

[0024] Figure 2 It is a schematic diagram of a spatial attention architecture provided by the embodiment of the present application;

[0025] Figure 3 It is a schematic diagram of a channel attention architecture provided by the embodiment of the present application;

[0026] Figure 4 It is a processing flowchart provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0028] The following terms are used in this article.

[0029] First, some embodiments of the present application provide a colony classification method based on an attention convolutional network (refer to Figure 4 ), including:

[0030] Obtain a colony image common in food;

[0031] Label and enhance the colony image to obtain an enhanced image;

[0032] Divide the enhanced image into a training set, a test set, and a validation set;

[0033] Train the attention convolutional network according to the training set and the test set;

[0034] Validate the trained attention convolutional network according to the validation set;

[0035] Use the attention convolutional network after passing the validation to classify the colonies to be detected.

[0036] It can be understood that through the attention convolutional network with an attention module in this application, it is possible to more effectively focus on the key features in the input data and flexibly adjust the focus of attention, thereby improving the accuracy of colony classification.

[0037] In an alternative embodiment of the present application, constructing an attention convolutional network includes the following steps:

[0038] Construct a spatial attention module, a self-attention module, and a channel attention module, and connect them in sequence to form an enhanced attention module (refer to Figure 1 ); embed the enhanced attention module into the Unet network to obtain the attention convolutional network for replacing the 3*3 convolutional layer in the expansion structure of the Unet network;

[0039] The enhanced attention module is used to process the input feature by the spatial attention module to obtain feature map A, feature map A is processed by the self-attention module to obtain feature map B, and the feature map is processed by the channel attention module to obtain feature map C, and feature map C is used as the output.

[0040] In an alternative embodiment of the present application,

[0041] The spatial attention module is used for:

[0042] Reshape and transpose the input feature to obtain the first feature map, reshape the input feature into the second feature map and the third feature map, multiply the first feature map and the second feature map, then process through the softmax function and multiply with the third feature map to obtain the fourth feature map, and add the fourth feature map to the input feature to obtain feature map A.

[0043] In an alternative embodiment of the present application,

[0044] Performing pooling convolution processing on the feature map A to obtain feature map B includes the following steps:

[0045] Perform first dilated convolution, global average pooling, and second dilated convolution processing on the feature map A respectively;

[0046] Multiply the feature maps obtained after the first dilated convolution and global average pooling processing once;

[0047] Multiply the feature maps obtained after the second dilated convolution and global average pooling processing twice;

[0048] Concatenate the feature maps obtained after the single multiplication and double multiplication processing to obtain the feature map B.

[0049] In an alternative embodiment of the present application,

[0050] Perform pooling convolution processing on the feature map B to obtain the feature map C, including the following steps:

[0051] Perform 1×1 convolution and softmax processing on the feature map C and then multiply it with the feature map C. Perform 1×1 convolution processing on the feature map obtained after multiplication and then add it to the feature map C to obtain the feature map C.

[0052] In an alternative embodiment of the present application,

[0053] Before classifying the colony to be detected using the trained attention convolutional network, first detect and segment the culture dish based on the Hough transform.

[0054] In an alternative embodiment of the present application,

[0055] The enhancement processing includes: randomly cropping, randomly rotating, randomly horizontally flipping, randomly changing brightness, randomly changing saturation, randomly blurring, randomly adding noise, randomly deforming, Mosaic enhancement, and copy enhancement to the colony image.

[0056] The training set in the present application includes 18 types of bacterial colonies, namely genus Pseudomonas, genus Alkaligenes, genus Salmonella, genus Vibrio, genus Aeromonas, genus Campylobacter, genus Paenibacillus, genus Moraxella, genus Brochothrix, genus Citrobacter, genus Carnobacterium, genus Corynebacterium, genus Alteromonas, genus Hafnia, Escherichia coli, Klebsiella pneumoniae, Staphylococcus aureus, and Pseudomonas aeruginosa. They all belong to the most common human pathogenic bacteria. The total number of each bacterial category in this dataset is 4982, and the images of each class are divided into a training set, a test set, and a validation set. The image percentages of the training set, test set, and validation set are 40%, 40%, and 20% respectively. The training set uses 1993 images, the test set uses 1993 images, and the validation set uses 996 images.

[0057] In an alternative embodiment of the present application, the training steps of the above attention convolutional network are as follows:

[0058] Input the training set into the above-mentioned attention convolutional network for iterative training. Optionally, through the backpropagation algorithm, by adjusting the identity function, each x' value in the output space is made similar to the x value in the input space. By minimizing the error function and updating the weights, the model can continuously improve its performance.

[0059] Input the test set into the attention convolutional-based neural network trained with the training set. If its performance meets the requirements, it is determined to meet the requirements and the training ends.

[0060] To evaluate the performance of the model, we used several parameters: true positive (TP), false positive (FP), true negative (TN), and false negative (FN). Sensitivity is the model that correctly classifies bacterial colonies into their respective species, which can be expressed as:

[0061] Sensitivity = 1 / 18 ∑(TP / (TP + FN));

[0062] The specificity of correctly rejecting bacteria from entering a classification they do not belong to can be expressed as:

[0063] Specificity = 1 / 18 ∑(TN / (TN + FP));

[0064] Precision and accuracy are respectively

[0065] Precision = 1 / 18 ∑(TP / (TP + FP));

[0066] Accuracy = 1 / 18 ∑((TP + TN) / (TP + TN + FP + FN));

[0067] In an alternative embodiment of the present application, the verified attention convolutional network is used to classify the colony to be detected, which specifically includes:

[0068] Image acquisition: Use an imaging system to capture a real-time image of the petri dish.

[0069] Image preprocessing: For the obtained petri dish data image, readjust the absorbance of the image through steps such as blurring and denoising, differential transformation, and threshold segmentation.

[0070] Image rough processing: Detect and segment the petri dish based on the Hough transform.

[0071] Image classification: Input the segmented petri dish area image into the trained attention convolutional network for classification.

[0072] In a second aspect, some embodiments of the present application further provide a colony classification device based on an attention convolutional network. The classification device includes:

[0073] An acquisition module for acquiring images of common colonies in food;

[0074] A preprocessing module for performing annotation and enhancement processing on the colony image to obtain an enhanced image;

[0075] A partitioning module for partitioning the enhanced image into a training set, a test set, and a validation set;

[0076] A training module for training the attention convolutional network according to the training set and the test set; validating the trained attention convolutional network according to the validation set;

[0077] A classification module for classifying the colonies to be detected by using the attention convolutional network that has passed the validation.

[0078] In a third aspect, some embodiments of the present application further provide a computer-readable medium, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the method as described above.

[0079] Advantageous effects:

[0080] The attention convolutional network designed in the present application integrates a spatial attention module, a self-attention module, and a channel attention module and embeds them into the unet network. The enhanced attention module is used to process the input features by the spatial attention module to obtain feature map A, feature map A is processed by the self-attention module to obtain feature map B, and the feature map is processed by the channel attention module to obtain feature map C, and feature map C is used as the output, which greatly improves the accuracy of colony classification.

[0081] The method and / or embodiment in the embodiments of the present application can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the method shown in the flowchart. When the computer program is executed by a processing unit, the above functions defined in the method of the present application are executed.

[0082] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, the computer-readable medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0083] In this application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination of the above.

[0084] The computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0085] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of devices, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0086] Computer-readable media includes both permanent and non-permanent, removable and non-removable media implemented by any method or technology for storing information. The information can be computer-readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device.

[0087] In addition, an embodiment of the present application also provides a computer program, which is stored in a computer device and enables the computer device to execute the method executed by the control code.

[0088] It should be noted that the present application can be implemented in software and / or a combination of software and hardware. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of the present application can be executed by a processor to implement the above steps or functions. Similarly, the software program (including related data structures) of the present application can be stored in a computer-readable recording medium, such as a RAM memory, a magnetic or optical drive, or a floppy disk and similar devices. Additionally, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with a processor to execute each step or function.

[0089] For those skilled in the art, it is obvious that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be construed as limiting the claims concerned. In addition, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. The multiple elements or devices recited in the apparatus claims can also be implemented by one element or device through software or hardware. The terms such as "first" and "second" are used to denote names and do not denote any particular order.

Claims

1. A method for classifying colonies based on an attention convolutional network, characterized in that obtain images of common colonies in food; perform annotation and enhancement processing on the colony images to obtain enhanced images; divide the enhanced images into a training set, a test set, and a validation set; train the attention convolutional network according to the training set and the test set; validate the trained attention convolutional network according to the validation set; use the attention convolutional network that has passed the validation to classify the colonies to be detected; wherein, the construction of the attention convolutional network includes the following steps: construct a spatial attention module, a self-attention module, and a channel attention module, and connect them in sequence to form an enhanced attention module; embed the enhanced attention module into the Unet network to obtain the attention convolutional network for replacing the 3*3 convolutional layer in the dilation structure of the Unet network; the enhanced attention module is used to process the input feature by the spatial attention module to obtain feature map A, feature map A is processed by the self-attention module to obtain feature map B, and the feature map is processed by the channel attention module to obtain feature map C, and use feature map C as the output.

2. The method according to claim 1, characterized in that the spatial attention module is used for: transpose the input feature after reshaping to obtain a first feature map, reshape the input feature into a second feature map and a third feature map, multiply the first feature map and the second feature map and then process through the softmax function and then multiply with the third feature map to obtain a fourth feature map, and add the fourth feature map to the input feature to obtain feature map A.

3. The method according to claim 1, characterized in that performing pooling convolution processing on feature map A to obtain feature map B includes the following steps: perform first dilated convolution, global average pooling, and second dilated convolution processing on feature map A respectively; multiply the feature maps obtained after the first dilated convolution and global average pooling processing once; multiply the feature maps obtained after the second dilated convolution and global average pooling processing twice; concatenate the feature maps obtained after the single multiplication and the double multiplication processing to obtain feature map B.

4. The method according to claim 1, characterized in that performing pooling convolution processing on feature map B to obtain feature map C includes the following steps: perform 1×1 convolution and softmax processing on feature map C and then multiply with feature map C, perform 1×1 convolution processing on the feature map obtained after multiplication and then add it to feature map C to obtain feature map C.

5. The method according to claim 1, characterized in that before using the attention convolutional network that has passed the validation to classify the colonies to be detected, first detect and segment the culture dish based on the Hough transform.

6. The method according to claim 1, characterized in that The enhancement processing includes: randomly cropping, randomly rotating, randomly horizontally flipping, randomly changing the brightness, randomly changing the saturation, randomly blurring, randomly adding noise, randomly deforming, Mosaic enhancement, and copy enhancement to the colony images.

7. A colony classification device based on an attention convolutional network, characterized in that The classification device includes: an acquisition module, configured to acquire the colony images common in food; a preprocessing module, configured to perform annotation and enhancement processing on the colony images to obtain enhanced images; a division module, configured to divide the enhanced images into a training set, a test set, and a validation set; a training module, configured to train the attention convolutional network according to the training set and the test set; and validate the trained attention convolutional network according to the validation set; a classification module, configured to classify the colony to be detected by using the attention convolutional network that has passed the validation.

8. The device according to claim 7, characterized in that, The device further includes one or more processors; and a memory storing computer program instructions.

9. A computer-readable medium, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the method according to any one of claims 1-6.

Images