Generative adversarial network and cyclic transfer learning-based common pathogenic bacterium detection and identification method

Abstract

The invention provides a common pathogenic bacterium detection and identification method based on a generative adversarial network and cyclic transfer learning, and belongs to the technical field of image processing and classification and identification. Comprising the following steps: circularly converting images; on the premise that information used for distinguishing is reserved, conversion of the microscopic image and the macroscopic image between the microscopic field and the macroscopic field is achieved; carrying out circular transfer learning; and guiding the microcosmic field by using the macroscopic field for the data after the image loop conversion processing. The model can circularly transfer knowledge between the macroscopic field and the microcosmic field, so that the model can effectively train similar target objects among different fields. By comparing a plurality of existing methods, the method disclosed by the invention can obtain a more excellent effect, which shows that the method disclosed by the invention is of great significance in classification and detection of pathogenic bacteria.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and classification recognition, and in particular, to a common pathogenic bacteria detection and recognition method based on generative adversarial networks and cyclic transfer learning. Background Art

[0002] In the fields of medicine, food, agriculture, etc., rapid identification of bacterial species is a common and important task. The process of strain identification in the laboratory based on traditional biochemical methods is cumbersome. It requires complex sample preparation, which is time-consuming and labor-intensive, that is, it requires a long preparation time and experienced biological experts to conduct identification. With the rapid development of deep learning technology, using computer technology to identify and classify bacteria is a promising solution. Compared with traditional methods, they are faster, more accurate and cheaper for bacterial strain classification.

[0003] At present, the methods used for bacterial species identification mainly include laboratory biochemical methods and deep learning methods using computer vision. In the field of biochemistry, traditional strain identification needs to be carried out in the laboratory, and the work process is very cumbersome. It requires a lot of time and manpower for sample preparation, and experienced biologists are required to carry out identification work. In the computer field, deep learning is first used to determine the basic characteristics of bacteria such as color and shape, and then manually classify and process them. In recent years, researchers have improved its ability to classify bacteria by constantly giving it new images to learn. There are deep learning models based on VGGNet and AlexNet to train the DIBaS dataset and classify different bacteria. There is a two-stage method for bacterial identification using data enhancement technology and a compact deep separable neural network structure, and there is also a subset of the DIBaS dataset that is classified using data enhancement technology and a custom neural network topology. Although the above methods have achieved good results, they are all traditional deep learning methods, and the classification and identification of bacterial species are completed by continuous training of the model. But there is a problem that they only learn features in the microscopic field, there will be interference between different categories, and the importance of shape in classification and recognition tasks is not taken into account. Summary of the invention

[0004] According to the technical problems mentioned in the above background technology, a common pathogen detection and identification method based on generative adversarial network and cyclic transfer learning is provided. The technical means adopted by the present invention are as follows:

[0005] The common pathogen detection and identification method based on generative adversarial network and cyclic transfer learning includes the following steps:

[0006] Step 1: Image cyclic conversion: Under the premise of retaining the information used for differentiation, the conversion of microscopic images and macroscopic images between the microscopic field and the macroscopic field is realized;

[0007] Step 2: Circular transfer learning: cyclically transform the processed data of the image and use the macro field to guide the micro field.

[0008] Furthermore, the image cyclic conversion in step 1 includes the following steps:

[0009] Step 11: Define the macro image as h and the micro image as w; set up two converters F and G and two discriminators D h and D w And the loss function

[0010] Step 12: Define the macro image as x h , the microscopic image is x w , the label of the macro image is l h ; The label of the microscopic image is l w ;

[0011] Step 13: Generate adversarial network for microscopic image x w and the macro image x h Make the conversion;

[0012]

[0013] in, An image that represents a microscopic image in the realm of a macroscopic image; An image that represents a macroscopic image in the realm of microscopic images;

[0014] Step 14: Add cycle consistency loss L cyc ;

[0015] Step 15: In order to distinguish between microscopic domain images and macroscopic domain images, two discriminators D h and D w Increase the target adversarial loss L wadv ;

[0016] Step 16: In the converter F and the discriminator D h Add source adversarial loss L hadv ;

[0017] Step 17: According to the three equations in steps 14, 15, and 16, unlabeled image conversion is achieved between micro and macro domains; for different categories in different domains, the recognition loss L is used i retain information used to distinguish between categories;

[0018] Step 18: Dropout is added to the converter and discriminator to avoid overfitting; this allows the microscopic image and the macroscopic image to be converted between the microscopic and macroscopic fields while retaining the information used for differentiation.

[0019] Furthermore, the additional cycle consistency loss L cyc for:

[0020]

[0021] Among them, p h represents the distribution in the macroscopic field; p w Represents the distribution of microscopic fields; It means to generate a fake macroscopic image from the generated microscopic image; It means regenerating a false microscopic image from a false macroscopic image.

[0022] Furthermore, in step 15, the target resistance loss L wadv for:

[0023]

[0024] Furthermore, in step 15, the source resistance loss L hadv for:

[0025]

[0026] Furthermore, the recognition loss L i for:

[0027]

[0028] in, Indicates that by x h The generated images of the microscopic field; Indicates that by x w Generate macroscopic images of the field.

[0029] Furthermore, the step 2 of cyclic transfer learning includes the following steps:

[0030] Step 21: Using the correlated micro and macro images that share the same label, define two feature extractors as f h and f w ;

[0031] Step 22: From the Image Extract feature vectors from

[0032] Step 23: Apply Softmax to the feature vector and get the output:

[0033] Step 24: Use the macro-guided loss L on ma Predict the categories to which the original image and the converted image belong; let f h and f w Two feature extractors for x h To correctly classify, x h To guide x through the transformation w Macroscopic images of feature representation;

[0034] Step 25: The classification loss L is used on the Softmax output c Predict the correct category of the conversion target;

[0035] Step 26: Optimize the entire model through the following function:

[0036] L=L cyc +L wadv +L hadv +λ 1 L i +λ 2 L ma +λ 3 L c .

[0037] Furthermore, the macroscopic guidance loss L ma for:

[0038]

[0039] Among them, l h Represents x h Label, H(p,q)=-∑ i p i log q i ; H(p,q) represents cross entropy.

[0040] Furthermore, the classification loss L c for:

[0041]

[0042] Where H(p,q)=-∑ i p i log q i represents cross entropy.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] The present invention guides the identification and detection of microscopic targets by transferring knowledge from the macroscopic field to the microscopic field. Past experiments have rarely paid attention to the impact of shape on the classification and identification of pathogenic bacteria, but the present invention uses shape as an important factor to complete the identification. By comparing with several models, the importance of shape in the classification and detection of pathogenic bacteria is proved. The model of the present invention can transfer knowledge between the macroscopic and microscopic fields in a cycle, so that it can effectively train similar target objects between different fields. By comparing with several existing methods, the method of the present invention can obtain better results, which shows the important significance of the present invention in the classification and detection of pathogenic bacteria. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 This is a structural diagram of the model of the present invention.

[0047] Figure 2 It is the micro-macro correspondence diagram of the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0050] like Figure 1-2 As shown, the present invention provides a common pathogenic bacteria detection and identification method based on generative adversarial network and cyclic transfer learning, comprising the following steps:

[0051] Step 1: Image cyclic conversion: Under the premise of retaining the information used for differentiation, the conversion of microscopic images and macroscopic images between the microscopic field and the macroscopic field is realized;

[0052] Step 2: Circular transfer learning: cyclically transform the processed data of the image and use the macro field to guide the micro field.

[0053] Preferably, in the present application, step 1 of image loop conversion includes the following steps:

[0054] Step 11: Define the macro image as h and the micro image as w; set up two converters F and G and two discriminators D h and D w And the loss function

[0055] Step 12: Define the macro image as x h , the microscopic image is x w , the label of the macro image is l h ; The label of the microscopic image is l w ;

[0056] Step 13: Generate adversarial network for microscopic image x w and the macro image x h Make the conversion;

[0057]

[0058] in, An image that represents a microscopic image in the realm of a macroscopic image; An image that represents a macroscopic image in the realm of microscopic images;

[0059] Step 14: Add cycle consistency loss L cyc ; plus cycle consistency loss L cyc for:

[0060]

[0061] Among them, p h represents the distribution in the macroscopic field; p w Represents the distribution of microscopic fields; It means to generate a fake macroscopic image from the generated microscopic image; It means regenerating a false microscopic image from a false macroscopic image.

[0062] Step 15: In order to distinguish between microscopic domain images and macroscopic domain images, two discriminators D h and D w Increase the target adversarial loss L wadv ; Target confrontation loss L wadv for:

[0063]

[0064] Step 16: In the converter F and the discriminator D h Add source adversarial loss L hadv ; Source adversarial loss L hadv for:

[0065]

[0066] Step 17: According to the three equations in steps 14, 15, and 16, unlabeled image conversion is achieved between micro and macro domains; for different categories in different domains, the recognition loss L is used i Retain information used to distinguish categories; identification loss L i for:

[0067]

[0068] in, Indicates that by x h The generated images of the microscopic field; Indicates that by x w Generate macroscopic images of the field.

[0069] Step 18: Dropout is added to the converter and discriminator to avoid overfitting; this allows the microscopic image and the macroscopic image to be converted between the microscopic and macroscopic fields while retaining the information used for differentiation.

[0070] In the present application, preferably, step 2 of cyclic transfer learning includes the following steps:

[0071] Step 21: Using the correlated micro and macro images that share the same label, define two feature extractors as f h and f w ;

[0072] Step 22: From the Image Extract feature vectors from

[0073] Step 23: Apply Softmax to the feature vector and get the output:

[0074] Step 24: Use the macro-guided loss L on ma Predict the categories to which the original image and the converted image belong; let f h and f w Two feature extractors for x h To correctly classify, x h To guide x through the transformation w Macroscopic image of feature representation; macroscopic guidance loss L ma for:

[0075]

[0076] Among them, l h Represents x h Label, H(p,q)=-∑ i p i log q i ; H(p,q) represents the cross entropy.

[0077] Step 25: The classification loss L is used on the Softmax output c Predict the correct category of the conversion target; classification loss L c for:

[0078]

[0079] Where H(p,q)=-∑ i p i log q i represents cross entropy.

[0080] Step 26: Optimize the entire model through the following function:

[0081] L=L cyc +L wadv +L hadv +λ 1 L i +λ 2 L ma +λ 3 L c .

[0082] The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. In the above-mentioned embodiments of the present invention, the descriptions of each embodiment have their own emphases. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A common pathogen detection and identification method based on generative adversarial networks and cyclic transfer learning, characterized in that: The following steps are involved: Step 1: Image loop conversion; Under the premise of retaining the information used for differentiation, the conversion between microscopic images and macroscopic images is realized between the microscopic field and the macroscopic field; Step 2: Cyclic transfer learning; The image is cyclically converted into processed data, and the macroscopic field is used to guide the microscopic field.

2. The common pathogenic bacteria detection and identification method based on generative adversarial network and cyclic transfer learning according to claim 1 is characterized in that: The image loop conversion in step 1 includes the following steps: Step 11: Define the macro image as h and the micro image as w; set up two converters F and G and two discriminators D h and D w And the loss function Step 12: Define the macro image as x h , the microscopic image is x w , the label of the macro image is l h ; The label of the microscopic image is l w ; Step 13: Generate adversarial network for microscopic image x w and the macro image x h Make the conversion; in, An image that represents a microscopic image in the realm of a macroscopic image; An image that represents a macroscopic image in the realm of microscopic images; Step 14: Add cycle consistency loss L cyc ; Step 15: In order to distinguish between microscopic domain images and macroscopic domain images, two discriminators D h and D w Increase the target adversarial loss L wadv ; Step 16: In the converter F and the discriminator D h Add source adversarial loss L hadv ; Step 17: According to the three equations in steps 14, 15, and 16, unlabeled image conversion is achieved between micro and macro domains; for different categories in different domains, the recognition loss L is used i retain information used to distinguish between categories; Step 18: Dropout is added to the converter and discriminator to avoid overfitting; this allows the microscopic image and the macroscopic image to be converted between the microscopic and macroscopic fields while retaining the information used for differentiation.

3. The common pathogenic bacteria detection and identification method based on generative adversarial network and cyclic transfer learning according to claim 2 is characterized in that: The additional cycle consistency loss L cyc for: Among them, p h represents the distribution in the macroscopic field; p w Represents the distribution of microscopic fields; It means to generate a fake macroscopic image from the generated microscopic image; It means regenerating a false microscopic image from a false macroscopic image.

4. The common pathogenic bacteria detection and identification method based on generative adversarial network and cyclic transfer learning according to claim 2 is characterized in that: In step 15, the target adversarial loss L wadv for:

5. The common pathogenic bacteria detection and identification method based on generative adversarial network and cyclic transfer learning according to claim 2 is characterized in that: In step 15, the source resistance loss L hadv for:

6. The common pathogenic bacteria detection and identification method based on generative adversarial network and cyclic transfer learning according to claim 2 is characterized in that: The recognition loss L i for: in, Indicates that through x h The generated images of the microscopic field; Indicates that through x w Generate macroscopic images of the field.

7. The common pathogenic bacteria detection and identification method based on generative adversarial network and cyclic transfer learning according to claim 1 is characterized in that: The step 2 cyclic transfer learning includes the following steps: Step 21: Using the correlated micro and macro images that share the same label, define two feature extractors as f h and f w ; Step 22: From the Image Extract feature vectors from Step 23: Apply Softmax to the feature vector and get the output: Step 24: Use the macro-guided loss L on ma Predict the categories to which the original image and the converted image belong; let f h and f w Two feature extractors for x h To correctly classify, x h To guide x through the transformation w Macroscopic images of feature representation; Step 25: The classification loss L is used on the Softmax output c Predict the correct category of the conversion target; Step 26: Optimize the entire model through the following function: L=L cyc +L wadv +L hadv +λ1L i +λ2L ma +λ3L c 。 8. The common pathogenic bacteria detection and identification method based on generative adversarial network and cyclic transfer learning according to claim 7 is characterized in that: The macroscopic guidance loss L ma for: Among them, l h Represents x h Label, H(p,q)=-∑ i p i logq i ; H(p,q) represents the cross entropy.

9. The common pathogenic bacteria detection and identification method based on generative adversarial network and cyclic transfer learning according to claim 7 is characterized in that: The classification loss L c for: Where H(p,q)=-∑ i p i logq i represents cross entropy.

Images