Apparatus and method for determining type of cultured microorganism

Abstract

Provided are an artificial intelligence-based microbial colony determination apparatus and method, wherein image raw data of a culture medium is acquired from one or more cameras, image data is generated by performing preprocessing on the image raw data, a colony of microorganisms corresponding to at least one of a plurality of candidate microorganisms is detected on the basis of the image data, the type of microorganisms having a matching probability equal to or greater than a threshold value is identified from among the plurality of candidate microorganisms by using one or more multi-layer neural networks, on the basis of the image data, the type of microorganisms is identified by using a multi-layer neural network other than the one or more multi-layer neural networks if the microorganisms having the matching probability equal to or greater than the threshold value are not identified from among the plurality of candidate microorganisms, and information about the type of microorganisms is output.

Description

Device and method for determining the type of cultured microorganism

[0001] Embodiments of the present invention relate to an apparatus and method for determining the type of microorganism based on the culture state of a microorganism cultured in a laboratory.

[0002] While various tests performed in diagnostic laboratory medicine laboratories have been automated, the field of microbiology testing is difficult to automate compared to other testing areas because there are processes where human laboratories must perform or interpret results at several testing steps.

[0003] Microbiological testing involves several steps, including inoculation into a culture medium, culture, identification, and antimicrobial susceptibility testing, after sample collection. During this process, the examiner must manually check the culture status at 24 or 48-hour intervals and identify the bacterial species based on the colony morphology and characteristics of the cultured bacteria.

[0004] In particular, while mass spectrometry equipment such as MALDI-TOF or automated equipment such as VITEK 2 are used during the identification process, human intervention is still required during the culture stage and some reading processes, resulting in high inspection time and costs. Existing culture monitoring systems can only confirm bacterial growth, and the identification stage is not automated. To address these issues, there is a growing need for technology that utilizes AI-based object detection and recognition techniques to monitor culture media in real time and automatically identify bacterial species by training on colony image data of cultured bacteria.

[0005] The present invention utilizes artificial intelligence-based object detection and object recognition technology to monitor a culture medium in real time and learns colony image data of cultured bacteria to automatically identify bacterial species.

[0006] In one embodiment of the present disclosure, an apparatus for identifying microbial colonies based on artificial intelligence is provided. The apparatus may include at least one processor comprising a memory for storing one or more instructions and a processing circuit. By executing one or more instructions by at least one processor, the apparatus may acquire image raw data of a culture site from one or more cameras. By executing one or more instructions by at least one processor, the apparatus may perform preprocessing on the image raw data to generate image data. The preprocessing may include at least one of a first preprocessing for removing the background of the culture site from the image raw data, a second preprocessing for resizing the resolution of the image raw data, and a third preprocessing for removing noise from the image raw data. By executing one or more instructions by at least one processor, the apparatus may detect a microbial colony corresponding to at least one of a plurality of candidate microorganisms based on the image data. By executing one or more instructions by at least one processor, the apparatus may identify the type of microorganism among a plurality of candidate microorganisms whose matching probability is greater than or equal to a threshold value using one or more multilayer neural networks based on the image data. One or more multilayer neural networks may include at least one of a first multilayer neural network that determines the type of microorganism corresponding to a detected colony based on the texture features of the detected colony, a second multilayer neural network that determines the type of microorganism corresponding to a detected colony based on the color features of the detected colony, and a third multilayer neural network that determines the type of microorganism corresponding to a detected colony based on the size features of the detected colony. When at least one processor executes one or more instructions, the device may identify the type of microorganism using a multilayer neural network other than the one or more multilayer neural networks if no microorganism among a plurality of candidate microorganisms has a matching probability greater than or equal to a threshold value.By having at least one processor execute one or more instructions, the device can output information regarding the type of microorganism.

[0007] In one embodiment of the present disclosure, by executing one or more instructions by at least one processor, the device can acquire a texture feature map related to at least one of surface smoothness, wrinkle pattern, and internal density of a detected colony based on image data. By executing one or more instructions by at least one processor, the device can input the texture feature map into a first multilayer neural network to determine the type of microorganism corresponding to the detected colony.

[0008] In one embodiment of the present disclosure, one or more cameras may include a first camera that captures a visible light region, a second camera that captures an infrared region, and a third camera that captures an ultraviolet region. The image raw data of the culture site may include a first image raw data captured through the visible light region, a second image raw data captured through the infrared region, and a third image raw data captured through the ultraviolet region.

[0009] In one embodiment of the present disclosure, one or more cameras may include a 3D camera. By executing one or more instructions, at least one processor can obtain 3D image raw data of a culture medium from the 3D camera. By executing one or more instructions, at least one processor can identify at least one of the diameter of a detected colony, the height of a colony, and the volume of a colony based on the 3D image raw data of the culture medium. By executing one or more instructions, at least one processor can determine the type of microorganism corresponding to the colony.

[0010] In one embodiment of the present disclosure, the device can acquire raw data of a culture image at a first time interval in a first time interval by executing one or more instructions by at least one processor. The device can acquire raw data of a culture image at a second time interval in a second time interval after the first time interval by executing one or more instructions by at least one processor. The second time interval may be longer than the first time interval.

[0011] In one embodiment of the present disclosure, the device can identify the number of microorganisms contained in a colony by executing one or more instructions by at least one processor. By executing one or more instructions by at least one processor, the device can determine the time when the number of microorganisms contained in a colony doubles. By executing one or more instructions by at least one processor, the device can determine the type of microorganism corresponding to the colony based on the time when the number of microorganisms contained in the colony doubles.

[0012] In one embodiment of the present disclosure, the output layer of one or more multilayer neural networks can input each output vector into a multiclass classification model by transforming it based on a softmax activation function.

[0013] In one embodiment of the present disclosure, a method for identifying microbial colonies based on artificial intelligence is provided. The method may include the step of acquiring raw image data of a culture medium from one or more cameras. The method may generate image data by performing preprocessing on the raw image data. The preprocessing may include a step comprising at least one of a first preprocessing step of removing the background of the culture medium from the raw image data, a second preprocessing step of resizing the resolution of the raw image data, and a third preprocessing step of removing noise from the raw image data. The method may include the step of detecting a microbial colony corresponding to at least one of a plurality of candidate microorganisms based on the image data. Based on the image data, the method may identify the type of microorganism among a plurality of candidate microorganisms whose matching probability is greater than or equal to a threshold value using one or more multilayer neural networks. One or more multilayer neural networks may include at least one of a first multilayer neural network that determines the type of microorganism corresponding to a detected colony based on texture features of the detected colony, a second multilayer neural network that determines the type of microorganism corresponding to a detected colony based on color features of the detected colony, and a third multilayer neural network that determines the type of microorganism corresponding to a detected colony based on size features of the detected colony. The method may include a step of identifying the type of microorganism using a multilayer neural network other than one or more multilayer neural networks when no microorganism with a matching probability greater than or equal to a threshold is identified among a plurality of candidate microorganisms. The method may include a step of outputting information regarding the type of microorganism.

[0014] In one embodiment of the present disclosure, the step of identifying the type of microorganism may include: acquiring a texture feature map related to at least one of surface smoothness, wrinkle pattern, and internal density of a detected colony based on image data; and inputting the texture feature map into a first multilayer neural network to determine the type of microorganism corresponding to the detected colony.

[0015] In one embodiment of the present disclosure, one or more cameras may include a first camera that captures a visible light region, a second camera that captures an infrared region, and a third camera that captures an ultraviolet region. The image raw data of the culture site may include a first image raw data captured through the visible light region, a second image raw data captured through the infrared region, and a third image raw data captured through the ultraviolet region.

[0016] In one embodiment of the present disclosure, one or more cameras may include a 3D camera. The method may include the step of acquiring 3D image raw data of a culture medium from a 3D camera. The method may include the step of identifying at least one of the diameter of a detected colony, the height of a colony, and the volume of a colony based on the 3D image raw data of the culture medium. The method may include the step of determining the type of microorganism corresponding to the colony.

[0017] In one embodiment of the present disclosure, the method may include the step of acquiring raw data of a culture image at a first time interval in a first time interval. The method may include the step of acquiring raw data of a culture image at a second time interval in a second time interval after the first time interval. The second time interval may be longer than the first time interval.

[0018] In one embodiment of the present disclosure, the method may include the step of identifying the number of microorganisms contained in a colony. The method may include the step of determining the time at which the number of microorganisms contained in a colony doubles. The method may include the step of determining the type of microorganism corresponding to the colony based on the time at which the number of microorganisms contained in the colony doubles.

[0019] In one embodiment of the present disclosure, one or more output layers of a multilayer neural network can input each output vector into a multiclass classification model by transforming it based on a Softmax activation function.

[0020] In one embodiment of the present disclosure, a non-transient computer-readable recording medium is provided. The non-transient computer-readable recording medium may record a program for executing a method for identifying microbial colonies based on artificial intelligence. A program recorded on the non-transient computer-readable recording medium may include the step of acquiring image raw data of a culture site from one or more cameras, which is executed by a processor. A program recorded on the non-transient computer-readable recording medium may include the step of generating image data by performing preprocessing on the image raw data, which is executed by a processor. The preprocessing may include at least one of a first preprocessing that removes the background of the culture site from the image raw data, a second preprocessing that resizes the resolution of the image raw data, and a third preprocessing that removes noise from the image raw data. A program recorded on the non-transient computer-readable recording medium may include the step of detecting a microbial colony corresponding to at least one of a plurality of candidate microorganisms based on the image data, which is executed by a processor. A program recorded on a non-transient computer-readable recording medium may include a step of being executed by a processor to identify, based on image data, the type of microorganism among a plurality of candidate microorganisms whose matching probability is greater than or equal to a threshold using one or more multilayer neural networks. The one or more multilayer neural networks may include a step of including at least one of a first multilayer neural network that determines the type of microorganism corresponding to a detected colony based on texture features of the detected colony, a second multilayer neural network that determines the type of microorganism corresponding to a detected colony based on color features of the detected colony, and a third multilayer neural network that determines the type of microorganism corresponding to a detected colony based on size features of the detected colony.A program recorded on a non-transient computer-readable recording medium may include a step of identifying the type of microorganism using a multilayer neural network other than one or more multilayer neural networks when the program is executed by a processor and, if no microorganism among a plurality of candidate microorganisms has a matching probability greater than or equal to a threshold, the program is executed by a processor and may include a step of outputting information regarding the type of microorganism.

[0021] The present invention utilizes an artificial intelligence model to monitor culture media in real time and automates identification without human intervention, thereby reducing inspection time and costs as well as improving accuracy.

[0022] FIG. 1 is a drawing illustrating an apparatus for providing an artificial intelligence-based method for identifying microbial colonies according to embodiments of the present invention.

[0023] FIG. 2 is a block diagram of an apparatus providing an artificial intelligence-based method for identifying microbial colonies according to embodiments of the present invention.

[0024] FIG. 3 is a flowchart of a method for identifying microbial colonies based on artificial intelligence according to embodiments of the present invention.

[0025] FIG. 4 is a flowchart of a method for identifying microbial colonies based on the texture of the colony according to embodiments of the present invention.

[0026] FIG. 5 is a flowchart of a method for identifying microbial colonies based on a three-dimensional image according to embodiments of the present invention.

[0027] FIG. 6 is a flowchart of a method for identifying microbial colonies based on the number of microorganisms included in a colony according to embodiments of the present invention.

[0028] FIG. 7 is a flowchart of a method for identifying microbial colonies based on a three-dimensional image according to embodiments of the present invention.

[0029] FIG. 8 is a block diagram illustrating the configuration of a device according to embodiments of the present invention.

[0030] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, various modifications may be made to the embodiments, and thus the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, and substitutions to the embodiments are included within the scope of the rights.

[0031] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed forms, and the scope of this specification includes modifications, equivalents, or substitutions that fall within the technical concept.

[0032] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0033] Furthermore, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly and specifically defined otherwise. In certain cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant explanatory sections. Accordingly, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0034] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or coupled with that other component, or that there may be other components in between.

[0035] The terms used in the embodiments are for illustrative purposes only and should not be interpreted as intended to be limiting. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprising" or "having" are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0036] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, the singular form used in this specification includes the plural form unless specifically stated otherwise in the text. Additionally, the expression "at least one of a, b, and c" described throughout this specification may encompass 'a alone,' 'b alone,' 'c alone,' 'a and b,' 'a and c,' 'b and c,' or 'a, b, and c all.'

[0037] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the embodiments pertain. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0038] Additionally, terms such as “…part,” “…module,” etc., as described in this specification refer to a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software. Furthermore, embodiments of the present disclosure may be represented by functional block configurations and various processing steps. These functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, embodiments of the present disclosure may employ integrated circuit configurations such as memory, processing, logic, look-up tables, etc., which can execute various functions under the control of one or more microprocessors or other control devices.

[0039] Each block of the process flow diagrams attached to this specification and combinations of the flow diagrams may be executed by computer program instructions. Since these computer program instructions may be loaded into the processor of a general-purpose computer, a computer for special purposes, or other programmable data processing equipment, the instructions executed through the processor of the computer or other programmable data processing equipment create means for performing the functions described in the flow diagram block(s).

[0040] These computer program instructions may be stored in computer-available or computer-readable memory that can be directed toward a computer or other programmable data processing equipment to implement a function in a specific way, and the instructions stored in said computer-available or computer-readable memory may also produce a manufactured item containing instruction means that performs the function described in the flowchart block(s).

[0041] Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that perform a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in the flowchart block(s).

[0042] Additionally, each block may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). Furthermore, in some alternative execution examples, the functions mentioned in the blocks may occur out of order. For instance, two blocks described in succession may actually be executed substantially simultaneously, or the blocks may be executed in reverse order according to their corresponding functions.

[0043] In addition, when describing with reference to the attached drawings, identical components are assigned the same reference numeral regardless of drawing symbols, and redundant descriptions thereof are omitted. In describing the embodiments, if it is determined that a detailed description of related prior art could unnecessarily obscure the essence of the embodiments, such detailed description is omitted.

[0044] FIG. 1 is a drawing illustrating an apparatus for providing an artificial intelligence-based method for identifying microbial colonies according to embodiments of the present invention.

[0045] Referring to FIG. 1, in one embodiment of the present disclosure, an apparatus (2000) is shown that provides a method for determining the type (130) of microorganisms contained in a culture medium (110) using an artificial intelligence model from image data (120) for a culture medium (110).

[0046] In one embodiment of the present disclosure, the device (2000) may have the form of a notebook, desktop, laptop, tablet PC, smartphone, etc., but is not limited thereto. Additionally, the device (2000) may have the form of an application server, computing server, database server, file server, web server, etc.

[0047] In one embodiment of the present disclosure, the device (2000) can acquire image data (120) regarding a culture site (110). For example, the device (2000) can acquire image data (120) by photographing the culture site (110) using a camera of the device (2000) or a camera connected to the device (2000). For example, the device (2000) can receive image data (120) captured by an external device.

[0048] In one embodiment of the present disclosure, the culture medium (110) may include at least one of a blood agar plate and a MacConkey agar plate, but is not limited thereto. The culture medium (110) may be referred to as a culture medium. The culture medium (110) may be cultured under specified conditions. For example, the culture environment may be an environment with a temperature of 35 to 37°C and 5% CO2. When microorganisms proliferate on the culture medium (110), they may form colonies. Colonies may have various shapes. For example, colonies may be classified into at least one of circular, irregular, filamentous, and rhizoid depending on their distribution pattern. For example, colonies may be classified according to their lateral shape into one of the following: raised, with a slightly raised surface; convex, with a rounded, raised shape that is highest in the center and gradually decreases toward the edges; flat, with a flat shape that is almost at the same height as the surface of the culture medium and has no noticeable change in height; umbonate, with a protruding center that is rounded in the middle; and crateriform, with a high edge and a concave center, but are not limited thereto. For example, colonies may be classified according to their peripheral shape into one of the following: entire, with a smooth and regular edge; irregular, with a bumpy and irregular edge; undulate, with a wavy edge; lobate, with a rounded protrusion; and filamentous, with a thread-like edge, but are not limited thereto.

[0049] In one embodiment of the present disclosure, the device (2000) may include an artificial intelligence model. In one embodiment of the present disclosure, the artificial intelligence model may be trained to determine the type (130) of microorganisms contained in the culture medium (110) based on information regarding image data (120) of the culture medium (110). The artificial intelligence model may include a multilayer neural network. The device (2000) may obtain the type of microorganism by inputting the image data (120) into the artificial intelligence model.

[0050] FIG. 2 is a block diagram of an apparatus providing an artificial intelligence-based method for identifying microbial colonies according to embodiments of the present invention.

[0051] In one embodiment of the present disclosure, the device (2000) can obtain the type of microorganism in the culture medium based on image raw data. Referring to FIG. 2, the device (2000) may include an input module (210), a preprocessing module (220), an object detection module (230), and a multilayer neural network (240).

[0052] In one embodiment of the present disclosure, the input module (210) can acquire raw data of the culture site. For example, the input module (210) can acquire image raw data of the culture site from one or more cameras. For example, the input module (210) can acquire image raw data of the culture site captured through a camera of the device (2000) or a camera connected to the device (2000).

[0053] In one embodiment of the present disclosure, the input module (210) may periodically acquire image raw data of the culture site. For example, the input module (210) may acquire image raw data at predetermined time intervals. Alternatively, for example, the input module (210) may acquire image raw data at first time intervals in a first time interval and acquire image raw data at second time intervals in a second time interval. The second time interval may be longer than the first time interval. The input module (210) may efficiently process images by acquiring image raw data at shorter time intervals in the early part of the shooting and acquiring image raw data at longer time intervals in the later part of the shooting.

[0054] In one embodiment of the present disclosure, the camera may include at least one of a first camera that captures a visible light region, a second camera that captures an infrared region, and a third camera that captures an ultraviolet region. The image raw data of the culture site may include at least one of a first image raw data captured through the visible light region, a second image raw data captured through the infrared region, and a third image raw data captured through the ultraviolet region. Since the way metabolic products, pigments, cell structures, etc., react to light of a specific wavelength differs depending on the type of microorganism, different features can be obtained depending on the infrared camera, the ultraviolet camera, and the visible light camera.

[0055] In one embodiment of the present disclosure, the preprocessing module (220) can generate image data by performing preprocessing on image raw data. For example, the preprocessing module (220) can perform preprocessing to remove the background of the culture site from the image raw data. For example, the preprocessing module (220) can perform preprocessing to resize the resolution of the image raw data. For example, the preprocessing module (220) can perform preprocessing to remove noise from the image raw data. The preprocessing module (220) can perform one or more of the aforementioned preprocessing. By processing the image raw data through preprocessing, the processing result using at least one of the object detection module (230) and the multilayer neural network (240) can be improved.

[0056] In one embodiment of the present disclosure, the object detection module (230) can detect a colony of microorganisms corresponding to at least one of a plurality of candidate microorganisms based on image data. The object detection module (230) may be an artificial intelligence model trained to identify a colony included in the image data. For example, the object detection module (230) can detect whether it corresponds to a colony of microorganisms included in the image data and the location of the colony of microorganisms.

[0057] In one embodiment of the present disclosure, a multilayer neural network (240) can identify the type of microorganism among a plurality of candidate microorganisms whose matching probability is greater than or equal to a threshold value using one or more multilayer neural networks based on image data. The multilayer neural network (420) may include a plurality of multilayer neural networks according to features.

[0058] In one embodiment of the present disclosure, if a microorganism with a matching probability greater than or equal to a threshold value among a plurality of candidate microorganisms is not identified, a multilayer neural network other than one or more multilayer neural networks (240) can identify the type of microorganism.

[0059] In one embodiment of the present disclosure, the multilayer neural network may include a first multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the texture features of the detected colony. The texture feature map is information related to at least one of the surface smoothness, wrinkle pattern, and internal density of the detected colony, and may be obtained based on image data. The first multilayer neural network may determine the type of microorganism corresponding to the detected colony by taking the texture feature map as input.

[0060] In one embodiment of the present disclosure, the multilayer neural network may include a second multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the color characteristics of the detected colony.

[0061] In one embodiment of the present disclosure, the multilayer neural network may include a third multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the size characteristics of the detected colony. For example, the colony size may be expressed as a size ratio to the entire culture medium or may refer to the absolute size of the colony. The third multilayer neural network may be trained to determine the type of microorganism using the colony size as input.

[0062] In one embodiment of the present disclosure, a multilayer neural network can obtain the type of microorganism using image data captured over time. For example, the multilayer neural network can obtain the type of microorganism using image data captured at fixed time intervals. By using multiple image data captured over time rather than using only a single image data, the multilayer neural network can infer the type of microorganism more accurately.

[0063] In one embodiment of the present disclosure, the device (2000) can output information regarding the type of microorganism. For example, information regarding the type of microorganism can be output through an output module of the device (2000).

[0064] FIG. 3 is a flowchart of a method for identifying microbial colonies based on artificial intelligence according to embodiments of the present invention.

[0065] In one embodiment of the present disclosure, a method for identifying an artificial intelligence-based microbial colony may be performed by an apparatus (2000). A processor of the apparatus (2000) may perform a method for identifying an artificial intelligence-based microbial colony by executing an instruction stored in memory.

[0066] In step S310, the device (2000) may acquire image raw data of a culture medium from one or more cameras. The culture medium may include at least one of blood agar medium and McConkey agar medium. The camera may include at least one of a visible light camera, an infrared camera, an ultraviolet camera, and a 3D camera. The visible light camera, the infrared camera, and the ultraviolet camera may each refer to a camera that captures images using wavelengths in the visible light region, the infrared region, and the ultraviolet region. The 3D camera may refer to a camera that captures 3D images.

[0067] In step S320, the device (2000) may generate image data by performing preprocessing on the image raw data. In one embodiment of the present disclosure, the preprocessing may include at least one of a first preprocessing that removes the background of the culture medium from the image raw data, a second preprocessing that resizes the resolution of the image raw data, and a third preprocessing that removes noise from the image raw data. The first preprocessing may set values ​​outside the culture medium in the image raw data to a predetermined value (e.g., 0). The second preprocessing may reduce the resolution of the image raw data to allow processing of less data. The third preprocessing may obtain clear image data by performing noise-enhancing filtering on the image raw data. However, the preprocessing may include a preprocessing that converts a color image into a black and white image, and is not limited thereto.

[0068] In step S330, the device (2000) can detect a colony of microorganisms corresponding to at least one of a plurality of candidate microorganisms based on image data. The device (2000) can detect the colony using an artificial intelligence model trained to detect a colony of microorganisms included in the image data. In one embodiment of the present disclosure, the artificial intelligence model is a model that determines only whether it is a microorganism colony, rather than the type of microorganism colony, and can detect a microorganism colony with a small amount of computation.

[0069] In one embodiment of the present disclosure, the colony may be determined to be either Gram-positive or Gram-negative bacteria based on the culture medium. For example, the colony may be determined to be either Gram-positive or Gram-negative bacteria based on whether colonies are formed in blood agar medium and McConkey agar medium.

[0070] In step S340, the device (2000) can identify the type of microorganism among a plurality of candidate microorganisms whose matching probability is greater than or equal to a threshold value using one or more multilayer neural networks based on image data. The plurality of candidate microorganisms may include microorganisms that can be identified in a culture medium. For example, the candidate microorganism may include at least one of Streptococcus pyogenes, Escherichia coli, Klebsiella pneumoniae, Enterobacter, Salmonella, Shigella, Pseudomonas aeruginosa, Acinetobacter baumanii, Bacillus anthracis, Helicobacter pylori, Staphylococcus aureus, Streptococcus pneumoniae, Escherichia coli, Klebsiella species, Enterobacter species, Serratia species, and Citrobacter species.

[0071] In one embodiment of the present disclosure, a multilayer neural network may include at least one of a first multilayer neural network that determines the type of microorganism corresponding to a detected colony based on texture features of the detected colony, a second multilayer neural network that determines the type of microorganism corresponding to a detected colony based on color features of the detected colony, and a third multilayer neural network that determines the type of microorganism corresponding to a detected colony based on size features of the detected colony. In one embodiment of the present disclosure, the output layer of the multilayer neural network may input each output vector into a multiclass classification model by transforming it based on a softmax activation function.

[0072] In one embodiment of the present disclosure, the device (2000) may determine the final microorganism type based on the types of microorganisms identified using a plurality of multilayer neural networks. For example, the device (2000) may determine the type of microorganism most frequently inferred among the types of microorganisms identified based on a plurality of multilayer neural networks as the final microorganism type. Alternatively, for example, the device (2000) may determine the final microorganism type by assigning weights to the inference results of a plurality of multilayer neural networks. The weights may be determined through additional training using a plurality of trained multilayer neural networks.

[0073] In step S350, if a microorganism with a matching probability greater than or equal to a threshold value among a plurality of candidate microorganisms is not identified, the device (2000) may identify the type of microorganism using a multilayer neural network other than one or more multilayer neural networks. For example, in step S340, if a microorganism with a matching probability greater than or equal to a threshold value among a plurality of candidate microorganisms is not identified by the device (2000) using a first multilayer neural network, the device may identify the type of microorganism using at least one of a second multilayer neural network and a third multilayer neural network. However, the device (2000) is not limited thereto and may identify the type of microorganism using a multilayer neural network other than the first multilayer neural network, the second multilayer neural network, and the third multilayer neural network.

[0074] In step S360, the device (2000) can output information regarding the type of microorganism. For example, the device (2000) can provide information regarding the type of microorganism through a display or through audio.

[0075] FIG. 4 is a flowchart of a method for identifying microbial colonies based on the texture of the colony according to embodiments of the present invention.

[0076] In one embodiment of the present disclosure, a method for identifying an artificial intelligence-based microbial colony may be performed by an apparatus (2000). A processor of the apparatus (2000) may perform a method for identifying an artificial intelligence-based microbial colony by executing an instruction stored in memory. In one embodiment of the present disclosure, step S340 may include steps S410 and S420.

[0077] In step S410, the device (2000) can acquire a texture feature map related to at least one of the surface smoothness, wrinkle pattern, and internal density of the detected colony based on image data. The texture feature map can be acquired using an artificial intelligence model trained with a first multilayer neural network. For example, the device (2000) can acquire the texture feature map by inputting image data into an artificial intelligence model trained to generate the texture feature map. In one embodiment of the present disclosure, the texture feature map may have the same size as the image data, but is not limited thereto.

[0078] In step S420, the device (2000) can input a texture feature map into a first multilayer neural network to determine the type of microorganism corresponding to the detected colony. Depending on the type of microorganism, the texture features of the colony may differ. The device (2000) can determine the type of microorganism based on the texture features of the colony.

[0079] FIG. 5 is a flowchart of a method for identifying microbial colonies based on a three-dimensional image according to embodiments of the present invention.

[0080] In one embodiment of the present disclosure, a method for identifying an artificial intelligence-based microbial colony may be performed by an apparatus (2000). A processor of the apparatus (2000) may perform a method for identifying an artificial intelligence-based microbial colony by executing an instruction stored in memory.

[0081] In step S510, the device (2000) can obtain raw data of a three-dimensional image of the culture site from a three-dimensional camera. A three-dimensional camera may refer to a camera that captures a three-dimensional image. In one embodiment of the present disclosure, the three-dimensional image may include at least one of a point cloud image represented as a set of points in three-dimensional space, a polygon mesh image represented as a combination of polygonal faces, and a voxel image represented by dividing three-dimensional space into small cube units.

[0082] In step S520, the device (2000) can identify at least one of the diameter of the detected colony, the height of the colony, and the volume of the colony based on the three-dimensional image raw data of the culture medium.

[0083] In one embodiment of the present disclosure, the device (2000) can obtain three-dimensional image data by performing preprocessing based on three-dimensional image raw data. For example, the device (2000) may include at least one of a first preprocessing that removes the background of the culture medium from the three-dimensional image raw data, a second preprocessing that resizes the resolution of the three-dimensional image raw data, and a third preprocessing that removes noise from the three-dimensional image raw data, just like a two-dimensional image.

[0084] In one embodiment of the present disclosure, the device (2000) can detect a colony based on image data. The device (2000) can detect a colony using an artificial intelligence model trained to detect a colony using three-dimensional image data. The device (2000) can identify at least one of the diameter of the detected colony, the height of the colony, and the volume of the colony based on the three-dimensional image.

[0085] In step S530, the device (2000) can determine the type of microorganism corresponding to the colony. The device (2000) can determine the type of microorganism using a fourth multilayer neural network trained to determine the type of microorganism by taking at least one of the diameter of the colony, the height of the colony, and the volume of the colony as input.

[0086] FIG. 6 is a flowchart of a method for identifying microbial colonies based on the number of microorganisms included in a colony according to embodiments of the present invention.

[0087] In one embodiment of the present disclosure, a method for identifying an artificial intelligence-based microbial colony may be performed by an apparatus (2000). A processor of the apparatus (2000) may perform a method for identifying an artificial intelligence-based microbial colony by executing an instruction stored in memory.

[0088] In step S610, the device (2000) can identify the number of microorganisms contained in the colony. For example, the device (2000) can determine the number of microorganisms using an artificial intelligence model trained to determine the number of individual microorganisms contained in the colony.

[0089] In step S620, the device (2000) can determine the time when the number of microorganisms contained in the colony doubles. The device (2000) can determine the time when the number of microorganisms doubles by periodically acquiring image data and identifying the number of microorganisms.

[0090] In step S630, the device (2000) can determine the type of microorganism corresponding to the colony based on the time it takes for the number of microorganisms contained in the colony to double. In one embodiment, the time it takes for the number of microorganisms to double may be referred to as generation time or doubling time. The time it takes for the number of microorganisms to double may vary depending on the species, environmental conditions, and nutrient supply status. For example, it may take about 20 minutes for E. coli to double under ideal conditions, and about 20 to 30 minutes for Staphylococcus aureus to double under ideal conditions. Also, for example, it may take about 30 minutes for Salmonella to double the number of microorganisms, and about 15 to 20 hours for Mycobacterium tuberculosis. Additionally, for example, it takes about 10 to 12 minutes for the number of microorganisms to double for cholera bacteria, about 50 minutes for Listeria, and about 8 to 10 minutes for gas gangrene bacteria.

[0091] In one embodiment of the present disclosure, the device (2000) can determine the type of microorganism based on the time when the microorganism doubles by a rule-based method, but is not limited thereto, and can infer it using a multilayer neural network.

[0092] FIG. 7 is a flowchart of a method for identifying microbial colonies based on a three-dimensional image according to embodiments of the present invention.

[0093] In one embodiment of the present disclosure, a method for identifying an artificial intelligence-based microbial colony may be performed by an apparatus (2000). A processor of the apparatus (2000) may perform a method for identifying an artificial intelligence-based microbial colony by executing an instruction stored in memory.

[0094] In step S710, the device (2000) can obtain raw data of a three-dimensional image of the culture site from a three-dimensional camera. In one embodiment of the present disclosure, step S710 may correspond to step S510 of FIG. 5. For example, the three-dimensional image may include at least one of a point cloud image, a polygon mesh image, and a voxel image.

[0095] In step S720, the device (2000) can acquire a texture feature map related to at least one of the surface smoothness, wrinkle pattern, and internal density of the detected colony based on three-dimensional image data. For example, the device (2000) can acquire the texture feature map by inputting the three-dimensional image data into an artificial intelligence model trained to generate the texture feature map. In one embodiment of the present disclosure, the device (2000) can extract a texture feature map with high accuracy by acquiring the texture feature map using a three-dimensional image.

[0096] In step S730, the device (2000) can determine the type of microorganism corresponding to the detected colony by inputting a texture feature map into a first multilayer neural network. The device (2000) can determine the type of microorganism based on the texture features of the identified colony using a three-dimensional image.

[0097] FIG. 8 is a block diagram illustrating the configuration of a device according to embodiments of the present invention.

[0098] In one embodiment, the device (2000) may include a memory (2010), a camera (2020), and a processor (2030). However, the present disclosure is not limited thereto. The device (2000) may be configured with some of the components shown in FIG. 8 omitted, or may be configured to include other components in addition to those shown in FIG. 8. For example, the device (2000) may further include an input / output interface and a communication interface.

[0099] In one embodiment, the memory (2010), camera (2020), and processor (2030) may each be physically / electrically connected to each other.

[0100] In one embodiment, the memory (2010) may include at least one type of storage medium among a flash memory type, a hard disk type, an SSD type (Solid State Disk type), an SSD type (Silicon Disk Drive type), a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), RAM (random access memory), SRAM (static random access memory), ROM (read-only memory), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory, and an optical disk.

[0101] In one embodiment, the memory (2010) may store data used in the device (2000). For example, the memory (2015) may include a microorganism candidate database (DB) (2015). The microorganism candidate DB (2015) may refer to a data set in which information about candidate microorganisms that can be identified in a culture medium is stored. For example, the microbial candidate database (2015) may include information regarding at least one of Streptococcus pyogenes, Escherichia coli, Klebsiella pneumoniae, Enterobacter, Salmonella, Shigella, Pseudomonas aeruginosa, Acinetobacter baumanii, Bacillus anthracis, Helicobacter pylori, Staphylococcus aureus, Streptococcus pneumoniae, Escherichia coli, Klebsiella species, Enterobacter species, Serratia species, and Citrobacter species. there is.

[0102] In one embodiment, the memory (2010) may store instructions, programs, or modules for the operation of the processor (2030). In one embodiment, the memory (2010) may include an input module (210), a preprocessing module (220), an object detection module (230), and a multilayer neural network (240). The input module (210), the preprocessing module (220), the object detection module (230), and the multilayer neural network (240) may consist of instructions or program code for performing the operation of the device (2000).

[0103] However, the present disclosure is not limited thereto, and it is understood that the memory (2010) may contain more models or modules than those shown in FIG. 8, or fewer models or modules. In one embodiment, FIG. 8 shows an input module (210), a preprocessing module (220), an object detection module (230), and a multilayer neural network (240) as separate models, but the present disclosure is not limited thereto. For example, the input module (210), the preprocessing module (220), and the object detection module (230) may be composed of a single module capable of inferring the type of microorganism based on image data.

[0104] In one embodiment of the present disclosure, the input module (210) can acquire raw data of the culture medium. For example, the input module (210) can acquire image raw data of the culture medium from a camera (2020). For example, the input module (210) can acquire image raw data of the culture medium captured through the camera (2020) of the device (2000) or a camera connected to the device (2000). In one embodiment of the present disclosure, the input module (210) can acquire image raw data at predetermined time intervals. For example, the input module (210) can acquire image raw data at first time intervals in a first time interval, and in a second time interval after the first time interval, acquire image raw data at second time intervals that are longer than the first time interval.

[0105] In one embodiment of the present disclosure, a preprocessing module (220) may generate image data by performing preprocessing on image raw data. For example, the preprocessing module (220) may perform a first preprocessing to remove the background excluding the culture medium from the image raw data. For example, the preprocessing module (220) may perform a second preprocessing to enlarge or reduce the resolution of the image raw data. For example, the preprocessing module (220) may perform a third preprocessing to remove noise from the image raw data. The preprocessing module (220) may perform one or more of the aforementioned preprocessing.

[0106] In one embodiment of the present disclosure, the object detection module (230) can detect a colony of microorganisms corresponding to at least one of a plurality of candidate microorganisms based on image data. The object detection module (230) may be an artificial intelligence model trained to identify a colony included in the image data. The object detection module (230) can detect whether it corresponds to a colony of microorganisms included in the image data and the location of the colony of microorganisms.

[0107] In one embodiment of the present disclosure, a multilayer neural network (240) can identify the type of microorganism among a plurality of candidate microorganisms whose matching probability is greater than or equal to a threshold value using one or more multilayer neural networks based on image data. The multilayer neural network (420) may include a plurality of multilayer neural networks according to features. In one embodiment of the present disclosure, the multilayer neural network may include at least one of a first multilayer neural network that determines the type of microorganism corresponding to a detected colony based on the texture features of the detected colony, a second multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the color features of the detected colony, a third multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the size features of the detected colony, and a fourth multilayer neural network trained to determine the type of microorganism by taking at least one of the diameter of the colony, the height of the colony, and the volume of the colony as input. In one embodiment of the present disclosure, the multilayer neural network may be referred to as an artificial intelligence model.

[0108] Training for an artificial intelligence model may be performed on a device containing an artificial intelligence model according to one embodiment. Alternatively, training may be performed through a separate server and / or system. Examples of algorithms for training include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0109] An artificial intelligence model can be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model.

[0110] For example, multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. The artificial neural network may include a Deep Neural Network (DNN), and may include, for example, a Convolutional Neural Network (CNN), 3D-CNN, Recurrent Neural Network (RNN), Generative Adversarial Networks (GAN), Transformer model, Variant Auto Encoder (VAE) model, or Diffusion model, but is not limited to any one of them.

[0111] In one embodiment, the camera (2020) may mean a device for acquiring one or more frames. Here, the frames may be represented as still images, videos, or photographs.

[0112] In one embodiment, the camera (2020) may photograph the front, side, or rear of the device (2000). In one embodiment, the camera (2020) may include at least one of a wide-angle camera or a telephoto camera. In one embodiment, the camera (2020) may include at least one of a micro-camera or a pinhole camera. In one embodiment, the camera (2020) may include at least one of an infrared camera, an ultraviolet camera, and a visible light camera. In one embodiment, the camera (2020) may include a three-dimensional camera. In one embodiment, the device (2000) may include a plurality of cameras (2020).

[0113] The camera (2020) may include at least one of an image sensor and an image signal processor (ISP). The image sensor may refer to a sensor that identifies light transmitted through the lens of the camera (2020). The image signal processor may refer to a processor that processes the signal identified by the image sensor. In one embodiment, the image signal processor may perform correction for distortion caused by the camera (2020).

[0114] In one embodiment, the processor (2030) may include a general-purpose processor such as a CPU (Central Processing Unit), AP (Application Processor), DSP (Digital Signal Processor), or a neural network processing processor such as an NPU (Neural Processing Unit). In one embodiment, the processor (2030) may be divided by one or more processors to perform operations. In one embodiment, the processor (2030) may control the operation of the device (2000). The processor (2030) may control the operation of the device (2000) according to instructions stored in memory (2010).

[0115] In one embodiment, the processor (2030) may be implemented as a memory that stores data for an algorithm or a program that reproduces the algorithm for controlling the operation of components within the device (2000) of the present disclosure, and as a processor that performs the aforementioned operation using the data stored in the memory. In this case, the memory and the processor may each be implemented as separate chips. Alternatively, the memory and the processor may be implemented as a single chip.

[0116] In addition, the processor may control one or a combination of the components described above in order to implement various embodiments according to the present disclosure through the device (2000).

[0117] In one embodiment, the device (2000) may be connected to various types of external devices through an input / output interface. In one embodiment, the input / output interface may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module (SIM), an audio I / O (Input / Output) port, or a video I / O (Input / Output) port. In one embodiment, the input / output interface may include a USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), or DVI (Digital Visual Interface), etc.

[0118] In one embodiment, the communication interface may include one or more components that enable communication between the device (2000) and an external server or external electronic device. In one embodiment, the communication interface may include at least one of a wired communication module or a wireless communication module.

[0119] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), DVI (Digital Visual Interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service).

[0120] The wireless communication module may include a wireless communication module that supports at least one of a wireless communication method including WiBro (Wireless broadband), GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, 6G, Bluetooth™, or Wi-Fi (Wireless-Fidelity).

[0121] In one embodiment, the processor (2030) can obtain information from an external electronic device or server or provide information through a communication interface.

Claims

1. In an artificial intelligence-based device for identifying microbial colonies, Memory for storing one or more instructions; and It includes at least one processor including a processing circuit, and By the above at least one processor executing the above one or more instructions, the device, Acquire raw image data of the culture site from one or more cameras, and Image data is generated by performing preprocessing on the above image raw data, and the preprocessing includes at least one of a first preprocessing for removing the background of the culture medium from the above image raw data, a second preprocessing for resizing the resolution of the above image raw data, and a third preprocessing for removing noise from the above image raw data. Based on the above image data, detect a colony of a microorganism corresponding to at least one of a plurality of candidate microorganisms, and Based on the above image data, the type of microorganism among the plurality of candidate microorganisms whose matching probability is greater than or equal to a threshold is identified using one or more multilayer neural networks, wherein the one or more multilayer neural networks include at least one of a first multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the texture features of the detected colony, a second multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the color features of the detected colony, and a third multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the size features of the detected colony. If no microorganism among the above plurality of candidate microorganisms has a matching probability greater than or equal to a threshold, the type of microorganism is identified using a multilayer neural network other than the one or more of the above multilayer neural networks, and Outputting information regarding the types of the above microorganisms, AI-based microbial colony identification device.

2. In Paragraph 1, By the above at least one processor executing the above one or more instructions, the device, Based on the above image data, a texture feature map related to at least one of the surface smoothness, wrinkle pattern, and internal density of the detected colony is obtained, and Inputting the above texture feature map into the above first multilayer neural network to determine the type of microorganism corresponding to the detected colony, AI-based microbial colony identification device.

3. In Paragraph 1, The above one or more cameras include a first camera that captures a visible light region, a second camera that captures an infrared region, and a third camera that captures an ultraviolet region, and The image raw data of the culture site includes a first image raw data captured through the visible light region, a second image raw data captured through the infrared region, and a third image raw data captured through the ultraviolet region, AI-based microbial colony identification device.

4. In Paragraph 1, The above one or more cameras include a 3D camera, and By the above at least one processor executing the above one or more instructions, the device, 3D image raw data of the culture site is obtained from the above 3D camera, and Based on the 3D image raw data of the culture medium, at least one of the diameter of the detected colony, the height of the colony, and the volume of the colony is identified, and Determining the type of microorganism corresponding to the above colony, AI-based microbial colony identification device.

5. In Paragraph 1, By the above at least one processor executing the above one or more instructions, the device, Raw data of culture images is acquired at first time intervals during the first time interval, and Raw data of the culture image is acquired at a second time interval in the second time interval following the first time interval above, and Characterized that the second time interval is longer than the first time interval, AI-based microbial colony identification device.

6. In Paragraph 1, By the above at least one processor executing the above one or more instructions, the device, Identify the number of microorganisms included in the above colony, and Determine the time when the number of microorganisms included in the above colony doubles, and Determining the type of microorganism corresponding to the colony based on the time when the number of microorganisms included in the colony doubles, AI-based microbial colony identification device.

7. In Paragraph 1, The output layer of the above-mentioned one or more multilayer neural networks is characterized by transforming each output vector based on a softmax activation function and inputting it into a multiclass classification model. AI-based microbial colony identification device.

8. In a method for identifying microbial colonies based on artificial intelligence, A step of acquiring raw image data of a culture site from one or more cameras; A step of generating image data by performing preprocessing on the above image raw data, wherein the preprocessing comprises at least one of a first preprocessing for removing the background of the culture medium from the above image raw data, a second preprocessing for resizing the resolution of the above image raw data, and a third preprocessing for removing noise from the above image raw data; A step of detecting a colony of a microorganism corresponding to at least one of a plurality of candidate microorganisms based on the above image data; Based on the image data above, the method identifies the type of microorganism among the plurality of candidate microorganisms whose matching probability is greater than or equal to a threshold using one or more multilayer neural networks, wherein the one or more multilayer neural networks include at least one of a first multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the texture features of the detected colony, a second multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the color features of the detected colony, and a third multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the size features of the detected colony; If no microorganism with a matching probability greater than or equal to a threshold is identified among the plurality of candidate microorganisms, a step of identifying the type of microorganism using a multilayer neural network other than the one or more multilayer neural networks; and A method comprising the step of outputting information regarding the type of microorganism mentioned above. AI-based microbial colony identification method.

9. In Paragraph 8, The step of identifying the type of microorganism mentioned above is, A step of obtaining a texture feature map related to at least one of surface smoothness, wrinkle pattern, and internal density of the detected colony based on the image data above; and The method includes the step of inputting the texture feature map into the first multilayer neural network to determine the type of microorganism corresponding to the detected colony. AI-based microbial colony identification method.

10. In Paragraph 8, The above one or more cameras include a first camera that captures a visible light region, a second camera that captures an infrared region, and a third camera that captures an ultraviolet region, and The image raw data of the culture site includes a first image raw data captured through the visible light region, a second image raw data captured through the infrared region, and a third image raw data captured through the ultraviolet region, AI-based microbial colony identification method.

11. In Paragraph 8, The above one or more cameras include a 3D camera, and A step of acquiring raw 3D image data of the culture site from the above 3D camera; A step of identifying at least one of the diameter of the detected colony, the height of the colony, and the volume of the colony based on the three-dimensional image raw data of the culture medium; A method comprising the step of determining the type of microorganism corresponding to the above colony, AI-based microbial colony identification method.

12. In Paragraph 8, A step of acquiring raw data of culture images at a first time interval in a first time interval; and It includes the step of acquiring raw data of culture images at a second time interval in a second time interval after the first time interval above, and Characterized that the second time interval is longer than the first time interval, AI-based microbial colony identification method.

13. In Paragraph 8, A step of identifying the number of microorganisms included in the above colony; A step of determining the time for the number of microorganisms included in the above colony to double; and A step comprising determining the type of microorganism corresponding to the colony based on the time at which the number of microorganisms included in the colony doubles, AI-based microbial colony identification method.

14. In Paragraph 8, The output layer of the above-mentioned one or more multilayer neural networks is characterized by transforming each output vector based on a softmax activation function and inputting it into a multi-class classification model. AI-based microbial colony identification method.

15. In a non-transient computer-readable recording medium, A step of acquiring raw image data of a culture site from one or more cameras; A step of generating image data by performing preprocessing on the above image raw data, wherein the preprocessing comprises at least one of a first preprocessing for removing the background of the culture medium from the above image raw data, a second preprocessing for resizing the resolution of the above image raw data, and a third preprocessing for removing noise from the above image raw data; A step of detecting a colony of a microorganism corresponding to at least one of a plurality of candidate microorganisms based on the above image data; Based on the image data above, the method identifies the type of microorganism among the plurality of candidate microorganisms whose matching probability is greater than or equal to a threshold using one or more multilayer neural networks, wherein the one or more multilayer neural networks include at least one of a first multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the texture features of the detected colony, a second multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the color features of the detected colony, and a third multilayer neural network that determines the type of microorganism corresponding to the detected colony based on the size features of the detected colony; If no microorganism with a matching probability greater than or equal to a threshold is identified among the plurality of candidate microorganisms, a step of identifying the type of microorganism using a multilayer neural network other than the one or more multilayer neural networks; and A non-transient computer-readable recording medium having a program for executing a step of outputting information regarding the type of microorganism mentioned above.

Images