Computer program, information processing method, and information processing apparatus

Abstract

To provide an information processing method capable of identifying a colony image of a new strain by appropriately discriminating strains even if the new strain whose stain cannot be discriminated is included in a culture medium in the case of discriminating the strains on the basis of the colony image included in a culture medium image.SOLUTION: An information processing method acquires a culture medium image obtained by imaging a culture medium including a plurality of colonies formed by a plurality of kinds of cultured bacteria, extracts a plurality of colony images included in the acquired culture medium image, and identifies colony images of a known strain whose strain can be discriminated and of a new strain whose strain cannot be discriminated by utilizing both processing for clustering the plurality of extracted colony images on the basis of feature amounts of the colony images, and processing for discriminating strains forming a colony on the basis of the colony images.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a computer program, an information processing method, and an information processing apparatus.

Background Art

[0002] Infectious diseases are caused by bacteria and the like invading the body. In medical institutions such as hospitals and inspection institutions, a specimen suspected of containing the causative bacterium of an infectious disease is collected from a patient, the specimen is cultured in a growth medium on a petri dish, and an inspection is performed to identify the bacteria contained in the specimen.

[0003] Patent Document 1 discloses a system that images colonies of bacteria formed on a medium and identifies the bacteria that form the colonies based on the captured image. It is also conceivable to identify the bacterial species from the image of the colony using a learned learning model.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when bacteria other than the learned bacterial species are contained in the medium, there is a problem that the bacterial species cannot be appropriately identified.

[0006] An object of the present invention is to appropriately identify the bacterial species and identify the colony image of a novel bacterial species even when a novel bacterial species for which the bacterial species cannot be identified is contained in the medium when identifying the bacterial species based on the colony image included in the medium image. It is to provide a computer program, an information processing method, and an information processing apparatus capable of doing so.

Means for Solving the Problems

[0007] The computer program according to this aspect causes a computer to execute a process of identifying a colony image of an existing bacterial species whose bacterial species can be identified and a colony image of a novel bacterial species whose bacterial species cannot be identified, by combining a process of obtaining a medium image obtained by imaging a medium containing a plurality of colonies formed by a plurality of types of cultured bacteria, a process of extracting a plurality of colony images included in the obtained medium image, a process of clustering the extracted plurality of colony images based on feature amounts of the colony images, and a process of identifying a bacterial species forming a colony based on the colony image.

[0008] The information processing method according to this aspect obtains a medium image obtained by imaging a medium containing a plurality of colonies formed by a plurality of types of cultured bacteria, extracts a plurality of colony images included in the obtained medium image, and combines a process of clustering the extracted plurality of colony images based on feature amounts of the colony images and a process of identifying a bacterial species forming a colony based on the colony image, to identify an existing bacterial species whose bacterial species can be identified and a colony image of a novel bacterial species whose bacterial species cannot be identified.

[0009] The information processing apparatus according to this aspect includes an acquisition unit that acquires a medium image obtained by imaging a medium containing a plurality of colonies formed by a plurality of types of cultured bacteria, and an arithmetic unit. The arithmetic unit extracts a plurality of colony images included in the acquired medium image, and combines a process of clustering the extracted plurality of colony images based on feature amounts of the colony images and a process of identifying a bacterial species forming a colony based on the colony image, to execute a process of identifying a colony image or its image feature amount and an existing bacterial species (hereinafter referred to as an existing bacterial species) whose bacterial species name is pre-associated, and a colony image of a bacterial species (hereinafter referred to as a novel bacterial species) whose bacterial species cannot be identified because its bacterial species name is not pre-associated with the colony image or its image feature amount.

Advantages of the Invention

[0010] According to the above, when identifying the bacterial species based on the colony images included in the culture medium image, even if the culture medium contains a novel bacterial species for which the bacterial species cannot be identified, the bacterial species can be appropriately identified and the colony image of the novel bacterial species can be specified.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Modes for Carrying Out the Invention

[0012] Specific examples of a computer program, an information processing method, a learning model generation method, and an information processing apparatus according to embodiments of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these examples, and is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. Also, at least a part of the embodiments described below may be arbitrarily combined.

[0013] (Embodiment 1) The information processing apparatus according to Embodiment 1 is an apparatus that identifies the bacterial species contained in a specimen collected from a patient. The specimen collected from the patient is cultured on a growth medium in a petri dish. Bacterial colonies are formed on the cultured medium. The information processing apparatus acquires an image obtained by imaging the medium (hereinafter referred to as a medium image), and based on the acquired medium image, executes a process of identifying the type (bacterial species) of the bacteria forming the colonies. However, when the specimen contains motile bacteria, it becomes difficult to identify the bacterial species. The information processing apparatus according to Embodiment 1 can improve the accuracy of identifying bacterial species by detecting such motile bacteria, and can streamline the bacteriological examination by early detection or mechanical automatic detection of motile bacteria. Also, the information processing apparatus according to Embodiment 1 performs bacterial species determination using a learning model, and can accumulate a medium image or a colony image that does not contain motile bacteria as training data for additional learning or test data. Furthermore, even when a new bacterial species is contained in the medium, it is an apparatus that can appropriately identify the bacterial species and specify the colony image of the new bacterial species. Furthermore, when performing additional learning of the learning model, the information processing apparatus according to Embodiment 1 evaluates the learning model using an evaluation index that takes into account clinical utility, rather than statistically used evaluation indices such as sensitivity, specificity, positive likelihood ratio, ROAUC, and PR-AUC, and can update the learning model.

[0014] <Configuration of Information Processing Apparatus> FIG. 1 is a block diagram showing an information processing apparatus 1. The information processing apparatus 1 is a computer such as a personal computer or a server apparatus. The information processing apparatus 1 includes an arithmetic unit 11, a memory 12, a storage unit 13, an operation unit 14, a display unit 15, a notification unit 16, and an acquisition unit 17. Note that the information processing apparatus 1 may be a multi-computer including a plurality of computers. Also, it may be a server-client system, a cloud server, or a virtual machine virtually constructed by software. In the following description, the information processing apparatus 1 is described as being a single computer.

[0015] The arithmetic unit 11 is an arithmetic processing apparatus such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-purpose computing on graphics processing units), or TPU (Tensor Processing Unit). Note that the arithmetic unit 11 may be configured using a quantum computer. By reading and executing the computer program 131 stored in the storage unit 13, the arithmetic unit 11 implements the information processing method according to the first embodiment, such as the identification process of the bacterial species included in the culture medium image, the detection process of motile bacteria, the identification of a new bacterial species, and the update process of the learning model.

[0016] The memory 12 is a volatile memory such as a DRAM (Dynamic RAM) or an SRAM (Static RAM), and temporarily stores the computer program 131 read from the storage unit 13 when the arithmetic processing of the arithmetic unit 11 is executed, or various data generated by the arithmetic processing of the arithmetic unit 11.

[0017] The storage unit 13 is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable ROM), or a flash memory. The storage unit 13 stores various programs executed by the arithmetic unit 11 and various data necessary for the processing of the arithmetic unit 11. In the first embodiment, the storage unit 13 stores a computer program 131 executed by the arithmetic unit 11, a motile bacterium recognition learning model 2 and a motility recognition learning model 3 for detecting motile bacteria from a culture medium image, a bacterial species identification learning model 4 for identifying the bacterial species included in the culture medium image, an existing bacterial species confirmation learning model 5, and a weight coefficient table 6 for evaluating additional learning of the bacterial species identification learning model 4.

[0018] The computer program 131 is recorded in a computer-readable manner on a recording medium 10, for example. The storage unit 13 stores the computer program 131 read from the recording medium 10 by a reading device (not shown). The recording medium 10 is a semiconductor memory such as a flash memory, an optical disk, a magnetic disk, a magneto-optical disk, or the like. Further, the computer program 131 may be stored in the storage unit 13 at the manufacturing stage of the information processing apparatus 1. Furthermore, the computer program 131 according to the first embodiment may be downloaded from an external server (not shown) connected to a communication network and stored in the storage unit 13.

[0019] The weight coefficient table 6 stores information indicating the clinical importance of each of a plurality of bacteria. The information indicating the clinical importance is a weight coefficient having a larger value as the bacterium has a greater severity of influence on the human body. Details of the weight coefficient will be described later.

[0020] The operation unit 14 is an input device that receives operations from an operator such as a medical technician. The input device is, for example, a keyboard or a pointing device.

[0021] The display unit 15 is an output device that outputs information such as a culture image and the identification result of bacteria. The output device is, for example, a liquid crystal display or an EL display.

[0022] The notification unit 16 is a device that notifies the presence of motile bacteria when motile bacteria are detected during bacterial culture. The notification unit 16 is a lamp, a speaker, a communication device, or the like.

[0023] The acquisition unit 17 is an interface that acquires a medium image obtained by imaging the medium during bacterial culture using the imaging device 9. The imaging device 9 performs imaging continuously or intermittently during bacterial culture and outputs the medium image to the information processing device 1. The acquisition unit 17 acquires the medium image output from the imaging device 9 each time, and the calculation unit 11 executes a process of determining the presence or absence of motile bacteria.

[0024] <Motile Bacteria Recognition Learning Model 2> FIG. 2 is a conceptual diagram showing a configuration example of the motile bacteria recognition learning model 2. The motile bacteria recognition learning model 2 is an image recognition model that outputs the presence or absence of motile bacteria that may be contained in the medium when a single medium image obtained by imaging the medium during bacterial culture is input. Note that the single medium image may be a so-called color image or a grayscale image. A color image is an image composed of three channels of red, green, and blue. A grayscale image is an image composed of a single channel having values from 0 to 255, for example.

[0025] In the first embodiment, the motile bacteria recognition learning model 2 is, for example, a neural network having an input layer 21 that receives an input of a medium image, an intermediate layer 22 that extracts feature amounts of bacterial images contained in the medium image, and an output layer 23 that outputs information indicating the presence or absence of motile bacteria that may be contained in the medium. The motile bacteria recognition learning model 2 of the first embodiment is a CNN (Convolution Neural Network) such as ResNet or DenseNet, or Attention. The information processing device 1 generates the motile bacteria recognition learning model 2 by performing deep learning for learning the relationship between the medium image and the presence or absence of motile bacteria with respect to the CNN model.

[0026] The input layer 21 of the neural network has a plurality of nodes that receive the input of the pixel values of each pixel constituting the culture medium image, and passes the input pixel values to the intermediate layer 22. The intermediate layer 22 has a plurality of nodes that extract the feature amounts of the culture medium image, and passes the extracted feature amounts to the output layer 23. For example, when the motile bacteria recognition learning model 2 is a CNN, the intermediate layer 22 has a configuration in which a plurality of convolutional layers that convolve the pixel values of each pixel input from the input layer 21 and a pooling layer that maps the pixel values convolved by the convolutional layer are connected in series, and finally extracts the feature amounts of the culture medium image while compressing the pixel information of the culture medium image. The output layer 23 has nodes that output information indicating the presence or absence of motile bacteria. Note that the activation function of the output layer 23 is, for example, a sigmoid function. The information indicating the presence or absence of motile bacteria is, for example, the probability that motile bacteria are present.

[0027] Figure 3 is a flowchart showing a method for generating the motile bacteria recognition learning model 2. Here, an example in which the information processing apparatus 1 generates the motile bacteria recognition learning model 2 will be described, but machine learning may be configured to be executed on another computer. The storage unit 13 stores training data in which a single culture medium image obtained by imaging a culture medium coated with at least one of arbitrary bacteria and motile bacteria is associated with information (teacher data) indicating the presence or absence of motile bacteria that may be contained in the culture medium. Needless to say, the training data includes a plurality of sets of data in which a single culture medium image and teacher data are associated. The culture medium images of the training data desirably include culture medium images captured at various times during bacterial culture. Further, the culture medium images of the training data desirably include culture medium images containing only motile bacteria, culture medium images not containing motile bacteria, and culture medium images in which motile bacteria and bacteria are mixed. Furthermore, the culture medium images of the training data desirably include various culture medium images such as selective media, non-selective media, and identification media.

[0028] The arithmetic unit 11 of the information processing apparatus 1 acquires training data from the storage unit 13, that is, training data in which a single medium image and teacher data indicating the presence or absence of motile bacteria are associated (step S11). Then, the arithmetic unit 11 generates a motile bacteria recognition learning model 2 by performing machine learning on an unlearned neural network using the acquired training data (step S12). That is, when a single medium image obtained by imaging the medium during bacterial culture is input, the neural network is machine-learned so that information indicating the presence or absence of motile bacteria is output. Specifically, the parameters of the neural network are optimized so that the difference between the information indicating the presence or absence of motile bacteria output when a single medium image, which is the training data, is input to the unlearned neural network and the teacher data associated with the medium image of the training data becomes small. The parameters are, for example, the weights (connection coefficients) between nodes. The method for optimizing the parameters is not particularly limited. For example, the arithmetic unit 11 optimizes various parameters using the steepest descent method or the like.

[0029] In the present Embodiment 1, it is assumed that the motile bacteria recognition learning model 2 is a CNN such as DenseNet, but the configuration of the model is not limited to CNN. The motile bacteria recognition learning model 2 may be a learning model composed of, for example, a neural network other than CNN, Vision Transformer, SVM (Support Vector Machine), Bayesian network, or a decision tree such as XGBoost.

[0030] <Motility Recognition Learning Model 3> FIG. 4 is a conceptual diagram showing a configuration example of the migration recognition learning model 3. The migration recognition learning model 3 is an image recognition model that outputs the presence or absence of motile bacteria that may be contained in a medium when a plurality of medium images obtained by imaging the same medium at different time points (i.e., a plurality of different culture times) during bacterial culture are input. The above-described motile bacteria recognition learning model 2 determines the presence or absence of motile bacteria using a single medium image, whereas the migration recognition learning model 3 is configured in a multi-channel in which a plurality of medium images obtained by imaging the same medium at a plurality of different culture times are simultaneously input, and is different in that it determines the presence or absence of motile bacteria using a plurality of medium images. The feature of the migration recognition learning model 3 is that a plurality of medium images derived from the same specimen are included in the same batch and input. The migration recognition learning model 3 is useful when the growth area of bacteria is small. A detailed description of the configuration common to the motile bacteria recognition learning model 2 will be omitted. The migration recognition learning model 3 includes, for example, an object detection learning model 31 for detecting the position and range of colony images included in a medium image, and an image recognition learning model 32 for recognizing the presence or absence of motile bacteria that may be contained in the medium based on a plurality of colony images extracted from the medium image based on the detection result of the object detection learning model 31.

[0031] As shown in the upper part of Fig. 4, the object detection learning model 31 is an object detection model such as YOLOv3, U-Net, Faster R-CNN, SSD, etc., which detects colony images included in the culture medium images. The object detection learning model 31 is, for example, a neural network having an input layer 31a that receives an input of a culture medium image, an intermediate layer 31b that extracts feature amounts of the culture medium image, and an output layer 31c that outputs information indicating the position and range of an image of a colony (hereinafter referred to as a colony image) formed by bacteria included in the culture medium image. Multiple culture medium images captured at different time points during bacterial culture are sequentially input to the object detection learning model 31, and the position and range of the colony image included in the culture medium image are detected. Using the information on the position and range of the colony image output from the object detection learning model 31, the colony image can be extracted from the culture medium image. In the migration recognition learning model 3, the object detection learning model 31 extracts a plurality of colony images from each of the plurality of culture medium images obtained by imaging the same culture medium at a plurality of different culture times.

[0032] As shown in the lower part of Fig. 4, the image recognition learning model 32 is a neural network having an input layer 32a into which a plurality of colony images extracted from a plurality of culture medium images captured at different time points during bacterial culture are input, an intermediate layer 32b that extracts feature amounts of the plurality of colony images, and an output layer 32c that outputs information indicating the presence or absence of motile bacteria that may be included in the culture medium.

[0033] The input layer 32a of the neural network has a plurality of nodes that receive the input of the pixel values of each pixel constituting the colony image, and passes the input pixel values to the intermediate layer 32b. The input layer 32a receives, for example, a plurality of colony images captured at a plurality of time points and extracted from each medium image. The intermediate layer 32b has a plurality of nodes that extract the feature amounts of the plurality of colony images extracted from the medium image, and passes the extracted feature amounts to the output layer 32c. The output layer 32c has a node that outputs information indicating the presence or absence of motile bacteria. Note that the activation function of the output layer 32c is, for example, a sigmoid function. The information indicating the presence or absence of motile bacteria is, for example, the probability that motile bacteria are present. Further, the output layer 32c may be configured as a softmax function to perform multi-class classification of bacteria. Note that although an example in which the motility recognition learning model 3 is configured by the object detection learning model 31 and the image recognition learning model 32 has been described, it may be configured by one learning model.

[0034] FIG. 5 is a flowchart showing a method for generating the motility recognition learning model 3, and FIG. 6 is an explanatory diagram showing a method for generating the motility recognition learning model 3. Here, it is assumed that the object detection learning model 31 has been pre-trained using a known machine learning method so as to be able to detect colony images. The storage unit 13 stores training data in which a plurality of medium images obtained by imaging a medium coated with at least one of arbitrary bacteria and motile bacteria at different time points during bacterial culture are associated with information (teacher data) indicating the presence or absence of motile bacteria that may be contained in the medium. The plurality of medium images obtained by imaging at a plurality of time points are a plurality of images obtained by intermittently imaging the same medium. Needless to say, the training data includes a plurality of sets of data in which a plurality of medium images are associated with teacher data.

[0035] The arithmetic unit 11 of the information processing apparatus 1 acquires training data in which the training data stored in the storage unit 13, that is, a plurality of medium images obtained by imaging at a plurality of time points, and teacher data indicating the presence or absence of motile bacteria that may be contained in the medium are associated with each other (step S31). Then, the arithmetic unit 11 generates a migration recognition learning model 3 by performing machine learning on an unlearned neural network using the acquired training data (step S32). That is, when a plurality of medium images obtained by imaging at different time points during bacterial culture are input, the neural network is machine-learned so that information indicating the presence or absence of motile bacteria is output. Specifically, when a plurality of colony images extracted from a plurality of medium images, which are training data, using the object detection learning model 31 are input to the unlearned neural network of the image recognition learning model 32, the difference between the information indicating the presence or absence of motile bacteria output and the teacher data associated with the medium image of the training data is minimized by optimizing the parameters of the neural network. The parameters are, for example, the weights (coupling coefficients) between nodes. The method for optimizing the parameters is not particularly limited. For example, the arithmetic unit 11 optimizes various parameters using the steepest descent method or the like.

[0036] The learning of the migration recognition learning model 3 may be performed by batch processing called mini-batch learning as shown in FIG. 4. In the same batch of training data for learning the image recognition learning model 32 of the migration recognition learning model 3, a plurality of colony images derived from the same specimen are included. The plurality of colony images are images extracted from medium images obtained by intermittently imaging the medium of the same specimen at a plurality of different time points. Since the weight parameters of the neural network are updated using the training data derived from the same specimen, it is possible to effectively learn the image feature amounts common to the same specimen.

[0037] When the migration recognition learning model 3 is configured as a multi-class classification model, training data is created by associating medium images captured under a plurality of different culture conditions (different media, etc.), medium images captured under a plurality of different imaging conditions (different illumination methods or imaging at different wavelengths), a plurality of medium images obtained by imaging at different time points during bacterial culture, and information (teacher data) indicating one or more bacterial species that may be contained in the medium. Then, when a plurality of medium images obtained by imaging at different time points during bacterial culture are input, all single colony images on the culture dish are extracted in such a way that the image is cut out with a square containing one single colony, and the neural network is machine-learned so that information indicating the probability of the presence of each of the plurality of bacteria in each of the extracted images is output. Specifically, the parameters of the neural network are optimized so that the difference between the probability of the presence of each of the plurality of bacteria output when a plurality of medium images, which are training data, are input to an untrained neural network and the correct bacterial species information (specifically, a vector data (multi-hot vector) having binary values of 1 when each of the plurality of bacteria is present and 0 when not present as vector components, with the total number of bacterial species as the dimension of the vector, generally called the correct label data) associated with the colony image of the training data is reduced. Note that the activation function of the output layer of the neural network is the softmax function.

[0038] Also, the migration recognition learning model 3 may be configured with a learning model such as a recurrent neural network (RNN), LSTM, or Vision Transformer.

[0039] <Bacterial species identification learning model 4> The bacterial species identification learning model 4 includes, for example, an object detection learning model 41 for detecting the position and range of a colony image (hereinafter referred to as a colony image) included in a medium image, and an image recognition learning model 42 for recognizing the type of bacteria forming the colony based on the colony image.

[0040] FIG. 7 is a conceptual diagram showing a configuration example of the bacterial species identification learning model 4. The bacterial species identification learning model 4 includes, for example, an object detection learning model 41 for detecting the position and range of colony images included in a culture medium image, and an image recognition learning model 42 for recognizing the bacterial species included in the culture medium based on the colony images extracted from the culture medium image based on the detection result of the object detection learning model 41. The object detection learning model 41 shown in the upper part of FIG. 7 is an object detection model such as YOLOv3, U-Net, Faster R-CNN, SSD, etc. that detects colony images included in a culture medium image. The object detection learning model 41 is a neural network having, for example, an input layer 41a that receives an input of a culture medium image, an intermediate layer 41b that extracts feature amounts of the culture medium image, and an output layer 41c that outputs information indicating the position and range of an image of a colony (hereinafter referred to as a colony image) formed by bacteria included in the culture medium image. Note that the object detection model does not need to recognize the bacterial species of the colony. One or more culture medium images are extracted from the culture medium image based on the information on the position and range output from the object detection learning model 41.

[0041] FIG. 7 is a conceptual diagram showing a configuration example of the image recognition learning model 42 shown in the lower part of the figure. The image recognition learning model 42 is an image recognition model such as VGG, ResNet, DenseNet, Vision Transformer, etc. that performs image recognition processing on a colony image when the colony image is input and identifies the bacterial species forming the colony image. The image recognition learning model 42 is a neural network having, for example, an input layer 42a that receives an input of a colony image, an intermediate layer 42b that extracts feature amounts of the colony image, and an output layer 42c that outputs information indicating the type of bacteria forming the colony. The information indicating the type of bacteria is, for example, the accuracy of each of a plurality of bacteria forming a colony. Note that when inputting a colony image to the image recognition learning model 42, it may be configured to adjust the size of the colony image to a predetermined size.

[0042] Note that although an example in which the bacterial species identification learning model 4 is configured by the object detection learning model 41 and the image recognition learning model 42 has been described, it may be configured by a single learning model. In addition, the image recognition learning model 42 may also have a multi-channel configuration, and when colony images captured at multiple time points during bacterial culture are input, it may be configured to output information indicating the type of bacteria forming the colony. In this case, the image recognition learning model 42 may be configured with a recurrent neural network, LSTM, Vision Transformer, etc.

[0043] FIG. 8 is a conceptual diagram showing a configuration example of the existing bacterial species confirmation learning model 5. When a colony image is input, the existing bacterial species confirmation learning model 5 performs image recognition processing on the colony image and outputs information indicating whether the bacterial species forming the colony is an existing bacterial species, in other words, whether it is a novel bacterial species. The existing bacterial species are bacterial species that can be identified by the image recognition learning model 42 of the bacterial species identification learning model 4, that is, bacterial species that the image recognition learning model 42 has learned. The existing bacterial species confirmation learning model 5 includes an input layer 51a, an intermediate layer 51b, and an output layer 51c similar to those of the image recognition learning model 42. However, the activation function of the output layer 51c is a sigmoid function.

[0044] <Information Processing Method: Detection of Motile Bacteria and Bacterial Species Identification Processing> FIG. 9 is a flowchart showing the processing procedure of bacterial species identification and model update. An operator of a bacterial examination performs bacterial culture of a specimen collected from a patient (step S51) and starts imaging of the culture medium using an imaging device 9 (step S52). The specimen may contain motile bacteria.

[0045] Next, the arithmetic unit 11 of the information processing apparatus 1 acquires a culture medium image from the imaging device 9 and executes a process of identifying the type of bacteria contained in the specimen (step S53). The process of bacterial identification includes a process of determining the presence or absence of motile bacteria during bacterial culture.

[0046] FIGs. 10 to 12 are flowcharts showing the processing procedure for bacterial species identification according to Embodiment 1. The arithmetic unit 11 acquires the culture medium image output from the imaging device 9 by the acquisition unit 17 (step S71), and stores the acquired culture medium image in the storage unit 13 (step S72). The imaging device 9 captures the culture medium image continuously or intermittently during the bacterial culture, and the acquisition unit 17 acquires the culture medium image each time the culture medium image is output from the imaging device 9.

[0047] Next, the arithmetic unit 11 inputs the acquired single culture medium image into the motile bacteria recognition learning model 2, and primarily determines the presence or absence of motile bacteria based on the information output from the motile bacteria recognition learning model 2 (step S73). In this Embodiment 1, the presence or absence of motile bacteria is determined by a plurality of different methods, and the final determination of the presence or absence of motile bacteria is made in the process of step S80.

[0048] Next, the arithmetic unit 11 inputs a plurality of culture medium images acquired at different time points during the bacterial culture into the motility recognition learning model 3, and primarily determines the presence or absence of motile bacteria based on the information output from the motility recognition learning model 3 (step S74).

[0049] Next, the arithmetic unit 11 inputs each of the plurality of culture medium images acquired at different time points during the bacterial culture into the object detection learning model 31, and detects colony images included in the plurality of culture medium images based on the information output from the object detection learning model 31 (step S75). The arithmetic unit 11 can recognize the position and range of the colony image in the culture medium image.

[0050] Next, the arithmetic unit 11 calculates the amount of movement (position change amount) of the colonies by comparing the positions of the colony images in a plurality of culture medium images (step S76). Then, the arithmetic unit 11 makes a primary determination of the presence or absence of motile bacteria based on the calculated amount of movement (step S77). Specifically, the arithmetic unit 11 determines whether or not the amount of movement of the colonies is equal to or greater than a first threshold value. If the amount of colony movement is equal to or greater than the first threshold value, it is determined that there are motile bacteria. Usually, a plurality of colony images are included in the culture medium image, but if there is even one colony whose movement amount is equal to or greater than the first threshold value, it may be configured to determine that there are motile bacteria.

[0051] Note that the center coordinates of the colonies in a plurality of culture medium images may be calculated, and it may be determined whether the center coordinates of the colonies at the second time point have moved outside the detection range of the colonies at the first time point (a time point before the second time point), and the presence or absence of motile bacteria may be determined. Whether or not the colonies have moved outside the above detection range may be determined by whether or not the ratio of the colonies existing within the detection range is less than a threshold value.

[0052] Next, the arithmetic unit 11 calculates the area expansion rate (size change amount) of the colonies by comparing the sizes, for example, the areas, of the colony images in a plurality of culture medium images (step S78). Then, the arithmetic unit 11 makes a primary determination of the presence or absence of motile bacteria based on the calculated area expansion rate (step S79). Specifically, the arithmetic unit 11 determines whether or not the area expansion rate of the colonies is equal to or greater than a second threshold value. If the area expansion rate of the colonies is equal to or greater than the second threshold value, it is determined that there are motile bacteria. Usually, a plurality of colony images are included in the culture medium image, but if there is even one colony whose area expansion rate is equal to or greater than the second threshold value, it may be configured to determine that there are motile bacteria.

[0053] It is preferable to use the lower limit value of the 95% confidence interval (the 2.5 percentile point from the lower side of the values) as the second threshold value from the distribution of the values of the area expansion ratio per unit time of the bacterial population by culturing motile bacteria alone in a plurality of culture dishes. In addition, in a sample in which motile bacteria and non-motile bacteria coexist, a histogram regarding the rate of area expansion of the bacterial population is created, and a threshold value that divides into two groups by one set threshold value is searched for a separation ability (correct answer rate, sensitivity, specificity, etc. of both) regarding the ratio of motile bacteria and non-motile bacteria contained, and it is preferable to use the value obtained by the search as the second threshold value.

[0054] Next, based on the primary determination results of step S73, step S74, step S77, and step S79, the arithmetic unit 11 comprehensively determines the presence or absence of motile bacteria (step S80). For example, the arithmetic unit 11 may determine the presence or absence of motile bacteria by logical sum. That is, among a plurality of primary determination results, if there is even one primary determination indicating the presence of motile bacteria, it is finally determined that there are motile bacteria. Note that the arithmetic unit 11 may be configured to finally determine that there are motile bacteria if there are a predetermined number or more of primary determinations indicating the presence of motile bacteria among a plurality of primary determination results. Further, the arithmetic unit 11 obtains a loss function including the value output from the motile bacteria recognition learning model 2, the value output from the motility recognition learning model 3, the difference between the colony movement amount and the first threshold value, and the difference between the colony area expansion rate and the second threshold value, a weighted sum of these numerical values, etc., and may be configured to finally determine the presence or absence of motile bacteria based on the value of the loss function and the value of the weighted sum.

[0055] In addition, in the present embodiment, an example of primarily determining the presence or absence of motile bacteria by four methods has been described, but when motile bacteria are detected by the motile bacteria recognition learning model 2 or the motility recognition learning model 3, the processes of step S75 to step S79 may be skipped. Furthermore, the order of performing the four primary determinations is not particularly limited. When the four primary determinations are executed in order, if motile bacteria are detected, the remaining primary determination processes may be skipped.

[0056] Next, if it is determined that motile bacteria are present as a result of the process in step S80 (step S81: YES), the arithmetic unit 11 notifies the notification unit 16 of the presence of motile bacteria (step S82). For example, the arithmetic unit 11 may turn on a warning lamp, output a sound, or transmit information indicating the presence of motile bacteria to the communication terminal of the operator. This notification is performed at the stage when motile bacteria are detected during bacterial culture. The operator can know the presence of motile bacteria at an early stage. By detecting motile bacteria at an early stage, an inspector or a colony picker device can pick the bacteria and separately culture the motile bacteria and the bacteria other than the motile bacteria. Also, if necessary, an operator such as an inspector can repeat the bacterial culture. Therefore, an increase in inspection costs can be suppressed, and an increase in TAT (Turn Around Time) representing the inspection time can be suppressed.

[0057] Next, the arithmetic unit excludes the culture medium image in which motile bacteria are detected from the plurality of culture medium images acquired from the imaging device 9 (step S83). By this exclusion process, the culture medium image including motile bacteria is excluded from the target of bacterial species identification. Also, by this exclusion process, the culture medium image including motile bacteria is excluded from the training data and the test data for additional learning of the object detection learning model 41, the image recognition learning model 42, and the like.

[0058] When the process in step S83 is completed, or when it is determined in step S81 that motile bacteria are not present (step S81: NO), or when the process in step S83 is completed, the arithmetic unit 11 determines whether the bacterial culture is completed (step S84). If it is determined that the bacterial culture is in progress (step S84: NO), the arithmetic unit 11 returns the process to step S71 and continues the process of determining the presence or absence of motile bacteria.

[0059] When it is determined that the bacterial culture has been completed (step S84: YES), the arithmetic unit 11 inputs a plurality of medium images obtained by imaging the same medium, which are medium images in which motile bacteria have not been detected, into the object detection learning model 41, thereby recognizing the position and range of the colony image and extracting the colony image from each medium image (step S85). A plurality of colony images are extracted in the process of step S85.

[0060] Next, as a provisional bacterial species identification process, regardless of the presence or absence of a new bacterial species, the arithmetic unit 11 inputs the extracted colony image into the image recognition learning model 42 to identify the bacterial species of the colony image (step S86). The activation function of the output layer 42c of the image recognition learning model 42 is softmax, and the output layer 42c outputs a numerical value indicating the probability corresponding to each of a plurality of bacterial species.

[0061] Next, the arithmetic unit 11 determines whether the extracted colony image is a colony image of an existing bacterial species or a new bacterial species (step S87). Specifically, the arithmetic unit 11 inputs the extracted colony image into the existing bacterial species confirmation learning model 5 to determine whether the colony image is a colony image of an existing bacterial species or a colony image of a new bacterial species. Note that the arithmetic unit 11 may determine whether it is a new bacterial species based on the numerical value output from the image recognition learning model 42.

[0062] Then, the arithmetic unit 11 determines whether a new bacterial species exists (step S88). When it is determined that no new bacterial species exists (step S88: NO), the arithmetic unit 11 executes a bacterial species identification process using the colony image extracted from one or more medium images in which motile bacteria have not been detected (step S89). Specifically, the arithmetic unit 11 inputs the colony image extracted from the medium image into the image recognition learning model 42 to identify the type of bacteria forming the colony. The arithmetic unit 11 executes a process of identifying the bacterial species for all colony images extracted from at least the most recently imaged medium image. Note that the type of bacteria forming the colony may be identified by inputting colony images captured at multiple time points during bacterial culture into the multi-channel image recognition learning model 42.

[0063] Next, the arithmetic unit 11 executes a process of converting the accuracy output from the image recognition learning model 42 into the probability that a specific bacterium exists in the medium (step S90). The accuracy output from the image recognition learning model 42 is a numerical value that serves as a classification index for the bacterium species identification performed by the image recognition learning model 42, and is a real number in the range from 0 to 1, but does not necessarily match the probability that the bacterium actually exists in the medium. Therefore, the arithmetic unit 11 uses a pre-stored conversion function to convert the accuracy into a probability. That is, the arithmetic unit 11 calibrates the accuracy output from the image recognition learning model 42 to the probability that the bacterium species actually exists. The conversion function is a function that converts the accuracy (the accuracy of being a colony image formed by a specific bacterium species) output from the image recognition learning model 42 into the probability that the colony is actually formed by the specific bacterium species. The conversion function may be a table associating the accuracy and the probability.

[0064] The method for creating the conversion function is as follows. First, for a plurality of colonies, a frequency distribution of the accuracy is created from the set of accuracies that the bacterium species contained in the colony belongs to a specific bacterium species. For example, a frequency distribution of the accuracy is created in intervals of 0.1. On the other hand, for a plurality of colonies having the same frequency in the same interval in the above frequency distribution, the actual probability that the bacterium species contained in the colony belongs to a specific bacterium species, that is, the probability based on the judgment of the inspection technician or the probability based on the determination result of other inspections is calculated. Then, a conversion function associating the above accuracy and the probability is determined. The conversion function uses a logistic regression function, an isotonic regression function, or the like. By using the conversion function created in this way, the probability can be calculated from the accuracy value.

[0065] In step S88, when it is determined that a new bacterial species exists (step S88: YES), the arithmetic unit 11 calculates the feature amounts of the plurality of colony images extracted in step S85 (step S91).

[0066] The method for calculating the feature amount of the colony image is not particularly limited, but it may be calculated using the image recognition learning model 42. Since the intermediate layer 42b of the image recognition learning model 42 outputs a numerical value representing the features of the colony image, the numerical value (image feature amount) calculated and output by the intermediate layer 42b may be used as the feature amount. Further, a vector composed of the probabilities output from the output layer 42c of the image recognition learning model 42 may be used as the feature amount. Since the probabilities corresponding to each of the plurality of existing bacterial species are output from the output layer 42c, a vector having the dimension of the number of existing bacterial species can be obtained. Also, if the probability output from the output layer 42c is binarized using an appropriate threshold value to 0 or 1 and used as a component of the vector, the feature amount representing similar bacterial species in a hot vector is obtained. Furthermore, the feature amount of the colony image may be calculated using a learning model (not shown) that outputs an embedded vector obtained by dimension compression of the features of the colony image, for example, the feature amount of the colony image.

[0067] Then, the computing unit 11 clusters the plurality of colony images based on the feature amounts of the plurality of extracted colony images and the feature amounts of the colony images of the existing bacterial species identified and confirmed in steps S86 and S87 (step S92). This clustering process is a semi-supervised clustering process that uses the feature amounts of the colony images of the existing bacterial species with the bacterial species information labeled as teacher data. Since the bacterial species information is labeled for some of the feature amounts, the feature amounts of the plurality of colony images are more accurately clustered. The feature amounts labeled with the information of the same bacterial species are classified into the same class, and the feature amounts labeled with the information of different bacterial species are classified into different classes. There is a high possibility that the plurality of colony images include colony images of new bacterial species for which the bacterial species could not be identified in steps S86 and S87. Such colony images are classified into classes different from those of the existing bacterial species when their feature amounts are approximate to each other and not approximate to the feature amounts of the classes of the existing bacterial species. The new bacterial species may be of one type or multiple types, and depending on the features of the colony images, the colony images of the new bacterial species are classified into one or more classes. The colony images of the existing bacterial species are usually classified into one or more classes for each bacterial species.

[0068] Note that the method of clustering is not particularly limited, but it is advisable to perform semi-supervised clustering on the feature amounts of the colony images using a mixture distribution model such as K-means or the EM algorithm, tSNE, PCA, etc. Also, self-supervised learning (contrastive learning such as DeepCluster or Sela) may be used to cluster the feature amounts of the colony images.

[0069] Next, the arithmetic unit 11 inputs the colony images of each class that have been clustered into the image recognition learning model 42 to identify the types of bacteria that form the colonies or to identify novel bacterial species (step S93). For a colony image of a class corresponding to an existing bacterial species, one bacterial species can be identified based on the accuracy output from the image recognition learning model 42. When it is a colony image of a class corresponding to a novel bacterial species, one bacterial species cannot be uniquely identified based on the accuracy output from the image recognition learning model 42, and it is identified as a novel bacterial species. Here too, a colony image considered to be a novel bacterial species may be input into the existing bacterial species confirmation learning model 5 to reconfirm that it is a novel bacterial species.

[0070] Next, the arithmetic unit 11 specifies the number of novel bacterial species based on the clustering processing result of step S92 and the bacterial species identification result of step S93 (step S94). By clustering the feature amounts of the colony images, the number of clustered groups corresponds to the total number of bacterial species present in the medium. Note that the parameters during clustering can be adjusted so that the number of clustered groups corresponds to the total number of bacterial species present in the medium. On the other hand, the number of bacterial species that can be identified by the process of step S93 corresponds to the number of existing bacterial species present in the medium. The arithmetic unit 11 can specify the number of novel bacterial species by subtracting the number of existing bacterial species from the total number of bacterial species present in the medium. The arithmetic unit 11 may automatically assign a provisional bacterial species name or bacterial species group name to the specified novel bacterial species and the colony images of the novel bacterial species.

[0071] Note that the above vector, which is a feature amount of the colony image, may be configured to be clustered using the K-means, EM algorithm, etc. and identified as a novel bacterial species when classified into a cluster different from the existing bacterial species.

[0072] Next, the arithmetic unit 11 calculates the reliability of the determination that a specific colony image is a colony image of a new bacterial species (step S95). For example, the arithmetic unit 11 may calculate the reciprocal of the distance between the feature amount of a specific colony image and the center of each cluster as the reliability or similarity. More specifically, the kernel density estimate value and the reciprocal of the local outlier factor (LOF) may be calculated as the reliability of being a new bacterial species. The arithmetic unit 11 may also be configured to calculate, as the reliability, the reciprocal of a known degree of abnormality representing the degree of deviation from each cluster or the similarity.

[0073] In addition, the arithmetic unit 11 performs a process of converting the accuracy output from the image recognition learning model 42 for the colony image of the existing bacterial species into the probability that a specific bacterium exists in the medium (step S96).

[0074] Next, the arithmetic unit 11 executes a registration process for the new bacterial species (step S97). An inspection technician can visually confirm the official bacterial species name of the new bacterial species or by using a bacterial test kit or the like, and input the new bacterial species name into the information processing apparatus 1 at the operation unit 14. The arithmetic unit 11 registers information about the new bacterial species, such as the official bacterial species name input by the operation of the operation unit 14. Information such as the medium image, colony image, and bacterial species name including the new bacterial species can be used for additional learning or improvement of the bacterial species identification learning model 4.

[0075] The arithmetic unit 11 outputs, as processing results, the colony image of the new bacterial species, the number of new bacterial species, other information about the new bacterial species, and the reliability of the determination that it is a new bacterial species in the process of step S54 described in Embodiment 1. The arithmetic unit 11 also outputs, as a processing result, the probability that the identified existing bacterial species exists in the medium.

[0076] Note that the information processing apparatus 1 may be configured to pre-store in the storage unit 13 combinations of bacterial species grown by various specimens and media, and determine the validity of the identification result of the bacterial species based on the identification result of the bacterial species and the combinations of bacterial species stored in the storage unit 13. The validity can be calculated based on, for example, the similarity, statistical distance, outlier, etc. between the identified combination of bacterial species and the combination of bacterial species stored in the storage unit 13. For example, for a vector having the dimension of the number of bacterial species, the vector components are set to represent the presence or absence of each bacterial species, the inner product of the vectors is the similarity, and the absolute value of the difference between the vectors represents the Euclidean distance. At this time, when the dimensions of the number of bacterial species do not match, the missing bacterial species components of the vector with fewer bacterial species are interpolated with zeros. The calculation unit 11 may be configured to determine the validity of the identification result and the content of the newly registered bacterial species based on the combination of the identified bacterial species and the newly registered bacterial species and the combination of bacterial species stored in the storage unit 13.

[0077] After finishing the bacterial species identification process, the calculation unit 11 executes the result output and correction process of the bacterial species identification (step S54). For example, the calculation unit 11 displays on the display unit 15 the medium image, the center position and image range of the colony region detected by the process of step S53, the type of bacteria forming the colony, etc. Further, the calculation unit 11 accepts the correction of the inspection result at the operation unit 14, and changes and stores the center position and image range of the colony region, the type of bacteria, etc.

[0078] Then, as shown in FIG. 9, the calculation unit 11 associates the medium image with the appropriately corrected detection result and stores it in the storage unit 13 (step S55), and ends the process. For example, the storage unit 13 stores in association with each other the medium image obtained by imaging the medium, the information indicating the center position and image range of the colony region included in the medium image, and the information indicating the type of bacteria forming the colony. The information serves as training data and test data for additional learning of the object detection learning model 41, the image recognition learning model 42, and the existing bacterial species confirmation learning model 5.

[0079] <Information Processing Method: Additional Learning, Evaluation, and Update Processing of Learning Models> Next, a method for additional learning, evaluation, and update of the bacterial species identification learning model 4 will be described. First, the arithmetic unit 11 of the information processing apparatus 1 reads the stored data such as the culture medium image and the detection result stored in the storage unit 13 (step S61), and based on the read stored data, creates training data and test data for additional learning (step S62). If it is training data for the object detection learning model 41, the arithmetic unit 11 creates training data and test data in which the culture medium image is associated with information indicating the center position and the image range of the colony region included in the culture medium image. If it is training data for the image recognition learning model 42, the arithmetic unit 11 creates training data and test evaluation data in which the colony image is associated with information indicating the type of bacteria that formed the colony.

[0080] Next, the arithmetic unit 11 uses the created training data to perform additional machine learning on the bacterial species identification learning model 4, such as the object detection learning model 41 and the image recognition learning model 42 (step S63). Note that the storage unit 13 holds the learning model before the additional learning.

[0081] Then, the arithmetic unit 11 evaluates the bacterial species identification learning model 4 in order to determine whether to update the bacterial species identification learning model 4 before the additional learning to the bacterial species identification learning model 4 after the additional learning.

[0082] The arithmetic unit 11 calculates an evaluation index based on the information output from the bacterial species identification learning model 4 by inputting the test culture medium image or the colony image into the bacterial species identification learning model 4 before the additional learning, the information output from the bacterial species identification learning model 4 by inputting it into the bacterial species identification learning model 4 after the additional learning, and the information related to the clinical importance (step S64).

[0083] For example, the arithmetic unit 11 refers to the weight coefficient table 6 and calculates an evaluation index E represented by the following formula (1). In the evaluation index E represented by the following formula (1), when the value of pi is larger and the correct answer rate is higher, the smaller the value of the evaluation index E, the better the evaluation. E = Σwi(1 - pi) = Σwi × qi…(1) However, E: Evaluation index wi: Weight coefficient with values varying according to the type of bacteria Subscript i: Value indicating multiple types of bacteria, different values for each bacterial species pi: Correct rate, etc. when identifying the bacterial species represented by subscript i qi: Error rate, etc. when identifying the bacterial species represented by subscript i

[0084] The value of the weight coefficient wi is larger as the bacteria have a greater severity of impact on the human body. For example, the weight coefficient wi for pathogenic bacteria with drug resistance is a larger value than the weight coefficient wi for pathogenic bacteria without drug resistance and having a similar genomic sequence. Also, the weight coefficient wi for pathogenic bacteria (non-resident bacteria) is a larger value than the weight coefficient wi for resident bacteria. Specifically, the weight coefficient wi for pathogenic bacteria with drug resistance is 10, the weight coefficient wi for pathogenic bacteria without drug resistance is 5, and the weight coefficient wi for resident bacteria is 1. By setting the weight coefficient in this way, a greater weight is given to the evaluation of the identification accuracy of drug-resistant bacteria compared to the evaluation of the identification accuracy of non-drug-resistant bacteria. As a result, an evaluation index is calculated that is more strongly influenced by the identification accuracy of drug-resistant bacteria compared to non-drug-resistant bacteria. Also, a greater weight is given to the evaluation of the identification accuracy of pathogenic bacteria compared to the evaluation of the identification accuracy of resident bacteria, and an evaluation index is calculated that is more strongly influenced by the identification accuracy of pathogenic bacteria compared to resident bacteria.

[0085] Note that although the correct rate was described as the value pi indicating the general evaluation of the bacterial species identification learning model 4, it is not particularly limited to this, and sensitivity, specificity, positive likelihood ratio, ROAUC, PR-AUC, precision, recall, F-value, etc. may also be used.

[0086] Also, the following formula (2) may be used as qi.

Number

[0087] Note that NRI linearly adds (with a coefficient of 1) the evaluation indicators so that the value increases as the number of positive test results increases in the case of true positives and the value increases as the number of negative test results increases in the case of true negatives, and calculates the difference before and after the update of the prediction model.

[0088] Also, the following formula (3) may be used as qi. NB(v)=TP(v)-{v / (1-v)}FP(v)…(3) However, TP: True positive rate FP: False positive rate v: Predetermined threshold

[0089] Note that the threshold v is a value set in advance in the information processing apparatus 1 and stored in the storage unit 13. If you want to increase the sensitivity, set v to a low value (a value close to 0), and if you want to increase the specificity, set v to a high value (a value close to 1). Since the value of v is directly related to the prediction error rate, in practice, the threshold v is adjusted by setting it to a relatively high error rate that the inspection site can tolerate (tolerating retesting or the associated costs and delivery times) or setting it to a relatively low error rate to avoid losses due to retesting.

[0090] Furthermore, instead of the above evaluation indicator E, the evaluation indicator represented by the following formula (4) may be used to evaluate the bacterial species discrimination learning model 4.

Number

[0091] Note that the weight coefficient w k , w 1k , w 2k , w 3k , w 4k is the same as the weight coefficient wi, and the greater the severity of the impact on the human body, the greater the value.

[0092] After finishing the process of step S64, the arithmetic unit 11 similarly inputs an image of the test medium or an image of colonies into the bacterial species discrimination learning model 4 after additional learning, and calculates an evaluation index based on the information output from the bacterial species discrimination learning model 4 and the information related to the clinical importance (step S65). Note that the order of the processes of step S64 and step S65 is not particularly limited.

[0093] Then, the arithmetic unit 11 determines whether the evaluation index of the bacterial species discrimination learning model 4 after additional learning has improved compared to the evaluation index of the bacterial species discrimination learning model 4 before additional learning (step S66). If it is determined that the evaluation index has not improved (step S66: NO), the arithmetic unit 11 ends the process without updating the bacterial species discrimination learning model 4.

[0094] If it is determined that the evaluation index has improved (step S66: YES), the bacterial species discrimination learning model 4 before additional learning is updated to the bacterial species discrimination learning model 4 after additional learning (step S67). Thereafter, the arithmetic unit 11 executes processes such as bacterial discrimination using the bacterial species discrimination learning model 4 after additional learning.

[0095] According to the information processing apparatus 1 configured as described above, it is possible to determine the presence or absence of motile bacteria contained in the medium during bacterial culture.

[0096] In addition, when motile bacteria are detected, the information processing apparatus 1 notifies the presence of the motile bacteria, so that the operator can detect the motile bacteria at an early stage. By detecting the motile bacteria at an early stage, an inspection technician or a colony picker device can pick the bacteria and separately culture the motile bacteria and the bacteria other than the motile bacteria. Further, if necessary, the bacterial culture can be repeated, and an increase in inspection costs and an extension of the TAT (Turn Around Time) can be suppressed.

[0097] Furthermore, the information processing apparatus 1 can exclude the medium images containing motile bacteria and store the medium images and inspection results. Therefore, it is possible to create training data or test data for additional learning based on the medium images or colony images that do not contain motile bacteria. Thus, the bacterial species identification learning model 4 can be appropriately additionally learned.

[0098] Furthermore, when performing additional learning of the bacterial species identification learning model 4, the bacterial species identification learning model 4 can be evaluated by an evaluation index that takes into account clinical utility, and the bacterial species identification learning model 4 can be automatically updated.

[0099] Furthermore, even when a new bacterial species is included in the medium, the bacterial species of the colony images of the existing bacterial species can be identified, and the colony images of the new bacterial species can be specified. Since the new bacterial species are specified based on the medium images that do not include motile bacteria recognition, the colony images of the new bacterial species can be specified more accurately.

[0100] Furthermore, since it is configured to specify the colony images of the new bacterial species by semi-supervised clustering processing, the existing bacterial species and the new bacterial species can be more accurately identified and specified.

[0101] Furthermore, the number of the identified new bacterial species can be specified. And kana is assigned to the colony images of the new bacterial species. An operator can newly register a new bacterial species by identifying the type of bacteria forming the colony images specified as the new bacterial species and assigning a bacterial species name. The information of the registered new bacterial species can be used for updating the bacterial species identification learning model 4 and the existing bacterial species confirmation learning model 5.

[0102] Furthermore, it is possible to output the reliability of the determination result that a specific colony image is a colony of a new bacterial species.

[0103] Regarding the existing bacterial species, instead of the accuracy output from the image recognition learning model 42, it can be converted into the probability that the existing bacterial species actually exists in the medium and output.

[0104] In addition, in this embodiment, an example of determining the presence or absence of motile bacteria and identifying the bacterial species using a plurality of medium images obtained by imaging at different time points during bacterial culture has been described. However, it may be configured to determine the presence or absence of motile bacteria and identify the bacterial species using a plurality of medium images captured at different time points and under imaging conditions. Further, it may be configured to determine the presence or absence of motile bacteria and identify the bacterial species using a plurality of medium images obtained by imaging a medium in which bacteria are cultured using a plurality of different types of media.

[0105] (Embodiment 2) The information processing apparatus 1 and the like according to Embodiment 2 have a processing procedure for identifying a new bacterial species that is different from that of Embodiment 1. Since the other configurations of the information processing apparatus 1 are the same as those of the information processing apparatus 1 according to Embodiment 1, the same reference numerals are given to the same parts, and detailed descriptions thereof are omitted.

[0106] It is assumed that the storage unit 13 of the information processing apparatus 1 according to Embodiment 2 stores the feature amounts of the colony images of existing bacteria as information for executing seed clustering. For example, the storage unit 13 stores the feature amounts of colony images with known bacterial species used for the learning of the bacterial species identification learning model 4. Information on the bacterial species is labeled in the feature amounts.

[0107] FIG. 13 is a flowchart showing the processing procedure for bacterial species identification according to Embodiment 2. The arithmetic unit 11 executes processing such as determining the presence or absence of motile bacteria by executing the processing of steps S71 to S85 in the same manner as in Embodiment 1, and extracts a plurality of colony images from a plurality of medium images in which motile bacteria have not been detected.

[0108] After finishing the processing of step S85, the arithmetic unit 11 calculates the feature amounts of the plurality of colony images extracted in step S85 (step S286).

[0109] Next, the arithmetic unit 11 reads out the feature amounts of the colony images of the existing bacteria from the storage unit 13, and performs seed clustering on the plurality of colony images based on the feature amounts of the plurality of colony images extracted in step S286 and the feature amounts of the colony images of the existing bacterial species read out from the storage unit 13 (step S287). This clustering process is a semi-supervised clustering process that uses the feature amounts of the colony images of the existing bacterial species with the information of the bacterial species labeled as teacher data.

[0110] By inputting the colony images of each class after clustering into the image recognition learning model 42, the type of bacteria forming the colony is identified, or a new bacterial species is specified (step S288). This is the same process as step S93. Here too, a colony image considered to be a new bacterial species may be configured to be reconfirmed as a new bacterial species by inputting it into the existing bacterial species confirmation learning model 5.

[0111] Next, the arithmetic unit 11 determines whether a new bacterial species exists (step S289). If it is determined that no new bacterial species exists (step S289: NO), the arithmetic unit 11 executes a process of converting the output value (accuracy) output from the image recognition learning model 42 in the bacterial species identification process of step S288 into the probability that a specific bacterium exists in the culture medium (step S290).

[0112] If it is determined that a new bacterial species exists (step S289: YES), the arithmetic unit 11 executes the same processes as steps S94 to S97 of Embodiment 1 in steps S291 to S294 to specify the number of new bacterial species, calculate the reliability of the new bacterial species, calculate the existence probability of the existing bacterial species, and execute the registration process of the new bacterial species.

[0113] According to the information processing apparatus 1 and the like according to Embodiment 2, since the configuration is such that new bacterial species are specified by seed clustering, the existing bacterial species can be identified and the colony images of the new bacterial species can be specified with higher accuracy than in Embodiment 1.

[0114] (Appendix 1) A computer program for causing a computer to execute an update process of a learning model that outputs information for identifying the type of bacteria contained in a medium when a medium image obtained by imaging a bacterially cultured medium is input, performing additional learning on the learning model, calculating an evaluation index based on information output from the learning model by inputting a test medium image into the learning model before additional learning and information related to clinical importance, calculating an evaluation index based on information output from the learning model by inputting a test medium image into the learning model after additional learning and information related to clinical importance, updating the learning model before additional learning to the learning model after additional learning according to a comparison result between the evaluation index of the learning model after additional learning and the evaluation index of the learning model before additional learning A computer program for causing the computer to execute the process. (Appendix 2) The evaluation index is calculated by assigning a greater weight to the evaluation regarding the identification accuracy of drug-resistant bacteria than to the evaluation regarding the identification accuracy of non-drug-resistant bacteria. The computer program according to Appendix 1. (Appendix 3) The evaluation index is calculated by assigning a greater weight to the evaluation regarding the identification accuracy of opportunistic bacteria than to the evaluation regarding the identification accuracy of commensal bacteria. The computer program according to Appendix 1 or Appendix 2 for causing the computer to execute the process. (Appendix 4) An information processing method for updating a learning model that outputs information for identifying the type of bacteria contained in a medium when a medium image obtained by imaging a bacterially cultured medium is input, performing additional learning on the learning model, calculating an evaluation index based on information output from the learning model by inputting a test medium image into the learning model before additional learning and information related to clinical importance, An evaluation index is calculated based on information output from the learning model after additional learning by inputting a culture medium image for testing into the learning model, and information related to clinical importance. Update the learning model before additional learning to the learning model after additional learning according to the comparison result between the evaluation index of the learning model after additional learning and the evaluation index of the learning model before additional learning. Information processing method. (Appendix 5) An information processing apparatus for updating a learning model that outputs information for identifying the type of bacteria contained in a culture medium when a culture medium image obtained by imaging a bacterially cultured culture medium is input, A learning processing unit for additional learning of the learning model, A first evaluation index calculation unit that calculates an evaluation index based on information output from the learning model by inputting a culture medium image for testing into the learning model before additional learning and information related to clinical importance, A second evaluation index calculation unit that calculates an evaluation index based on information output from the learning model by inputting a culture medium image for testing into the learning model after additional learning and information related to clinical importance, An update processing unit that updates the learning model before additional learning to the learning model after additional learning according to the comparison result between the evaluation index calculated by the second evaluation index calculation unit and the evaluation index calculated by the first evaluation index calculation unit An information processing apparatus comprising the above.

Explanation of Signs

[0115] 1 Information processing apparatus 2 Motile bacteria recognition learning model 3 Migration recognition learning model 31 Object detection learning model 32 Image recognition learning model 4 Bacterial species identification learning model 41 Object detection learning model 42 Image recognition learning model 5 Existing bacterial species confirmation learning model 6 Weight coefficient table 10 Recording medium 11 Calculation unit 12 Memory 13 Storage unit 14 Operation unit 15 Display unit 16 Notification unit 17 Acquisition unit 131 Computer program

Claims

1. Obtain a medium image obtained by imaging a medium containing a plurality of colonies formed by a plurality of types of cultured bacteria, Extract a plurality of colony images included in the obtained medium image, Based on the feature amounts of the extracted plurality of colony images and the feature amounts of the colony images of existing bacterial species whose bacterial species can be identified, cluster the extracted plurality of colony images, Based on the colony images belonging to each class after clustering, identify the bacterial species forming the colonies, and specify the colony images of the classes for which the bacterial species could not be identified as colony images of new bacterial species A computer program for causing a computer to execute the processing, When a colony image is input, by inputting each of the extracted plurality of colony images into an image recognition learning model that outputs information indicating the type of bacteria forming the colony, tentatively identify the bacterial species forming the colony of each colony image, When a colony image is input, by inputting each of the extracted plurality of colony images into an existing bacterial species confirmation learning model that outputs information indicating whether the bacterial species forming the colony image is a predetermined existing bacterial species, determine whether the bacterial species forming the colony of each colony image is a predetermined existing bacterial species, Using the feature amount of the colony image determined to be the predetermined existing bacterial species and for which the bacterial species has been identified as the feature amount of the colony image of the existing bacterial species, cluster the plurality of colony images extracted from the medium image A computer program for causing the computer to execute the processing.

2. Obtain a predetermined colony image stored in the storage unit, Using the feature amount of the colony image obtained from the storage unit as the feature amount of the colony image of the existing bacterial species, cluster the plurality of colony images extracted from the medium image The computer program according to claim 1 for causing the computer to execute the processing.

3. Specify the number of new bacterial species by subtracting the number of classes of colony images that could be identified from the number of classes after clustering The computer program according to claim 1 or claim 2 for causing the computer to execute the processing.

4. When a colony image is input, by inputting the colony image extracted from the medium image into an image recognition learning model that outputs information indicating the bacterial species forming the colony, the bacterial species forming the colony of the colony image is identified. The computer program according to any one of claims 1 to 3 for causing the computer to execute the process.

5. Based on the feature amount of a specific colony image determined to be a new bacterial species and the feature amounts of other colony images of the class to which the colony image belongs, calculate the reliability that the specific colony image is a colony image of a new bacterial species. The computer program according to any one of claims 1 to 4 for causing the computer to execute the process.

6. The arithmetic unit of the computer acquires a medium image obtained by imaging a medium containing a plurality of colonies formed by a plurality of types of cultured bacteria. The arithmetic unit extracts a plurality of colony images included in the acquired medium image. The arithmetic unit clusters the extracted plurality of colony images based on the feature amounts of the extracted plurality of colony images and the feature amounts of colony images of existing bacterial species that can be identified by bacterial species. The arithmetic unit identifies the bacterial species forming the colony based on the colony images belonging to each class after clustering, and specifies the colony images of the class for which the bacterial species could not be identified as colony images of a new bacterial species. An information processing method, When a colony image is input, the arithmetic unit tentatively identifies the bacterial species forming the colony of each colony image by inputting each of the extracted plurality of colony images into an image recognition learning model that outputs information indicating the type of bacteria forming the colony. When a colony image is input, the arithmetic unit determines whether the bacterial species forming the colony of each colony image is a predetermined existing bacterial species by inputting each of the extracted plurality of colony images into an existing bacterial species confirmation learning model that outputs information indicating whether the bacterial species forming the colony image is a predetermined existing bacterial species. The arithmetic unit uses the feature amount of the colony image determined to be the predetermined existing bacterial species and for which the bacterial species has been identified as the feature amount of the colony image of the existing bacterial species, and clusters the plurality of colony images extracted from the medium image. Information processing method.

7. An acquisition unit that acquires a medium image obtained by imaging a medium containing a plurality of colonies formed by a plurality of types of cultured bacteria, An arithmetic unit, and is provided with, The arithmetic unit, extracts a plurality of colony images included in the acquired medium image, The arithmetic unit clusters the extracted plurality of colony images based on the feature amounts of the extracted plurality of colony images and the feature amounts of the colony images of existing bacterial species that can be identified by bacterial species, The arithmetic unit identifies the bacterial species forming the colonies based on the colony images belonging to each of the clustered classes, and specifies the colony images of the classes for which the bacterial species could not be identified as the colony images of new bacterial species, An information processing apparatus configured to execute the process, The arithmetic unit, When a colony image is input, by inputting each of the extracted plurality of colony images to an image recognition learning model that outputs information indicating the type of bacteria forming the colony, tentatively identify the bacterial species forming the colonies of each colony image, When a colony image is input, by inputting each of the extracted plurality of colony images to an existing bacterial species confirmation learning model that outputs information indicating whether or not the bacterial species forming the colony image is a predetermined existing bacterial species, determine whether or not the bacterial species forming the colonies of each colony image is a predetermined existing bacterial species, Using the feature amount of the colony image determined to be the predetermined existing bacterial species and for which the bacterial species has been identified as the feature amount of the colony image of the existing bacterial species, cluster the plurality of colony images extracted from the medium image, Information processing apparatus.

Images