Computer program, information processing method, learning model generation method, and information processing device

Abstract

To provide information processing methods capable of determining the presence or absence of migratory bacteria that can be contained in a medium during bacterial culture.SOLUTION: A medium during bacterial culture is imaged to obtain a medium image, and the presence or absence of migratory bacteria that may be contained in the medium is determined based on the acquired medium image.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a computer program, an information processing method, a learning model generation method, and an information processing device. [Background technology]

[0002] Infectious diseases are caused by bacteria and other microorganisms entering the body. In medical institutions such as hospitals and testing centers, specimens suspected to contain the causative bacteria of the infection are collected from patients suspected of having an infectious disease, cultured in a growth medium on a petri dish, and then tested to identify the bacteria contained in the specimen.

[0003] Patent Document 1 discloses a system for capturing images of bacterial colonies formed on a culture medium and identifying the bacteria that form the colonies based on the captured images. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-135240 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if a specimen contains motile bacteria, determining the bacterial species can be difficult. During bacterial culture, motile bacteria often spread onto other bacterial colonies, making it difficult to isolate the causative bacteria. This can make it impossible to correctly identify the bacterial species from the colony image. In such cases, the specimen must be cultured and the test redone. Conventionally, selective isolation culture has been performed by adding medium components (such as bile salts) that suppress migration to identify bacterial species, but this increases the cost of the culture medium. Also, differential isolation culture has been performed using a differential medium (such as a chromogenic enzyme substrate medium) to identify bacterial species, but this increases the cost of the enzyme.

[0006] An object of the present invention is to provide a computer program, an information processing method, a learning model generation method, and an information processing device that can determine the presence or absence of migratory bacteria that may be contained in a medium during bacterial culture. [Means for solving the problem]

[0007] The computer program according to this embodiment acquires a medium image by photographing a medium during bacterial culture, and causes the computer to execute a process of determining the presence or absence of migratory bacteria that may be contained in the medium based on the acquired medium image.

[0008] The information processing method according to this aspect acquires a medium image obtained by photographing a medium during bacterial culture, and determines the presence or absence of migratory bacteria that may be contained in the medium based on the acquired medium image.

[0009] The learning model generation method according to this embodiment acquires training data that associates a medium image obtained by photographing a medium during bacterial culture with information indicating the presence or absence of migratory bacteria that may be contained in the medium, and generates, through machine learning using the acquired training data, a migratory bacteria recognition learning model that outputs the presence or absence of migratory bacteria when a single medium image obtained by photographing a medium during bacterial culture is input.

[0010] The learning model generation method according to this embodiment acquires training data that associates multiple culture medium images obtained by photographing a culture medium at multiple time points corresponding to multiple different culture times during bacterial culture with information indicating the presence or absence of migratory bacteria that may be contained in the culture medium, and generates a migration recognition learning model that outputs the presence or absence of migratory bacteria when multiple culture medium images photographed at multiple different time points during bacterial culture are input through machine learning using the acquired training data.

[0011] The information processing device according to this aspect includes an acquisition unit that acquires a medium image by capturing an image of a medium during bacterial culture, and a determination unit that determines the presence or absence of migratory bacteria based on the acquired medium image. [Effects of the Invention]

[0012] According to the above, it is possible to determine whether or not migratory bacteria are present in a medium during bacterial culture. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of an information processing device according to a first embodiment. [Figure 2] FIG. 1 is a conceptual diagram showing an example of the configuration of a migratory bacteria recognition learning model. [Figure 3] 1 is a flowchart showing a method for generating a learning model for recognizing migratory bacteria. [Figure 4] FIG. 1 is a conceptual diagram showing an example of the configuration of a migration recognition learning model. [Figure 5] 1 is a flowchart illustrating a method for generating a migration recognition learning model. [Figure 6] FIG. 1 is an explanatory diagram showing a method for generating a migration recognition learning model. [Figure 7] FIG. 1 is a conceptual diagram showing an example of the configuration of a bacterial species identification learning model. [Figure 8] 10 is a flowchart showing a processing procedure for bacterial species identification and model update. [Figure 9] 1 is a flowchart showing a processing procedure for identifying bacterial species according to the first embodiment. [Figure 10] 1 is a flowchart showing a processing procedure for identifying bacterial species according to the first embodiment. [Figure 11] 10 is a flowchart showing a processing procedure for identifying bacterial species according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Specific examples of a computer program, an information processing method, a learning model generation method, and an information processing device according to embodiments of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. Furthermore, at least some of the embodiments described below may be combined in any manner.

[0015] (Embodiment 1) The information processing device according to the first embodiment is a device for identifying the species of bacteria contained in a specimen collected from a patient. The specimen collected from the patient is cultured in a growth medium on a petri dish. Bacterial colonies form in the culture medium. The information processing device acquires an image of the culture medium (hereinafter referred to as a culture medium image), and executes a process for identifying the type (species) of bacteria forming the colony based on the acquired culture medium image. However, if the specimen contains migratory bacteria, it becomes difficult to identify the species of bacteria. The information processing device of this embodiment 1 is a device that can improve the accuracy of identifying bacterial species by detecting such migratory bacteria, and can make bacterial testing more efficient by early detection of migratory bacteria or mechanical automatic detection. In addition, the information processing device according to this embodiment 1 is a device that performs bacterial species determination using a learning model and can accumulate images of culture media or colonies that do not contain migratory bacteria as training data or test data for additional learning. Furthermore, even if a novel bacterial species is contained in the culture medium, the device can appropriately identify the bacterial species and specify a colony image of the novel bacterial species. Furthermore, the information processing device according to the first embodiment is a device that can evaluate and update a learning model using an evaluation index that takes into account clinical usefulness, rather than commonly used statistical evaluation indexes such as sensitivity, specificity, positive predictive value, ROAUC, and PR-AUC, when additional learning of the learning model is performed.

[0016] <Configuration of information processing device> FIG. 1 is a block diagram showing an information processing device 1. The information processing device 1 is a computer such as a personal computer or a server device. The information processing device 1 includes a calculation unit 11, a memory 12, a storage unit 13, an operation unit 14, a display unit 15, a notification unit 16, and an acquisition unit 17. The information processing device 1 may be a multi-computer including multiple computers. It may also be a server-client system, a cloud server, or a virtual machine virtually constructed by software. In the following description, the information processing device 1 is described as being a single computer.

[0017] The calculation unit 11 is an arithmetic processing device such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-purpose computing on graphics processing units), or TPU (Tensor Processing Unit). The calculation unit 11 may be configured using a quantum computer. The calculation unit 11 reads and executes a computer program 131 stored in the storage unit 13, thereby implementing the information processing method according to the first embodiment, such as identifying the bacterial species contained in the culture medium image, detecting migratory bacteria, identifying new bacterial species, and updating the learning model.

[0018] The memory 12 is a volatile memory such as a DRAM (Dynamic RAM) or an SRAM (Static RAM), and temporarily stores a computer program 131 read from the storage unit 13 when the calculation unit 11 executes calculation processing, or various data generated by the calculation processing of the calculation unit 11.

[0019] The storage unit 13 is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable ROM), or a flash memory. The storage unit 13 stores various programs executed by the calculation unit 11 and various data required for the processing of the calculation unit 11. In the first embodiment, the storage unit 13 stores a computer program 131 executed by the calculation unit 11, a migratory bacteria recognition learning model 2 and a migratory recognition learning model 3 for detecting migratory bacteria from a culture medium image, a bacterial species identification learning model 4 for identifying bacterial species contained in the culture medium image, and a weighting coefficient table 5 for evaluating additional learning of the bacterial species identification learning model 4.

[0020] The computer program 131 is recorded in a computer-readable manner on, for example, the recording medium 10. The storage unit 13 stores the computer program 131 read from the recording medium 10 by a reading device (not shown). The recording medium 10 is a semiconductor memory such as a flash memory, an optical disk, a magnetic disk, a magneto-optical disk, or the like. The computer program 131 may also be stored in the storage unit 13 during the manufacturing stage of the information processing device 1. Furthermore, the computer program 131 according to the first embodiment may also be downloaded from an external server (not shown) connected to a communication network and stored in the storage unit 13.

[0021] The weighting coefficient table 5 stores information indicating the clinical importance of each of a plurality of bacteria. The information indicating the clinical importance is a weighting coefficient that has a larger value for bacteria that have a more serious effect on the human body. The weighting coefficients will be described in detail later.

[0022] The operation unit 14 is an input device that receives operations from an operator such as a medical technician, etc. The input device is, for example, a keyboard or a pointing device.

[0023] The display unit 15 is an output device that outputs information such as culture images, bacterial identification results, etc. The output device is, for example, a liquid crystal display or an EL display.

[0024] The notification unit 16 is a device that notifies the presence of migratory bacteria when migratory bacteria are detected during bacterial culture. The notification unit 16 is a lamp, a speaker, a communication device, or the like.

[0025] The acquisition unit 17 is an interface that acquires a medium image obtained by capturing an image of the medium during bacterial culture using the imaging device 9. The imaging device 9 captures images continuously or intermittently during bacterial culture and outputs the medium images to the information processing device 1. The acquisition unit 17 acquires the medium image output from the imaging device 9 each time, and the calculation unit 11 executes processing to determine the presence or absence of migratory bacteria.

[0026] <Learning model for recognizing migratory bacteria 2> Figure 2 is a conceptual diagram showing an example of the configuration of the migratory bacteria recognition learning model 2. The migratory bacteria recognition learning model 2 is an image recognition model that, when a single medium image obtained by capturing an image of a medium in which bacteria are being cultured is input, outputs the presence or absence of migratory bacteria that may be contained in the medium. Note that the single medium image may be a so-called color image or a grayscale image. A color image is an image composed of three channels: red, green, and blue. A grayscale image is an image composed of a single channel with values ​​ranging from 0 to 255, for example.

[0027] In this embodiment 1, the migratory bacteria recognition learning model 2 is a neural network having, for example, an input layer 21 that receives input of a culture medium image, an intermediate layer 22 that extracts feature quantities of the bacterial image contained in the culture medium image, and an output layer 23 that outputs information indicating the presence or absence of migratory bacteria that may be contained in the culture medium. The migratory bacteria recognition learning model 2 in this embodiment 1 is a CNN (Convolution Neural Network) such as ResNet or DenseNet, or Attention, etc. The information processing device 1 generates the migratory bacteria recognition learning model 2 by performing deep learning on the CNN model to learn the relationship between the culture medium image and the presence or absence of migratory bacteria.

[0028] The input layer 21 of the neural network has multiple nodes that accept input of pixel values ​​for each pixel constituting the culture medium image and passes the input pixel values ​​to the intermediate layer 22. The intermediate layer 22 has multiple nodes that extract features of the culture medium image and passes the extracted features to the output layer 23. For example, if the migratory bacteria recognition learning model 2 is a CNN, the intermediate layer 22 has a configuration in which multiple convolution layers that convolve the pixel values ​​of each pixel input from the input layer 21 and pooling layers that map the pixel values ​​convolved in the convolution layers are connected together, compressing the pixel information of the culture medium image and ultimately extracting the features of the culture medium image. The output layer 23 has nodes that output information indicating the presence or absence of migratory bacteria. The activation function of the output layer 23 is, for example, a sigmoid function. The information indicating the presence or absence of migratory bacteria is, for example, the probability that migratory bacteria are present.

[0029] FIG. 3 is a flowchart showing a method for generating the migratory bacteria recognition learning model 2. Here, an example is described in which the information processing device 1 generates the migratory bacteria recognition learning model 2; however, machine learning may be performed on a separate computer. The storage unit 13 stores training data that associates a single medium image obtained by capturing an image of a medium on which at least one of any bacteria and migratory bacteria has been applied with information (teaching data) indicating the presence or absence of migratory bacteria that may be contained in the medium. Needless to say, the training data includes multiple pairs of data in which a single medium image is associated with the teaching data. The medium images in the training data preferably include medium images captured at various times during bacterial culture. Furthermore, the medium images in the training data preferably include medium images containing only migratory bacteria, medium images not containing migratory bacteria, and medium images containing a mixture of migratory bacteria and bacteria. Furthermore, the medium images in the training data preferably include various medium images, such as selective medium, non-selective medium, and authentic medium.

[0030] The calculation unit 11 of the information processing device 1 acquires training data from the storage unit 13, that is, training data associating a single medium image with teacher data indicating the presence or absence of migratory bacteria (step S11). Then, the calculation unit 11 generates a migratory bacteria recognition learning model 2 by performing machine learning on an untrained neural network using the acquired training data (step S12). That is, the calculation unit 11 trains the neural network so that, when a single medium image obtained by capturing an image of a medium during bacterial culture is input, information indicating the presence or absence of migratory bacteria is output. Specifically, the parameters of the neural network are optimized so that the difference between the information indicating the presence or absence of migratory bacteria output when a single medium image serving as training data is input to the untrained neural network and the teacher data associated with the medium image in the training data is minimized. The parameters include, for example, weights (coupling coefficients) between nodes. The parameter optimization method is not particularly limited, and the calculation unit 11 optimizes various parameters using, for example, the steepest descent method.

[0031] In this embodiment, the migratory bacteria recognition learning model 2 is a CNN such as DenseNet, but the model configuration is not limited to CNN. The migratory bacteria recognition learning model 2 may be a learning model configured, for example, by a neural network other than CNN, a Vision Transformer, an SVM (Support Vector Machine), a Bayesian network, or a decision tree such as XGBoost.

[0032] <Migration Recognition Learning Model 3> Figure 4 is a conceptual diagram showing an example of the configuration of the migration recognition learning model 3. The migration recognition learning model 3 is an image recognition model that, when multiple medium images obtained by capturing images of the same medium at different times during bacterial culture (i.e., multiple different culture times) are input, outputs the presence or absence of migratory bacteria that may be contained in the medium. While the above-mentioned migration recognition learning model 2 determines the presence or absence of migratory bacteria using a single medium image, the migration recognition learning model 3 is configured as a multi-channel model that simultaneously inputs multiple medium images obtained by capturing images of the same medium at multiple different culture times, and determines the presence or absence of migratory bacteria using the multiple medium images. A distinctive feature of the migration recognition learning model 3 is that multiple medium images derived from the same sample are included in the same batch and input. The migration recognition learning model 3 is useful when the bacterial growth area is small. Detailed explanations of the configuration common to the migration recognition learning model 2 will be omitted. The migration recognition learning model 3 includes, for example, an object detection learning model 31 for detecting the position and range of a colony image contained in a culture medium image, and an image recognition learning model 32 for recognizing the presence or absence of migratory bacteria that may be contained in the culture medium based on multiple colony images extracted from the culture medium image based on the detection results of the object detection learning model 31.

[0033] The object detection learning model 31 shown in the upper diagram of Figure 4 is an object detection model such as YOLOv3, U-Net, Faster R-CNN, or SSD that detects colony images contained in culture medium images. The object detection learning model 31 is a neural network that includes, for example, an input layer 31a that receives input of a culture medium image, a middle layer 31b that extracts features of the culture medium image, and an output layer 31c that outputs information indicating the position and range of images of colonies formed by bacteria contained in the culture medium image (hereinafter referred to as colony images). Multiple culture medium images captured at different time points during bacterial culture are sequentially input to the object detection learning model 31, and the position and range of the colony images contained in the culture medium image are detected. The position and range information of the colony images output from the object detection learning model 31 can be used to extract colony images from the culture medium image. In the migration recognition learning model 3, the object detection learning model 31 extracts multiple colony images from each of multiple culture medium images obtained by capturing images of the same culture medium at multiple different culture times.

[0034] The image recognition learning model 32 shown in the lower diagram of Figure 4 is a neural network having an input layer 32a to which multiple colony images extracted from multiple culture medium images taken at different times during bacterial culture are input, an intermediate layer 32b that extracts features of the multiple colony images, and an output layer 32c that outputs information indicating the presence or absence of migratory bacteria that may be contained in the culture medium.

[0035] The input layer 32a of the neural network has multiple nodes that accept input of pixel values ​​of each pixel constituting the colony image and passes the input pixel values ​​to the middle layer 32b. The input layer 32a receives, for example, multiple colony images captured at multiple time points and extracted from each culture medium image. The middle layer 32b has multiple nodes that extract features of the multiple colony images extracted from the culture medium images and passes the extracted features to the output layer 32c. The output layer 32c has a node that outputs information indicating the presence or absence of migratory bacteria. The activation function of the output layer 32c is, for example, a sigmoid function. The information indicating the presence or absence of migratory bacteria is, for example, the probability that migratory bacteria are present. The output layer 32c may also be configured as a softmax function to perform multi-class classification of bacteria. Although an example in which the migration recognition learning model 3 is configured with the object detection learning model 31 and the image recognition learning model 32 has been described, it may be configured with a single learning model.

[0036] FIG. 5 is a flowchart showing a method for generating the migration recognition learning model 3, and FIG. 6 is an explanatory diagram showing the method for generating the migration recognition learning model 3. Here, it is assumed that the object detection learning model 31 has been trained in advance using a known machine learning method so that it can detect colony images. The memory unit 13 stores training data that associates multiple medium images obtained by capturing images of a medium coated with at least one of any bacteria and migratory bacteria at multiple different time points during bacterial culture with information (teaching data) indicating the presence or absence of migratory bacteria that may be contained in the medium. The multiple medium images captured at multiple time points are multiple images obtained by intermittently capturing images of the same medium. Needless to say, the training data includes multiple sets of data that associate multiple medium images with teaching data.

[0037] The calculation unit 11 of the information processing device 1 acquires training data stored in the storage unit 13, i.e., training data that associates multiple culture medium images captured at multiple time points with teacher data indicating the presence or absence of migratory bacteria that may be contained in the culture medium (step S31). The calculation unit 11 then generates a migration recognition learning model 3 by performing machine learning on an untrained neural network using the acquired training data (step S32). That is, the calculation unit 11 trains the neural network so that, when multiple culture medium images captured at multiple different time points during bacterial culture are input, information indicating the presence or absence of migratory bacteria is output. Specifically, when multiple colony images extracted from multiple culture medium images, which are training data, using the object detection learning model 31 are input to the untrained neural network of the image recognition learning model 32, the parameters of the neural network are optimized so that the difference between the information indicating the presence or absence of migratory bacteria output and the teacher data associated with the culture medium images in the training data is minimized. The parameters include, for example, weights (coupling coefficients) between nodes. The method for optimizing the parameters is not particularly limited, but for example, the calculation unit 11 uses the steepest descent method or the like to optimize the various parameters.

[0038] The migration recognition learning model 3 may be trained using batch processing known as mini-batch learning, as shown in Figure 4. A single batch, which serves as training data for training the image recognition learning model 32 of the migration recognition learning model 3, includes multiple colony images derived from the same specimen. The multiple colony images are extracted from culture medium images obtained by intermittently capturing images of the culture medium of the same specimen at multiple different time points. The weight parameters of the neural network are updated using training data derived from the same specimen, enabling effective learning of image features common to the same specimen.

[0039] When the migration recognition learning model 3 is configured as a model capable of multi-class classification, training data is created that associates culture medium images captured under a plurality of different culture conditions (such as different culture media), culture medium images captured under a plurality of different imaging conditions (such as different lighting methods or imaging at different wavelengths), and culture medium images captured at a plurality of different time points during bacterial culture with information (teaching data) indicating one or more bacterial species that may be contained in the culture medium. When a plurality of culture medium images captured at a plurality of different time points during bacterial culture are input, the neural network can be trained to extract all single colony images on the culture dish by cutting out the image into a square containing one single colony, and output information indicating the probability that each of a plurality of bacteria is present in each extracted image. Specifically, the parameters of the neural network are optimized so as to minimize the difference between the probability of the existence of each of the multiple bacteria output when multiple culture medium images, which are training data, are input into an untrained neural network, and the correct bacterial species information associated with the colony image in the training data (specifically, vector data (multi-hot vector) with a binary vector component of 1 when each of the multiple bacteria is present and 0 when not present, with the total number of bacterial species as the vector dimension, and data generally referred to as the correct label).The activation function of the output layer of the neural network is a softmax function.

[0040] The migration recognition learning model 3 may also be configured using a learning model such as a recurrent neural network (RNN), LSTM, or Vision Transformer.

[0041] <Bacteria species identification learning model 4> The bacterial species identification learning model 4 includes, for example, an object detection learning model 41 for detecting the position and range of a colony image (hereinafter referred to as a colony image) contained in a culture medium image, and an image recognition learning model 42 for recognizing the type of bacteria forming the colony based on the colony image.

[0042] 7 is a conceptual diagram showing an example of the configuration of the bacterial species identification learning model 4. The bacterial species identification learning model 4 includes, for example, an object detection learning model 41 for detecting the position and range of a colony image included in a culture medium image, and an image recognition learning model 42 for recognizing the bacterial species included in the culture medium based on a colony image extracted from the culture medium image based on the detection result of the object detection learning model 41. The object detection learning model 41 shown in the upper diagram of Figure 7 is an object detection model such as YOLOv3, U-Net, Faster R-CNN, or SSD that detects colony images contained in a culture medium image. The object detection learning model 41 is a neural network that has, for example, an input layer 41a that receives input of a culture medium image, an intermediate layer 41b that extracts features of the culture medium image, and an output layer 41c that outputs information indicating the position and range of images of colonies formed by bacteria contained in the culture medium image. Note that the object detection model does not need to recognize the bacterial species of the colonies. One or more culture medium images are extracted from the culture medium image based on the position and range information output from the object detection learning model 41.

[0043] The image recognition learning model 42 shown in the lower diagram of Figure 7 is an image recognition model such as VGG, ResNet, DenseNet, or Vision Transformer that performs image recognition processing on a colony image when the colony image is input and identifies the bacterial species that form the colony image. The image recognition learning model 42 is a neural network that includes, for example, an input layer 42a that receives the input colony image, a middle layer 42b that extracts features of the colony image, and an output layer 42c that outputs information indicating the type of bacteria that form the colony. The information indicating the type of bacteria is, for example, the probability that each of multiple bacteria forms a colony. Note that when a colony image is input to the image recognition learning model 42, it is preferable to configure the model to adjust the size of the colony image to a predetermined size.

[0044] Although the example in which the bacterial species identification learning model 4 is configured by the object detection learning model 41 and the image recognition learning model 42 has been described, it may be configured by a single learning model. The image recognition learning model 42 may also be configured as a multi-channel model, so that when colony images captured at multiple points during bacterial culture are input, the model outputs information indicating the type of bacteria forming the colony. In this case, the image recognition learning model 42 may be configured using a recurrent neural network, LSTM, Vision Transformer, etc.

[0045] <Information processing method: detection of swarming bacteria and bacterial species identification processing> 8 is a flowchart showing the processing procedure for bacterial species identification and model updating. The operator of the bacterial test performs bacterial culture of a specimen collected from a patient (step S51), and starts capturing an image of the culture medium using the imaging device 9 (step S52). The specimen may contain migratory bacteria.

[0046] Next, the calculation unit 11 of the information processing device 1 acquires the culture medium image from the imaging device 9 and executes a process of identifying the type of bacteria contained in the specimen (step S53). The process of identifying bacteria includes a process of determining the presence or absence of migratory bacteria in the bacterial culture.

[0047] 9 and 10 are flowcharts showing the processing procedure for bacterial species identification according to embodiment 1. The calculation unit 11 acquires a medium image output from the imaging device 9 using the acquisition unit 17 (step S71), and stores the acquired medium image in the storage unit 13 (step S72). The imaging device 9 captures medium images continuously or intermittently during bacterial culture, and the acquisition unit 17 acquires the medium image each time it is output from the imaging device 9.

[0048] Next, the calculation unit 11 inputs the acquired single medium image into the migratory bacteria recognition learning model 2, and makes a primary determination of the presence or absence of migratory bacteria based on the information output from the migratory bacteria recognition learning model 2 (step S73). In this embodiment 1, the presence or absence of migratory bacteria is determined using a plurality of different methods, and a final determination of the presence or absence of migratory bacteria is made in the processing of step S80.

[0049] Next, the calculation unit 11 inputs multiple culture medium images acquired at different times during bacterial culture into the migration recognition learning model 3, and makes an initial determination of the presence or absence of migratory bacteria based on the information output from the migration recognition learning model 3 (step S74).

[0050] Next, the calculation unit 11 inputs each of the plurality of culture medium images acquired at different times during bacterial culture into the object detection learning model 31, and detects colony images included in the plurality of culture medium images based on information output from the object detection learning model 31 (step S75). The calculation unit 11 can recognize the position and range of the colony image in the culture medium image.

[0051] Next, the calculation unit 11 calculates the amount of colony movement (amount of position change) by comparing the positions of the colony images in the multiple culture medium images (step S76). Then, the calculation unit 11 makes a primary determination of the presence or absence of migratory bacteria based on the calculated amount of movement (step S77). Specifically, the calculation unit 11 determines whether the amount of colony movement is equal to or greater than a first threshold. If the amount of colony movement is equal to or greater than the first threshold, it is determined that migratory bacteria are present. Normally, a culture medium image contains multiple colony images, but it may be configured so that if there is even one colony whose amount of movement is equal to or greater than the first threshold, it is determined that migratory bacteria are present.

[0052] Alternatively, the presence or absence of migratory bacteria may be determined by calculating the center coordinates of colonies in multiple culture medium images and determining whether the center coordinates of colonies at the second time point have moved outside the detection range of colonies at the first time point (a time point prior to the second time point).Whether or not colonies have moved outside the detection range may be determined by determining whether the proportion of colonies present within the detection range is less than a threshold value.

[0053] Next, the calculation unit 11 calculates the area expansion rate (size change amount) of the colony by comparing the size, for example, area, of the colony images in the multiple culture medium images (step S78). Then, the calculation unit 11 makes a primary determination of the presence or absence of migratory bacteria based on the calculated area expansion rate (step S79). Specifically, the calculation unit 11 determines whether the area expansion rate of the colony is equal to or greater than a second threshold. If the area expansion rate of the colony is equal to or greater than the second threshold, it is determined that migratory bacteria are present. Normally, a culture medium image contains multiple colony images, but it may be configured such that if there is even one colony whose area expansion rate is equal to or greater than the second threshold, it is determined that migratory bacteria are present.

[0054] It is recommended to culture migratory bacteria singly on multiple culture dishes and use the lower limit of the 95% confidence interval (2.5 percentile from the lowest value) from the distribution of values ​​for the rate of area expansion of the bacterial population per unit time as the second threshold. In addition, a histogram of the area expansion rate of bacterial populations can be created for samples containing a mixture of migratory and non-migratory bacteria, and a threshold value can be set to find one that provides excellent separation ability (accuracy rate, sensitivity, specificity, etc.) for the rate at which migratory and non-migratory bacteria are contained in two groups separated by a single threshold value. The value obtained through this search can be used as the second threshold value.

[0055] Next, the calculation unit 11 comprehensively determines the presence or absence of migratory bacteria based on the primary determination results of steps S73, S74, S77, and S79 (step S80). For example, the calculation unit 11 may determine the presence or absence of migratory bacteria by logical sum. That is, if at least one of the multiple primary determination results indicates the presence of migratory bacteria, a final determination is made that migratory bacteria are present. The calculation unit 11 may also be configured to make a final determination that migratory bacteria are present if a predetermined number or more of the multiple primary determination results indicate the presence of migratory bacteria. The calculation unit 11 may also be configured to calculate a loss function including the value output from the migratory bacteria recognition learning model 2, the value output from the migratory bacteria recognition learning model 3, the difference between the colony movement amount and a first threshold, and the difference between the colony area expansion rate and a second threshold, as well as a weighted sum of these values, and to make a final determination of the presence or absence of migratory bacteria based on the value of the loss function and the value of the weighted sum.

[0056] In addition, in this embodiment, an example has been described in which the presence or absence of migratory bacteria is primarily determined using four methods, but if migratory bacteria are detected using migratory bacteria recognition learning model 2 or migratory bacteria recognition learning model 3, the processing of steps S75 to S79 ​​may be skipped. Furthermore, the order in which the four primary determinations are performed is not particularly limited. When the four primary determinations are performed in order, if a migratory bacterium is detected, the remaining primary determination processes may be skipped.

[0057] Next, if the processing in step S80 determines that migratory bacteria are present (step S81: YES), the calculation unit 11 notifies the notification unit 16 of the presence of migratory bacteria (step S82). For example, the calculation unit 11 may turn on a warning lamp, output a sound, or transmit information indicating the presence of migratory bacteria to the operator's communication terminal. This notification is made when migratory bacteria are detected during bacterial culture. This allows the operator to know the presence of migratory bacteria early. By detecting migratory bacteria early, a laboratory technician or colony picker device can pick up the bacteria and separate and culture the migratory bacteria from bacteria other than migratory bacteria. Furthermore, if necessary, the laboratory technician or other operator can redo the bacterial culture. This prevents increases in testing costs and the turn-around time (TAT), which represents the testing time, from increasing.

[0058] Next, the arithmetic device excludes medium images in which migratory bacteria have been detected from the plurality of medium images acquired from the imaging device 9 (step S83). By this exclusion process, medium images containing migratory bacteria are excluded from the targets for bacterial species identification. Furthermore, by this exclusion process, medium images containing migratory bacteria are excluded from the training data and test data for additional learning of the object detection learning model 41, the image recognition learning model 42, etc.

[0059] When the processing of step S83 is completed or when it is determined in step S81 that no migratory bacteria are present (step S81: NO), the calculation unit 11 determines whether bacterial culture is complete (step S84). When it is determined that bacterial culture is in progress (step S84: NO), the calculation unit 11 returns the processing to step S71 and continues the process of determining the presence or absence of migratory bacteria. When it is determined that bacterial culture is complete (step S84: YES), the calculation unit 11 performs bacterial species determination using one or more culture medium images in which no migratory bacteria are detected (step S85). Specifically, the calculation unit 11 inputs the culture medium image into the object detection learning model 41 to recognize the center position and image range of the colony region contained in the culture medium image. Then, the calculation unit 11 extracts colony images from the culture medium image and inputs the extracted colony images into the bacteria recognition learning model to identify the type of bacteria forming the colony. The calculation unit 11 performs the process of identifying bacterial species for all colony images extracted from the culture medium image. It is also possible to input colony images captured at multiple points during bacterial cultivation into the multi-channel image recognition learning model 42, thereby identifying the type of bacteria forming the colony.

[0060] After completing the bacterial species identification process, the calculation unit 11 outputs the results of the bacterial species identification and executes a correction process (step S54). For example, the calculation unit 11 displays the medium image, the center position and image range of the colony area detected by the process of step S53, the type of bacteria forming the colony, etc. on the display unit 15. The calculation unit 11 also accepts corrections to the test results via the operation unit 14, changes the center position and image range of the colony area, the type of bacteria, etc., and stores them.

[0061] Then, the calculation unit 11 stores the medium image and the appropriately corrected detection result in association with each other in the storage unit 13 (step S55), and ends the process. For example, the storage unit 13 stores the medium image obtained by capturing an image of the medium, information indicating the center position and image range of the colony area contained in the medium image, and information indicating the type of bacteria forming the colony, in association with each other. This information is data that serves as the basis for training data and test data for additional learning of the object detection learning model 41 and the image recognition learning model 42.

[0062] <Information processing method: Additional learning, evaluation, and update processing of learning model> Next, the methods for additional learning, evaluation, and updating of the bacterial species identification learning model 4 will be described. First, the calculation unit 11 of the information processing device 1 reads out stored data such as culture medium images and detection results stored in the memory unit 13 (step S61), and creates training data and test data for additional learning based on the read-out stored data (step S62). In the case of training data for the object detection learning model 41, the calculation unit 11 creates training data and test data in which culture medium images are associated with information indicating the center positions and image ranges of colony areas contained in the culture medium images. In the case of training data for the image recognition learning model 42, the calculation unit 11 creates training data and test evaluation data in which colony images are associated with information indicating the types of bacteria that formed the colonies.

[0063] Next, the calculation unit 11 uses the created training data to perform additional machine learning on the bacterial species identification learning model 4, such as the object detection learning model 41 and the image recognition learning model 42 (step S63). Note that the storage unit 13 holds the learning models before the additional learning.

[0064] Then, the calculation unit 11 evaluates the bacterial species discrimination learning model 4 to determine whether or not to update the bacterial species discrimination learning model 4 before the additional learning to the bacterial species discrimination learning model 4 after the additional learning.

[0065] The calculation unit 11 inputs the test culture medium image or colony image into the bacterial species identification learning model 4 before additional learning, and calculates the evaluation index based on the information output from the bacterial species identification learning model 4 and information related to clinical importance (step S64).

[0066] For example, the calculation unit 11 refers to the weighting coefficient table 5 and calculates the evaluation index E expressed by the following formula (1). If the larger the value of pi, the higher the accuracy rate, the smaller the value of the evaluation index E expressed by the following formula (1), the better the evaluation. E = Σwi(1-pi) = Σwi × qi…(1) however, E: Evaluation indicators wi: weighting coefficient that varies depending on the type of bacteria Subscript i: A value indicating multiple types of bacteria, with different values ​​for each bacterial species pi: The accuracy rate when identifying the bacterial species represented by the subscript i qi: Error rate when identifying the bacterial species represented by subscript i

[0067] The value of the weighting coefficient wi increases as the bacteria have a more severe impact on the human body. For example, the weighting coefficient wi for pathogens with drug resistance is greater than the weighting coefficient wi for pathogens without drug resistance but with a similar genome sequence. Furthermore, the weighting coefficient wi for pathogens (non-resident bacteria) is greater than the weighting coefficient wi for resident bacteria. Specifically, the weighting coefficient wi for pathogens with drug resistance is 10, the weighting coefficient wi for pathogens without drug resistance is 5, and the weighting coefficient wi for resident bacteria is 1. By setting the weighting coefficients in this way, the evaluation of the discrimination accuracy for drug-resistant bacteria is weighted more heavily than the evaluation of the discrimination accuracy for non-drug-resistant bacteria, resulting in a calculated evaluation index that is more likely to be affected by the discrimination accuracy for drug-resistant bacteria than for non-drug-resistant bacteria.Furthermore, the evaluation of the discrimination accuracy for pathogenic bacteria is weighted more heavily than the evaluation of the discrimination accuracy for resident bacteria, resulting in a calculated evaluation index that is more likely to be affected by the discrimination accuracy for pathogenic bacteria than for resident bacteria.

[0068] Although the accuracy rate has been described as the value pi indicating a general evaluation of the bacterial species identification learning model 4, it is not limited to this, and sensitivity, specificity, positive predictive value, ROAUC, PR-AUC, precision, recall, F-value, etc. may also be used.

[0069] Alternatively, the following formula (2) may be used for qi.

number

[0070] The NRI is calculated by linearly adding the evaluation index (with a coefficient of 1) so that the value increases as the number of positive tests increases in the case of true positives, and the value increases as the number of negative tests increases in the case of true negatives, and then calculating the difference before and after updating the prediction model.

[0071] Alternatively, the following formula (3) may be used for qi. NB(v)=TP(v)-{v / (1-v)}FP(v)…(3) however, TP: true positive rate FP: false positive rate v: a predetermined threshold

[0072] The threshold value v is a value that is set in advance in the information processing device 1 and is stored in the storage unit 13. If sensitivity is to be increased, v is set to a low value (close to 0), and if specificity is to be increased, v is set to a high value (close to 1). Since the value of v is directly linked to the predicted error rate, the threshold value v is adjusted by setting it to a higher error rate that the testing site can actually tolerate (to allow for retesting or the associated costs and delivery time), or by setting it to a lower error rate to avoid losses due to retesting.

[0073] Furthermore, instead of the evaluation index E, the bacterial species identification learning model 4 may be evaluated using an evaluation index expressed by the following formula (4).

number

[0074] In addition, the weighting coefficient w k ,w 1k ,w 2k ,w 3k ,w 4k As with the weighting coefficient wi, the more severe the impact of bacteria on the human body, the larger the value of

[0075] After completing the processing of step S64, the calculation unit 11 similarly inputs the test culture medium image or colony image into the bacterial species identification learning model 4 after additional learning, and calculates the evaluation index based on the information output from the bacterial species identification learning model 4 and the information related to clinical importance (step S65). Note that the processing order of steps S64 and S65 is not particularly limited.

[0076] Then, the calculation unit 11 determines whether the evaluation index of the bacterial species identification learning model 4 after the additional learning has improved compared to the evaluation index of the bacterial species identification learning model 4 before the additional learning (step S66). If it is determined that the evaluation index has not improved (step S66: NO), the calculation unit 11 ends the process without updating the bacterial species identification learning model 4.

[0077] If it is determined that the evaluation index has improved (step S66: YES), the bacterial species identification learning model 4 before the additional learning is updated to the bacterial species identification learning model 4 after the additional learning (step S67). Thereafter, the calculation unit 11 executes processing such as bacteria identification using the bacterial species identification learning model 4 after the additional learning.

[0078] According to the information processing device 1 and the like configured in this manner, it is possible to determine the presence or absence of migratory bacteria contained in a medium during bacterial culture.

[0079] Furthermore, if migratory bacteria are detected, the information processing device 1 notifies the operator of the presence of migratory bacteria, allowing the operator to detect the migratory bacteria early. Early detection of migratory bacteria allows the laboratory technician or colony picker device to pick up the bacteria and separate and culture the migratory bacteria from bacteria other than migratory bacteria. Furthermore, bacterial culture can be redone as necessary, preventing increases in testing costs and extensions of TAT (Turn Around Time).

[0080] Furthermore, the information processing device 1 can store culture medium images and test results excluding culture medium images containing migratory bacteria. Therefore, training data or test data for additional learning can be created based on culture medium images or colony images that do not contain migratory bacteria. This allows the bacterial species identification learning model 4 to perform additional learning appropriately.

[0081] Furthermore, when additional learning is performed on the bacterial species discrimination learning model 4, the bacterial species discrimination learning model 4 can be evaluated using an evaluation index that takes into account clinical usefulness, and the bacterial species discrimination learning model 4 can be automatically updated.

[0082] In this embodiment, an example has been described in which the presence or absence of migratory bacteria is determined and bacterial species are identified using multiple medium images captured at multiple different times during bacterial culture, but the presence or absence of migratory bacteria may be determined and bacterial species identified using multiple medium images captured at multiple different times and under different imaging conditions.Furthermore, the presence or absence of migratory bacteria may be determined and bacterial species identified using multiple medium images captured by capturing images of media in which bacteria have been cultured using multiple different types of media.

[0083] (Embodiment 2) The information processing device 1 etc. according to the second embodiment differs from the first embodiment in the processing procedure for bacterial species identification. The information processing device 1 according to the second embodiment can detect the presence and number of novel bacterial species in addition to the presence or absence of migratory bacteria, and calculate the reliability of the determination result that the bacterial species is novel. The other configurations of the information processing device 1 are the same as those of the information processing device 1 according to the first embodiment, so the same reference numerals are used for the same parts and detailed description will be omitted.

[0084] 11 is a flowchart showing the processing procedure for bacterial species identification according to embodiment 2. The calculation unit 11 executes the processes of steps S71 to S84 in the same manner as in embodiment 1, thereby executing processes such as determining the presence or absence of migratory bacteria.

[0085] When the cultivation is completed, the calculation unit 11 inputs the culture medium image in which no migratory bacteria were detected into the object detection learning model 41, thereby recognizing the position and range of the colony image and extracting the colony image (step S285).

[0086] Next, as a provisional bacterial species identification process, the calculation unit 11 inputs the extracted colony image into the image recognition learning model 42, regardless of whether a new bacterial species is present, to identify the bacterial species of the colony image (step S286). The activation function of the output layer 42c of the image recognition learning model 42 is softmax, and the output layer 42c outputs a numerical value indicating the probability that each of the multiple bacterial species corresponds to the colony image.

[0087] Next, the calculation unit 11 determines whether the extracted colony image is a colony image of a new bacterial species (step S287). The calculation unit 11 may determine whether the colony image is a colony image of a new bacterial species, for example, using a new bacterial species determination learning model for determining whether the colony image is a new bacterial species. The new bacterial species determination learning model is an image recognition model that, when a colony image is input, outputs information indicating whether the colony image is an existing bacterial species that has been learned by the bacterial species identification learning model 4. The activation function of the output layer of the new bacterial species determination learning model is a sigmoid function. The calculation unit 11 may determine whether or not the bacterial species is a novel one based on a numerical value output from the image recognition learning model 42.

[0088] Then, the calculation unit 11 determines whether or not a new bacterial species is present (step S288). If it is determined that a new bacterial species is not present (step S288: NO), the calculation unit 11 performs bacterial species determination using one or more culture medium images in which no migratory bacteria are detected, in the same process as in embodiment 1 (step S289).

[0089] Next, the calculation unit 11 executes a process of converting the output value (accuracy) output from the image recognition learning model 426 into the probability that a specific bacterium is present in the culture medium (step S290). The accuracy output from the image recognition learning model 426 is a numerical value that serves as a classification index for the bacterial species identification performed by the image recognition learning model 42, and is a real number ranging from 0 to 1. However, it does not necessarily correspond to the probability that the specific bacterium actually exists in the culture medium. Therefore, the calculation unit 11 converts the accuracy into a probability using a pre-stored conversion function. In other words, the calculation unit 11 calibrates the accuracy output from the image recognition learning model 42 to the probability that the specific bacterial species actually exists. The conversion function is a function that converts the accuracy output from the image recognition learning model 42 (the accuracy that the colony image is formed by a specific bacterial species) into the probability that the colony is actually formed by the specific bacterial species. The conversion function may be a table that associates accuracy with probability.

[0090] The conversion function is created as follows. First, a frequency distribution of probabilities is created for multiple colonies from a set of probabilities that the bacterial species contained in the colonies belongs to a specific bacterial species. For example, a frequency distribution of probabilities is created in intervals of 0.1. Meanwhile, for multiple colonies that have frequencies in the same interval in the frequency distribution, the actual probability that the bacterial species contained in the colony belongs to a specific bacterial species is calculated, that is, the probability based on the judgment of a laboratory technician or the confirmed results of other tests. Then, a conversion function that relates the above probabilities to the probabilities is determined. The conversion function may be a logistic regression function, an isotonic regression function, or the like. By using the conversion function created in this way, it is possible to calculate the probability from the probability value.

[0091] If it is determined in step S288 that a new bacterial species is present (step S288: YES), the calculation unit 11 extracts colony images from multiple culture medium images obtained by capturing images of the same culture medium, and identifies the number of new bacterial species by clustering the feature amounts of the extracted multiple colony images (step S291). The number of groups clustered by clustering the feature amounts of the colony images corresponds to the total number of bacterial species present in the culture medium. Note that parameters for clustering can be adjusted so that the number of clustered groups corresponds to the total number of bacterial species present in the culture medium. Meanwhile, the number of bacterial species identified in the process of step S286 corresponds to the number of existing bacterial species present in the culture medium. The calculation unit 11 can identify the number of new bacterial species by subtracting the number of existing bacterial species from the total number of bacterial species present in the culture medium. The calculation unit 11 may automatically assign a provisional bacterial species name or bacterial species group name to the identified new bacterial species.

[0092] The method for calculating the feature values ​​of the colony images to be clustered is not particularly limited, and may be calculated using the image recognition learning model 42. The intermediate layer 42b of the image recognition learning model 42 outputs numerical values ​​representing the features of the colony images, and the numerical values ​​(image feature values) calculated and output by the intermediate layer 42b may be used as the feature values. Alternatively, a vector having the probability output from the output layer 42c of the image recognition learning model 42 as its components may be used as the feature values. Because the output layer 42c outputs the probability corresponding to each of the multiple existing bacterial species, a vector having a dimension equal to the number of existing bacterial species is obtained. Alternatively, the probability output from the output layer 42c may be binarized using 0 or 1 using an appropriate threshold value, and the binarized values ​​may be used as the components of a vector, resulting in a feature value representing similar bacterial species as a hot vector.

[0093] The clustering method is not particularly limited, but it is advisable to cluster the features of colony images using a mixture distribution model such as K-means or EM algorithm, tSNE, PCA, etc. Furthermore, the features of colony images may be clustered using self-supervised learning (contrastive learning such as DeepCluster or Sela).

[0094] The vectors, which are the feature quantities of the colony images, may be clustered using K-means, EM algorithm, or the like, and if the colony is classified into a cluster different from existing bacterial species, it may be identified as a new bacterial species.

[0095] Furthermore, while the example described here is one in which colony image features are clustered based on medium images obtained by capturing images of the same medium currently being inspected, seed clustering, a type of semi-supervised learning, may also be performed by using colony image features extracted from medium images captured by capturing images of other media as seeds for clustering. Specifically, seed clustering may be performed as follows: An input dataset is created by mixing multiple colony images previously confirmed to be of existing bacterial species with colony images whose bacterial species are to be identified. The multiple colony images include colony images of all types of existing bacterial species. Then, by using the colony image features of the input dataset in the above-described method, clustering and identification of new bacterial species can be performed with greater accuracy.

[0096] Next, the calculation unit 11 calculates the reliability of the determination that the specific colony image is a colony image of a novel bacterial species (step S292). For example, the calculation unit 11 may calculate the inverse of the distance between the feature amount of the specific colony image and the center of each cluster as the reliability. More specifically, the calculation unit 11 may calculate the inverse of the kernel density estimate or the local outlier (LOF) as the reliability that the specific colony image is a novel bacterial species. Alternatively, the calculation unit 11 may be configured to calculate the inverse of a known anomaly that indicates the degree of deviation from each cluster, or the similarity, as the reliability.

[0097] Next, the calculation unit 11 executes a registration process for the new bacterial species (step S293). A laboratory technician can confirm the official species name of the new bacterial species by visual inspection or using a bacterial testing kit, etc., and input the new species name into the information processing device 1 using the operation unit 14. The calculation unit 11 registers information about the new bacterial species, such as the official species name input by operating the operation unit 14. Information such as culture medium images, colony images, and species names containing the new bacterial species can be used for additional learning or improvement of the bacterial species identification learning model 4.

[0098] The calculation unit 11 outputs, as processing results, the colony image of the new bacterial species, the number of the new bacterial species, other information about the new bacterial species, and the reliability of the determination that the species is a new bacterial species in step S54 described in embodiment 1. The calculation unit 11 also outputs, as processing results, the probability that the identified existing bacterial species is present in the culture medium.

[0099] The information processing device 1 may be configured to store combinations of bacterial species grown in various specimens and culture media in advance in the memory unit 13, and determine the validity of the bacterial species identification results based on the bacterial species identification results and the bacterial species combinations stored in the memory unit 13. The validity can be calculated based on the similarity, statistical distance, outliers, etc. between the identified bacterial species combinations and the bacterial species combinations stored in the memory unit 13. For example, a vector having a dimension of the number of bacterial species is set so that vector components represent the presence or absence of each bacterial species, the dot product of the vectors represents the similarity, and the absolute value of the vector difference represents the Euclidean distance. In this case, if the dimensions of the number of bacterial species do not match, the missing bacterial species components of the vector with the fewer bacterial species are interpolated as zero. The calculation unit 11 may be configured to determine the validity of the identification results and the content of the newly registered bacterial species based on the combinations of the identified bacterial species and the registered new bacterial species and the bacterial species combinations stored in the memory unit 13.

[0100] The information processing device 1 according to the second embodiment can detect novel bacterial species and specify the number of novel bacterial species. In addition, it can output the reliability of the determination result that the bacterial species is novel. Furthermore, instead of the accuracy output from the image recognition learning model 42, it is possible to convert it into the probability that the existing bacterial species actually exists in the culture medium and output it. [Explanation of symbols]

[0101] 1. Information processing equipment 2. Learning model for recognizing migratory bacteria 3. Migration Recognition Learning Model 31 Object detection learning model 32 Image Recognition Learning Model 4. Bacteria species identification learning model 41 Object detection learning model 42 Image Recognition Learning Model 5 Weighting Factor Table 10 Recording media 11 Arithmetic section 12 Memory 13 Storage section 14 Control section 15 Display section 16 Notification Department 17 Acquisition Department 131 Computer Programs

Claims

1. Acquiring a medium image by capturing an image of the medium during bacterial culture; determining the presence or absence of migratory bacteria that may be contained in the culture medium by inputting the acquired culture medium image into a migratory bacteria recognition learning model that has been generated by machine learning using training data that associates medium images obtained by photographing the culture medium during bacterial culture with information indicating the presence or absence of migratory bacteria that may be contained in the culture medium, so that when a single culture medium image obtained by photographing the culture medium during bacterial culture is input, the presence or absence of migratory bacteria that may be contained in the culture medium is output; When a medium image obtained by photographing a medium in which bacteria are cultured is input, the system outputs information for identifying the type of bacteria contained in the medium, and the bacterial species identification learning model is generated by machine learning using the medium image and information indicating the type of bacteria forming the colony image contained in the medium image. The system additionally trains the model using a medium image determined to be free of migratory bacteria. A computer program for causing a computer to execute a process, an evaluation index is calculated by inputting a test culture medium image into the bacterial species discrimination learning model before additional learning, and multiplying the accuracy rate or error rate of bacterial species discrimination based on information output from the bacterial species discrimination learning model by a weighting coefficient having a greater value for discrimination accuracy of drug-resistant bacteria or non-resident bacteria than for discrimination accuracy of non-drug-resistant bacteria or resident bacteria; inputting a test culture medium image into the bacterial species identification learning model after additional learning, and multiplying the accuracy rate or error rate of bacterial species identification based on information output from the bacterial species identification learning model by the weighting coefficient to calculate an evaluation index; When the evaluation index of the bacterial species discrimination learning model after the additional learning is improved compared to the evaluation index of the bacterial species discrimination learning model before the additional learning, the bacterial species discrimination learning model before the additional learning is updated to the bacterial species discrimination learning model after the additional learning. A computer program that causes a computer to execute a process.

2. Acquiring a plurality of medium images obtained by imaging the medium in the bacterial culture at different times during the bacterial culture; a migration recognition learning model generated by machine learning using training data that associates multiple medium images obtained by photographing a medium at multiple different times during bacterial culture with information indicating the presence or absence of migratory bacteria that may be contained in the medium, so that when multiple medium images taken at multiple different times during bacterial culture are input, the model outputs the presence or absence of migratory bacteria that may be contained in the medium; When a medium image obtained by photographing a medium in which bacteria are cultured is input, the system outputs information for identifying the type of bacteria contained in the medium, and the bacterial species identification learning model is generated by machine learning using the medium image and information indicating the type of bacteria forming the colony image contained in the medium image. The system additionally trains the model using a medium image determined to be free of migratory bacteria. A computer program for causing a computer to execute a process, an evaluation index is calculated by inputting a test culture medium image into the bacterial species discrimination learning model before additional learning, and multiplying the accuracy rate or error rate of bacterial species discrimination based on information output from the bacterial species discrimination learning model by a weighting coefficient having a greater value for discrimination accuracy of drug-resistant bacteria or non-resident bacteria than for discrimination accuracy of non-drug-resistant bacteria or resident bacteria; inputting a test culture medium image into the bacterial species identification learning model after additional learning, and multiplying the accuracy rate or error rate of bacterial species identification based on information output from the bacterial species identification learning model by the weighting coefficient to calculate an evaluation index; When the evaluation index of the bacterial species discrimination learning model after the additional learning is improved compared to the evaluation index of the bacterial species discrimination learning model before the additional learning, the bacterial species discrimination learning model before the additional learning is updated to the bacterial species discrimination learning model after the additional learning. A computer program that causes a computer to execute a process.

3. Acquiring a plurality of medium images obtained by imaging the medium in the bacterial culture at different times during the bacterial culture; Detecting the position of an image of a colony formed by the cultured bacteria based on the acquired medium image; Calculating the amount of change in position of the detected colony image; determining whether the calculated amount of position change is equal to or greater than a predetermined threshold value, thereby determining the presence or absence of migratory bacteria that may be contained in the medium; When a medium image obtained by photographing a medium in which bacteria are cultured is input, the system outputs information for identifying the type of bacteria contained in the medium, and the bacterial species identification learning model is generated by machine learning using the medium image and information indicating the type of bacteria forming the colony image contained in the medium image. The system additionally trains the model using a medium image determined to be free of migratory bacteria. A computer program for causing a computer to execute a process, an evaluation index is calculated by inputting a test culture medium image into the bacterial species discrimination learning model before additional learning, and multiplying the accuracy rate or error rate of bacterial species discrimination based on information output from the bacterial species discrimination learning model by a weighting coefficient having a greater value for discrimination accuracy of drug-resistant bacteria or non-resident bacteria than for discrimination accuracy of non-drug-resistant bacteria or resident bacteria; inputting a test culture medium image into the bacterial species identification learning model after additional learning, and multiplying the accuracy rate or error rate of bacterial species identification based on information output from the bacterial species identification learning model by the weighting coefficient to calculate an evaluation index; When the evaluation index of the bacterial species discrimination learning model after the additional learning is improved compared to the evaluation index of the bacterial species discrimination learning model before the additional learning, the bacterial species discrimination learning model before the additional learning is updated to the bacterial species discrimination learning model after the additional learning. A computer program that causes a computer to execute a process.

4. Acquiring a plurality of medium images obtained by imaging the medium at different times during bacterial culture; Detecting the size of the image of the colony formed by the cultured bacteria based on the acquired medium image; Calculating the amount of change in size of the detected colony image; The presence or absence of migratory bacteria that may be contained in the culture medium is determined by determining whether the calculated amount of size change is equal to or greater than a predetermined threshold. The computer program according to any one of claims 1 to 3, for causing the computer to execute processing.

5. If it is determined that chemotactic bacteria are present, a notification is sent that chemotactic bacteria have been detected.

5. A computer program according to claim 1, for causing a computer to execute a process.

6. The medium images determined to contain chemotactic bacteria are excluded, and the medium images determined to contain no chemotactic bacteria are stored.

6. A computer program according to claim 1, for causing a computer to execute a process.

7. If it is determined that there are no migratory bacteria, The image of a colony contained in the acquired medium image is input to the bacterial species identification learning model, thereby identifying the bacterial species of the colony.

7. A computer program product according to claim 1, for causing a computer to execute a process.

8. Determine whether there is a colony image of a new bacterial species whose species cannot be identified; When there is a colony image of a new bacterial species that cannot be identified, the image features of multiple colonies contained in the acquired culture medium image are clustered, Output information about the clustering results 8. A computer program product according to claim 7, for causing the computer to execute a process.

9. Acquiring a medium image by capturing an image of the medium during bacterial culture; determining the presence or absence of migratory bacteria that may be contained in the culture medium by inputting the acquired culture medium image into a migratory bacteria recognition learning model that has been generated by machine learning using training data that associates medium images obtained by photographing the culture medium during bacterial culture with information indicating the presence or absence of migratory bacteria that may be contained in the culture medium, so that when a single culture medium image obtained by photographing the culture medium during bacterial culture is input, the presence or absence of migratory bacteria that may be contained in the culture medium is output; When a medium image obtained by photographing a medium in which bacteria are cultured is input, the system outputs information for identifying the type of bacteria contained in the medium, and the bacterial species identification learning model is generated by machine learning using the medium image and information indicating the type of bacteria forming the colony image contained in the medium image. The system additionally trains the model using a medium image determined to be free of migratory bacteria. An information processing method, comprising: an evaluation index is calculated by inputting a test culture medium image into the bacterial species discrimination learning model before additional learning, and multiplying the accuracy rate or error rate of bacterial species discrimination based on information output from the bacterial species discrimination learning model by a weighting coefficient having a greater value for discrimination accuracy of drug-resistant bacteria or non-resident bacteria than for discrimination accuracy of non-drug-resistant bacteria or resident bacteria; inputting a test culture medium image into the bacterial species identification learning model after additional learning, and multiplying the accuracy rate or error rate of bacterial species identification based on information output from the bacterial species identification learning model by the weighting coefficient to calculate an evaluation index; When the evaluation index of the bacterial species discrimination learning model after the additional learning is improved compared to the evaluation index of the bacterial species discrimination learning model before the additional learning, the bacterial species discrimination learning model before the additional learning is updated to the bacterial species discrimination learning model after the additional learning. Information processing methods.

10. Acquiring a plurality of medium images obtained by imaging the medium in the bacterial culture at different times during the bacterial culture; a migration recognition learning model generated by machine learning using training data that associates multiple medium images obtained by photographing a medium at multiple different times during bacterial culture with information indicating the presence or absence of migratory bacteria that may be contained in the medium, so that when multiple medium images taken at multiple different times during bacterial culture are input, the model outputs the presence or absence of migratory bacteria that may be contained in the medium; When a medium image obtained by photographing a medium in which bacteria are cultured is input, the system outputs information for identifying the type of bacteria contained in the medium, and the bacterial species identification learning model is generated by machine learning using the medium image and information indicating the type of bacteria forming the colony image contained in the medium image. The system additionally trains the model using a medium image determined to be free of migratory bacteria. An information processing method, comprising: an evaluation index is calculated by inputting a test culture medium image into the bacterial species discrimination learning model before additional learning, and multiplying the accuracy rate or error rate of bacterial species discrimination based on information output from the bacterial species discrimination learning model by a weighting coefficient having a greater value for discrimination accuracy of drug-resistant bacteria or non-resident bacteria than for discrimination accuracy of non-drug-resistant bacteria or resident bacteria; inputting a test culture medium image into the bacterial species identification learning model after additional learning, and multiplying the accuracy rate or error rate of bacterial species identification based on information output from the bacterial species identification learning model by the weighting coefficient to calculate an evaluation index; When the evaluation index of the bacterial species discrimination learning model after the additional learning is improved compared to the evaluation index of the bacterial species discrimination learning model before the additional learning, the bacterial species discrimination learning model before the additional learning is updated to the bacterial species discrimination learning model after the additional learning. Information processing methods.

11. Acquiring a plurality of medium images obtained by imaging the medium in the bacterial culture at different times during the bacterial culture; Detecting the position of an image of a colony formed by the cultured bacteria based on the acquired medium image; Calculating the amount of change in position of the detected colony image; determining whether the calculated amount of position change is equal to or greater than a predetermined threshold value, thereby determining the presence or absence of migratory bacteria that may be contained in the medium; When a medium image obtained by photographing a medium in which bacteria are cultured is input, the system outputs information for identifying the type of bacteria contained in the medium, and the bacterial species identification learning model is generated by machine learning using the medium image and information indicating the type of bacteria forming the colony image contained in the medium image. The system additionally trains the model using a medium image determined to be free of migratory bacteria. An information processing method, comprising: an evaluation index is calculated by inputting a test culture medium image into the bacterial species discrimination learning model before additional learning, and multiplying the accuracy rate or error rate of bacterial species discrimination based on information output from the bacterial species discrimination learning model by a weighting coefficient having a greater value for discrimination accuracy of drug-resistant bacteria or non-resident bacteria than for discrimination accuracy of non-drug-resistant bacteria or resident bacteria; inputting a test culture medium image into the bacterial species identification learning model after additional learning, and multiplying the accuracy rate or error rate of bacterial species identification based on information output from the bacterial species identification learning model by the weighting coefficient to calculate an evaluation index; When the evaluation index of the bacterial species discrimination learning model after the additional learning is improved compared to the evaluation index of the bacterial species discrimination learning model before the additional learning, the bacterial species discrimination learning model before the additional learning is updated to the bacterial species discrimination learning model after the additional learning. Information processing methods.

12. Acquiring a medium image by capturing an image of the medium during bacterial culture; determining the presence or absence of migratory bacteria that may be contained in the culture medium by inputting the acquired culture medium image into a migratory bacteria recognition learning model that has been generated by machine learning using training data that associates medium images obtained by photographing the culture medium during bacterial culture with information indicating the presence or absence of migratory bacteria that may be contained in the culture medium, so that when a single culture medium image obtained by photographing the culture medium during bacterial culture is input, the presence or absence of migratory bacteria that may be contained in the culture medium is output; When a medium image obtained by photographing a medium in which bacteria are cultured is input, the system outputs information for identifying the type of bacteria contained in the medium, and the bacterial species identification learning model is generated by machine learning using the medium image and information indicating the type of bacteria forming the colony image contained in the medium image. The system additionally trains the model using a medium image determined to be free of migratory bacteria. An information processing device including a calculation unit that executes processing, The calculation unit an evaluation index is calculated by inputting a test culture medium image into the bacterial species discrimination learning model before additional learning, and multiplying the accuracy rate or error rate of bacterial species discrimination based on information output from the bacterial species discrimination learning model by a weighting coefficient having a greater value for discrimination accuracy of drug-resistant bacteria or non-resident bacteria than for discrimination accuracy of non-drug-resistant bacteria or resident bacteria; inputting a test culture medium image into the bacterial species identification learning model after additional learning, and multiplying the accuracy rate or error rate of bacterial species identification based on information output from the bacterial species identification learning model by the weighting coefficient to calculate an evaluation index; When the evaluation index of the bacterial species discrimination learning model after the additional learning is improved compared to the evaluation index of the bacterial species discrimination learning model before the additional learning, the bacterial species discrimination learning model before the additional learning is updated to the bacterial species discrimination learning model after the additional learning. Information processing device.

13. Acquiring a plurality of medium images obtained by imaging the medium in the bacterial culture at different times during the bacterial culture; a migration recognition learning model generated by machine learning using training data that associates multiple medium images obtained by photographing a medium at multiple different times during bacterial culture with information indicating the presence or absence of migratory bacteria that may be contained in the medium, so that when multiple medium images taken at multiple different times during bacterial culture are input, the model outputs the presence or absence of migratory bacteria that may be contained in the medium; When a medium image obtained by photographing a medium in which bacteria are cultured is input, the system outputs information for identifying the type of bacteria contained in the medium, and the bacterial species identification learning model is generated by machine learning using the medium image and information indicating the type of bacteria forming the colony image contained in the medium image. The system additionally trains the model using a medium image determined to be free of migratory bacteria. An information processing device including a calculation unit that executes processing, The calculation unit an evaluation index is calculated by inputting a test culture medium image into the bacterial species discrimination learning model before additional learning, and multiplying the accuracy rate or error rate of bacterial species discrimination based on information output from the bacterial species discrimination learning model by a weighting coefficient having a greater value for discrimination accuracy of drug-resistant bacteria or non-resident bacteria than for discrimination accuracy of non-drug-resistant bacteria or resident bacteria; inputting a test culture medium image into the bacterial species identification learning model after additional learning, and multiplying the accuracy rate or error rate of bacterial species identification based on information output from the bacterial species identification learning model by the weighting coefficient to calculate an evaluation index; When the evaluation index of the bacterial species discrimination learning model after the additional learning is improved compared to the evaluation index of the bacterial species discrimination learning model before the additional learning, the bacterial species discrimination learning model before the additional learning is updated to the bacterial species discrimination learning model after the additional learning. Information processing device.

14. Acquiring a plurality of medium images obtained by imaging the medium in the bacterial culture at different times during the bacterial culture; Detecting the position of an image of a colony formed by the cultured bacteria based on the acquired medium image; Calculating the amount of change in position of the detected colony image; determining whether the calculated amount of position change is equal to or greater than a predetermined threshold value, thereby determining the presence or absence of migratory bacteria that may be contained in the medium; When a medium image obtained by photographing a medium in which bacteria are cultured is input, the system outputs information for identifying the type of bacteria contained in the medium, and the bacterial species identification learning model is generated by machine learning using the medium image and information indicating the type of bacteria forming the colony image contained in the medium image. The system additionally trains the model using a medium image determined to be free of migratory bacteria. An information processing device including a calculation unit that executes processing, The calculation unit an evaluation index is calculated by inputting a test culture medium image into the bacterial species discrimination learning model before additional learning, and multiplying the accuracy rate or error rate of bacterial species discrimination based on information output from the bacterial species discrimination learning model by a weighting coefficient having a greater value for discrimination accuracy of drug-resistant bacteria or non-resident bacteria than for discrimination accuracy of non-drug-resistant bacteria or resident bacteria; inputting a test culture medium image into the bacterial species identification learning model after additional learning, and multiplying the accuracy rate or error rate of bacterial species identification based on information output from the bacterial species identification learning model by the weighting coefficient to calculate an evaluation index; When the evaluation index of the bacterial species discrimination learning model after the additional learning is improved compared to the evaluation index of the bacterial species discrimination learning model before the additional learning, the bacterial species discrimination learning model before the additional learning is updated to the bacterial species discrimination learning model after the additional learning. Information processing device.

Images