Computer program, antibiotic effectiveness estimation system, and method for estimating antibiotic effectiveness

Abstract

To provide a computer program capable of inexpensively and quickly estimating an antibiotic effectiveness, an antibiotic effectiveness estimation system, and a method for estimating the antibiotic effectiveness.SOLUTION: A computer program causes a computer to executes processes of acquiring an image of a test bacterial liquid containing an antibiotic that has been incubated for a required incubation time, inputting the acquired image of the test bacterial liquid containing the antibiotic into a trained model that outputs information regarding an antibiotic effectiveness when the image of the test bacterial liquid containing the antibiotic is input, and estimating the antibiotic effectiveness for the incubation time.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a computer program, an antimicrobial effect estimation system, and an antimicrobial effect estimation method. [Background technology]

[0002] The effectiveness of drugs against bacteria is confirmed by observing the growth, inhibition, or death of bacteria using culture methods using liquid culture medium and agar culture medium, and confirmation takes more than 24 hours. In emergency situations such as medical settings, drugs that are thought to be effective against many bacteria are selected, and the use of multiple drugs in combination leads to increased medical costs and the development of bacterial resistance.

[0003] Patent Document 1 discloses a testing system that can rapidly confirm the susceptibility of bacteria to antibiotics by incubating a mixture of an antibiotic and a liquid of the test bacteria in a flow channel of a microdevice and observing the observation area in the flow channel under a microscope. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6032575 Summary of the Invention [Problem to be solved by the invention]

[0005] However, microdevices require the formation of complex and minute flow paths, which increases costs. In addition, each container in the observation area has an extremely small volume (for example, about 1 μml), making it difficult to control the concentration of the bacterial liquid and antibiotics at a constant level, and the concentrations cannot be reproduced. Furthermore, there was a risk that stable evaluation results would not be obtained by adopting a new container and antibiotic mixing design, compared to the stable and proven evaluation results achieved by mixing drugs and bacteria that had been cultivated using conventional culture methods. In addition, although it is stated that sensitivity can be judged from differences in images, there is no indication of the percentage or numerical value of changes in the number of bacteria or shape, and it is not made clear what changes can be correlated with the results of antimicrobial susceptibility tests, so it is unclear whether a specific judgment can be made.

[0006] The present disclosure has been made in consideration of the above circumstances, and discloses a computer program, an antimicrobial effect estimation system, and an antimicrobial effect estimation method that can estimate the effectiveness of an antimicrobial drug quickly and inexpensively. [Means for solving the problem]

[0007] The present application includes multiple means for solving the above-mentioned problems. One example is to have a computer acquire an image of a test bacterial liquid containing an antibacterial agent that has been cultured for a culture time shorter than a predetermined culture time, input the image of the test bacterial liquid containing an antibacterial agent acquired by the process into a trained model that outputs information regarding the effectiveness of the antibacterial agent when the image of the test bacterial liquid containing an antibacterial agent is input, and execute a process to estimate the effectiveness of the antibacterial agent. Effect of the Invention

[0008] According to the present disclosure, the effectiveness of an antibacterial drug can be estimated inexpensively and quickly. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing an example of the configuration of an antibacterial effect estimation system according to the present embodiment. FIG. [Diagram 2] FIG. 1 is a schematic diagram showing the effectiveness of an antibacterial drug. [Diagram 3] FIG. 2 is a schematic diagram showing an example of training data and validation data. [Figure 4] FIG. 1 is a schematic diagram showing a first example of a CNN learning method. [Diagram 5] FIG. 13 is a schematic diagram showing an example of a learning result. [Figure 6] FIG. 13 is an explanatory diagram showing an example of a method for determining an incubation time range. [Figure 7] FIG. 13 is a schematic diagram showing an example of a learning result. [Figure 8] FIG. 13 is a schematic diagram showing a second example of a CNN learning method. [Figure 9] FIG. 13 is a schematic diagram showing a third example of a CNN learning method. [Figure 10] FIG. 13 is a schematic diagram showing a fourth example of a CNN learning method. [Figure 11] FIG. 13 is a schematic diagram showing a fifth example of a CNN learning method. [Figure 12] FIG. 13 is a schematic diagram showing a sixth example of a CNN learning method. [Figure 13] FIG. 1 is a schematic diagram showing a first example of a method for estimating the effectiveness of an antibiotic using a trained model. [Figure 14] FIG. 11 is a schematic diagram showing an example of a method for selecting a trained model. [Figure 15] FIG. 13 is a schematic diagram showing a second example of a method for estimating the effectiveness of an antibiotic using a trained model. [Figure 16] FIG. 13 is an explanatory diagram showing the evaluation results of the accuracy rate based on the validation data of each trained model. [Figure 17] FIG. 11 is a block diagram showing a first other example of the estimation device. [Figure 18] FIG. 11 is a block diagram showing another second example of the estimation device. [Figure 19] FIG. 13 is a schematic diagram showing a first example of a display screen at the start of measurement when culturing and observing in the culture device. [Figure 20] FIG. 13 is a schematic diagram showing a second example of a display screen at the start of measurement. [Figure 21] FIG. 13 is a schematic diagram showing an example of a display screen after measurement is completed. [Figure 22] FIG. 13 is a schematic diagram showing an example of a list display of measurement results. [Figure 23] FIG. 13 is a block diagram showing another example of the configuration of the estimation device. [Figure 24] 1 is a flowchart showing a first example of a processing procedure of a method for generating a learning model by an estimation device. [Diagram 25]13 is a flowchart showing a second example of the processing steps of the method for generating a learning model by the estimation device. [Figure 26] 1 is a flowchart showing an example of a processing procedure of a method for estimating the effect of an antimicrobial agent by an estimation device. [Figure 27] FIG. 1 is a schematic diagram showing a first example of the configuration of an imaging system of an imaging device. [Figure 28] FIG. 4 is a schematic diagram showing a second example of the configuration of an imaging system of the imaging device. [Figure 29] FIG. 1 is a schematic diagram showing an example of a system for estimating the effect of an antimicrobial agent having a culture device. [Diagram 30] FIG. 2 is a schematic diagram showing an example of the configuration of a culture device and an imaging device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram showing an example of the configuration of an antibacterial effect estimation system of this embodiment. The antibacterial effect estimation system includes an estimation device 50 and an imaging device. The imaging device includes a light source 10, an observation container 20, and an imaging unit 30. The observation container 20 has a plurality of wells 21 (also called holes or containers) arranged on the bacteria observation surface of a plate-shaped container body. The wells 21 have a circular opening in a plan view, a diameter of about 5 mm to 8 mm, a depth of about 1 cm, and a pitch between adjacent wells 21 that can be about 1 cm, but are not limited to these. One well 21 has a volume of about 300 μl. Note that in FIG. 1, the number of wells 21 is four for convenience, but the number of wells 21 is not limited to four. For example, the number of wells 21 may be six, ten, or twenty, and 96 wells 21 may be arranged in one observation container 20. Each well 21 can accommodate a test bacteria liquid having different conditions.

[0011] An imaging unit 30 is disposed below the observation container 20, and a light source 10 is disposed above (on one side of) the observation container 20. The imaging unit 30 is disposed on the opposite side of the light source 10 with the observation container 20 in between. The imaging unit 30 has a plurality of imaging devices 31 disposed on its light receiving surface. Each of the plurality of imaging devices 31 is disposed corresponding to one well 21. Specifically, the imaging device 31 is disposed directly below the well 21. The imaging device 31 has a plurality of imaging elements (e.g., photodiodes) disposed therein.

[0012] The light source 10 can irradiate straight light (e.g., laser light). By irradiating light (e.g., straight light) from the light source 10 toward the test bacterial liquid contained in each of the multiple wells 21, an image of the test bacterial liquid can be acquired by the imaging device 31. Furthermore, by arranging a plurality of imaging devices 31 corresponding to each of the multiple wells 21, for example, by relatively moving either a single imaging device or an observation container (moving in the XY direction) to image the test bacterial liquid in multiple wells, a drive system that is required for this purpose is not required, and blurring of the image due to the drive system can be prevented. Furthermore, an expensive optical microscope is not required.

[0013] In addition, since the volume of the well 21 is as large as the volume of a well provided in a general observation container, there is a risk of out-of-focus occurring when imaging the test bacteria liquid. Therefore, in order to prevent out-of-focus in the optical axis direction (Z direction), for example, a drive system or imaging method that enables relative movement of the light source 10, the observation container 20, and the imaging unit 30 along the Z axis direction can be adopted. In this case, movement in the Z axis direction can be performed at a pitch of several μm. For example, the image processing unit 53 extracts focused images from images captured while repeatedly moving in the Z axis direction at a pitch of several μm, or only focused areas from one image, and the estimation device 50 can estimate the effect of the antibacterial drug.

[0014] In addition, in this embodiment, since the volume of the well 21 is large, the concentration adjustment of the antibacterial agent is stabilized, and appropriate effect estimation can be performed. Specifically, the amount of the antibacterial agent fixed in the well 21 can be increased, and the amount of the liquid to be mixed can also be increased, making it possible to stabilize the effect estimation. In addition, since the mixing design with the container and the antibacterial agent is similar to existing ones, it is possible to fully ensure compatibility with conventional evaluations.

[0015] The estimation device 50 includes a control unit 51 that controls the entire device, an interface unit 52, an image processing unit 53, a storage unit 54, a learning device 55 as a learning model, a display panel 56, and an operation unit 57. The control unit 51 can be configured with a CPU, a ROM, a RAM, etc. The estimation device 50 can also include a communication unit (not shown) that can transmit and receive information to and from an external device via a communication network.

[0016] The interface section 52 has an interface function with the imaging unit 30. The interface section 52 can acquire an image captured by the imaging device 31 (image data).

[0017] The image processing unit 53 can perform required image processing. The image processing unit 53 will be described in detail later.

[0018] The storage unit 54 can be configured with a semiconductor memory, a hard disk, or the like, and can store images acquired via the interface unit 52. The storage unit 54 can also store data obtained as a result of processing performed by the estimation device 50, and data in the middle of processing.

[0019] The learning device 55 can be configured, for example, with a multi-layered neural network (deep learning), and can use, for example, a convolutional neural network, but other machine learning methods may also be used. The learning device 55 can be configured, for example, by combining hardware such as a CPU (for example, a multi-processor having multiple processor cores), GPUs (Graphics Processing Units), DSPs (Digital Signal Processors), and FPGAs (Field-Programmable Gate Arrays). Details of the learning device 55 will be described later. Using the learning device 55, a multi-layered neural network learns by linking images of bacteria with labels of effective or ineffective based on the results of 24-hour cultivation, making it possible to infer which images are effective and which are ineffective.

[0020] The display panel 56 can be configured with a liquid crystal panel, an organic EL (Electro Luminescence) display, etc. The operation unit 57 can be configured with, for example, a touch panel incorporated in the display panel 56, and allows a predetermined operation to be performed on icons, etc. displayed on the display panel 56. The operation unit 57 may also be a hardware keyboard and mouse, etc.

[0021] Next, drug susceptibility testing will be explained.

[0022] FIG. 2 is a schematic diagram showing the effect of an antibacterial drug. A drug susceptibility test is a test conducted to examine the susceptibility of bacteria (cells, which are a group of single-celled organisms, and fungi, which are a collective term for molds) to antibacterial drugs, and is used to detect resistant bacteria. For example, liquid culture medium, agar culture medium, etc. are used to culture bacteria. A certain amount of bacteria is inoculated into a culture medium containing an antibacterial drug (drug), and after culturing for a predetermined culture time (for example, 24 hours, which is one culture time adopted by CLSI / Clinical and Laboratory Standards Institute, which proposes standards for antibacterial drug susceptibility evaluation), the presence or absence of growth of the bacteria can be visually observed to confirm the effect of the antibacterial drug. In this specification, the predetermined culture time is described as 24 hours, which is adopted by CLSI / Clinical and Laboratory Standards Institute. For example, as shown in FIG. 2, if the test bacteria liquid at the start of culture is cloudy overall, it can be determined that the antibacterial drug is ineffective. Similarly, if the bacteria settle, it can be determined that the antibacterial drug is ineffective. Note that at the start of culture, the concentration of bacteria is generally low, so that almost no bacteria are visible in the captured image. In addition, the bacteria shown in Fig. 2 are shown diagrammatically and are different from the actual shape of the bacteria. The test bacteria are, for example, bacteria to be tested that are thought to be infectious disease causative bacteria isolated from a test sample. In addition, when the test bacteria liquid at the start of the culture becomes almost transparent, as shown diagrammatically by the dashed line in Fig. 2, although some bacteria remain as residues, it can be determined that the antibacterial drug is effective.

[0023] The estimation device 50 of the present embodiment can estimate the effect of an antibacterial agent within a required culture time that is shorter than a predetermined culture time. As described above, the predetermined culture time can be the culture time required to visually determine the effect of an antibacterial agent by a predetermined method. The required culture time can be a culture time that enables estimation of the effect of an antibacterial agent (i.e., the visually determined effect of an antibacterial agent by a predetermined method) by image-based determination. The estimation device 50 of the present embodiment can estimate the effect of an antibacterial agent in a shorter time compared to the conventional visual method. Further, the estimation device 50 of the present embodiment can estimate the effect of an antibacterial agent at low cost. This will be specifically described below.

[0024] First, a learning method of the learning device 55, that is, a method for generating the learning device 55 (trained model) will be described.

[0025] FIG. 3 is a schematic diagram showing an example of training data and verification data. Let the culture times be t0, t1, t2, …, tn. Here, assume t0 < t1 < t2 < … < tn. The time t0 is a time point within the time when the effect of the antibacterial agent cannot be discriminated, and for example, can be the start time point of culture or a time point immediately after the start of culture (a culture time shorter than the required culture time). The time tn is, for example, a time when the effect of the antibacterial agent can be discriminated by the conventional visual method, and can be 24 hours, which is the predetermined culture time. The interval of the culture time (referred to as the unit time) can be, for example, 1 hour or 0.5 hour.

[0026] First, time series images (C0, C1, C2, ..., Cn) of a test bacterial liquid containing no antibiotics, which have been cultured for a plurality of different culture times (t0, t1, t2, ..., tn), are collected. The plurality of different culture times (t0, t1, t2, ..., tn) are, for example, culture times within a time span from the start time t0 of the culture to a predetermined culture time tn. The time series images (C0, C1, C2, ..., Cn) can be collected for various combinations of bacteria. Similarly, time series images (E0, E1, E2, ..., En) of a test bacterial liquid containing an effective antibiotic, which has been cultured for a plurality of different culture times (t0, t1, t2, ..., tn), and time series images (N0, N1, N2, ..., Nn) of a test bacterial liquid containing an ineffective antibiotic are collected. The time series images (E0, E1, E2, ..., En) and (N0, N1, N2, ..., Nn) can be collected by combining various bacteria and various antibacterial drugs. In the time series images (C0, C1, C2, ..., Cn) and the time series images (N0, N1, N2, ..., Nn) of the test bacteria liquid containing an ineffective antibacterial drug, the bacteria increase in the form of tiny particles as the culture progresses. On the other hand, in the time series images (E0, E1, E2, ..., En) of the test bacteria liquid containing an effective antibacterial drug, while some bacteria increase as the culture progresses, the bacteria that were swollen or granular are lysed and elongated due to the effect of the antibacterial drug. In addition, the bacteria shown in Figure 3 are illustrated diagrammatically and differ from the actual shape of the bacteria. Note that the time series images used as training data and the time series images used as verification data are distinguished.

[0027] When the size of the image captured by the imaging device 31 is large, the image processing unit 53 can cut out a part of the image to generate training and verification images. It is desirable to collect a large number of training and verification images. The collected images may be stored in the storage unit 54, or may be stored in an external database (not shown) accessible from the estimation device 50.

[0028] FIG. 4 is a schematic diagram showing a first example of a learning method of the CNN 55a. In this specification, the learning device 55 is described as a CNN 55a (Convolutional Neural Network). First, the CNN 55a is trained using each image at the culture time t(n-1) closest to the culture time tn. Specifically, the CNN 55a is trained using an image C0 of the test bacterial liquid not containing an antibacterial agent at the culture time t0, an image C(n-1) of the test bacterial liquid not containing an antibacterial agent at the culture time t(n-1), an image E(n-1) of the test bacterial liquid containing an effective antibacterial agent, and an effective label (effective label) as training data, and similarly, the CNN 55a is trained using an image C0 of the test bacterial liquid not containing an antibacterial agent at the culture time t0, an image C(n-1) of the test bacterial liquid not containing an antibacterial agent at the culture time t(n-1), an image N(n-1) of the test bacterial liquid containing an ineffective antibacterial agent, and an ineffective label (ineffective label) as training data. For the trained CNN 55a, validation data (each image different from the training data) at incubation time t(n-1) is given to the trained CNN 55a, and the accuracy rate is calculated. The accuracy rate can be calculated as the reliability R(n-1) of the estimation result of the CNN 55a. Since the incubation time t(n-1) of each image used for learning is close to time tn, the accuracy rate, i.e., the reliability, is high.

[0029] Next, the CNN 55a is trained using as training data the image C0 of the test bacterial liquid not containing an antibacterial agent at the culture time t0, the image C(n-2) of the test bacterial liquid not containing an antibacterial agent at the culture time t(n-2) where the culture time is shortened by a unit time, the image E(n-2) of the test bacterial liquid containing an effective antibacterial agent, and the effective label (effective label), and similarly, the CNN 55a is trained using as training data the image C0 of the test bacterial liquid not containing an antibacterial agent at the culture time t0, the image C(n-2) of the test bacterial liquid not containing an antibacterial agent at the culture time t(n-2), the image N(n-2) of the test bacterial liquid containing an ineffective antibacterial agent, and the ineffective label (invalid label). The trained CNN 55a is given validation data (each image different from the training data) at the culture time t(n-2) to obtain the accuracy rate. The accuracy rate can be calculated as the reliability R(n-2) of the estimation result of the CNN 55a. Since the incubation time t(n-2) of each image used for learning is far from the time tn, it is considered that the reliability R(n-2) is lower than the reliability R(n-1).

[0030] Similarly, the culture time is shortened and the same process is repeated. That is, the CNN 55a is trained using the image C0 of the test bacterial liquid not containing an antibacterial agent at the culture time t0, the image C(nd) of the test bacterial liquid not containing an antibacterial agent at the culture time t(nd), the image E(nd) of the test bacterial liquid containing an effective antibacterial agent, and the effective label (effective label) as training data, and similarly, the CNN 55a is trained using the image C0 of the test bacterial liquid not containing an antibacterial agent at the culture time t0, the image C(nd) of the test bacterial liquid not containing an antibacterial agent at the culture time t(nd), the image N(nd) of the test bacterial liquid containing an ineffective antibacterial agent, and the ineffective label (invalid label) as training data. Here, d is d=1, 2, ..., (n-1). The trained CNN 55a is given validation data (each image different from the training data) at the culture time t(nd) to obtain the accuracy rate. The accuracy rate can be calculated as the reliability R(nd) of the estimation result of the CNN 55a. As d increases (i.e., as the incubation time decreases), the confidence level is expected to decrease gradually.

[0031] Furthermore, in the above example, when evaluating the reliability of the culture time t(nd), instead of using each image at the culture time t(nd) (image C(nd), image E(nd), image N(nd)), each image at a plurality of culture times including the culture time t(nd) can be used together.

[0032] For example, the CNN 55a can be trained using as training data an image C0 of the test bacterial liquid containing no antibiotic at incubation time t0, an image C(n-d+1) of the test bacterial liquid containing no antibiotic and an image E(n-d+1) of the test bacterial liquid containing an effective antibiotic at incubation time t(n-d+1), an image C(nd) of the test bacterial liquid containing no antibiotic and an image E(nd) of the test bacterial liquid containing an effective antibiotic at incubation time t(nd), an image C(nd-1) of the test bacterial liquid containing no antibiotic and an image E(nd-1) of the test bacterial liquid containing an effective antibiotic at incubation time t(nd-1), and a label indicating that the antibiotic is effective (effective label). Similarly, the CNN 55a can be trained using the image C0 of the test bacterial liquid not containing an antibacterial agent at the culture time t0, the image C(n-d+1) of the test bacterial liquid not containing an antibacterial agent at the culture time t(n-d+1) and the image N(n-d+1) of the test bacterial liquid containing an ineffective antibacterial agent at the culture time t(nd), the image C(nd) of the test bacterial liquid not containing an antibacterial agent and the image N(nd) of the test bacterial liquid containing an ineffective antibacterial agent at the culture time t(nd-1), the image C(nd-1) of the test bacterial liquid not containing an antibacterial agent and the image N(nd-1) of the test bacterial liquid containing an ineffective antibacterial agent at the culture time t(nd-1), and the label of no effect (effective label) as training data. Here, d can be changed as appropriate. In the above example, images of three times, t(n-d+1), t(nd), and t(nd-1), were used as the multiple culture times, but the multiple culture times are not limited to three. Two or four or more may be used.

[0033] As shown in FIG. 4, when the reliability Rk is the required reliability or the reliability within the allowable range (above a threshold value) and the incubation time tk is the shortest incubation time, the shortest incubation time can be obtained within the range where the required reliability can be obtained, so that the learning result at the shortest incubation time tk can be adopted to generate the CNN 55a. This allows the adoption of an incubation time that is both appropriate and shortest. In this specification, the reliability can be, for example, the approval standard for in vitro diagnostic drugs, and the required reliability means, for example, that the detection results of the same sample for the comparative product (already approved) and the product under investigation are listed in an appropriate table (m×n contingency table, for example, 2×2 contingency table), and the agreement rate between the two is 90% or more, more preferably 95% or more, and furthermore, that the sensitivity (no false positives) is close to 100%.

[0034] FIG. 5 is a schematic diagram showing an example of a learning result. As shown in FIG. 5, the types of bacteria are represented as A1, A2, ..., the types of antibiotics are represented as F1, F2, F3, F4, F5, ..., and the concentrations of the antibiotics are represented as α1, α2, α3, β1, .... Antibiotics F1, F2, and F3 are effective against bacteria A1, and antibiotics F1 are effective when the antibiotic concentrations are α1, α2, and α3, and the shortest incubation times for each can be determined as tk1, tk2, and tk3. Similarly, antibiotics F2 are effective when the antibiotic concentrations are β1, β2, and β3, and the shortest incubation times for each can be determined as tk4, tk5, and tk6. F1, F2, ... may have some in common, antibiotic concentrations α1, α2, ... may have some in common, and the incubation times tk1, tk2, ... may also have some in common.

[0035] 4 and 5 are configurations in which the reliability for each culture time is compared with a threshold value, but the present invention is not limited to this, and a statistical value of the reliability over a certain time span across multiple culture times can also be compared with a threshold value. The following describes a case in which a statistical value of the reliability over a certain time span across multiple culture times is used.

[0036] FIG. 6 is an explanatory diagram showing an example of a method for determining the culture time width. First, FIG. 6A will be described. The culture times are culture times t(n-1), t(n-2), t(n-3), t(n-4), ..., t1, as in the case of FIG. 4. As the statistical value of the reliability, for example, the average reliability of each of the last three culture times is used. For example, as shown in FIG. 6A, the average reliability RA(n-3) at the culture time t(n-3) is set to the average reliability R(n-1), R(n-2), and R(n-3) at the culture times t(n-1), t(n-2), and t(n-3). That is, RA(n-3)={R(n-1)+R(n-2)+R(n-3)} / 3. The average reliability after the incubation time t(n-4) can be calculated in the same manner, and the average reliability RA1 at the incubation time t1 can be calculated by the formula RA1 = {R3 + R2 + R1} / 3.

[0037] Then, when RAp is equal to or greater than the threshold, more specifically, when RAp≧threshold and RA(p−1)<threshold, the time span corresponding to the average reliability RAp can be determined as the culture time span T. Here, the culture time span T can be between culture times tp and t(p+2). The culture time span T may be between culture times tp and t(p+1), or between culture times t(p+1) and t(p+2). In addition, any of the culture times t(p+2), t(p+1), or tp within the culture time span T may be adopted. In the above example, the most recent three culture times are used, but the number is not limited to three, and two or four or more may be used.

[0038] Next, FIG. 6B will be described. For example, the average reliability at the culture time before and after the culture time is used as the statistical value of reliability. For example, as shown in FIG. 6B, the average reliability RA(n-2) at the culture time t(n-2) is the average reliability R(n-1), R(n-2), and R(n-3) at the culture times t(n-1), t(n-2), and t(n-3). That is, RA(n-2)={R(n-1)+R(n-2)+R(n-3)} / 3. The average reliability after the culture time t(n-3) can be calculated in the same manner.

[0039] Then, when RAq is equal to or greater than the threshold, more specifically, when RAq≧threshold and RA(q-1)<threshold, the time span corresponding to the average reliability RAq can be determined as the culture time span T. Here, the culture time span T can be between culture times t(q-1) and t(q+1). Note that the culture time span T may be between culture times t(q-1) and tq, or between culture times tq and t(q+1). Also, any of the culture times t(q+1), tq, or t(q-1) within the culture time span T may be adopted.

[0040] In Figures 6A and 6B as well, the reliability at a certain incubation time can be obtained based on the results of collectively learning each image (Image C, Image E, Image N) for each of multiple incubation times including that incubation time.

[0041] FIG. 7 is a schematic diagram showing an example of a learning result. As shown in FIG. 7, the types of bacteria are represented by B1, B2, ..., the types of antibiotics are represented by V1, V2, V3, V4, V5, ..., and the concentrations of the antibiotics are represented as α11, α12, α13, β11, .... Antibiotics V1, V2, and V3 are effective against bacteria B1, and antibiotics V1 are effective when the antibiotic concentrations are α11, α12, and α13, and the respective incubation time spans can be determined as T1, T2, and T3. Similarly, antibiotics V2 are effective when the antibiotic concentrations are β11, β12, and β13, and the respective incubation time spans can be determined as T3, T2, and T4. V1, V2, ... may have some in common. Antibiotic concentrations α11, α12, α13, β11, ... may have some in common. The culture time range may be, for example, a culture time range of 1 hour to 24 hours, a culture time range of 1 hour to 8 hours, or a culture time range of 2 hours to 6 hours.

[0042] FIG. 8 is a schematic diagram showing a second example of the learning method of the CNN 55a. The second example is a learning method in the case where the culture time for learning is known in advance by some means. As shown in FIG. 8, training data combining an image Eta of a test bacterial liquid containing an effective antibacterial agent cultured for a required culture time ta (a time shorter than the specified culture time tn) and a valid label corresponding to the image is provided to the CNN 55a to learn the CNN 55a. Similarly, training data combining an image Nta of a test bacterial liquid containing an ineffective antibacterial agent cultured for a required culture time ta (a time shorter than the specified culture time tn) and an invalid label corresponding to the image is provided to the CNN 55a to learn the CNN 55a. Note that the training data can be a combination of various bacteria and various antibacterial agents.

[0043] As a result, when an image of a test bacterial liquid containing an antibiotic at a required incubation time ta is input, CNN 55a can output information regarding the effectiveness of the antibiotic (e.g., effective, ineffective, effective level, etc.), and can estimate the results when the antibiotic is incubated for a specified incubation time tn at the time of the required incubation time ta.

[0044] FIG. 9 is a schematic diagram showing a third example of the learning method of the CNN 55a. The third example is a learning method in the case where the incubation time for learning is known in advance by some means, as in the second example. The difference from the second example is that an image Cta of a test bacterial liquid containing no antibiotics, which has been cultured for a required incubation time ta, is further used as training data. That is, the CNN 55a is trained by providing training data that combines an image Cta of a test bacterial liquid containing no antibiotics, which has been cultured for a required incubation time ta, an image Eta of a test bacterial liquid containing an effective antibiotic, and a valid label corresponding to the image, to the CNN 55a. Similarly, the CNN 55a is trained by providing training data that combines an image Cta of a test bacterial liquid containing no antibiotics, which has been cultured for a required incubation time ta, an image Nta of a test bacterial liquid containing an ineffective antibiotic, and a valid label corresponding to the image, to the CNN 55a.

[0045] The CNN 55a can also learn information about the difference between an image of a test bacterial liquid that does not contain an antibacterial agent and an image of a test bacterial liquid that contains an antibacterial agent (effective antibacterial agent and ineffective antibacterial agent). As a result, when an image of a test bacterial liquid that does not contain an antibacterial agent and an image of a test bacterial liquid that contains an antibacterial agent at a required incubation time ta are input, the CNN 55a can output information about the effect of the antibacterial agent (e.g., effective, ineffective, effective level, etc.). Furthermore, by taking into account the image of the test bacterial liquid that does not contain an antibacterial agent, the result when the antibacterial agent is incubated for a predetermined incubation time tn can be estimated with even greater accuracy at the required incubation time ta.

[0046] Fig. 10 is a schematic diagram showing a fourth example of the learning method of the CNN 55a. Like the third example, the fourth example is a learning method in which the incubation time for learning is known in advance by some means. The difference from the third example is that an image C0 of the test bacteria liquid containing no antibiotic, captured at the incubation time t0 (the initial stage of incubation), is further used as training data.

[0047] That is, the CNN 55a is trained by providing training data combining an image C0 of a test bacterial liquid containing no antibacterial agent captured at a culture time t0, an image Cta of a test bacterial liquid containing no antibacterial agent cultured for a required culture time ta, an image Eta of a test bacterial liquid containing an effective antibacterial agent, and a valid label corresponding to the image to the CNN 55a. Similarly, the CNN 55a is trained by providing training data combining an image C0 of a test bacterial liquid containing no antibacterial agent captured at a culture time t0, an image Cta of a test bacterial liquid containing no antibacterial agent cultured for a required culture time ta, an image Nta of a test bacterial liquid containing an ineffective antibacterial agent, and a valid label corresponding to the image to the CNN 55a. Note that the training data may include an image E0 of a test bacterial liquid containing an effective antibacterial agent captured at a culture time t0, and an image N0 of a test bacterial liquid containing an ineffective antibacterial agent. Since this is immediately after the start of cultivation, from the perspective of the state of the test bacteria, image E0 of the test bacterial liquid containing an effective antibiotic and image N0 of the test bacterial liquid containing an ineffective antibiotic are considered to be identical to image C0 of the test bacterial liquid containing no antibiotic, so only image C0 of the test bacterial liquid containing no antibiotic may be used.

[0048] CNN55a can also learn information regarding differences in images of test bacterial liquids that do not contain an antibiotic, differences in images of test bacterial liquids that contain an effective antibiotic, and differences in images of test bacterial liquids that contain an ineffective antibiotic, which are caused by the time difference between the start of cultivation and the time when the required cultivation time has elapsed.

[0049] As a result, when an image of the test bacterial liquid not containing an antibiotic taken at the incubation time t0, an image of the test bacterial liquid not containing an antibiotic at the required incubation time ta, and an image of the test bacterial liquid containing an antibiotic are input, the CNN 55a can output information regarding the effectiveness of the antibiotic (e.g., effective, ineffective, effective level, etc.). By also taking into consideration the image of the test bacterial liquid not containing an antibiotic taken at the incubation time t0, the result when the antibiotic is incubated for a predetermined incubation time tn can be estimated with even greater accuracy at the required incubation time ta.

[0050] In the above-mentioned examples of Figures 8 to 10, a trained model is generated using an image at a constant culture time ta, but the present invention is not limited to this, and a trained model can be generated using images cultured for a plurality of different culture times within the required time range. Note that the examples of Figures 8 to 10 can be said to be cases in which an image cultured for one culture time within the required time range is used.

[0051] FIG. 11 is a schematic diagram showing a fifth example of a learning method for the CNN 55a. In the fifth example, the CNN 55a is generated using images from the incubation time range tb to tn. As shown in FIG. 11, the CNN 55a is given training data combining an image C0 of a test bacterial liquid not containing an antibacterial agent captured at the incubation time t0, images Cn, C(n-1), ..., Cb of a test bacterial liquid not containing an antibacterial agent at the incubation times tn, t(n-1), ..., tb, images En, E(n-1), ..., Eb of a test bacterial liquid containing an effective antibacterial agent at the incubation times tn, t(n-1), ..., tb, and effective labels corresponding to the images En, E(n-1), ..., Eb, to learn the CNN 55a. Similarly, the CNN 55a is provided with training data combining an image C0 of the test bacterial liquid not containing an antibacterial agent captured at the culture time t0, images Cn, C(n-1), ..., Cb of the test bacterial liquid not containing an antibacterial agent at the culture times tn, t(n-1), ..., tb, images Nn, N(n-1), ..., Nb of the test bacterial liquid containing an ineffective antibacterial agent at the culture times tn, t(n-1), ..., tb, and invalid labels corresponding to the images Nn, N(n-1), ..., Nb, to train the CNN 55a. The time span from the culture time tb to tn may be, for example, in the range of 3 hours to 24 hours, or may be in another range.

[0052] In addition, when providing training data to the CNN 55a, a culture time range flag can be input to the CNN 55a. The culture time range flag is a flag for making the CNN 55a recognize the culture time range, and may be, for example, the time range from culture time tb to tn, or each culture time from culture time tb to tn.

[0053] Since there are almost no bacteria in the image C0 of the test bacteria liquid not containing an antibacterial agent captured at the incubation time t0, an image captured at an early stage of incubation outside the required time range (for example, an image Ctt at the incubation time tt) may be used instead of the image C0. <tbである。

[0054] As a result, when images of a test bacterial liquid that does not contain an antibiotic and images of a test bacterial liquid that contains an antibiotic are input, CNN 55a can output information regarding the effectiveness of the antibiotic (e.g., effective, ineffective, effective level, etc.).

[0055] Even if a site where the effectiveness of an antimicrobial agent is tested is recommended to test by culturing for a certain shortest incubation time, it may be difficult to test for the recommended incubation time due to various situations and conditions at the site. In the example of Figure 11, the recommended incubation time can be set to a required range, improving the convenience of testing at the site.

[0056] In the example of Figure 11, the corresponding incubation times of the images of the test bacterial liquid containing an effective antibiotic and the images of the test bacterial liquid containing an ineffective antibiotic are the same, such as images En, E(n-1), ..., Eb of the test bacterial liquid containing an effective antibiotic at incubation times tn, t(n-1), ..., tb, and images Nn, N(n-1), ..., Nb of the test bacterial liquid containing an ineffective antibiotic at incubation times tn, t(n-1), ..., tb. However, this is not limited to this, and the corresponding incubation times of the images of the test bacterial liquid containing an effective antibiotic and the images of the test bacterial liquid containing an ineffective antibiotic may be different, and the intervals between adjacent incubation times may also be different.

[0057] FIG. 12 is a schematic diagram showing a sixth example of the learning method of the CNN 55a. In the sixth example, the CNN 55a is generated using images from the incubation time range te to th. As shown in FIG. 12, the CNN 55a is given training data combining an image C0 of the test bacterial liquid not containing an antibacterial agent captured at the incubation time t0, images Ch, C(h-1), ..., Ce of the test bacterial liquid not containing an antibacterial agent at the incubation times th, t(h-1), ..., te, images Eh, E(h-1), ..., Ee of the test bacterial liquid containing an effective antibacterial agent at the incubation times th, t(h-1), ..., te, and effective labels corresponding to the images Eh, E(h-1), ..., Ee, to learn the CNN 55a. Similarly, the CNN 55a is trained by providing training data combining an image C0 of the test bacterial liquid not containing an antibacterial agent captured at the culture time t0, images Ch, C(h-1), ..., Ce of the test bacterial liquid not containing an antibacterial agent at the culture times th, t(h-1), ..., te, images Nh, N(h-1), ..., Ne of the test bacterial liquid containing an ineffective antibacterial agent at the culture times th, t(h-1), ..., te, and invalid labels corresponding to the images Nh, N(h-1), ..., Ne to the CNN 55a. The time span from the culture time te to th may be, for example, in the range of 2 hours to 6 hours, or may be in another range.

[0058] In addition, when training data is provided to the CNN 55a, a culture time range flag can be input to the CNN 55a. The culture time range flag is a flag for making the CNN 55a recognize the culture time range, and may be, for example, the time range from culture time te to th, or each culture time from culture time te to th.

[0059] Since there are almost no bacteria in the image C0 of the test bacteria liquid not containing an antibacterial agent captured at the incubation time t0, an image captured at an early stage of incubation outside the required time range (for example, an image Ctt at the incubation time tt) may be used instead of the image C0. <teである。

[0060] As a result, when images of a test bacterial liquid that does not contain an antibiotic and images of a test bacterial liquid that contains an antibiotic are input, CNN 55a can output information regarding the effectiveness of the antibiotic (e.g., effective, ineffective, effective level, etc.).

[0061] Even if it is recommended to culture and test for a certain shortest incubation time at a site where the effectiveness of an antibiotic is tested, it may be difficult to test for the recommended incubation time due to various situations and conditions at the site. In the example of Figure 12, the recommended incubation time can be set to a required range, improving the convenience of on-site testing. In addition, generally, the detection accuracy is higher when images cultured for a longer incubation time (e.g., 16 hours, 18 hours, 24 hours, etc.) are used, but if the incubation time used at the site is about 3 to 6 hours, it is more accurate to use a trained model trained with the incubation time range used at the site, and it becomes possible to use a trained model that suits the situation at the site.

[0062] In the example of Figure 12, the corresponding incubation times of the images of the test bacterial liquid containing an effective antibiotic and the images of the test bacterial liquid containing an ineffective antibiotic are the same, such as images Eh, E(h-1), ..., Ee of the test bacterial liquid containing an effective antibiotic at incubation times th, t(h-1), ..., te, and images Nh, N(h-1), ..., Ne of the test bacterial liquid containing an ineffective antibiotic at incubation times th, t(h-1), ..., te. However, this is not limited to this, and the corresponding incubation times of the images of the test bacterial liquid containing an effective antibiotic and the images of the test bacterial liquid containing an ineffective antibiotic may be different, and the intervals between adjacent incubation times may also be different.

[0063] Next, the estimation of the effect of an antibacterial agent by the estimation device 50 will be described.

[0064] FIG. 13 is a schematic diagram showing a first example of a method for estimating the effect of an antibiotic using a trained model 55a. The trained model 55a is obtained by training the CNN 55a using the above-mentioned training method. The control unit 51 (estimation device 50) acquires an image of a test bacterial liquid containing an antibiotic, which has been cultured for a culture time tk1 shorter than a predetermined culture time tn. When an image of a test bacterial liquid containing an antibiotic, which has been cultured for a required culture time tk1, is input, the CNN 55a can output information about the effect of the antibiotic (e.g., effective). "Effective" may literally mean effective, or may be an effective level indicating the degree of the effect (the higher the level, the more effective, and the lower the level, the less effective). The CNN 55a may also output "ineffective" as information about the effect of the antibiotic.

[0065] The control unit 51 can estimate the effect of the antibacterial agent obtained after a predetermined incubation time tn in the case of a conventional visual inspection method at an early stage at the required incubation time tk1.

[0066] The above-mentioned configuration allows the effectiveness of an antibacterial drug to be estimated in a short time compared to the conventional visual inspection method. In addition, since a special container such as a microdevice is not required and a conventional container can be used as is, the effectiveness of an antibacterial drug can be estimated inexpensively and quickly.

[0067] When the trained model 55a shown in FIG. 13 is generated by the method shown in FIG. 9, the estimation device 50 can estimate the effect of the antibiotic as follows. That is, the control unit 51 further acquires an image of the test bacterial liquid containing no antibiotic, which is cultured for a culture time tk1 shorter than the predetermined culture time tn. When an image of the test bacterial liquid containing no antibiotic, which is cultured for the required culture time tk1, is further input, the trained model 55a can output information on the effect of the antibiotic (for example, effective). "Effective" may literally mean effective, or may be an effective level indicating the degree of the effect (the higher the level, the more effective, and the lower the level, the less effective). Although not shown, the trained model 55a may also output "ineffective" as information on the effect of the antibiotic.

[0068] The trained model 55a can output information regarding the effectiveness of the antibiotic, taking into account information regarding the image difference obtained by comparing an image of a test bacterial liquid that does not contain an antibiotic with an image of a test bacterial liquid that contains an antibiotic.

[0069] The control unit 51 can estimate the effect of the antibacterial agent obtained after a predetermined incubation time tn in the case of a conventional visual inspection method earlier and more accurately at the required incubation time tk1.

[0070] When the trained model 55a shown in FIG. 13 is generated by the method shown in FIG. 4 or FIG. 10, the estimation device 50 can estimate the effect of the antibiotic as follows. That is, the control unit 51 further acquires an image of the test bacterial liquid containing an antibiotic and an image of the test bacterial liquid not containing an antibiotic, which are captured at a culture time t0 (at the initial stage of culture). When an image of the test bacterial liquid containing an antibiotic and an image of the test bacterial liquid not containing an antibiotic, which are captured at the initial stage of culture, are further input, the trained model 55a can output information on the effect of the antibiotic (for example, effective). "Effective" may literally mean effective, or may be an effective level indicating the degree of the effect (the higher the level, the more effective, and the lower the level, the less effective). The trained model 55a can also output "ineffective" as information on the effect of the antibiotic.

[0071] The trained model 55a can output information regarding the effectiveness of the antibiotic, taking into account information regarding the difference between images of the test bacterial liquid containing the antibiotic at each of the culture times t0 and tk1, and information regarding the difference between images of the test bacterial liquid not containing the antibiotic at each of the culture times t0 and tk1.

[0072] The control unit 51 can estimate the effect of the antibacterial agent obtained after a predetermined incubation time tn in the case of a conventional visual inspection method earlier and more accurately at the required incubation time tk1.

[0073] FIG. 14 is a schematic diagram showing an example of a method for selecting a trained model. As shown in FIG. 14, the trained models include trained models 551, 552, and 553. The trained model 551 is a trained model generated by the method illustrated in FIG. 6, FIG. 11, or FIG. 12, for example, and is a trained model for a culture time range T1. Similarly, the trained model 552 is a trained model for a culture time range T2, and the trained model 553 is a trained model for a culture time range T3. The culture time ranges T1, T2, and T3 can be in the range of 1 hour to 24 hours, 1 hour to 8 hours, and 2 hours to 6 hours, respectively, but are not limited thereto.

[0074] The operation unit 57 receives the required culture time range selected by the user. The culture time range can be displayed on the selection screen so that the user can select it. The operation unit 57 outputs the received culture time range to the control unit 51. The control unit 51 can select one of the trained models 551, 552, and 553 according to the acquired culture time range. This allows the use of a trained model that is optimal for the culture time desired by the user. Note that a plurality of culture times may be displayed to the user so that the culture time selected by the user is within the culture time range and a CNN with a culture time range that is closest to the longest culture time of the culture time range may be selected. For example, if the culture time selected by the user is 5 hours, a trained model with a culture time range ranging from 2 hours to 6 hours can be selected from the above-mentioned culture time ranges.

[0075] FIG. 15 is a schematic diagram showing a second example of a method for estimating the effect of an antibacterial agent using a trained model 551. The example of FIG. 15 corresponds to a case where the trained model 551 is selected from the trained models 551, 552, and 553 in the example of FIG. 14. The control unit 51 (estimation device 50) acquires an image of the test bacterial liquid containing an antibacterial agent in the culture time range T1. Although only one image is illustrated in FIG. 15, each image when cultured for one or more culture times within the culture time range T1 can actually be acquired. When an image of the test bacterial liquid containing an antibacterial agent cultured for one or more culture times within the culture time range T1 is input, the trained model 551 can output information regarding the effect of the antibacterial agent (for example, effective). "Effective" may literally mean effective, or may be an effective level indicating the degree of the effect (the higher the level, the more effective, and the lower the level, the less effective). The trained model 551 may also output "ineffective" as information regarding the effect of the antibacterial agent.

[0076] The control unit 51 can estimate the effectiveness of the antibacterial agent based on the image of the test bacteria liquid containing the antibacterial agent cultured for a culture time within the required culture time range. This makes it possible to estimate the effectiveness of the antibacterial agent in the test bacteria liquid cultured for a desired culture time according to the situation and conditions at the site.

[0077] Although not shown in Figure 15, as in the case of Figure 13, images of the test bacterial liquid containing an antibiotic and images of the test bacterial liquid not containing an antibiotic taken at the early stage of cultivation can also be used.

[0078] Next, the evaluation results of the estimation of the effect of an antibacterial agent by the estimation device 50 of this embodiment will be described.

[0079] FIG. 16 is an explanatory diagram showing the evaluation results of the accuracy rate based on the validation data of each trained model. The method of acquiring the training data for generating each trained model is as follows. That is, the strains used were Escherichia coli (NBRC15034) and Staphylococcus aureus (ATCC25923). The bacterial liquid was prepared by adjusting the concentration to about 1.5×10 to the power of 8 (cfu / ml) and then mixing with about 12 ml of liquid medium to prepare the bacterial liquid. The culture method was to use a dry plate made by Eiken DP31, and drop 0.1 ml of the adjusted bacterial liquid into the wells of CAZ (ceftazidime) and MINO (minocycline) for both Escherichia coli and Staphylococcus aureus, and then dissolve the antibiotic over 15 minutes. These were transferred to a container that could be observed, and the state of the bacteria was photographed every 15 minutes while culturing in a time-lapse incubator (manufactured by ASTEC, CCM-iBIS). To determine the effectiveness, after 24 hours of incubation, each well was visually observed to determine the presence or absence of turbidity and bacterial clumps. If there were no turbidity or bacterial clumps, the well was labeled as effective (susceptible), and if there were, the well was labeled as effective (not susceptible).

[0080] The trained models are the 3H model, the 2H model, the 2H30MIN-3H30MIN model, and the 2H30MIN-6H model. Each model is explained below.

[0081] The learning method of the 3H model was to link images taken at 3 hours of incubation with control images and labels of effective or ineffective labels obtained by visual observation after 24 hours, and to train the CNN several hundred times. The training data consisted of half images labeled as effective and half labeled as ineffective. The verification method of the 3H model was to output prediction data of effective or ineffective for verification images taken every 15 minutes with the 3H model, and to verify the accuracy rate for each time period against the actual results after 24 hours. As shown in Figure 16, the accuracy rate exceeded 90% from 2 hours 15 minutes to 3 hours, and it can be judged that it can be used to estimate the effectiveness of antibiotics.

[0082] The difference between the 2H model and the 3H model is that the images taken at 2 hours of incubation were used, and the rest are the same as the 3H model. As shown in Figure 16, the accuracy rate exceeded 90% from 2 hours to 3 hours, and it can be judged that it can be used to estimate the effectiveness of antibiotics.

[0083] The 2H30MIN-3H30MIN model differs from the 3H model in that it uses images taken every 15 minutes between 2h30min and 3h30min of incubation time, and is otherwise the same as the 3H model. As shown in Figure 16, the accuracy rate exceeds 90% from 2h30min to 3h45min, and it can be determined that it can be used to estimate the effectiveness of antibiotics. It is believed that expanding the time range of the learning data can improve the detection accuracy over a wider range of time.

[0084] The difference between the 2H30MIN-6H model and the 3H model is that it uses images taken every 15 minutes between the incubation time of 2h30min and 5h45min, and the rest is the same as the 3H model. As shown in Figure 16, the accuracy rate exceeds 90% from 2h45min to 5h45min, and it can be judged that it can be used to estimate the effectiveness of antibiotics. It is thought that expanding the time range of the learning data can improve the detection accuracy over a wider range of time.

[0085] As Comparative Example 1, visual observation was performed for 3 hours and 5 hours 30 minutes using the same method as the conventional culture method (a method without using the estimation device 50 of this embodiment), but it was found that all the culture fluids in the wells were transparent, and the effects of the antibiotics could not be determined, or that all were determined to be effective. In other words, while the conventional visual method does not determine the effects of the antibiotics or erroneously estimates that the antibiotics are effective, the estimation device 50 of this embodiment makes it possible to estimate the effects of the antibiotics with high accuracy.

[0086] In Comparative Example 2, without assistance from a trained model, the presence or absence of differences between images of wells containing an antibiotic and those of a control was visually judged in photographed images. When the judgement was "effective if there is a difference" and "ineffective if there is no difference," there were cases where the accuracy rate of visual judgment by a human was significantly reduced because the antibiotic may not be effective even if some bacteria had changed or decreased. In contrast, the estimation device 50 of this embodiment can estimate the effectiveness of the antibiotic with high accuracy.

[0087] Fig. 17 is a block diagram showing a first other example of the estimation device 50. The difference from the estimation device 50 illustrated in Fig. 1 is that a trained model 550 is provided instead of the learning device 55, and other configurations are the same as those in Fig. 1. In the first example shown in Fig. 17, the estimation device 50 does not have a function for training a CNN, but has a trained model generated by another PC or the like equipped with a learning device. In this case, the estimation device 50 does not assume overwriting or modification of the trained model installed therein.

[0088] Fig. 18 is a block diagram showing a second example of the estimation device 50. The difference from the estimation device 50 illustrated in Fig. 1 is that a trained model 550 is provided instead of the learning device 55, and other configurations are the same as those in Fig. 1. Also, the difference from Fig. 17 is that the estimation device 50 can overwrite or modify the trained model installed therein. In this case, as shown in Fig. 18, an updated trained model can be obtained from a learning PC 60 equipped with a learning device.

[0089] Fig. 19 is a schematic diagram showing a first example of a display screen at the start of measurement when culturing and observing in a culture device. As shown in Fig. 19A, a light source 10, an observation container 20, and an imaging device 40 having an imaging unit 30 are integrated with the culture device. The imaging device 40 is connected to an external estimation device 50 (not shown). The culture device will be described in detail later.

[0090] A barcode, two-dimensional barcode, or IC tag attached to a blood, urine, or sputum sample is read by a reading device, and information such as the patient's name, patient ID, bacterial species, and gram positive / negative is input to the estimation device 50. At the same time, information on the reagent used (a test kit in which a bacterial liquid or test liquid is poured into an area where an antibacterial drug has been applied for testing), the name of the drug manufacturer, and the product name are also read, and information on which well 21 of the observation container 20 is applied with which type of antibacterial drug is also input to the estimation device 50. When a barcode or the like is read, the reading time may also be recorded or input. The measurement start time may also be recorded or input. At least one of the reading time and the measurement start time may be input from an input screen. It may also be possible to select whether to input the reading time or the measurement start time.

[0091] As shown in Fig. 19B, the input information is displayed on the display panel 56, and can be checked before starting measurement. If there are corrections or additions to be made to the displayed content, the input information can be corrected or added by operating the "Edit" icon. If there are no problems with the input information, the measurement can be started by operating the "OK" icon.

[0092] Fig. 20 is a schematic diagram showing a second example of the display screen at the start of measurement. As shown in Fig. 20, an image of each well 21 of the observation container 20 is displayed in the image display area 101. When a desired well 21 is selected on the selection screen 102, a selection screen for the type of antibiotic in the selected well 21 can be displayed. This allows the user to adjust the test contents by putting the desired type of antibiotic into each well in an empty container themselves, without using a commercially available observation container.

[0093] Although not shown in the figure, during measurement, for example, if any estimated results are obtained for the time elapsed since the start of measurement or for each well, the well number and estimated results can be displayed and a notification (using text, figures, audio, etc.) can be given to draw attention.

[0094] FIG. 21 is a schematic diagram showing an example of a display screen after the end of measurement. FIG. 21 shows an example of the output of the estimation result (measurement result) by the estimation device 50. The control unit 51 can display the estimation result as shown in FIG. 21 on the display panel 56. The control unit 51 can also output the estimation result to an external display device (not shown) so that it is displayed on the external display device. As shown in FIG. 21, the estimation result includes the patient name and the patient ID, and further includes an image display area 101 and an estimation result display area 103. The image display area 101 displays an image of each well 21 of the observation container 20 (in the example of FIG. 21, 20 wells are displayed together with their identification numbers (1 to 20). The well numbered 1 is a control well, and contains a test bacterium liquid that does not contain an antibacterial agent. The wells numbered 2 to 20 are test wells, and contain test bacterium liquids that contain different types and concentrations of antibacterial agents. It is possible to test at once what conditions of antibacterial agents are effective against the test bacterium.

[0095] Each of the wells numbered 2, 5, and 14 is displayed in a different display mode from the other wells (for example, it may be surrounded by a thick circle as shown in FIG. 21, or may be patterned or colored). It indicates that the antibiotics in each of the wells numbered 2, 5, and 14 are effective, and that the antibiotics in the other wells are ineffective. When the cursor is placed over each well and an operation is performed (for example, clicking, double clicking, etc.), an image of the test bacterial liquid in that well may be displayed. The desired culture time from the culture time t0 (the start time of culture) to the current culture time tk1 can be selected by moving the slider 102, for example, in units of 1 hour or 0.5 hour, and an image of the test bacterial liquid at the selected culture time can be displayed. This makes it easy to confirm the change in the image of the test bacterial liquid in each well.

[0096] The estimation result display area 103 displays the name of the test bacteria and detailed information of the test wells 2 to 20. For example, the detailed information includes the name of the antibacterial agent, the concentration of the antibacterial agent, whether or not it is effective, and an icon 104 for displaying a predicted image after culturing for a predetermined culturing time tn for each test well. The image processing unit 53 may use each collected image to generate a predicted image of the test bacteria liquid when the culture time reaches tn (for example, 24 hours) based on the image of the test bacteria liquid at the culture time tk1, or may select a predicted image from the collected images. The presence or absence of an effect may also include the reliability of the effect. The reliability may be an index (effectiveness level) that indicates the likelihood of the effect. The effective level may be expressed numerically, for example, 98%, 95%, 90%, 80%, ..., 20%. A higher effective level indicates "effective", and a lower effective level indicates "ineffective". This makes it possible to objectively indicate the likelihood of the estimation result.

[0097] The display screen displays an area 106 for inputting additional comments, and a "Print or Send" icon 107 for printing or sending the estimation results and comments.

[0098] FIG. 22 is a schematic diagram showing an example of a list display of measurement results. As shown in FIG. 22, the search setting area 107 displays selection or input fields for setting the measurement period, target patient, antibacterial drug manufacturer, antibacterial drug type, etc. as search conditions. When the search conditions are set, the measurement result list area 108 displays a list of measurement results that satisfy the search conditions. By setting the search conditions, the user can check the required measurement results from a large number of measurement results in a list. The display of the list of measurement results can be processed by the control unit 51. The list of measurement results is not limited to the example of FIG. 22.

[0099] Fig. 23 is a block diagram showing another example of the configuration of the estimation device 50. In Fig. 23, reference numeral 150 denotes a computer. The computer 150 includes a control unit 151, an interface unit 156, an operation unit 157, a display panel 158, a storage unit 159, and a recording medium reading unit 160. The control unit 151 includes a CPU (processor) 152, a ROM 153, a RAM 154, and an I / F (interface) 155.

[0100] 1, and the interface unit 52 corresponds to the operation unit 57, the display panel 56, and the storage unit 54. The control unit 151 corresponds to the control unit 51, the image processing unit 53, and the learning device 55 shown in FIG.

[0101] The recording medium reading unit 160 is capable of reading a computer program (for example, the processing procedures shown in FIGS. 24, 25, and 26 described later) from a recording medium M (for example, a medium such as a DVD) on which a computer program is recorded.

[0102] Although not shown, the computer program recorded on the recording medium M is not limited to one recorded on a medium that is easily portable, but may also include a computer program transmitted via the Internet or other communication lines.

[0103] Next, the operation of the estimation device 50 will be described.

[0104] 24 is a flowchart showing a first example of the processing procedure of the method for generating a learning model by the estimation device 50. In the following, for convenience, the control unit 51 will be described as the subject of the processing. The control unit 51 acquires time-series images of the test bacterial liquid containing no antibiotic, which has been cultured for a plurality of different culture times (S11). The time-series images of the test bacterial liquid containing no antibiotic are the time-series images (C0, C1, C2, ..., Cn) shown in FIG. 3 as an example.

[0105] The control unit 51 acquires time-series images of the test bacteria liquid containing an effective antibacterial agent, which has been cultured for a plurality of different culture times (S12). The time-series images of the test bacteria liquid containing an effective antibacterial agent are the time-series images (E0, E1, E2, ..., En) illustrated in FIG.

[0106] The control unit 51 acquires time-series images of the test bacteria liquid containing an ineffective antibacterial agent cultured for a plurality of different culture times (S13). The time-series images of the test bacteria liquid containing an ineffective antibacterial agent are the time-series images (N0, N1, N2, ..., Nn) illustrated in FIG.

[0107] The control unit 51 sets the longest incubation time (t(n-1)) (S14), and trains the learning model (CNN 55a) using as training data the images of the test bacterial liquid containing no antibacterial agent, the images of the test bacterial liquid containing an effective antibacterial agent, and the images of the test bacterial liquid containing an ineffective antibacterial agent, as well as the effective labels corresponding to the images of the test bacterial liquid containing an effective antibacterial agent and the invalid labels corresponding to the images of the test bacterial liquid containing an ineffective antibacterial agent, at the set incubation time (S15). A large number of images can be used for learning.

[0108] The control unit 51 calculates an estimation result of the learning model using the verification data (S16). For example, when 100 verification data are input to the CNN 55a, if 95 results are correct and 5 results are incorrect, the accuracy rate, i.e., the reliability, can be calculated as 95% (=95 / 100).

[0109] The control unit 51 determines whether the reliability is at a desired level (S17). Whether the reliability is at a desired level can be determined based on whether the reliability is equal to or greater than a threshold value.

[0110] If the reliability is at the desired level (YES in S17), the control unit 51 can determine that there is room to further shorten the incubation time, and so sets the incubation time shorter by a unit time (e.g., 1 hour, 0.5 hour, etc.) (S18), and continues processing from step S15 onwards.

[0111] If the reliability is not at a desired level (NO in S17), the control unit 51 can determine that the culture time longer by a unit time than the culture time set in step S18 is the shortest culture time, and therefore determines a culture time range including the culture time (S19), generates a learning model (S20), and ends the process. The culture time range can be determined by using, for example, a method as illustrated in FIG. 6.

[0112] In the processing procedure (step S15) shown in FIG. 24, an image C0 of the test bacteria liquid containing no antibacterial agent at the incubation time t0 can also be added to the training data.

[0113] Fig. 25 is a flowchart showing a second example of the processing procedure of the learning model generation method by the estimation device 50. The control unit 51 acquires an image of the test bacterial liquid containing an effective antibacterial agent and an image of the test bacterial liquid containing an ineffective antibacterial agent, which have been cultured for one or more different culture times within a required time span (S31). The image of the test bacterial liquid containing an effective antibacterial agent and the image of the test bacterial liquid containing an ineffective antibacterial agent are image Eta and image Nta, exemplified in Fig. 10.

[0114] The control unit 51 acquires an image of the test bacteria liquid containing no antibiotic, which has been cultured for one or more different culture times within the required time span (S32). The image of the test bacteria liquid containing no antibiotic is image Cta, as shown in FIG.

[0115] The control unit 51 acquires an image of the test bacterial liquid containing an effective antibacterial agent, an image of the test bacterial liquid containing an ineffective antibacterial agent, and an image of the test bacterial liquid containing no antibacterial agent, all of which were captured in the early stage of culture (S33). The image of the test bacterial liquid containing no antibacterial agent is image C0, as exemplified in Fig. 10. Note that the image of the test bacterial liquid containing an effective antibacterial agent and the image of the test bacterial liquid containing an ineffective antibacterial agent, all of which were captured in the early stage of culture, do not need to be used.

[0116] The control unit 51 trains a learning model (CNN 55a) using each acquired image and the valid or invalid label corresponding to each image (S34), and determines whether or not there is an image for learning (S35). If there is an image for learning (YES in S35), the control unit 51 continues the process from step S31 onwards, and if there is no image for learning (NO in S35), the control unit 51 generates a learning model (S36) and ends the process.

[0117] 26 is a flowchart showing an example of a processing procedure of a method for estimating the effect of an antibacterial agent by the estimation device 50. The control unit 51 acquires an image of a test bacterial liquid containing an antibacterial agent that has been cultured for a required culture time (S51), and acquires an image of a test bacterial liquid containing no antibacterial agent that has been cultured for the culture time (S52).

[0118] The control unit 51 acquires an image of the test bacterial liquid containing an antibacterial agent captured at an early stage of culture, and an image of the test bacterial liquid containing an antibacterial agent (S53). Note that the image of the test bacterial liquid containing an antibacterial agent captured at an early stage of culture does not need to be used.

[0119] The control unit 51 inputs each acquired image to the trained model (CNN 55a) (S54), outputs the estimation result of the trained model (S55), and ends the processing.

[0120] Next, a description will be given of the imaging device 40 of this embodiment. As described above, the imaging device 40 includes the light source 10, the observation container 20, and the imaging unit 30.

[0121] FIG. 27 is a schematic diagram showing a first example of the configuration of the imaging system of the imaging device 40. The distance L between the observation surface of the test bacteria liquid and the light receiving surface of the imaging element 311 (e.g., photodiode), and the incident angle θ of the light (straight light) from the light source on the light receiving surface are set to have a required relationship (required value). When the shape of the imaging element 311 in a plan view is a square, the vertical or horizontal dimension is 2S (e.g., 1.2 μm), and the shape of a bacterium of size P (e.g., about 1 μm) is captured, the condition for focusing can be tan θ=S / L, since it is sufficient to suppress the reflection of light on the adjacent imaging element 311. For example, when L=1000 μm, θ is 0.04°. That is, by controlling the incident angle θ to within ±0.04°, the test bacteria can be imaged in focus.

[0122] The distance between the bacteria observation surface of the observation container 20 and the imaging device 31 is L1, the vertical or horizontal dimension of the imaging device 31, which is a square in plan view, is W, and the distance between the light source 10 and the light receiving surface of the imaging device 31 is D. If the dimension W of the imaging device 31 is 144 μm, the distance L1 is 1000 μm, and the incident angle θ is 0.04°, the distance D can be about 20 cm. That is, by disposing the light receiving surface of the imaging device 31 at a position 1000 μm below the bacteria observation surface of the observation container 20 and disposing the light source 10 about 20 cm above the light receiving surface of the imaging device 31, it is possible to image the test bacteria in a focused state, and an expensive optical microscope is not required. Note that FIG. 27 is a schematic diagram of the imaging system, and is illustrated for convenience and ease of understanding, and therefore differs from the actual dimensions.

[0123] FIG. 28 is a schematic diagram showing a second example of the configuration of the imaging system of the imaging device 40. As shown in FIG. 28, when a plurality of imaging devices 31 are arranged adjacent to each other, the light from the light source 10 may interfere with the light receiving surface of the imaging device 31. If the distance between the light source 10 and the light receiving surface is D and the pitch of the adjacent imaging devices 31 is Q, the condition tanα=Q / D may be satisfied in order for the light from the light source 10 not to interfere with the light receiving surface of the adjacent imaging devices. If the distance D is 20 cm and the pitch Q is 1000 μm, the viewing angle α is 2.86°, so an optical film 36 (e.g., a louver film) that can be controlled to the viewing angle α (=2.86°) may be provided. The imaging device 40 of this embodiment may be configured using a normal optical microscope, and the optical measurement method is not limited to the examples of FIG. 27 and FIG. 28.

[0124] Next, the combination with a culture device will be described.

[0125] 29 is a schematic diagram showing an example of an antimicrobial effect estimation system including a culture device 70. The estimation device 50 is connected to a communication network. The communication network may be wireless or wired. An imaging device 40, a PC (information processing device) 80 that controls the culture device 70, an electronic medical record server 100, testing devices 91 and 92, and a terminal device 90 used by medical professionals such as doctors are connected to the communication network.

[0126] The culture device 70 includes a controller 71, a thermostatic chamber 72, a drive unit 75 for moving the image capture device 40 inside and outside the thermostatic chamber 72, and the like. The culture device 70 may also include an operation panel (not shown) for setting culture conditions and the like. Setting of culture conditions and operation of the image capture device 40 may be performed from a PC 80. The image capture device 40 is carried into the thermostatic chamber 72 by the drive unit 75, and images of the test bacteria liquid are repeatedly taken at each culture time while culturing. When the culture is completed, the image capture device 40 is carried out of the thermostatic chamber 72. The image of the test bacteria liquid captured by the image capture device 40 can be output to the estimation device 50 via the PC 80. It is not necessary to provide the PC 80. In that case, the image of the test bacteria liquid captured by the image capture device 40 can be directly output to the estimation device 50.

[0127] The electronic medical record server 100 is a server that manages the electronic medical records of patients, and can acquire test data from the testing devices 91, 92 used when testing patients, and record the data in the electronic medical records. The testing devices 91, 92 include, for example, blood test devices, urine test devices, ultrasound devices, X-ray devices, CT image testing devices, etc., but may also be testing devices with other functions. The terminal device 90 can be linked to the electronic medical records and allows the estimation results of the estimation device 50 to be viewed.

[0128] Furthermore, the estimation result by the estimation device 50 can be recorded on an external PC, a server on the cloud, or a recording medium.

[0129] As described above, the antimicrobial effect estimation system includes the incubation device 70 capable of accommodating the imaging device 40. The incubation device 70 may be configured so that the imaging device 40 is built into the thermostatic chamber 72.

[0130] Fig. 30 is a schematic diagram showing an example of the configuration of the culture device 70 and the imaging device 40. Fig. 30 shows a case where the culture device 70 and the imaging device 40 are configured as separate devices. The observation container 20, which has been cultured for the required culture time in the culture device 70, is removed from the culture device 70, and the removed observation container 20 is set in the imaging device 40 to start measurement. The other configuration is the same as in Fig. 29, so a description thereof will be omitted.

[0131] In this embodiment, there is no need to wait 24 hours or more for any changes that would allow visual detection of an increase or decrease in the number of bacteria to occur. Instead, after mixing with a specific drug, images that can be observed macroscopically are immediately obtained, and by linking these data with the results 24 hours later, the effectiveness of the antibiotic can be rapidly estimated.

[0132] In addition, the container to be observed does not require any special ingenuity, and for example, by preparing an imaging unit capable of imaging each well according to the number of wells in the observation container, the effect estimation method of this embodiment can be applied to any observation container, enabling simple and low-cost image acquisition. As a result, it is possible to acquire images and estimate effects in the form of reagents and containers that have been used for conventional judgments, and judgments and inferences based on past experience and performance can be made more quickly (in a short time) than before.

[0133] In this embodiment, the images used for training, verification, and effect estimation may be all or part of the images captured by the imaging device 40, or may be images obtained by processing the captured images using required processing. Processing includes, for example, processing to emphasize or de-emphasize the characteristics of the test bacteria.

[0134] In this embodiment, the image captured by the imaging device 40 is used for training, verification, and effect estimation, but the present invention is not limited to this. Features may be extracted from the image and the extracted features may be used for training, verification, and effect estimation. For example, parameters such as the number of bacteria, the shape of the bacteria, the size of the bacteria, the color of the bacteria, and the density of the bacteria that can be extracted from the image of the test bacteria liquid may be quantified as the feature. The feature extraction may be performed by the image processing unit 53.

[0135] In this embodiment, by acquiring an image at the bottom surface of the well, it is possible to easily adjust the focus on the bacteria and acquire a highly accurate image.

[0136] In this embodiment, the concentration of the test bacteria liquid can be photographed at 10 to the power of 5 to 10 to the power of 8. This means that there is a certain density and quantity when photographing, and it is possible to photograph a necessary and sufficient number of bacteria to estimate the effectiveness of an antibacterial drug.

[0137] The computer program of this embodiment causes a computer to acquire an image of a test bacterial liquid containing an antibiotic that has been cultured for a required culture time, and when an image of the test bacterial liquid containing an antibiotic is input, the acquired image of the test bacterial liquid containing an antibiotic is input into a trained model that outputs information regarding the effectiveness of the antibiotic, and the effectiveness of the antibiotic is estimated.

[0138] The method for estimating the effectiveness of an antibiotic in this embodiment acquires an image of a test bacterial liquid containing an antibiotic that has been cultured for a required culture time, inputs the acquired image of the test bacterial liquid containing an antibiotic into a trained model that outputs information regarding the effectiveness of the antibiotic when an image of the test bacterial liquid containing an antibiotic is input, and estimates the effectiveness of the antibiotic.

[0139] The learning model generation method of this embodiment acquires images of a test bacterial liquid containing an effective antibiotic that has been cultured for one or more different culture times within a required time range, acquires images of a test bacterial liquid containing an ineffective antibiotic that has been cultured for one or more different culture times within the required time range, acquires valid labels corresponding to the images of the test bacterial liquid containing the effective antibiotic and invalid labels corresponding to the images of the test bacterial liquid containing the ineffective antibiotic, and generates a learning model that outputs information regarding the effectiveness of the antibiotic using the acquired images, valid labels, and invalid labels.

[0140] The antibiotic effect estimation system of this embodiment includes an acquisition unit that acquires an image of a test bacterial liquid containing an antibiotic that has been cultured for a required culture time, and an estimation unit that inputs the image of the test bacterial liquid containing an antibiotic acquired by the acquisition unit into a trained model that outputs information regarding the effect of the antibiotic when an image of the test bacterial liquid containing an antibiotic is input, and estimates the effect of the antibiotic.

[0141] The computer acquires an image of the test bacteria liquid containing the antibacterial agent, which has been cultured for a required culture time. The required culture time may be, for example, equal to or less than the culture time specified in the drug susceptibility test. When the acquired image is input to the trained model, the trained model outputs information regarding the effect of the antibacterial agent, so that the computer can estimate the effect of the antibacterial agent.

[0142] The trained model can be generated as follows: Images of a test bacterial liquid containing an effective antibacterial drug and an image of a test bacterial liquid containing an ineffective antibacterial drug, both cultured for a required culture time, are obtained. Training data combining images of a test bacterial liquid containing an effective antibacterial drug and a corresponding effective label for the image, as well as training data combining images of a test bacterial liquid containing an ineffective antibacterial drug and a corresponding invalid label for the image, are provided to the learning model for training.

[0143] When an image of a test bacterial liquid containing an antibiotic that has been cultured for the required culture time is input, the trained model outputs information about the effectiveness of the antibiotic (for example, whether the antibiotic is effective or ineffective, or the effectiveness level (the higher the level, the more effective it is, and the lower the level, the less effective it is)), making it possible to estimate the effectiveness of the antibiotic obtained by conventional visual inspection methods for the required culture time.

[0144] The above-mentioned configuration allows the effectiveness of an antibacterial drug to be estimated in a short time compared to the conventional visual inspection method. In addition, since a special container such as a microdevice is not required and a conventional container can be used as is, the effectiveness of an antibacterial drug can be estimated inexpensively and quickly.

[0145] The computer program of this embodiment causes a computer to acquire an image of an antibiotic-free test bacterial liquid that has been cultured for the required culture time, and when an image of the antibiotic-free test bacterial liquid is further input, input the acquired image of the antibiotic-free test bacterial liquid into the trained model, which outputs information regarding the effectiveness of the antibiotic, and execute a process to estimate the effectiveness of the antibiotic.

[0146] The learning model generation method of this embodiment acquires images of antibiotic-free test bacterial liquid cultured for one or more different incubation times within the required time range, and generates the learning model using the acquired images of the antibiotic-free test bacterial liquid.

[0147] In the antibiotic effect estimation system of this embodiment, the acquisition unit acquires an image of the antibiotic-free test bacterial liquid that has been cultured for the required culture time, and the estimation unit inputs the image of the antibiotic-free test bacterial liquid acquired by the acquisition unit into the trained model, which outputs information regarding the antibiotic effect when an image of the antibiotic-free test bacterial liquid is further input, and estimates the antibiotic effect.

[0148] The computer acquires images of the test bacterial liquid containing no antibiotics that have been cultured for a required incubation time. When the acquired images are further input to the trained model, the trained model outputs information on the effectiveness of the antibiotics while also taking into account the images of the test bacterial liquid containing no antibiotics, allowing the computer to estimate the effectiveness of the antibiotics with even greater accuracy.

[0149] The learning model can be generated as follows. That is, images of test bacterial liquids containing no antibacterial agent, cultured for one or more different culture times within a required time span, are obtained. Training data combining images of test bacterial liquids containing no antibacterial agent and images of test bacterial liquids containing effective antibacterial agents with effective labels corresponding to the images, as well as training data combining images of test bacterial liquids containing no antibacterial agent and images of test bacterial liquids containing ineffective antibacterial agents with invalid labels corresponding to the images, are provided to the learning model for learning. The learning model can also learn information regarding the difference between images of test bacterial liquids containing no antibacterial agent and images of test bacterial liquids containing antibacterial agents (effective antibacterial agents and ineffective antibacterial agents).

[0150] When images of the test bacterial liquid containing no antibiotic that was cultured for the required culture time are further input, the learning model (trained model) outputs information regarding the effectiveness of the antibiotic (e.g., whether the antibiotic is effective or ineffective, or the effectiveness level (the higher the level, the more effective; the lower the level, the less effective)), thereby making it possible to more accurately estimate the effectiveness of the antibiotic obtained by conventional visual methods, taking into account images of the test bacterial liquid containing no antibiotic for the required culture time.

[0151] The computer program of this embodiment causes a computer to acquire images of a test bacterial liquid containing an antibiotic and an image of a test bacterial liquid not containing an antibiotic that have been cultured for a culture time shorter than the required culture time, and when further inputting images of a test bacterial liquid containing an antibiotic and an image of a test bacterial liquid not containing an antibiotic that have been cultured for a culture time shorter than the required culture time, input the acquired images of the test bacterial liquid containing an antibiotic and the image of the test bacterial liquid not containing an antibiotic into the trained model that outputs information regarding the effect of the antibiotic, and execute a process to estimate the effect of the antibiotic.

[0152] The learning model generation method of this embodiment acquires images of the test bacterial liquid containing the effective antibiotic, an image of the test bacterial liquid containing the ineffective antibiotic, and an image of the test bacterial liquid containing no antibiotic, all of which are captured at an early stage of cultivation outside the required time range, and generates the learning model using the acquired images of the test bacterial liquid containing the effective antibiotic, the image of the test bacterial liquid containing the ineffective antibiotic, and the image of the test bacterial liquid containing no antibiotic.

[0153] The computer acquires images of the test bacterial liquid containing an antibacterial agent and images of the test bacterial liquid not containing an antibacterial agent, which have been cultured for a culture time shorter than the required culture time. The culture time shorter than the required culture time can be, for example, the start time of culture. When the acquired images are further input to the trained model, the trained model takes into account images of the test bacterial liquid in which the effect of the antibacterial agent is not apparent, and outputs information regarding the effect of the antibacterial agent, so that the computer can estimate the effect of the antibacterial agent with even greater accuracy.

[0154] The learning model can be generated as follows. That is, images of the test bacterial liquid containing an effective antibacterial agent, an image of the test bacterial liquid containing an ineffective antibacterial agent, and an image of the test bacterial liquid containing no antibacterial agent, which are taken at an early stage of culture outside the required time range, are obtained. Training data combining images of the test bacterial liquid containing no antibacterial agent and an image of the test bacterial liquid containing an effective antibacterial agent taken at the early stage of culture, and images of the test bacterial liquid containing no antibacterial agent and an image of the test bacterial liquid containing an effective antibacterial agent cultured for the required culture time, and effective labels corresponding to the images, are provided to the learning model for learning, and training data combining images of the test bacterial liquid containing no antibacterial agent and an image of the test bacterial liquid containing an ineffective antibacterial agent taken at the early stage of culture, and images of the test bacterial liquid containing no antibacterial agent and an image of the test bacterial liquid containing an ineffective antibacterial agent cultured for the required culture time, and invalid labels corresponding to the images, are provided to the learning model for learning. The learning model can also learn information regarding, for example, differences between images of test bacterial liquids that do not contain an antibiotic, differences between images of test bacterial liquids that contain an effective antibiotic, and differences between images of test bacterial liquids that contain an ineffective antibiotic, which are caused by the time difference between the start of cultivation and the time when the required cultivation time has elapsed.

[0155] When images of a test bacterial liquid containing an antibiotic and an image of a test bacterial liquid not containing an antibiotic that have been cultured for a culture time shorter than the required culture time are further input, the learning model (trained model) outputs information regarding the effect of the antibiotic (e.g., whether the antibiotic is effective or ineffective, or the effectiveness level (the higher the level, the more effective it is, and the lower the level, the less effective it is)).Therefore, by taking into account images of a test bacterial liquid containing an antibiotic and an image of a test bacterial liquid not containing an antibiotic that have been taken for a culture time shorter than the required culture time (e.g., in the early stages of culture), it is possible to more accurately estimate the effect of the antibiotic obtained by conventional visual inspection methods.

[0156] The computer program of this embodiment causes a computer to execute a process of acquiring multiple images of test bacterial liquids containing antibiotics of different types and / or concentrations cultured for the incubation time, inputting the acquired multiple images into the trained model, and estimating the effects of each of the antibiotics.

[0157] The learning model generation method of this embodiment acquires images of test bacterial solutions containing antimicrobial agents of different types and / or concentrations placed in multiple containers, and generates the learning model using the acquired images.

[0158] The computer acquires multiple images of the test bacterial liquid containing antibacterial agents differing in at least one of type and concentration, which have been cultured for a required culture time. Different types and concentrations of antibacterial agents can be placed in each well of an observation container in which multiple wells (holes, containers) are arranged. The learning model is generated using images of the test bacterial liquid containing antibacterial agents differing in at least one of type and concentration, which are placed in each of the multiple containers. When the multiple acquired images are input, the learning model outputs information regarding the effectiveness of the antibacterial agent for each image, so that the computer can estimate the effectiveness of each antibacterial agent.

[0159] The computer program of this embodiment causes a computer to execute a process of outputting the reliability of the effect of the antimicrobial agent.

[0160] The computer outputs the reliability of the effectiveness of the antibiotic. The reliability may be, for example, the presence or absence of an effect, or an index (effectiveness level) that indicates the likelihood of the effect. The effectiveness level can be expressed numerically, for example, 98%, 95%, 90%, 80%, ..., 20%. A higher effectiveness level indicates "effectiveness," and a lower effectiveness level indicates "ineffectiveness." This makes it possible to objectively express the likelihood of the estimated result.

[0161] The computer program of this embodiment causes the computer to execute a process of receiving a required incubation time range, selecting a trained model corresponding to the received incubation time range, and estimating the effects of each of the antibiotics using the selected trained model.

[0162] This allows the user to use a trained model that is optimal for the culture time desired.

[0163] In the computer program of this embodiment, the required incubation time is shorter than a predetermined incubation time for estimating the effect of the antibacterial agent in a predetermined method.

[0164] The predetermined incubation time can be the incubation time required to visually determine the effect of the antibacterial agent using a predetermined method. The required incubation time can be the incubation time that allows the effect of the antibacterial agent due to the predetermined incubation time (i.e., the effect of the antibacterial agent by visual inspection using a predetermined method) to be estimated by image judgment. This makes it possible to estimate the effect of the antibacterial agent in a shorter time than with the conventional visual inspection method.

[0165] The learning model generation method of this embodiment acquires time series images of a test bacterial liquid containing no antibiotic cultured for a plurality of different culture times, acquires time series images of a test bacterial liquid containing an effective antibiotic cultured for the plurality of different culture times, and time series images of a test bacterial liquid containing an ineffective antibiotic cultured for the plurality of different culture times, and while changing the culture time from the longest to the shortest of the plurality of different culture times, generates a learning model using the images of the test bacterial liquid containing no antibiotic cultured for each of the culture times, the images of the test bacterial liquid containing an effective antibiotic cultured for each of the culture times, and valid labels corresponding to the images of the test bacterial liquid containing the effective antibiotic, and invalid labels corresponding to the images of the test bacterial liquid containing an ineffective antibiotic cultured for each of the culture times, and the images of the test bacterial liquid containing the ineffective antibiotic cultured for each of the culture times.

[0166] The method for generating a learning model first acquires time-series images (C0, C1, C2, …, Cn) of a test bacterial solution without an antibacterial agent cultured at a plurality of different culture times (for example, t0, t1, t2, …, tn). Here, let t0 < t1 < t2 < … < tn. t0 is, for example, the time point at the start of culture and is within a time period when the effect of the antibacterial agent cannot be determined. tn is, for example, a time when the effect of the antibacterial agent can be determined by a conventional visual method. Further, time-series images (E0, E1, E2, …, En) of a test bacterial solution containing an effective antibacterial agent and time-series images (N0, N1, N2, …, Nn) of a test bacterial solution containing an ineffective antibacterial agent, both cultured at a plurality of different culture times (for example, t0, t1, t2, …, tn), are acquired.

[0167] Next, while changing the culture time from the longer to the shorter among the plurality of different culture times, a learning model is generated using the image Ct of the test bacterial solution without an antibacterial agent cultured at each culture time t, the image Et of the test bacterial solution containing an effective antibacterial agent cultured at each culture time t and the reliability of the effect of the effective antibacterial agent, and the image Nt of the test bacterial solution containing an ineffective antibacterial agent cultured at each culture time t and the reliability of the effect of the ineffective antibacterial agent.

[0168] Specifically, the learning model can be learned using each image C(n - 1), E(n - 1), N(n - 1), and the reliability at the culture time t(n - 1), and by inputting verification data (for example, a plurality of images of a test bacterial solution containing an antibacterial agent at the culture time t(n - 1)) into the learning model, the correct answer rate can be obtained and calculated as the reliability R(n - 1). Since the culture time of each image used for learning is close to the time tn, the correct answer rate, that is, the reliability, becomes high.

[0169] Next, a learning model is trained using each image C(n-2), E(n-2), N(n-2) and the reliability at the culture time t(n-2) shortened by a unit time, and verification data (e.g., multiple images of the test bacterial liquid containing an antibiotic at the culture time t(n-2)) is input to the learning model to obtain the accuracy rate, which can be calculated as the reliability R(n-2). By gradually shortening the culture time, the time tn is moved away, and therefore the accuracy rate, i.e., the reliability, is considered to gradually decrease. Therefore, since the shortest culture time can be obtained within the range where the required reliability can be obtained, the learning result at the shortest culture time can be used to generate a learning model.

[0170] The learning model generation method of this embodiment determines an incubation time range that includes one or more different incubation times at which a desired confidence level in the effect of the effective antibiotic is obtained when the incubation time is changed from a long to a short incubation time.

[0171] The learning model generation method determines an incubation time range that includes one or more different incubation times at which a desired confidence level of an effective antimicrobial effect is obtained when the incubation time is changed from long to short, thereby making it possible to adopt a required incubation time with a reasonable confidence level.

[0172] The antibiotic effect estimation system of this embodiment includes an imaging device having a plurality of containers capable of holding test bacterial liquids that do not contain antibiotics and test bacterial liquids that contain antibiotics, a light source that irradiates light from one side of the plurality of containers, and a plurality of imaging devices that are positioned on the opposite side of the light source with the plurality of containers between them and are capable of imaging the test bacterial liquid in the containers for each container.

[0173] The antibacterial effect estimation system includes an imaging device. The imaging device includes a plurality of containers capable of containing a test bacterial liquid that does not contain an antibacterial agent and a test bacterial liquid that contains an antibacterial agent, a light source that irradiates light from one side of the plurality of containers, and a plurality of imaging devices that are arranged on the opposite side of the light source across the plurality of containers and can image the test bacterial liquid in each container. This allows the imaging device to acquire an image of the test bacterial liquid by irradiating light (e.g., straight light) from the light source toward the test bacterial liquid contained in each of the plurality of containers. In addition, by arranging a plurality of imaging devices corresponding to each of the plurality of containers, for example, a drive system that is required when imaging the test bacterial liquid in the plurality of containers by moving either a single imaging device or the containers relatively (moving in the XY direction) is not required, and blurring of the image due to the drive system can be prevented. In addition, an expensive microscope is not required.

[0174] In the antibiotic effectiveness estimation system of this embodiment, the distance between the observation surface of the test bacterial liquid and the imaging device, and the angle of incidence of light from the light source on the light receiving surface of the imaging device are set to required values.

[0175] The distance L between the observation surface of the test bacteria liquid and the imaging device, and the incident angle θ of the light from the light source on the light receiving surface of the imaging device are set to required values. If the light from the light source is a straight light (e.g. laser light), the light receiving surface is a square with a vertical or horizontal dimension of S, the condition for capturing an in-focus image of the bacteria on the imaging device can be expressed as tan θ=S / (2·L).

[0176] The antimicrobial effect estimation system of this embodiment includes an optical film on the emission side of the light source.

[0177] The antimicrobial efficacy estimation system is equipped with an optical film on the emission side of the light source. If the distance between the light source and the light receiving surface is D and the pitch of adjacent imaging devices is Q, then in order for the light from the light source to not interfere with the light receiving surfaces of adjacent imaging devices, the condition tan α=Q / D needs to be satisfied, and therefore an optical film (such as a louver film) that can control the viewing angle α should be provided. This makes it possible to prevent the light from the light source from interfering with the light receiving surfaces of the imaging devices.

[0178] The antimicrobial effect estimation system of this embodiment includes a culture device capable of accommodating the imaging device.

[0179] The antimicrobial effect estimation system includes an incubation device capable of accommodating an imaging device. The incubation device may have a thermostatic chamber and may be configured to incorporate the imaging device in the thermostatic chamber, or may be configured to be capable of loading and unloading the imaging device into and from the thermostatic chamber using a drive unit. [Explanation of symbols]

[0180] 10 light source 20 Observation Container 21 wells 30 Imaging unit 31 Imaging Device 311 Image sensor 36 Optical Film 40 Imaging device 50 Estimation device 51 Control section 52, 156 Interface section 53 Image processing section 54, 159 Memory section 55 Learning Unit 55a CNN 56, 158 Display panel 57, 157 Operation section 60 PC for learning 70 Culture equipment 71 Controller 72 Constant temperature bath 75 Drive unit 80 PC 90 Terminal Equipment 91, 92 Inspection equipment 100 Electronic medical record server 150 Computers 151 Control section 152 CPU 153 ROM 154 RAM 155 I / F 160 Recording medium reading unit

Claims

1. On the computer, Acquire an image of a test bacteria liquid containing an antibacterial agent that has been cultured for a required culture time that allows the effect of the antibacterial agent over time to be estimated by image judgment, and Obtaining an image of the test bacteria liquid containing no antibacterial agent cultured for the required incubation time; Obtaining images of a test bacterial liquid containing an antibacterial agent and an image of a test bacterial liquid not containing an antibacterial agent, the test bacterial liquid being cultured at the start of culture or immediately after the start of culture for a culture time in which the effect of the antibacterial agent cannot be determined; an image of a test bacterial liquid containing an antibacterial agent, an image of a test bacterial liquid not containing an antibacterial agent, and an image of a test bacterial liquid containing an antibacterial agent and an image of a test bacterial liquid not containing an antibacterial agent that have been cultured at the start of culture or immediately after the start of culture for a culture time at which the effect of the antibacterial agent cannot be determined, are input into a trained model that outputs information regarding the effect of the antibacterial agent when the acquired image of the test bacterial liquid containing an antibacterial agent, the acquired image of the test bacterial liquid not containing an antibacterial agent, and the acquired image of the test bacterial liquid containing an antibacterial agent and an image of the test bacterial liquid not containing an antibacterial agent that have been cultured at the start of culture or immediately after the start of culture for a culture time at which the effect of the antibacterial agent cannot be determined are input, and the effect of the antibacterial agent is estimated; A computer program that executes a process.

2. On the computer, Obtaining a plurality of images of test bacterial solutions containing different types and / or concentrations of antimicrobial agents cultured for the culture time; The acquired images are input into the trained model to estimate the effects of each of the antibiotics.

2. A computer program product as claimed in claim 1, which is adapted to carry out a process.

3. On the computer, The computer program according to claim 1 or 2, which executes a process of outputting a reliability of the effect of the antibacterial drug.

4. On the computer, Accept the required incubation time range, Select a trained model that corresponds to the received incubation time range, Using the selected trained model, estimate the effect of each of the antibiotics; A computer program product according to any one of claims 1 to 3, which is used to execute a process.

5. The computer program according to claim 1 , wherein the required incubation time is shorter than a predetermined incubation time for visually estimating the effect of the antibacterial agent.

6. Acquire an image of a test bacteria liquid containing an antibacterial agent that has been cultured for a required culture time that allows the effect of the antibacterial agent over time to be estimated by image judgment, and Obtaining an image of the test bacteria liquid containing no antibacterial agent cultured for the required incubation time; Obtaining images of a test bacterial liquid containing an antibacterial agent and an image of a test bacterial liquid not containing an antibacterial agent, the test bacterial liquid being cultured at the start of culture or immediately after the start of culture for a culture time in which the effect of the antibacterial agent cannot be determined; an image of a test bacterial liquid containing an antibacterial agent, an image of a test bacterial liquid not containing an antibacterial agent, and an image of a test bacterial liquid containing an antibacterial agent and an image of a test bacterial liquid not containing an antibacterial agent that have been cultured at the start of culture or immediately after the start of culture for a culture time at which the effect of the antibacterial agent cannot be determined, are input into a trained model that outputs information regarding the effect of the antibacterial agent when the acquired image of the test bacterial liquid containing an antibacterial agent, the acquired image of the test bacterial liquid not containing an antibacterial agent, and the acquired image of the test bacterial liquid containing an antibacterial agent and an image of the test bacterial liquid not containing an antibacterial agent that have been cultured at the start of culture or immediately after the start of culture for a culture time at which the effect of the antibacterial agent cannot be determined are input, and the effect of the antibacterial agent is estimated; Methods for estimating the effectiveness of antibiotics.

7. A first acquisition unit that acquires an image of a test bacteria liquid containing an antibacterial agent, the test bacteria liquid being cultured for a required culture time that allows the effect of the antibacterial agent over a period of time to be estimated by image judgment, and the effect of the antibacterial agent can be determined by a visual method; A second acquisition unit that acquires an image of the test bacteria liquid not containing an antibacterial agent that has been cultured for the required culture time; a third acquisition unit that acquires an image of the test bacterial liquid containing an antibacterial agent and an image of the test bacterial liquid not containing an antibacterial agent, the images being cultured at the start of culture or immediately after the start of culture for a culture time during which the effect of the antibacterial agent cannot be determined; an estimation unit which, when an image of a test bacterial liquid containing an antibacterial agent, an image of a test bacterial liquid not containing an antibacterial agent, and an image of a test bacterial liquid containing an antibacterial agent and an image of a test bacterial liquid not containing an antibacterial agent which have been cultured at the start of culture or immediately after the start of culture for a culture time at which the effect of the antibacterial agent cannot be determined are input, outputs information about the effect of the antibacterial agent to a trained model which, when the image of the test bacterial liquid containing an antibacterial agent acquired by the first acquisition unit, the image of the test bacterial liquid not containing an antibacterial agent acquired by the second acquisition unit, and the image of the test bacterial liquid containing an antibacterial agent and the image of the test bacterial liquid not containing an antibacterial agent which have been cultured at the start of culture or immediately after the start of culture for a culture time at which the effect of the antibacterial agent cannot be determined, are input to the trained model which outputs information about the effect of the antibacterial agent; An antibiotic effectiveness estimation system that includes:

8. A plurality of containers capable of accommodating a test bacteria liquid containing no antibacterial agent and a test bacteria liquid containing an antibacterial agent, A light source that irradiates light from one side of the plurality of containers; a plurality of image capturing devices arranged on the opposite side of the light source with the plurality of containers therebetween, capable of capturing an image of the test bacteria liquid in each of the containers; an imaging device having 8. The antibacterial effect estimation system according to claim 7, wherein the distance between the observation surface of the test bacterial liquid and the imaging device, and the angle of incidence of light from the light source on the light receiving surface of the imaging device are set to required values.

9. When the distance between the observation surface of the test bacteria liquid and the light receiving surface of the imaging device is L, the shape of the imaging device in a plan view is a square, the vertical or horizontal dimension of the square is 2S, and the incident angle is θ, tan θ=S / L. The antibacterial effect prediction system according to claim 8.

10. The antimicrobial effect estimation system according to claim 8 or claim 9, further comprising an optical film on the exit side of the light source.

11. The antimicrobial effect estimation system according to claim 8 , further comprising a culture device capable of accommodating the imaging device.

Images