Method for advancing diagnosis of mycobacterium tuberculosis on basis of artificial intelligence, and qc method for mycobacterium tuberculosis diagnosis device

Abstract

A method for diagnosing Mycobacterium tuberculosis according to an embodiment comprises the steps of: preparing a first diagnosis model trained for diagnosis of Mycobacterium tuberculosis classified as exhibiting normal growth; preparing a second diagnosis model trained for diagnosis of Mycobacterium tuberculosis classified as being affected by a cocktail containing antibiotics; preparing a third diagnosis model trained for diagnosis of other bacteria similar in shape to the Mycobacterium tuberculosis at or above a predetermined level without being affected by the antibiotics; and deriving a diagnosis result of Mycobacterium tuberculosis for a specimen by using a result of diagnosing the specimen acquired through a camera by at least one of the first diagnosis model, the second diagnosis model, and the third diagnosis model.

Description

Methods for improving tuberculosis diagnosis based on artificial intelligence and QC methods for tuberculosis diagnosis devices

[0001] The present invention relates to a method for improving tuberculosis diagnosis based on artificial intelligence and a QC method for a tuberculosis diagnosis device.

[0002] Several methods exist for diagnosing tuberculosis bacteria or testing drug susceptibility. Among them, MODS (microscopic observation drug susceptibility) is approved by the World Health Organization (WHO). This method is known to be rapid, inexpensive, and accurate.

[0003] However, these MODS have not been commercialized. This is due to several reasons, including the difficulty of training individuals to accurately visualize the growth of tuberculosis bacteria under a microscope. In particular, nontuberculous mycobacteria (NTM), which grow in a similar manner to tuberculosis bacteria, are difficult to inhibit with antibiotics and other treatments. Their growth patterns are also somewhat similar to those of tuberculosis bacteria, making the process of training individuals to distinguish them extremely challenging.

[0004] Accordingly, there is a need for a method to diagnose or test tuberculosis bacteria more quickly and accurately while minimizing the input of human resources by improving software or hardware (devices) for tuberculosis diagnosis.

[0005] The problem to be solved according to one embodiment includes suggesting a method for diagnosing or testing tuberculosis bacteria or testing drug susceptibility more quickly and accurately while minimizing the input of manpower.

[0006] However, the task according to one embodiment is not limited to the interpretation described above.

[0007] According to one embodiment, a method for diagnosing tuberculosis comprises the steps of: providing a first diagnostic model learned for diagnosing tuberculosis bacilli classified as having normal growth; providing a second diagnostic model learned for diagnosing tuberculosis bacilli classified as having been affected by a cocktail including antibiotics; providing a third diagnostic model learned for diagnosing other bacteria that are not affected by the antibiotics and have a shape similar to the tuberculosis bacilli to a predetermined level or more; and deriving a diagnosis result of tuberculosis bacilli for a sample obtained through a camera using a diagnosis result obtained by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0008] The above diagnostic results can be obtained by ensemble processing the diagnostic results from each of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0009] The above ensemble processing may include at least one of a voting method in which positive or negative is determined by majority vote, a weighted average method in which an average probability is calculated by reflecting the reliability of each model, and a stacking method in which a meta model relearns the outputs of the first diagnostic model, the second diagnostic model, and the third diagnostic model to make a final judgment.

[0010] The above specimen may be acquired based on the focusing method of the camera, which varies depending on the acquisition time of the specimen.

[0011] The above focusing method may include a Z-axis movement distance of a lens included in the camera. In this case, the Z-axis movement distance may be determined based on a Z-axis reference value set for each culture time point by monitoring the growth height of tuberculosis bacteria cultured in a liquid medium during each culture period.

[0012] The above camera may be inspected based on at least one of the accuracy of a captured image for a given shape viewed on a plane and the focusing accuracy for one or more points included in the shape.

[0013] The first diagnostic model may be learned as a feature that reflects changes in the length of the tubercle bacilli as they grow. In addition, the second diagnostic model may be learned as a feature that reflects changes in the length and shape of the tubercle bacilli as they are affected by antibiotics. In addition, the third diagnostic model may be learned as a feature that reflects the shape and length of other bacteria that are similar in shape to the tubercle bacilli to a predetermined degree or higher but have different lengths when grown.

[0014] According to one embodiment, a tuberculosis diagnosis device comprises a memory storing at least one command; and a processor, wherein the at least one command is executed by the processor, so that the device provides a first diagnostic model learned for diagnosing tuberculosis bacilli classified as having normal growth, provides a second diagnostic model learned for diagnosing tuberculosis bacilli classified as having been affected by a cocktail containing antibiotics, provides a third diagnostic model learned for diagnosing other bacteria that are not affected by the antibiotics and have a shape similar to the tuberculosis bacilli to a predetermined level or more, and derives a diagnosis result of tuberculosis bacilli for a sample obtained through a camera by using a result of diagnosis by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0015] A non-transitory computer-readable recording medium storing at least one computer-executable instruction according to one embodiment, wherein the at least one instruction, when executed by a processor, causes the processor to perform a method including: providing a first diagnostic model learned for diagnosing tuberculosis bacilli classified as having normal growth; providing a second diagnostic model learned for diagnosing tuberculosis bacilli classified as having been affected by a cocktail containing antibiotics; providing a third diagnostic model learned for diagnosing other bacteria that are not affected by the antibiotics and have a shape similar to the tuberculosis bacilli to a predetermined level or more; and deriving a diagnosis result of tuberculosis bacilli for a sample obtained through a camera using a diagnosis result obtained by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0016] A computer program stored in a non-transitory computer-readable recording medium according to one embodiment, wherein the computer program comprises instructions for causing the processor to perform a method, the method comprising: providing a first diagnostic model learned for diagnosing tuberculosis bacilli classified as having normal growth; providing a second diagnostic model learned for diagnosing tuberculosis bacilli classified as having been affected by a cocktail containing antibiotics; providing a third diagnostic model learned for diagnosing other bacteria that are not affected by the antibiotics and have a shape similar to the tuberculosis bacilli to a predetermined level or more; and deriving a diagnosis result of tuberculosis bacilli for a sample obtained through a camera using a diagnosis result obtained by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0017] According to one embodiment, by using an artificial intelligence model targeted to distinguish between normal tuberculosis bacteria, tuberculosis bacteria affected by antibiotics, or other bacteria that are difficult to suppress with antibiotics, that is, by using software designed based on artificial intelligence to diagnose or test for drug susceptibility of tuberculosis bacteria, tuberculosis bacteria can be diagnosed or tested for more quickly and accurately while minimizing human input.

[0018] In addition, since the artificial intelligence model described above is a model that receives and processes images as input, quality control (QC) is essential for a tuberculosis diagnosis device equipped with an image capturing camera. If the QC method according to one embodiment is followed, QC for the device can be performed more easily, quickly, and accurately.

[0019] Figure 1 illustrates two types of MODS.

[0020] Figure 2 illustrates the concept of direct MODS.

[0021] Figure 3 illustrates the concept of indirect MODS.

[0022] Figure 4 illustrates an example of a component that can be used in both direct / indirect MODS.

[0023] Figure 5 illustrates an example of an inoculation plate that can be prepared when using the component of Figure 4.

[0024] Figure 6 illustrates an example of a diagnosis result for a given specimen or sample displayed as an image.

[0025] Figure 7 illustrates an example of the interpreted results for the diagnostic results for a given specimen or sample displayed on an image.

[0026] Figure 8 illustrates an image of a tuberculosis diagnosis device and a specimen or sample handled by the device.

[0027] Figure 9 illustrates an example of tuberculosis bacteria being cultured and provided to the tuberculosis diagnosis device of Figure 8.

[0028] Figure 10 illustrates an internal structural diagram of the tuberculosis diagnosis device of Figure 8.

[0029] Figure 11 illustrates an example of a lens positioned at the bottom to observe and photograph the MODS Plate.

[0030] Figure 12 illustrates an example of an LED light that provides light to each well of a 24-well plate to ensure sufficient light quantity.

[0031] Figure 13 illustrates an example UI process for a tuberculosis diagnosis device.

[0032] Figure 14 illustrates an example of a setting process for a tuberculosis diagnosis device.

[0033] Figure 15 illustrates an example of a setting process for the tuberculosis diagnosis device of Figure 14, in which an examiner selects and designates a predetermined pointer from among a maximum of 215 points.

[0034] Figure 16 illustrates an example of Acquisition Mode.

[0035] Figure 17 illustrates an example for Manual Mode.

[0036] Figure 18 illustrates an example of a launcher.

[0037] Figure 19 illustrates an example of the Report Form function according to each rule.

[0038] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims.

[0039] When describing embodiments of the present invention, detailed descriptions of known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined in light of their functions in the embodiments of the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0040] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0041] First, MODS (microscopic observation drug susceptibility) is one of the tuberculosis diagnosis methods approved by WHO, as shown in the figure below.

[0042] MODS is a manual liquid technique and can perform drug susceptibility testing for INH (Isoniazid) and RIF (Rifampicin). In other words, MODS can determine whether tuberculosis bacteria react to these two drugs through susceptibility testing.

[0043] However, in the case of MODS, a human inspector must use a microscope to confirm the presence or growth of tuberculosis bacteria. However, for various reasons, meticulous training of such personnel is challenging. Therefore, in one embodiment, an automated method for diagnosing tuberculosis bacteria, a device implementing such a method, and a technology for quality control of such a device are presented.

[0044] Figure 1 illustrates two types of MODS. Referring to Figure 1, one is direct MODS and the other is indirect MODS.

[0045] The embodiments described below are applicable to both direct MODS and indirect MODS.

[0046] Figure 2 illustrates the concept of direct MODS, Figure 3 illustrates the concept of indirect MODS, Figure 4 illustrates components that can be used for both direct / indirect MODS, and Figure 5 illustrates an inoculation plate that can be prepared when using the components of Figure 4.

[0047] Referring to Figure 5, PNB powder kills tuberculosis bacilli but does not act on nontuberculous mycobacteria (NTM). Therefore, if a well containing PNB powder is negative but a well containing a sample is positive, this can be diagnosed as a positive specimen without contamination.

[0048] Conversely, if the test is positive in a well containing PNB powder and also positive in a well containing a sample, it may be diagnosed as contamination.

[0049] Here, NTM has a characteristic of not being easily killed even with general antibiotics, and in order to prevent the accuracy of the diagnosis result for tuberculosis bacteria from being reduced due to NTM, in one embodiment, the following computer program is proposed.

[0050] Such computer programs may be computer programmed to perform steps including:

[0051] - A step of preparing a first diagnostic model learned for diagnosing tuberculosis bacteria classified as normal growth;

[0052] - A step of preparing a second diagnostic model learned for diagnosing tuberculosis bacteria classified as being affected by a cocktail containing antibiotics;

[0053] - A step of preparing a third diagnostic model learned for diagnosing other bacteria that are similar in shape to the tuberculosis bacteria to a certain level or more while not being affected by the antibiotic; and

[0054] - A step of deriving a diagnosis result of tuberculosis bacteria for a sample obtained through a camera by using the diagnosis result obtained by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0055]

[0056] That is, a first diagnostic model that has learned the features of tuberculosis bacilli classified as normal growth, a second diagnostic model that has learned the features of tuberculosis bacilli that have not grown or have died due to the influence of a cocktail containing antibiotics, and a third diagnostic model that has learned the features of other bacteria that have a shape similar to tuberculosis bacilli to a certain degree or more can be prepared. Here, the cocktail can include a culture medium mixed with several types of antituberculosis drugs. Such a cocktail can be used to determine whether cultured tuberculosis bacilli are inhibited from growing or killed by a specific combination (cocktail) of antibiotics.

[0057] In addition, by comprehensively considering the diagnostic results of these first to third diagnostic models for a specimen or sample, a diagnostic result for the specimen or sample can be derived.

[0058] Here, in the first diagnostic model, the code or length formed as the tubercle bacilli grow can be learned as a feature and used for subsequent diagnosis. The code here could refer, for example, to the unique filamentous colony structure that appears when tubercle bacilli grow in liquid medium. Accordingly, the feature could be a vector embedding of that colony structure, but the present invention is not limited to this.

[0059] In addition, in the second diagnostic model, the shape or length of tuberculosis bacteria when affected by antibiotics, etc. can be learned as a feature and used for subsequent diagnosis.

[0060] In addition, in the third diagnostic model, the shape or length of bacteria that are similar to Mycobacterium tuberculosis to a certain level or more but have different lengths when grown, such as shorter NTM, can be learned as features. Here, various variations are possible for this third diagnostic model. For example, the third diagnostic model itself may not be used for the above-mentioned tuberculosis diagnosis, in which case only the first or second diagnostic models can be used for the diagnosis. Alternatively, the third diagnostic model may be learned for all 200 or more types of NTM, but depending on the embodiment, the features may be learned for some NTM that are reported to generate codes. Here, some NTM may be judged to have a cord formation similar to Mycobacterium tuberculosis to a certain level or more, but the present invention is not limited thereto.

[0061] Here, the first through third diagnostic models may be image-based deep learning or machine learning models, and the model architecture is not limited to any one of them. Furthermore, the data used to train each model may also undergo certain preprocessing steps. Furthermore, the diagnostic results from each model for a specimen or sample may be used to derive a tuberculosis diagnosis result for the specimen or sample after undergoing ensemble or various postprocessing steps.

[0062] In one embodiment, a plurality of independent diagnostic models are used for the above-described tuberculosis diagnosis, and the respective diagnostic results of these multiple diagnostic models may be combined through ensemble processing to obtain a more stable and accurate prediction result than a single model. For example, the diagnostic results of each of the first to third diagnostic models may be combined through an ensemble method, such as a voting method in which positive or negative is determined by majority vote, a weighted average method in which an average probability is calculated by reflecting the reliability of each model, or a stacking method in which a meta-model relearns the outputs of the three models and makes a final judgment, to derive a final tuberculosis diagnosis result.

[0063] In other embodiments, a post-processing step may be applied to the diagnosis results of each model for a specimen or sample to correct them for clinical or experimental data criteria or to apply additional rules. Such post-processing steps may include a cutoff application method that determines no growth if the growth area is below a set percentage, a PNB well comparison method that corrects for contamination by comparing the results of a drug-free control group and a PNB (para-nitrobenzoic acid) treated group, a method to prevent NTM confusion by downgrading or indicating the need for retesting the positive results of the first and second diagnostic models if a third model gives a high score, or a time condition reflection method that applies a penalty to the reliability of the prediction results if the incubation period is too short or too long.

[0064] Fig. 6 illustrates an example of a diagnosis result for a given specimen or sample displayed on an image, and Fig. 7 illustrates an example of an interpretation result for a diagnosis result for a given specimen or sample displayed on an image.

[0065] The computer program described above can be installed and operated on a tuberculosis diagnostic device. This will allow doctors and others to experience an automated diagnostic process using such a device.

[0066] That is, according to one embodiment, by using an artificial intelligence model targeted to distinguish between normal tuberculosis bacteria, tuberculosis bacteria affected by antibiotics, or other bacteria that are difficult to inhibit by antibiotics, that is, by using software designed based on artificial intelligence to diagnose or test for drug sensitivity of tuberculosis bacteria, tuberculosis bacteria can be diagnosed or tested for more quickly and accurately while minimizing human resources.

[0067] Meanwhile, the aforementioned tuberculosis diagnostic device uses images of specimens or samples for diagnosis. This requires a camera, a means of capturing images. As such, these cameras are optical devices and must be inspected to ensure they capture accurate images of the specimen or sample. This necessitates a specific quality control (QC) process.

[0068] Of course, the aforementioned tuberculosis diagnostic device does not include a camera, and embodiments in which images captured by an external camera are provided to the tuberculosis diagnostic device are not excluded from the present invention. However, the following description assumes that the tuberculosis diagnostic device includes a camera, and thus the camera calibration and QC described below are applied to the camera installed in the tuberculosis diagnostic device or to the device itself.

[0069] FIG. 8 illustrates an image of a tuberculosis diagnosis device and a specimen or sample handled by the device, FIG. 9 illustrates an image of tuberculosis bacteria cultured and provided to the tuberculosis diagnosis device of FIG. 8, and FIG. 10 illustrates an internal structure diagram of the tuberculosis diagnosis device of FIG. 8.

[0070] Hereinafter, let us examine the features of these devices by way of example with reference to FIGS. 11 and 12.

[0071] Figure 11 illustrates an example of a MODS Plate being observed and photographed with a lens positioned at the bottom, and Figure 12 illustrates an example of an LED light providing light to each well of a 24 well plate to secure sufficient light quantity.

[0072]

[0073] 1. Plate loading section (Stage); The stage is manufactured using sapphire material to prevent scratches on the 24-well plate.

[0074] 2. Lens and XYZ moving; Like a conventional microscope (inverted microscope), the lens is positioned at the bottom to observe and photograph the MODS Plate. It has an autofocusing function and an XYZ moving function that allows the lens to move from well to well.

[0075] 3. Light sources; Design LED lights to fit each well of a 24-well plate to ensure sufficient light.

[0076] 4. Closed structure of plate loading part for infection control and UV irradiation; Since the growth of pathogens such as tuberculosis bacteria is measured, the space of the device where the MODS Plate is placed is a closed structure and has an infection control function by allowing UV irradiation.

[0077] 5. QC Kit Creation: QC kits can be created to check the following four main functions:

[0078] (1) Auto focusing

[0079] (2) Well-to-well movement of the optic lens on the plate

[0080] (3) Sending images to Launcher

[0081] (4) After the inspection is completed, move the lens to the first well (origin) of the plate.

[0082]

[0083] In particular, for auto-focusing, the following process can be performed.

[0084] - Create three transparent balls. (The use of three balls and the number of balls used here are just examples; various variations are possible.)

[0085] - The ball must fit exactly into the well of the plate (width x length x height).

[0086] - And 3 points are engraved at different heights (depths) on the inside of each ball.

[0087]

[0088] Additionally, the QC process using this can proceed as follows.

[0089] 1. Insert the ball into each of the three wells (A1, A2, D6) of the plate.

[0090] 2. Operate the device in Auto mode to check whether the three imprinted points of the ball in the A1 well are accurately autofocused (Autofocusing QC)

[0091] 3. After automatic movement to A2 well from the device, check whether the three imprinted points of the ball in A2 well are accurately autofocused (Autofocusing QC + Well to Well movement QC)

[0092] 4. After the device automatically moves to the D6 well, check whether the three imprinted points of the ball in the D6 well are accurately autofocused (Autofocusing QC + Well-to-Well movement QC recheck)

[0093] 5. Click Termination on the device to check whether the image is transferred to the Launcher.

[0094] 6. Check if the lens moves to the A1 well (origin) and autofocuses on the three imprinted points of the ball.

[0095]

[0096] Here, in the QC process, not only the shape of the engraved three points but also the shape of a specific shape can be determined, and whether the three points are well focused can also be determined on the image. Specifically, the three points can form a certain shape when viewed on a plane, but when viewed three-dimensionally, they can have depth. Whether the focusing is good or not can be determined as pass / fail of QC depending on whether each point has clarity and an image is formed by reflecting the depth information.

[0097] Here, pass / fail QC can be performed by a pre-trained deep learning model, but alternatively, it can be performed by a human. In the latter case, a numerical value for determining focus accuracy is presented as a predetermined range, allowing the human to determine pass / fail based on this range. This range can be determined in various ways and may depend on various factors, such as the shape and material of the two or more engraved dots or the ball, as described above.

[0098] That is, according to one embodiment, since the first to third diagnostic models used for diagnosing tuberculosis are artificial intelligence models that input and process images, QC for a tuberculosis diagnostic device equipped with an image capturing camera is essential. If the QC method according to one embodiment is followed, QC for the device (the capturing module or camera) can be performed more easily, quickly, and accurately.

[0099] Meanwhile, QC may be performed in addition to or in addition to the aforementioned ball. For example, in the aforementioned plate, two or more wells, for example, three wells, may be filled with a solid material with little or no movement, but marked at different depths to ensure that such markings reflect the depth and properly focus. However, this is not a limitation.

[0100] Meanwhile, the following will describe additional descriptions and drawings of the tuberculosis diagnosis device described so far.

[0101] Figure 13 illustrates an example UI process for a tuberculosis diagnosis device.

[0102] We will look at these UI processes in more detail below, but the following is only an example.

[0103] ① Turn on the device and check whether the system is operating normally on the System Check screen.

[0104] ② When the System Check is completed normally, you will automatically be taken to the Log in screen.

[0105] ③ After logging in, the main screen appears.

[0106] ④ Click the Setting button on the Main screen to set the Well plate and Position.

[0107] ⑤ Mount the MODS plate on the device and click the barcode reading button to check the mounted barcode number, then activate Acquisition Mode (Auto) and Manual Mode.

[0108] ⑥ Automatically acquire images in Acquisition Mode through Launcher Software, and the software retrieves images in real time, automatically reads them through AI, and then saves the results.

[0109] ⑦ If you select Manual Mode, manually acquire images and use the Termination function to call the MycoScan Launcher Software to automatically save the results.

[0110] ⑧ After completing the inspection, turn off the device. The UV lamp will operate for 10 to 15 minutes and then automatically turn off.

[0111]

[0112] Fig. 14 illustrates an example of a setting process for a tuberculosis diagnosis device, and Fig. 15 illustrates an example of a setting process for a tuberculosis diagnosis device of Fig. 14, in which an examiner selects and designates a predetermined pointer from among a maximum of 215 points.

[0113] ① Well Plate Selection Function: 1, 6, 12, 24, or 96 wells

[0114] ② Position selection function: Standard1, Standard2, Random Select

[0115] 1) Standard1: Center 9 pointers

[0116] 2) Standard2: Center, Middle 9 center pointers

[0117] 3) The inspector selects up to 48 pointers from a maximum of 215 points.

[0118]

[0119] Figure 16 illustrates an example of Acquisition Mode.

[0120] ① Real-time acquisition image location display

[0121] - You can check the area corresponding to the position set in Settings and the shooting point (the shooting position blinks).

[0122] - Plate displays the location of the shooting well and the inspection progress status (color)

[0123] ② Displays the progress of plate image acquisition. Total work time and progress time are displayed, and the progress is displayed in a graph.

[0124] ③ Display of images acquired in real time

[0125] ④ Controls Information (lens) Displays shooting conditions

[0126]

[0127] Figure 17 illustrates an example for Manual Mode.

[0128] ① Used when detailed shooting is desired and image re-acquisition is required. Displays the currently selected well (multi-select wells requiring manual shooting).

[0129] - Forward: Includes a start signal. Moves quickly to the next well in sequence (when the Well Plate is installed, the lens moves to the center position of A01).

[0130] - Backward: Move quickly to the previous Well in sequence (center of Well)

[0131] - Number of Images: Displays the number of images currently generated from the well being captured. A single well can only capture up to 100 images, and the currently displayed example means that 2 images out of 100 have been captured.

[0132] - The Termination button terminates the manual shooting mode and operates as follows:

[0133] * Pressing the button will exit manual shooting mode and automatically call up the Launcher Software.

[0134] * Launcher Software automatically reads images acquired from a tuberculosis diagnostic device using AI and automatically saves the results.

[0135] * Plate selection area is initialized upon exit

[0136] ② Set the light source adjustment options (Lamp Power, Exposure, Contrast, Gain, Offset)

[0137] ③ Auto Focus device ON / OFF function

[0138] ④ Function to set shooting magnification: x100, x40

[0139] ⑤ Displays the image currently being shot: coordinates, current focus value

[0140]

[0141] Figure 18 illustrates an example of a launcher. Here, the launcher refers to a computer program or software for driving and controlling a device.

[0142] Referring to Figure 18, 1) MycoScan device (tuberculosis diagnosis device), 2) Launcher Server (Launcher / viewer), and 3) AI server can be included as components of the system.

[0143] Specifically, when a researcher places the AI-MODS Plate prepared for a tuberculosis resistance test on the MycoScan device (hereinafter referred to as the same object as the tuberculosis bacteria diagnosis device), the MycoScan device automatically conducts the test to acquire a tuberculosis bacteria growth image, and the acquired image is transmitted to the AI ​​Server in real time by the Launcher Server to perform AI automatic interpretation. After the AI ​​automatic interpretation, the Launcher Server (Viewer Function) reports the results to the examiner.

[0144]

[0145] MycoScan Launcher Software

[0146] ※ Software Launcher: This refers to a software tool that helps to quickly and easily launch applications (device operation and AI reading SW) on a computer or mobile device. It also refers to software designed to facilitate the operation of the MycoScan device and allow users to easily review image data analyzed by artificial intelligence and check the results.

[0147]

[0148] Such software may include, but is not limited to, plate barcode functionality, MycoScan device setup functionality, or Report Form functionality.

[0149]

[0150] #. Plate Barcode function

[0151] 1) Verify the valid barcode using the Plate Barcode registration function in Launcher Software.

[0152] 2) Automatically link and manage patient and specimen information (Sample Number, Patient Test ID, Patient Name, Type of Tuberculosis Drug, QC Progress, Specimen Type (Sputum or Tuberculosis Strain), Specimen Isolation Date, and Culture Start Date) associated with the Plate through the barcode.

[0153] 3) Automatic termination function for plate barcodes that have passed a certain number of days from the start date of the culture start date mapped to the plate barcode.

[0154] 4) Terminated plates cannot be reused.

[0155]

[0156] #. MycoScan device settings function

[0157] 1) Device execution through automatic positive result judgment and result display for each well of the plate and setting of whether or not to acquire MycoScan device image.

[0158] 2) Real-time process processing between MycoScan device, Launcher, and AI.

[0159]

[0160] Figure 19 illustrates an example of the Report Form function according to each rule. Referring to Figure 19, the interpretation according to each rule is illustrated.

[0161]

[0162] Next, the aforementioned tuberculosis diagnostic device may store various types of software or computer programs for tuberculosis diagnosis, one of which is a two-step AI algorithm. Let's examine this in more detail below.

[0163]

[0164] #. Problem

[0165] - The contamination rate is being checked through internal QC in the laboratory.

[0166] - However, current AI methods cannot read and calculate contamination rates.

[0167] - Currently, AI training only selects M. tb (M. tuberculosis) positive objects, so even if contamination occurs in the sample well, the AI ​​determines it as M. tb negative, making it impossible to confirm contamination.

[0168]

[0169] # Solution: Two-Step AI Algorithm

[0170] 1) 1st step AI: It should be possible to determine whether something has grown or not in the image acquired from Optic.

[0171] Even if no growth has occurred, there may be fine medium debris, so a method that allows for cutoff application is required. In other words, if something has grown, the area shown in the image is used for confirmation (Area; Set cutoff by clear zone vs. growing). Data is secured by culturing bacteria other than tuberculosis.

[0172] - Increase in the area of ​​the object in the MycoScan image due to the growth of other bacteria before M. tb positivity (if it is cut off or more, it is considered contamination)

[0173] If the incubation period is long, M. tb will grow and the area will eventually exceed the cutoff. However, in this case, since it has already been determined to be positive, no further imaging will be performed. Therefore, there is no problem.

[0174] 2) 2nd step AI: If something has grown in the image (if it is more than the cut off),

[0175] - If AI catches M. tb, it is judged as M. tb positive (interpretation)

[0176] - Susceptible if AI catches M. tb killed by drug (objects selection and training required)

[0177] - If AI fails to detect M. tb, it is judged as contamination.

[0178] - M. tb detection by AI is determined by the length of the cord, the angle of the cord, and the number of clumps.

[0179]

[0180] In one embodiment, a method for diagnosing tuberculosis may include an autofocus operation for stably capturing cord and clump shapes of tuberculosis bacteria growing in a liquid culture environment at high magnification. Specifically, since the growth height of tuberculosis bacteria in a liquid state is not constant and changes over the culture period, it is difficult to achieve accurate focus with a simple fixed Z-axis value. To solve this problem, a Z-axis reference value for each culture period is secured in advance, and autofocusing may be performed based on the Z-axis reference value secured in advance at each photographing point. Here, a cord is a filamentous colony structure of M. tb, and its length or angle distribution can be utilized as a diagnostic feature. In addition, a clump is a mass-like shape of a cell colony, and its number or shape can be included as a feature quantity.

[0181] First, as a plate preparation step, tuberculosis bacteria can be cultured by adding liquid medium (e.g., 1 ml) to each well of a 24-well plate and then inoculating the tuberculosis bacteria.

[0182] After a 24-well plate is uploaded to the MycoScan device during the culture period, the MycoScan's lens can be moved according to the set values ​​for each well (the X-axis and Y-axis values ​​can be set to be positioned at the center of the well). Additionally, the Z-axis reference value can be set so that the lens moves to a reference height for autofocusing corresponding to a given magnification (e.g., 100x or 40x).

[0183] Afterwards, according to the growth time of the tuberculosis bacteria in the liquid medium, a preset number of X / Y-positions (e.g., 9) per well are set and the lens can be moved. At this time, a Z-axis reference value can be set to enable autofocusing at each position. Here, the growth height of the tuberculosis bacteria for each culture period can be monitored, and the Z-axis reference value corresponding to each period can be obtained in advance. The Z-axis reference value thus obtained is set as a default value according to the growth of the tuberculosis bacteria and can be used as the initial focusing position in the next shooting cycle, thereby enabling autofocusing on the cords and clumps of the tuberculosis bacteria in the liquid medium, and minimizing the optical distortion and surface wave effects of the liquid medium.

[0184] As described above, according to one embodiment of the present invention, by using an artificial intelligence model targeted to distinguish between normal tuberculosis bacteria, tuberculosis bacteria affected by antibiotics, or other bacteria that are difficult to suppress with antibiotics, that is, by using software designed based on artificial intelligence to diagnose tuberculosis bacteria or test for drug sensitivity, tuberculosis bacteria can be diagnosed or tested for more quickly and accurately while minimizing the input of manpower.

[0185] In addition, since the artificial intelligence model described above is a model that receives and processes images as input, quality control (QC) is essential for a tuberculosis diagnosis device equipped with an image capturing camera. If the QC method according to one embodiment is followed, QC for the device can be performed more easily, quickly, and accurately.

[0186] The combinations of each block of the block diagram and each step of the flowchart of the present invention described above may be performed by computer program instructions. These computer program instructions may be installed in an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the instructions executed by the encoding processor of the computer or other programmable data processing device create a means for performing the functions described in each block of the block diagram or each step of the flowchart. These computer program instructions may also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing device to implement a function in a specific manner, so that the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes an instruction means for performing the functions described in each block of the block diagram or each step of the flowchart. Since the computer program instructions can also be installed on a computer or other programmable data processing device, a series of operational steps are performed on the computer or other programmable data processing device to create a computer-executable process, and the instructions that cause the computer or other programmable data processing device to perform the steps for performing the functions described in each block of the block diagram and each step of the flowchart can also provide steps for performing the functions described in each block of the block diagram and each step of the flowchart.

[0187] Additionally, each block or step may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps depicted in succession may actually be performed substantially concurrently, or the blocks or steps may sometimes be performed in reverse order, depending on the functionality they perform.

[0188] The above description is merely an illustrative illustration of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential quality of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

Claims

1. In a tuberculosis diagnosis method performed by a tuberculosis diagnosis device, A step of establishing a first diagnostic model learned for diagnosing tuberculosis bacteria classified as normal growth; A step of preparing a second diagnostic model trained to diagnose tuberculosis bacteria classified as being affected by a cocktail containing antibiotics; A step of preparing a third diagnostic model learned for diagnosing other bacteria that are similar in shape to the tuberculosis bacteria to a predetermined level or more while not being affected by the antibiotic; and A step of deriving a diagnosis result of tuberculosis bacteria for a sample obtained through a camera by using the diagnosis result obtained by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model. Methods for diagnosing tuberculosis.

2. In paragraph 1, The above diagnostic results are, Obtained by ensemble processing the results diagnosed from each of the first diagnostic model, the second diagnostic model, and the third diagnostic model. Methods for diagnosing tuberculosis.

3. In paragraph 2, The above ensemble processing is, It includes at least one of a voting method in which positive or negative is determined by majority vote, a weighted average method in which an average probability is calculated by reflecting the reliability of each model, and a stacking method in which a meta model relearns the outputs of the first diagnostic model, the second diagnostic model, and the third diagnostic model to make a final decision. Methods for diagnosing tuberculosis.

4. In paragraph 1, The above specimen is, It is obtained based on the focusing method of the camera, which varies depending on the acquisition time of the above specimen. Methods for diagnosing tuberculosis.

5. In paragraph 4, The above focusing method is, Includes the Z-axis movement distance of the lens included in the above camera, The above Z-axis movement distance is, The growth height of tuberculosis bacteria cultured in liquid medium is monitored over the culture period, and is determined based on the Z-axis reference value set for each culture time point. Methods for diagnosing tuberculosis.

6. In paragraph 1, The above camera, It is inspected based on at least one of the accuracy of the photographed image for a given shape viewed on a plane and the focusing accuracy for one or more points included in the shape. Methods for diagnosing tuberculosis.

7. In paragraph 1, The above first diagnostic model is, The change in length according to the growth of tuberculosis bacteria is learned as a feature. The above second diagnostic model is, The changes in the length and shape of the tuberculosis bacteria according to the effect of antibiotics are learned as features. The third diagnostic model above is, The shape and length of the above-mentioned bacteria, which are similar in shape to the above-mentioned tuberculosis bacteria to a certain degree or more but have different lengths when grown, are learned as features. Methods for diagnosing tuberculosis.

8. As a tuberculosis diagnosis device, a memory storing at least one instruction; and Includes a processor, By executing at least one instruction by the processor, the device, A first diagnostic model was prepared for the diagnosis of tuberculosis bacteria classified as normal growth. A second diagnostic model trained to diagnose tuberculosis bacteria classified as being affected by a cocktail containing antibiotics is provided. A third diagnostic model is prepared for diagnosing other bacteria that are similar in shape to the tuberculosis bacteria to a certain degree or more while not being affected by the above antibiotics. A diagnosis result for tuberculosis bacteria for a sample obtained through a camera is derived by using the results of diagnosis by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model. Tuberculosis diagnostic device.

9. In paragraph 8, The above diagnostic results are, Obtained by ensemble processing the results diagnosed from each of the first diagnostic model, the second diagnostic model, and the third diagnostic model. Tuberculosis diagnostic device.

10. In paragraph 9, The above ensemble processing is, It includes at least one of a voting method in which positive or negative is determined by majority vote, a weighted average method in which an average probability is calculated by reflecting the reliability of each model, and a stacking method in which a meta model relearns the outputs of the first diagnostic model, the second diagnostic model, and the third diagnostic model to make a final decision. Tuberculosis diagnostic device.

11. In paragraph 8, The above specimen is, It is obtained based on the focusing method of the camera, which varies depending on the acquisition time of the above specimen. Tuberculosis diagnostic device.

12. In paragraph 11, The above focusing method is, Includes the Z-axis movement distance of the lens included in the above camera, The above Z-axis movement distance is, The growth height of tuberculosis bacteria cultured in liquid medium is monitored over the culture period, and is determined based on the Z-axis reference value set for each culture time point. Tuberculosis diagnostic device.

13. In paragraph 8, The above camera, It is inspected based on at least one of the accuracy of the photographed image for a given shape viewed on a plane and the focusing accuracy for one or more points included in the shape. Tuberculosis diagnostic device.

14. A non-transitory computer-readable recording medium storing at least one computer-executable instruction, wherein the at least one instruction, when executed by a processor, A step of establishing a first diagnostic model learned for diagnosing tuberculosis bacteria classified as normal growth; A step of preparing a second diagnostic model trained to diagnose tuberculosis bacteria classified as being affected by a cocktail containing antibiotics; A step of preparing a third diagnostic model learned for diagnosing other bacteria that are similar in shape to the tuberculosis bacteria to a predetermined level or more while not being affected by the antibiotic; and A method for causing the processor to perform a method including a step of deriving a diagnosis result of tuberculosis bacteria for a sample obtained through a camera using a diagnosis result obtained by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model. Non-transitory computer-readable recording medium.

Images