Method for enhancing mycobacterium tuberculosis diagnosis based in artificial intelligence and quality contol method for mycobacteria diagnostic device

Abstract

A method for diagnosing tuberculosis bacteria according to one embodiment comprises: a step of preparing a first diagnostic model trained for diagnosing tuberculosis bacteria classified as normal growth; a step of preparing a second diagnostic model trained for diagnosing tuberculosis bacteria classified as being affected by a cocktail containing antibiotics; a step of preparing a third diagnostic model trained for diagnosing other bacteria that are not affected by the antibiotics and have a shape similar to the tuberculosis bacteria at a predetermined level or higher; and a step of deriving a diagnostic result of tuberculosis bacteria for a sample obtained through a camera using a result in which the sample is diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

Description

Technology Field

[0001] The present invention relates to a method for enhancing the diagnosis of tuberculosis bacteria based on artificial intelligence and a QC method for a tuberculosis bacteria diagnostic device. Background Technology

[0002] There are several methods for diagnosing tuberculosis bacteria or testing drug susceptibility. Among them, MODS (microscopic observation drug susceptibility) is a method approved by the WHO (World Health Organization). This method is known to be rapid, inexpensive, and accurate.

[0003] However, these MODS have not been commercialized. There are several reasons for this, for instance, that it is not easy to train humans to accurately observe the growth of tuberculosis bacteria with the naked eye through a microscope. In particular, nontuberculous mycobacteria (NTM), which grow in a form similar to tuberculosis bacteria, are difficult to inhibit with antibiotics, and since their growth patterns are also somewhat similar to tuberculosis bacteria, a great deal of effort is required in the training process to distinguish them.

[0004] Accordingly, there is a need for measures to diagnose or test for tuberculosis bacteria more quickly and accurately while minimizing the input of human resources by advancing software or hardware (devices) for tuberculosis diagnosis. The problem to be solved

[0005] The problem to be solved according to one embodiment includes presenting a method for diagnosing tuberculosis bacteria or testing drug susceptibility, which allows for more rapid and accurate diagnosis or testing while minimizing the input of manpower.

[0006] However, the problem according to one embodiment is not limited to the interpretation described above. means of solving the problem

[0007] A method for diagnosing tuberculosis bacteria according to one embodiment comprises: a step of preparing a first diagnostic model trained for diagnosing tuberculosis bacteria classified as normal growth; a step of preparing a second diagnostic model trained for diagnosing tuberculosis bacteria classified as being affected by a cocktail containing antibiotics; a step of preparing a third diagnostic model trained for diagnosing other bacteria that are not affected by the antibiotics and have a shape similar to the tuberculosis bacteria at a predetermined level or higher; and a step of deriving a diagnostic result of tuberculosis bacteria for a sample obtained through a camera using a result in which the sample is diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0008] The above diagnostic result can be obtained by ensemble processing the results diagnosed from each of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0009] The above ensemble processing may include at least one of a voting method in which a positive or negative result is determined by a majority vote, a weighted average method in which an average probability is calculated by reflecting the reliability of each model, and a stacking method in which a meta-model relearns the outputs of the first diagnostic model, the second diagnostic model, and the third diagnostic model to make a final decision.

[0010] The above sample may be obtained based on the focusing method of the camera, which varies depending on the time of acquisition of the above sample.

[0011] The above focusing method may include the Z-axis movement distance of the lens included in the camera. In this case, the Z-axis movement distance may be determined based on a Z-axis reference value set for each culture time point by monitoring the growth height of tuberculosis bacteria cultured in a liquid medium according to the culture period.

[0012] The above camera may be inspected based on at least one of the accuracy of a captured image of a predetermined shape viewed on a plane and the focusing accuracy of one or more points included in the shape.

[0013] The first diagnostic model may be previously learned as a feature of the change in length according to the growth of tuberculosis bacteria. Additionally, the second diagnostic model may be previously learned as a feature of the change in length and shape of the tuberculosis bacteria according to the influence of antibiotics. Additionally, the third diagnostic model may be previously learned as a feature of the shape and length of other bacteria that are similar in shape to the tuberculosis bacteria to a certain level or more but have different lengths when grown.

[0014] A tuberculosis bacterium diagnostic device according to one embodiment includes a memory that stores at least one instruction; and a processor, wherein the at least one instruction is executed by the processor, the device provides a first diagnostic model learned for diagnosing tuberculosis bacteria classified as normal growth, provides a second diagnostic model learned for diagnosing tuberculosis bacteria classified as affected by a cocktail containing antibiotics, provides a third diagnostic model learned for diagnosing other bacteria that are not affected by the antibiotics and have a shape similar to the tuberculosis bacteria at a predetermined level or higher, and derives a diagnosis result of tuberculosis bacteria for the sample obtained through a camera using the result of the first diagnostic model, the second diagnostic model, and the third diagnostic model being diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0015] A computer-readable recording medium storing at least one computer-executable instruction according to one embodiment, wherein the at least one instruction, when executed by a processor, comprises the steps of: providing a first diagnostic model learned for the diagnosis of tuberculosis bacteria classified as normal growth; providing a second diagnostic model learned for the diagnosis of tuberculosis bacteria classified as affected by a cocktail containing antibiotics; providing a third diagnostic model learned for the diagnosis of other bacteria that are not affected by the antibiotics and have a shape similar to the tuberculosis bacteria at a predetermined level or higher; and deriving a diagnostic result of tuberculosis bacteria for a sample obtained through a camera using a result in which the sample is diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0016] A computer program stored on a computer-executable or at least computer-readable recording medium according to one embodiment, wherein the computer program, when executed by a processor, comprises instructions for the processor to perform a method including: a step of providing a first diagnostic model learned for the diagnosis of tuberculosis bacteria classified as normal growth; a step of providing a second diagnostic model learned for the diagnosis of tuberculosis bacteria classified as affected by a cocktail containing antibiotics; a step of providing a third diagnostic model learned for the diagnosis of other bacteria that are not affected by the antibiotics and have a shape similar to the tuberculosis bacteria at a predetermined level or higher; and a step of deriving a diagnostic result of tuberculosis bacteria for a sample obtained through a camera using a result in which the sample is diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model. Effects of the invention

[0017] According to one embodiment, by using an artificial intelligence model targeted to distinguish between normal tuberculosis bacteria, tuberculosis bacteria affected by antibiotics, or other bacteria that are difficult to suppress by antibiotics, that is, by using software designed based on artificial intelligence for the diagnosis of tuberculosis bacteria or drug susceptibility testing, tuberculosis bacteria can be diagnosed or tested more quickly and accurately while minimizing the input of manpower.

[0018] In addition, since the aforementioned artificial intelligence model is a model that receives and processes images, quality control (QC) for a tuberculosis bacterium diagnostic device equipped with an image capture camera is essential; however, by following the QC method according to one embodiment, QC for the device can be performed more easily, quickly, and accurately. Brief explanation of the drawing

[0019] Figure 1 illustrates two methods of MODS. Figure 2 illustrates the concept of direct MODS. Figure 3 illustrates the concept of indirect MODS. Figure 4 illustrates an exemplary component that can be used in both direct and indirect MODS. FIG. 5 illustrates an exemplary inoculation plate that can be prepared when using the component of FIG. 4. FIG. 6 exemplarily illustrates the result of diagnosis for a given specimen or sample being displayed on an image. FIG. 7 exemplarily illustrates the interpreted result of a diagnosis result for a given specimen or sample displayed on an image. FIG. 8 illustrates an exemplary image of a tuberculosis bacterium diagnostic device and a specimen or sample to be handled by such a device. FIG. 9 illustrates, exemplarily, tuberculosis bacteria being cultured and provided to the tuberculosis bacteria diagnostic device of FIG. 8. FIG. 10 illustrates an exemplary internal structure diagram of the tuberculosis bacterium diagnostic device of FIG. 8. Figure 11 illustrates, as an example, that the lens is positioned at the bottom to observe and photograph the MODS Plate. FIG. 12 illustrates an exemplary LED light that provides light to each well of a 24-well plate to ensure sufficient light intensity. Figure 13 illustrates an exemplary UI process for a tuberculosis bacterium diagnostic device. FIG. 14 illustrates an exemplary example of a setting process for a tuberculosis bacterium diagnostic device. FIG. 15 illustrates, as an example of a setting process for the tuberculosis bacterium diagnostic device of FIG. 14, that an examiner selects and designates a specific pointer among up to 215 points. Figure 16 illustrates an exemplary Acquisition Mode. Figure 17 illustrates an exemplary Manual Mode. FIG. 18 illustrates an exemplary launcher. Figure 19 illustrates the Report Form function according to each rule as an example. Specific details for implementing the invention

[0020] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but can be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0021] In describing the embodiments of the present invention, specific descriptions of known functions or configurations will be omitted if it is determined that such detailed descriptions could unnecessarily obscure the essence of the invention. Furthermore, the terms described below are defined in consideration of their functions in the embodiments of the present invention, and these definitions may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification.

[0022] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0023] First, MODS (microscopic observation drug susceptibility) is one of the diagnostic methods for tuberculosis approved by the WHO, as shown in the figure below.

[0024] MODS is a manual liquid technique that enables drug susceptibility testing for INH (Isoniazid) and RIF (Rifampicin). In other words, MODS can determine whether tuberculosis bacteria respond to these two drugs through susceptibility testing.

[0025] However, in the case of such MODS, an examiner—that is, a human—must confirm the presence or growth of tuberculosis bacteria using a microscope; however, it is difficult to precisely train such a person for various reasons. Accordingly, in one embodiment, we intend to present an automated method for diagnosing tuberculosis bacteria, a device in which such a method is implemented, and a technology for quality control of such a device.

[0026] Figure 1 illustrates two types of MODS. Referring to Figure 1, one is direct MODS and the other is indirect MODS.

[0027] The embodiments described below are applicable to direct MODS as well as indirect MODS.

[0028] FIG. 2 illustrates the concept of direct MODS, FIG. 3 illustrates the concept of indirect MODS, FIG. 4 illustrates a component that can be used for both direct and indirect MODS, and FIG. 5 illustrates an exemplary inoculation plate that can be prepared when using the component of FIG. 4.

[0029] Referring to Figure 5, PNB powder kills tuberculosis bacteria but does not affect nontuberculous mycobacteria (NTM). Therefore, if the well containing PNB powder is negative but the well containing the sample is positive, it can be diagnosed as a positive specimen without contamination.

[0030] In contrast, if the well containing PNB powder is positive and the well containing the sample is also positive, it can be diagnosed as contamination.

[0031] Here, NTM has the characteristic of not being easily killed even with the use of general antibiotics. In order to prevent the accuracy of diagnostic results for tuberculosis bacteria from decreasing due to such NTM, one embodiment proposes the following computer program.

[0032] Such a computer program may be programmed to perform the following steps.

[0033] - A step of preparing a first diagnostic model trained for the diagnosis of tuberculosis bacteria classified as normal growth;

[0034] - A step of preparing a second diagnostic model trained for the diagnosis of tuberculosis bacteria classified as affected by an antibiotic cocktail;

[0035] - A step of providing a third diagnostic model trained for the diagnosis of other bacteria that are not affected by the above antibiotic and have a shape similar to the above tuberculosis bacteria at a predetermined level or higher; and

[0036] - A step of deriving a diagnosis result of tuberculosis bacteria for a sample obtained through a camera by using the result of the sample being diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model.

[0038] That is, a first diagnostic model that learns features of tuberculosis bacteria classified as normal growth, a second diagnostic model that learns features of tuberculosis bacteria that do not grow and are killed or do not grow under the influence of a cocktail containing antibiotics, and a third diagnostic model that learns features of other bacteria, such as NTM, that are similar in shape to tuberculosis bacteria to a certain level or higher. Here, the cocktail may include a culture medium mixed with various types of antituberculosis drugs. Such a cocktail may be used to determine whether cultured tuberculosis bacteria are inhibited from growing or killed by a specific antibiotic combination (cocktail).

[0039] In addition, the results of the first to third diagnostic models diagnosing the specimen or sample can be comprehensively considered to derive a diagnostic result for the specimen or sample.

[0040] Here, in the first diagnostic model, the code or length formed as the tuberculosis bacteria grow can be learned as a feature and used for subsequent diagnosis. Here, the code may, for example, refer to the characteristic fibrous colony structure that appears when tuberculosis bacteria grow in a liquid medium, and accordingly, the feature may be a vector embedding such colony structure, but is not limited thereto.

[0041] In addition, in the second diagnostic model, the shape or length of the tuberculosis bacteria, which is affected by antibiotics, etc., can be learned as a feature and used for subsequent diagnosis.

[0042] In addition, the third diagnostic model may learn the shape or length of bacteria, such as shorter NTMs, which have a shape similar to tuberculosis bacteria to a predetermined level but differ in length when grown. Here, various variations are possible for this third diagnostic model. For example, the third diagnostic model itself may not be used for the diagnosis of the tuberculosis bacteria mentioned above, in which case only the first or second diagnostic model may be used for diagnosis. Alternatively, the third diagnostic model may be learned against all 200 or more types of NTMs, but depending on the embodiment, its features may be learned against some NTMs that are reported to generate a cord. The some NTMs mentioned here may be those judged to have cord formation similar to tuberculosis bacteria to a predetermined level, but are not limited thereto.

[0043] Here, the first to third diagnostic models may be image-based deep learning or machine learning models, and the architecture of the models is not limited to any one. In addition, the data used for training each model may also undergo a predetermined preprocessing process. Furthermore, the results of each model's diagnosis of a specimen or sample may be used to derive a diagnosis result of tuberculosis bacteria for the specimen or sample by undergoing ensemble or various post-processing processes.

[0044] In one embodiment, a plurality of independent diagnostic models are used for the diagnosis of tuberculosis bacteria described above. In order to obtain a more stable and accurate prediction result than a single model, the diagnostic results of each of these plurality of diagnostic models can be combined through ensemble processing. For example, the diagnostic results of each of the first to third diagnostic models can be combined through ensemble methods such as a voting method in which positive or negative is determined by a majority vote, a weighted average method in which an average probability is calculated by reflecting the reliability of each model, or a stacking method in which a meta-model relearns the outputs of the three models to make a final judgment, to derive a final diagnosis result of tuberculosis bacteria.

[0045] In other embodiments, a post-processing process may be applied to correct the results diagnosed by each model for a specimen or sample according to clinical or experimental data standards, or to apply additional rules. Such post-processing processes may include a cutoff application method that determines no growth if the growth area is less than a set ratio; a PNB well comparison method that corrects for contamination by comparing the results of a drug-free control group with a PNB (para-nitrobenzoic acid) treatment group; an NTM confusion prevention method that lowers the positive results of the first and second diagnostic models or indicates the need for re-examination if the third model gives a high score; or a time condition reflection method that applies a penalty to the reliability of the prediction result if the culture period is too short or too long.

[0046] FIG. 6 exemplarily illustrates a result of diagnosis for a predetermined specimen or sample displayed on an image, and FIG. 7 exemplarily illustrates a result of interpretation of a result of diagnosis for a predetermined specimen or sample displayed on an image.

[0047] The computer program described so far can be installed and operated on a tuberculosis diagnostic device. Through this, doctors and others can experience an automated diagnostic process using such a device.

[0048] That is, according to one embodiment, by using an artificial intelligence model targeted to distinguish between normal tuberculosis bacteria, tuberculosis bacteria affected by antibiotics, or other bacteria that are difficult to inhibit by antibiotics, that is, by using software designed based on artificial intelligence for the diagnosis of tuberculosis bacteria or drug susceptibility testing, tuberculosis bacteria can be diagnosed or tested more quickly and accurately while minimizing human resources.

[0049] Meanwhile, the aforementioned tuberculosis diagnostic device diagnoses using images of specimens or samples. A means of capturing images, namely a camera, is required; since such a camera is an optical device, it must be inspected to ensure that it can capture accurate images of specimens or samples. In other words, a specific QC process is required.

[0050] Of course, embodiments in which the aforementioned tuberculosis bacterium diagnostic device is not equipped with a camera and images captured by an external camera are provided to such device are not excluded from the present invention. However, as described below, the tuberculosis bacterium diagnostic device is equipped with a camera, and the camera calibration, QC, etc. described below will be explained on the premise that they are applied to the camera equipped with such device or to the device itself.

[0051] FIG. 8 illustrates an exemplary image of a tuberculosis bacterium diagnostic device and a specimen or sample handled by such device, FIG. 9 illustrates an exemplary case of tuberculosis bacteria being cultured and provided to the tuberculosis bacterium diagnostic device of FIG. 8, and FIG. 10 illustrates an exemplary internal structural diagram of the tuberculosis bacterium diagnostic device of FIG. 8.

[0052] Below, let us look at the features of such a device by way of example with reference to FIGS. 11 and FIGS. 12.

[0053] FIG. 11 illustrates, in an exemplary manner, a lens positioned at the bottom to observe and photograph a MODS Plate, and FIG. 12 illustrates, in an exemplary manner, an LED light providing light to each well of a 24-well plate to ensure sufficient light intensity.

[0055] 1. Plate loading section (Stage); The stage is fabricated using sapphire material to prevent scratching of the 24-well plate.

[0056] 2. Lens and XYZ moving; Like a conventional inverted microscope, the lens is positioned at the bottom to observe and image the MODS Plate. It features an autofocus function and an XYZ moving function that allows the lens to move well to well.

[0057] 3. Light sources; LED lights are designed to fit each well of the 24-well plate to ensure sufficient light output.

[0058] 4. Closed structure of plate loading part for infection control and UV irradiation; Since the growth of pathogens such as Mycobacterium tuberculosis is measured, the space where the MODS plate is placed is designed as a closed structure, and an infection control function is provided by enabling UV irradiation.

[0059] 5. QC Kit Production: The QC kit can be produced to verify the following four key functions.

[0060] (1) Auto-focusing

[0061] (2) Well-to-well movement of the optic lens on the plate

[0062] (3) Send Image to Launcher

[0063] (4) After the examination is completed, move the lens to the first well (origin) of the plate.

[0065] In particular, for autofocusing, the following process may be performed.

[0066] - Create 3 transparent balls. (The use of balls and the number being 3 are merely examples, and various variations are possible.)

[0067] - The ball must fit precisely into the well of the plate (width x length x height).

[0068] - And imprint 3 points on the inside of each ball at different heights (depths).

[0070] In addition, the QC process using this can proceed as follows.

[0071] 1. Insert balls into the three wells (A1, A2, D6) of the plate.

[0072] 2. Operate the device in Auto mode to verify that it accurately autofocuses the three imprinted points of the ball in the A1 well (Autofocusing QC)

[0073] 3. Verify that the device accurately autofocuses on the 3 engraved points of the ball inside the A2 well after automatic movement to the A2 well (Autofocusing QC + Well-to-Well Movement QC)

[0074] 4. Verify that the device accurately autofocuses on the three engraved points of the ball inside the D6 well after automatically moving to the D6 well (Autofocusing QC + Re-verify Well-to-Well movement QC).

[0075] 5. Click Termination on the device to check if the image has been sent to the Launcher.

[0076] 6. Move the lens to the A1 well (origin) and check if it autofocuses the three imprinted points of the ball.

[0078] In the QC process, not only can the shape of a specific figure be determined as well as the three imprinted points, but the focus of the three points can also be determined on the image. Specifically, while the three points may form a specific figure when viewed in a planar view, they may possess depth when viewed in three dimensions. The pass / fail of the QC can be determined based on whether the image is formed with clarity for each point by reflecting the depth information, such as whether the focus is well done.

[0079] Here, the pass / fail of QC can be performed by a pre-trained deep learning model, but alternatively, it can also be performed by a human. In the latter case, for a human to determine pass / fail, numerical values ​​for determining whether the focus is correct are presented as a predetermined range, and the human can determine pass / fail by looking at such a range. The range here can be determined in various ways and can be determined dependently on various factors such as the shape or material of the aforementioned two or more imprinted points or balls.

[0080] That is, according to one embodiment, since the first to third diagnostic models used for diagnosing tuberculosis bacteria are artificial intelligence models that receive and process images, quality control (QC) for a tuberculosis bacteria diagnostic device equipped with an image capture camera is essential; however, if the QC method according to one embodiment is followed, QC for the device (the capture module or camera) can be performed more easily, quickly, and accurately.

[0081] Meanwhile, QC may be performed in addition to the aforementioned ball. For example, in the plate described above, two or more wells, such as three wells, may be filled with a solid material that has little or no movement, but are marked with different depths to check whether the markings reflect the depths and focus properly, but this is not limited thereto.

[0082] Meanwhile, the following describes additional explanations and drawings regarding the tuberculosis bacterium diagnostic device described so far.

[0083] Figure 13 illustrates an exemplary UI process for a tuberculosis bacterium diagnostic device.

[0084] We will examine these UI processes in more detail below, but the following is merely an example.

[0085] ① Turn on the device and check whether the system is operating normally on the System Check screen.

[0086] ② Once the System Check is successfully completed, it will automatically redirect to the Log In screen.

[0087] ③ The main screen is displayed after logging in.

[0088] ④ Click the Setting button on the Main screen to set the Well plate and Position.

[0089] ⑤ Mount the MODS plate on the device, click the Barcode Reading button to verify the mounted barcode number, and then activate Acquisition Mode (Auto) and Manual Mode.

[0090] ⑥ Images are automatically acquired in Acquisition Mode via the Launcher Software; the software retrieves images in real time, automatically analyzes them using AI, and saves the results.

[0091] ⑦ If Manual Mode is selected, manually acquire images and use the Termination function to call MycoScan Launcher Software to automatically save the results.

[0092] ⑧ When the device is turned off after the inspection is complete, the UV lamp operates for 10 to 15 minutes and then the power turns off automatically.

[0094] FIG. 14 illustrates an example of a setting process for a tuberculosis bacterium diagnostic device, and FIG. 15 illustrates an example of a setting process for a tuberculosis bacterium diagnostic device of FIG. 14 in which an examiner selects and designates a specific pointer among up to 215 points.

[0095] ① Well Plate Selection Function: 1, 6, 12, 24, or 96 wells

[0096] ② Position Selection Function: Distinction between Standard1, Standard2, and Random Select

[0097] 1) Standard1: Center 9 pointers

[0098] 2) Standard2: Center, Middle, 9 central pointers

[0099] 3) The inspector selects and designates up to 48 pointers out of a maximum of 215 points.

[0101] Figure 16 illustrates an exemplary Acquisition Mode.

[0102] ① Display of real-time acquired image location

[0103] - Displays the area corresponding to the position set in Settings and allows you to check the shooting point (shooting location blinks)

[0104] - The plate indicates the position of the imaging well and the inspection progress status (color).

[0105] ② Displays the progress status of plate image acquisition. Displays the total work time and progress time, and shows the progress status as a graph.

[0106] ③ Display of images acquired in real time

[0107] ④ Controls information (lens) displays shooting conditions

[0109] Figure 17 illustrates an exemplary Manual Mode.

[0110] ① Used when you want to take detailed shots or need to re-acquire images. Displays the currently selected Well (multiple Wells requiring manual shooting can be selected).

[0111] - Forward: Includes start signal. Moves at high speed sequentially to the next well (when a well plate is mounted, the lens moves to the center position of A01).

[0112] - Backward: Moves to the previous Well in sequence at high speed (Well center position)

[0113] - Number of Images: Displays the number of images currently generated from the well being captured. A maximum of 100 images can be captured from a single well, and the example currently displayed indicates that 2 images have been captured out of 100.

[0114] - The Termination button functions to exit manual shooting mode and operates according to the following procedure.

[0115] Pressing the * button exits manual shooting mode and automatically calls the Launcher Software.

[0116] The Launcher Software imports images acquired from the tuberculosis diagnostic device, automatically interprets them using AI, and automatically saves the results.

[0117] The Plate selection area is reset upon exit.

[0118] ② Set light source control options (Lamp Power, Exposure, Contrast, Gain, Offset)

[0119] ③ Turn Auto Focus device ON / OFF function

[0120] ④ Shooting magnification setting function: x100, x40

[0121] ⑤ Display the image currently being captured: Coordinates, current Focus value

[0123] FIG. 18 illustrates an exemplary launcher. Here, a launcher refers to a computer program or software for operating and controlling a device.

[0124] Referring to Fig. 18, 1) a MycoScan device (tuberculosis diagnostic device), 2) a Launcher Server (Launcher / viewer), and 3) an AI server may be included as components of the system.

[0125] Specifically, when a researcher places the AI-MODS Plate prepared for tuberculosis resistance testing onto the MycoScan device (hereinafter referred to as the same entity as the tuberculosis diagnostic device), the MycoScan device automatically performs the test to acquire images of tuberculosis bacterial growth. The Launcher Server then transmits the acquired images in real-time to the AI ​​Server for automatic AI interpretation. Following the automatic AI interpretation, the Launcher Server (Viewer Function) reports the results to the examiner.

[0127] MycoScan Launcher Software

[0128] ※ Software Launcher: Refers to a software tool that helps users easily and quickly launch applications (device operation and AI analysis software) on computers or mobile devices. It specifically refers to software designed to facilitate the operation of MycoScan devices and allow users to easily review image data analyzed by artificial intelligence and verify the results.

[0130] Such software may include the following functions. For example, plate barcode functions, MycoScan device configuration functions, or Report Form functions, but are not limited thereto.

[0132] #. Plate Barcode Function

[0133] 1) Verify a valid barcode through the Plate Barcode registration function in the Launcher Software.

[0134] 2) Automatically link and manage patient and specimen information (Sample Number, Patient Test ID, Patient Name, Type of Tuberculosis Drug, QC Status, Specimen Type (Sputum or Tuberculosis Strain), Specimen Separation Date, Culture Start Date) associated with the corresponding Plate via a barcode.

[0135] 3) Automatic plate barcode termination function for a specific number of days past the culture start date mapped to the plate barcode.

[0136] 4) Terminated plates cannot be reused.

[0138] #. MycoScan Device Configuration Function

[0139] 1) Automatic determination of positive results for each plate well and display of results, and execution of the device through the function to set whether to acquire images using the MycoScan device.

[0140] 2) Real-time process processing between MycoScan device, Launcher, and AI.

[0142] FIG. 19 illustrates the Report Form function according to each rule. Referring to FIG. 19, the interpretation according to each rule is illustrated.

[0144] Next, the aforementioned tuberculosis diagnostic device may store various types of software or computer programs for diagnosing tuberculosis bacteria, one of which is a two-step AI algorithm. Let us examine this in more detail below.

[0146] #. Problem

[0147] The laboratory is checking the contamination rate using Internal QC.

[0148] However, current AI methods cannot read and calculate the contamination rate.

[0149] - Currently, AI training selects only M. tb (M. tuberculosis) positive objects; therefore, even if contamination occurs in a sample well, the AI ​​determines it as M. tb negative, making it impossible to confirm the presence of contamination.

[0151] #. Solution: Two-Step AI Algorithm

[0152] 1) 1st step AI: Must be able to determine whether something has grown or not in the image acquired from the optic.

[0153] - Even if growth has not occurred, there may be minute traces of medium debris, so the method must allow for the application of a cut-off. In other words, if growth has occurred, the area displayed in the image must be checked (Area; set the cut-off based on Clear Zone vs. Growing, and secure data by culturing bacteria other than Mycobacterium tuberculosis).

[0154] - Increase in the area of ​​an object in a MycoScan image due to the growth of other bacteria prior to a positive M. tb result (considered contamination if the cut-off level is higher)

[0155] If the culture period is prolonged, M. tb will grow and eventually exceed the cut-off area. However, since it has already been determined to be positive, no further images will be acquired. In other words, this is not a problem.

[0156] 2) 2nd step AI: If something has grown in the image (if it exceeds the cut-off),

[0157] - If AI detects M. tb, it is determined as M. tb positive (interpretation)

[0158] - Susceptible if AI catches M. tb killed by medication (object selection and training required)

[0159] - If the AI ​​fails to catch M. tb, it is determined to be contamination.

[0160] - M. tb detection by AI is determined by cord length, cord angle, and the number of clumps.

[0162] In one embodiment, the method for diagnosing Mycobacterium tuberculosis may include an autofocusing operation to stably capture the cord and clumping shapes of Mycobacterium tuberculosis growing in a liquid culture environment at high magnification. Specifically, since the growth height of Mycobacterium tuberculosis in a liquid state is not constant and changes with the culture period, it is difficult to achieve accurate focus using a simple fixed Z-axis value; to address this, Z-axis reference values ​​for each culture period are secured in advance, and autofocusing can be performed at each imaging point based on the Z-axis reference values ​​secured in advance. Here, the cord is a fibrous colony structure of M. tb, and its length or angular distribution can be utilized as a diagnostic feature. Additionally, the clump is a mass-like form of cell colonies, and its number or shape may be included as a characteristic quantity.

[0163] First, as a plate preparation step, tuberculosis bacteria can be cultured by adding liquid medium (e.g., 1 ml) to each well of a 24-well plate and then inoculating the tuberculosis bacteria.

[0164] After a 24-well plate is uploaded to the MycoScan device during the culture period, the lens of MycoScan can be moved according to the set value of each well (X-axis and Y-axis values ​​can be set so as to be positioned at the center of the well). In addition, a Z-axis reference value can be set so that the lens moves to a reference height to enable autofocus corresponding to a predetermined magnification (e.g., 100x or 40x).

[0165] Subsequently, a pre-set number of X / Y-positions (e.g., 9) per well can be set according to the growth time of Mycobacterium tuberculosis in the liquid medium, allowing the lens to be moved. At this time, a Z-axis reference value can be set to enable autofocusing at each position. Here, the growth height of Mycobacterium tuberculosis over the culture period is monitored, and the Z-axis reference value corresponding to each period can be obtained in advance. The Z-axis reference value thus obtained is set as a default value based on the growth of Mycobacterium tuberculosis and can be used as the initial focusing position in the next imaging cycle. This enables autofocusing on the codes and clumps of Mycobacterium tuberculosis in the liquid medium, while minimizing optical distortion and the influence of surface waves in the liquid medium.

[0166] As described above, according to one embodiment of the present invention, by using an artificial intelligence model targeted to distinguish between normal tuberculosis bacteria, tuberculosis bacteria affected by antibiotics, or other bacteria that are difficult to suppress by antibiotics, that is, by using software designed based on artificial intelligence for the diagnosis of tuberculosis bacteria or drug susceptibility testing, tuberculosis bacteria can be diagnosed or tested more quickly and accurately while minimizing the input of manpower.

[0167] In addition, since the aforementioned artificial intelligence model is a model that receives and processes images, quality control (QC) for a tuberculosis bacterium diagnostic device equipped with an image capture camera is essential; however, by following the QC method according to one embodiment, QC for the device can be performed more easily, quickly, and accurately.

[0168] The combinations of each block of the block diagram of the present invention and each step of the flowchart described above may be executed by computer program instructions. Since these computer program instructions may be loaded into an encoding processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, the instructions executed through the encoding processor of the computer or other programmable data processing device create means for performing the functions described in each block of the block diagram or each step of the flowchart. Since these computer program instructions may also be stored in computer-available or computer-readable memory that can be directed toward a computer or other programmable data processing device to implement the function in a specific way, the instructions stored in the computer-available or computer-readable memory may also produce a manufactured item containing instruction means for performing the function described in each block of the block diagram or each step of the flowchart. Since computer program instructions can be loaded onto a computer or other programmable data processing device, instructions that perform a series of operation steps on a computer or other programmable data processing device to create a process executed by the computer and to execute the computer or other programmable data processing device may also provide steps for executing the functions described in each block of the block diagram and each step of the flowchart.

[0169] Additionally, each block or each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). Also, it should be noted that in some alternative embodiments, the functions mentioned in the blocks or steps may occur out of order. For example, two blocks or steps described in succession may actually be performed substantially simultaneously, or the blocks or steps may sometimes be performed in reverse order according to the corresponding function.

[0170] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the present invention.

Claims

Claim 1 A method for diagnosing tuberculosis bacteria performed by a tuberculosis bacteria diagnostic device comprises: a step of providing a first diagnostic model learned as features of length change according to the growth of tuberculosis bacteria, for diagnosing tuberculosis bacteria classified as having normal growth; a step of providing a second diagnostic model learned as features of length change and shape change of tuberculosis bacteria according to the influence of said antibiotic, for diagnosing tuberculosis bacteria classified as having been affected by a cocktail containing an antibiotic; and a step of providing a third diagnostic model learned as features of length change and shape change according to the growth of said other bacteria, for diagnosing other bacteria that are not affected by said antibiotic, have a shape similar to said tuberculosis bacteria to a predetermined level or higher, but have a length change pattern during growth different from said tuberculosis bacteria. A method for diagnosing tuberculosis bacteria, comprising the step of deriving a diagnosis result of tuberculosis bacteria for a sample obtained through a camera using a result diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model, wherein the first diagnostic model, the second diagnostic model, and the third diagnostic model are each independently learned. Claim 2 A method for diagnosing tuberculosis bacteria according to claim 1, wherein the diagnostic result is obtained by ensemble processing the diagnostic results from each of the first diagnostic model, the second diagnostic model, and the third diagnostic model. Claim 3 In paragraph 2, the ensemble processing comprises at least one of a voting method in which a positive or negative result is determined by a majority vote, a weighted average method in which an average probability is calculated by reflecting the reliability of each model, and a stacking method in which a meta-model relearns the outputs of the first diagnostic model, the second diagnostic model, and the third diagnostic model to make a final decision. Claim 4 A method for diagnosing tuberculosis bacteria according to claim 1, wherein the specimen is obtained based on the focusing method of the camera which varies differently depending on the time of securing the specimen. Claim 5 In claim 4, the focusing method includes a Z-axis movement distance of a lens included in the camera, and the Z-axis movement distance is determined based on a Z-axis reference value set for each culture time by monitoring the growth height of tuberculosis bacteria cultured in a liquid medium. Claim 6 A method for diagnosing tuberculosis bacteria according to claim 1, wherein the camera is inspected based on at least one of the accuracy of a captured image of a predetermined shape viewed on a plane and the focusing accuracy of one or more points included in the shape. Claim 7 A method for diagnosing tuberculosis bacteria according to claim 1, wherein the first diagnostic model is learned as a feature of the change in length according to the growth of tuberculosis bacteria, the second diagnostic model is learned as a feature of the change in length and shape of tuberculosis bacteria according to the influence of antibiotics, and the third diagnostic model is learned as a feature of the shape and length of other bacteria that are similar in shape to tuberculosis bacteria by a predetermined level or more but have different lengths when grown. Claim 8 A tuberculosis bacterium diagnostic device comprising: a memory storing at least one instruction; The device comprises a processor, wherein at least one instruction is executed by the processor, and the device comprises: a first diagnostic model learned as features of length change according to the growth of tuberculosis bacteria classified as normal growth for the diagnosis of tuberculosis bacteria; a second diagnostic model learned as features of length change and shape change of tuberculosis bacteria according to the influence of the antibiotic for the diagnosis of tuberculosis bacteria classified as affected by a cocktail containing the antibiotic; a third diagnostic model learned as features of length change and shape change according to the growth of other bacteria that are not affected by the antibiotic and have a shape similar to tuberculosis bacteria by a predetermined level or more, but have a length change pattern during growth different from tuberculosis bacteria for the diagnosis of other bacteria; and deriving a diagnostic result of tuberculosis bacteria for a sample obtained through a camera using a result diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model, wherein the first diagnostic model, the second diagnostic model, and the third diagnostic model are each independently A tuberculosis bacterium diagnostic device characterized by learned properties. Claim 9 In claim 8, the above diagnostic result is a tuberculosis bacterium diagnostic device obtained by ensemble processing the diagnostic results from each of the first diagnostic model, the second diagnostic model, and the third diagnostic model. Claim 10 A tuberculosis bacterium diagnostic device according to claim 9, wherein the ensemble processing comprises at least one of a voting method in which a positive or negative result is determined by a majority vote, a weighted average method in which an average probability is calculated by reflecting the reliability of each model, and a stacking method in which a meta-model relearns the outputs of the first diagnostic model, the second diagnostic model, and the third diagnostic model to make a final decision. Claim 11 In claim 8, the tuberculosis bacterium diagnostic device, wherein the specimen is obtained based on the focusing method of the camera which varies differently depending on the time of securing the specimen. Claim 12 A tuberculosis diagnostic device according to claim 11, wherein the focusing method includes a Z-axis movement distance of a lens included in the camera, and the Z-axis movement distance is determined based on a preset Z-axis reference value for each culture time by monitoring the growth height of tuberculosis bacteria cultured in a liquid medium according to the culture period. Claim 13 A tuberculosis bacterium diagnostic device according to claim 8, wherein the camera is inspected based on at least one of the accuracy of a captured image of a predetermined shape viewed on a plane and the focusing accuracy of one or more points included in the shape. Claim 14 A computer-readable recording medium storing at least one computer-executable instruction, wherein the at least one instruction, when executed by a processor, comprises: a step of providing a first diagnostic model learned as features of length change according to the growth of tuberculosis bacteria for the diagnosis of tuberculosis bacteria classified as normal growth; a step of providing a second diagnostic model learned as features of length change and shape change of tuberculosis bacteria according to the influence of said antibiotic for the diagnosis of tuberculosis bacteria classified as affected by a cocktail containing an antibiotic; and a step of providing a third diagnostic model learned as features of length change and shape change according to the growth of said other bacteria for the diagnosis of other bacteria that are not affected by said antibiotic, have a shape similar to said tuberculosis bacteria by a predetermined level or more, but have a length change pattern during growth different from said tuberculosis bacteria. A computer-readable recording medium that enables a processor to perform a method comprising the step of deriving a diagnosis result of tuberculosis bacteria for a sample obtained through a camera using a result diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model, wherein the first diagnostic model, the second diagnostic model, and the third diagnostic model are each independently learned. Claim 15 A computer program stored on a computer-readable recording medium, wherein, when executed by a processor, the computer program comprises: a step of providing a first diagnostic model learned as features of length change according to the growth of tuberculosis bacteria classified as normal growth for the diagnosis of tuberculosis bacteria; a step of providing a second diagnostic model learned as features of length change and shape change of tuberculosis bacteria according to the influence of said antibiotic for the diagnosis of tuberculosis bacteria classified as affected by a cocktail containing an antibiotic; and a step of providing a third diagnostic model learned as features of length change and shape change according to the growth of said other bacteria for the diagnosis of other bacteria that are not affected by said antibiotic, have a shape similar to said tuberculosis bacteria by a predetermined level or more, but have a length change pattern during growth different from said tuberculosis bacteria. A computer program stored on a computer-readable recording medium, comprising instructions for a processor to perform a method including the step of deriving a diagnosis result of tuberculosis bacteria for a sample obtained through a camera using a result diagnosed by at least one of the first diagnostic model, the second diagnostic model, and the third diagnostic model, wherein the first diagnostic model, the second diagnostic model, and the third diagnostic model are each independently learned.

Images