Classification apparatus and classification method based on artificial intelligence model for pulmonary fibrosis status of stem cell derived alveolar organoid

Abstract

According to some embodiments of the present invention, an artificial intelligence model-based classification device for the pulmonary fibrosis state of a stem cell-derived alveolar organoid comprises: a processor; and a memory configured to store instructions that cause the processor to perform operations when executed by the processor, wherein the operations include: an operation of extracting image features from a classification target image representing an alveolar organoid derived from a stem cell using a backbone network of an artificial intelligence neural network-based classification model, and an operation of classifying the state of the classification target image corresponding to the image features into a state of pulmonary fibrosis occurrence or a state of non-pulmonary fibrosis occurrence using a head network of the classification model, wherein the classification model is trained based on a training dataset, and the training dataset includes a first image dataset regarding an alveolar organoid in a first state in which pulmonary fibrosis has not occurred and a second image dataset regarding an alveolar organoid in a second state in which pulmonary fibrosis has occurred due to processing of a pulmonary fibrosis-inducing factor.

Description

Technology Field

[0001] The present invention relates to a classification device and a classification method capable of classifying the state of alveolar organoids (AOs) derived from induced pluripotent stem cells (iPSCs) as normal or pulmonary fibrosis using an artificial intelligence model. Background Technology

[0002] Pulmonary fibrosis is characterized by complex lung scar formation, and it may be necessary to effectively reproduce its pathology for the study of pulmonary fibrosis (PF) and the development of therapeutic agents. Alveolar organoid (AO) models can be utilized to mimic the structure of lung tissue; however, AO models may lack real-time dynamic analysis capabilities, which can reduce the efficiency of tracking disease progression and evaluating treatment responses. In particular, invasive procedures such as cell staining may be required to observe physiological responses occurring in AOs; however, these invasive processes can affect the condition of the AOs, thereby reducing the real-time capability and accuracy of monitoring cellular and molecular dynamics. The problem to be solved

[0003] One of the objectives of the present invention is to provide an artificial intelligence model-based classification technique capable of analyzing the condition of alveolar organoids (AOs) in real-time and dynamically in a non-invasive manner, in order to prevent the monitoring of the condition of alveolar organoids (AOs) from being affected by invasive procedures such as cell staining. The technical objectives of the present invention are not limited to the problems mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0004] According to some embodiments of the present invention, an artificial intelligence model-based classification device for the pulmonary fibrosis state of a stem cell-derived alveolar organoid comprises: a processor; and a memory configured to store instructions that cause the processor to perform operations when executed by the processor, wherein the operations include: an operation of extracting image features from a classification target image representing an alveolar organoid derived from a stem cell using a backbone network of an artificial intelligence neural network-based classification model, and an operation of classifying the state of the classification target image corresponding to the image features into a state of pulmonary fibrosis occurrence or a state of non-pulmonary fibrosis occurrence using a head network of the classification model, wherein the classification model is trained based on a training dataset, and the training dataset includes a first image dataset regarding an alveolar organoid in a first state in which pulmonary fibrosis has not occurred and a second image dataset regarding an alveolar organoid in a second state in which pulmonary fibrosis has occurred due to processing of a pulmonary fibrosis-inducing factor.

[0005] According to some embodiments of the present invention, the training dataset is formed through cell separation, region of interest extraction, pulmonary fibrosis labeling, and data augmentation for original images.

[0006] According to some embodiments of the present invention, the data augmentation for the original images comprises at least one of horizontal flipping, vertical flipping, clockwise rotation, counterclockwise rotation, horizontal distortion, and vertical distortion for the original images.

[0007] According to some embodiments of the present invention, the pulmonary fibrosis-inducing factor comprises transforming growth factor-beta-1 (TGF-β1), and the second image dataset shows symptoms of pulmonary fibrosis occurring in alveolar organoids due to treatment with TGF-β1.

[0008] According to some embodiments of the present invention, the operations further include a non-invasive operation of performing a real-time dynamic analysis of morphological changes occurring in alveolar organoids due to the treatment of TGF-β1 based on the result of classifying the state of the image to be classified using the classification model.

[0009] According to some embodiments of the present invention, the training dataset used for training the classification model comprises a first dataset relating to the original of the first image dataset and the second image dataset, a second dataset in which the alveolar region background is removed from the original of the first image dataset and the second image dataset, and a third dataset combining the first dataset and the second dataset, and the classification model comprises a first model trained based on the first dataset, a second model trained based on the second dataset, and a third model trained based on the third dataset, and the operations further include an operation of non-invasively performing the real-time dynamic analysis by comparing the classification result of the first model, the classification result of the second model, and the classification result of the third model for the image to be classified.

[0010] According to some embodiments of the present invention, an artificial intelligence model-based classification method for the pulmonary fibrosis state of a stem cell-derived alveolar organoid comprises: a step of extracting image features from a classification target image representing an alveolar organoid derived from a stem cell using a backbone network of an artificial intelligence neural network-based classification model; and a step of classifying the state of the classification target image corresponding to the image features into a state of pulmonary fibrosis occurrence or a state of non-pulmonary fibrosis occurrence using a head network of the classification model; wherein the classification model is trained based on a training dataset, and the training dataset includes a first image dataset regarding an alveolar organoid in a first state in which pulmonary fibrosis has not occurred and a second image dataset regarding an alveolar organoid in a second state in which pulmonary fibrosis has occurred due to the treatment of a pulmonary fibrosis-inducing factor.

[0011] According to some embodiments of the present invention, the training dataset is formed through cell separation, region of interest extraction, pulmonary fibrosis labeling, and data augmentation for original images.

[0012] According to some embodiments of the present invention, the data augmentation for the original images comprises at least one of horizontal flipping, vertical flipping, clockwise rotation, counterclockwise rotation, horizontal distortion, and vertical distortion for the original images.

[0013] According to some embodiments of the present invention, the pulmonary fibrosis-inducing factor comprises transforming growth factor-beta-1 (TGF-β1), and the second image dataset shows symptoms of pulmonary fibrosis occurring in alveolar organoids due to treatment with TGF-β1.

[0014] According to some embodiments of the present invention, the classification method further comprises the step of non-invasively performing a real-time dynamic analysis of morphological changes occurring in alveolar organoids due to the treatment of TGF-β1 based on the result of classifying the state of the image to be classified using the classification model.

[0015] According to some embodiments of the present invention, the training dataset used for training the classification model comprises a first dataset relating to the original of the first image dataset and the second image dataset, a second dataset in which the alveolar region background is removed from the original of the first image dataset and the second image dataset, and a third dataset combining the first dataset and the second dataset; the classification model comprises a first model trained based on the first dataset, a second model trained based on the second dataset, and a third model trained based on the third dataset; and the classification method further comprises the step of non-invasively performing the real-time dynamic analysis by comparing the classification result of the first model, the classification result of the second model, and the classification result of the third model for the image to be classified. Effects of the invention

[0016] According to embodiments of the present invention, an artificial intelligence model-based classification technique can be provided to analyze the condition of an alveolar organoid (AO) in real-time and dynamically in a non-invasive manner to prevent the monitoring of the condition of the alveolar organoid (AO) from being affected by invasive procedures such as cell staining.

[0017] The technical effects according to the embodiments of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art in accordance with the disclosure of this document. Brief explanation of the drawing

[0018] FIG. 1 illustrates the operation of an artificial intelligence model-based classification device for the pulmonary fibrosis status of stem cell-derived alveolar organoids according to some embodiments. FIG. 2 illustrates elements constituting an artificial intelligence model-based classification device for the pulmonary fibrosis status of stem cell-derived alveolar organoids according to some embodiments. FIGS. 3a to 3c illustrate a workflow for classifying alveolar organoids (AO) based on a deep learning algorithm according to some embodiments. FIGS. 4a to 4d illustrate the formation process and molecular dynamic characteristics of an alveolar organoid (AO) derived from human induced pluripotent stem cells (hiPSC) according to some embodiments. FIGS. 5a through 5c illustrate pulmonary fibrosis (PF) in alveolar organoids (AO) induced by TGFβ1 treatment and subsequent molecular dynamics characteristics according to some embodiments. FIGS. 6a and 6b illustrate a process flow for training a deep neural network for image classification of an alveolar organoid (AO) according to some embodiments. FIGS. 7a through 7c illustrate performance indicators and visual results for cell classification through deep learning across multiple datasets according to some embodiments. FIG. 8 illustrates steps constituting an artificial intelligence model-based classification method for the pulmonary fibrosis status of stem cell-derived alveolar organoids according to some embodiments. Specific details for implementing the invention

[0019] Embodiments of the present invention will be described in detail below with reference to the drawings. The description below is intended only to illustrate the embodiments and is not intended to limit or restrict the scope of the rights according to the present invention. Anything that can be easily inferred by a person skilled in the art from the detailed description and embodiments of the invention should be interpreted as falling within the scope of the rights according to the present invention. Detailed descriptions of matters widely known to those skilled in the art regarding the present invention are omitted.

[0020] The terms used in this invention are described as general terms widely used in the technical field relating to this invention; however, the meaning of the terms used in this invention may vary depending on the intent of those skilled in the field, the emergence of new technologies, examination standards, or case law. Some terms may be selected at the discretion of the applicant, and in such cases, the meaning of the arbitrarily selected terms will be explained in detail. The terms used in this invention should be interpreted not merely in their dictionary meanings, but in a sense that reflects the overall context of the specification.

[0021] FIG. 1 illustrates the operation of an artificial intelligence model-based classification device for the pulmonary fibrosis status of stem cell-derived alveolar organoids according to some embodiments.

[0022] Referring to FIG. 1, an artificial intelligence model-based classification device (100) for the pulmonary fibrosis state of a stem cell-derived alveolar organoid can receive a classification target image (10) as input and classify the pulmonary fibrosis state (20) of the classification target image (10).

[0023] The classification target image (10) may be an image of an alveolar organoid (AO) that is the subject of classification. The classification target image (10) may be generated based on various cell imaging techniques. An organoid is an organoid that performs functions similar to human organs and may be generated based on stem cells, etc. An alveolar organoid (AO) may be an analog that mimics the shape and function of alveoli based on stem cells, etc.

[0024] The pulmonary fibrosis state (20) may be a normal state in which pulmonary fibrosis (PF) has not progressed, or a state in which pulmonary fibrosis (PF) has progressed. For example, if the image to be classified (10) is similar to a training image of an alveolar organoid (AO) that has not been treated with a pulmonary fibrosis (PF) inducing factor, it may be classified as a normal state. Or, if the image to be classified (10) is similar to a training image of an alveolar organoid (AO) that has been treated with a induced factor such as TGF-β1, it may be classified as a state in which pulmonary fibrosis is present.

[0025] Pulmonary fibrosis (PF) is characterized by complex lung scar formation, and there can be significant challenges in effectively reproducing its pathology for research and therapeutic development. Recently, there have been attempts to better mimic lung tissue structures using alveolar organoid models; however, these models often lack dynamic and real-time analysis capabilities, which may limit the tracking of disease progression and the evaluation of treatment responses. According to the present invention, a laboratory model capable of mimicking pulmonary fibrosis (PF) by modifying alveolar organoids derived from induced pluripotent stem cells (iPSCs) with TGFβ1 and monitoring them in real-time via an AI-based deep neural network (DNN) can be developed. Research results confirm that the alveolar organoid (AO) and DNN system can monitor and precisely detect minute morphological changes in real-time with approximately 98% accuracy. This approach can contribute to understanding fibrotic changes at the cellular level and provide a strong foundation for preclinical trials. The synergy between iPSC technology and machine learning can accelerate the transition of laboratory research results to clinical applications and lead to significant advancements in the study of fibrosis-related diseases and the development of potential treatments.

[0026] Pulmonary fibrosis (PF) can be a serious respiratory disease characterized by progressive and irreversible decline in lung function. It primarily occurs when interactions between cellular components and the extracellular matrix are disrupted and may manifest as features such as excessive extracellular matrix accumulation, increased collagen deposition, extensive tissue remodeling, myofibroblast proliferation, and epithelial-mesenchymal transition (EMT). Reproducing these complex biological processes in a laboratory setting can be very difficult. Furthermore, the combined effects of genetic factors and environmental factors, such as exposure to harmful substances and autoimmune responses, can make the development of accurate models and the design of targeted therapies challenging.

[0027] To address these issues, advanced models may be required to discover effective treatments. Alveolar structures, which play a crucial role in pulmonary fibrosis (PF) research, may be primarily composed of Type 1 (AEC1) and Type 2 (AEC2) alveolar epithelial cells. AEC1 cells can form a fragile barrier essential for gas exchange, while AEC2 cells play a vital role in maintaining alveolar homeostasis by being responsible for surfactant secretion and regeneration. According to the present invention, induced pluripotent stem cells (iPSCs) can be differentiated into alveolar epithelial cells and utilized by utilizing the latest biotechnology.

[0028] Recent advancements in organoid technology can enable the development of sophisticated laboratory models that closely mimic the complex cellular structures and physiological responses of human tissues, particularly the lungs. These organoid models can be considered useful tools for disease modeling, drug testing, and personalized medicine. However, existing organoid analysis methods often involve invasive procedures, such as cell staining, which can interfere with the functional status and morphological structure of the organoids. Furthermore, since these methods generally provide only static information, they may have limitations in capturing dynamic changes in progressive diseases such as PF.

[0029] Interactions between various cell types within lung tissue and their microenvironments can complicate PF modeling. The extracellular matrix (ECM) can be a complex network composed of filaments, proteins, and signaling molecules that provides structural support and can significantly influence cellular behavior and disease progression. Advanced imaging techniques and computational modeling offer valuable insights into these interactions, providing opportunities for a deeper understanding of disease mechanisms. This can be crucial for designing effective targeted interventions for the disease at multiple levels, ranging from molecular interactions to cellular responses.

[0030] However, current invasive techniques may have the potential to distort interactions by disrupting the natural state of tissues and cells. These invasive methods can complicate the accurate modeling of dynamic and progressive diseases such as PF, which may increase the demand for non-invasive methodologies capable of monitoring and analyzing cellular and molecular dynamics in real time.

[0031] To overcome the limitations of existing MTT analysis methods and other invasive techniques, the present invention can analyze organoid images under processed and unprocessed conditions using an AI-based platform. The present invention can identify and classify subtle morphological changes indicating the progression of fibrosis by utilizing a deep learning algorithm trained on a high-resolution organoid image dataset. The AI ​​application can be configured to non-invasively monitor the state of fibrosis and organoid viability while preserving structural and cellular integrity.

[0032] The integration of AI and advanced organoid models represents a significant advance in regenerative medicine and can provide a new perspective for dynamically observing disease processes in real time. The present invention can transform conventional static analysis methods into dynamic, AI-enhanced methodologies and can innovatively advance the modeling, monitoring, and therapeutic potential of complex diseases such as PF.

[0033] To explore the potential applications of hiPSCs (induced pluripotent stem cells) in a pulmonary fibrosis (PF) organoid model, alveolar organoids can be developed using a 25-day differentiation protocol (see Fig. 6a). Initially, hiPSCs can be maintained under culture conditions that preserve a colony-like morphology, which is important for maintaining pre-differentiation pluripotency (Fig. 6b; D0). On day 14, these progenitor cells differentiate into iAECs (induced alveolar epithelial cells) and can subsequently be isolated (Fig. 6b; D13). The isolated cells can aggregate within a specialized extracellular matrix protein mixture to mimic the lung basement membrane microenvironment. This matrix can be removed the following day to induce self-organization into spherical structures.

[0034] At this stage, differentiating cells may begin to express markers indicating transformation into the lung epithelial lineage. On day 15, high expression of transcription factors essential for lung development, such as GATA6, NKX2.1, and SOX9, may be observed, while the expression of the pluripotency marker SOX2 may decrease (Fig. 6c). These molecular changes may represent key points in the differentiation trajectory that guide the subsequent process of organoid maturation into functional lung tissue.

[0035] Between days 21 and 25, differentiating spheroids may exhibit notable morphological changes. The organoids formed robust membrane structures, and transparent spaces reminiscent of alveolar structures appeared, which may indicate maturing alveolar epithelial cells (Fig. 6b; D21-D25). Additionally, a slight increase in size was observed, suggesting that cell expansion and extracellular matrix remodeling were taking place within the organoid structure. On day 25, sustained expression of NKX2.1 could be confirmed in the organoids, and high levels of expression of SFTPC and SFTPB, markers associated with terminal alveolar epithelial cells, could be observed (Fig. 6c).

[0036] Immunostaining performed on day 25 additionally confirmed the presence of AEC1 markers RAGE and AQP5, AEC2 marker SFTPC, and the epithelial cell-wide marker EpCAM. These expression profiles may support the successful differentiation and functional maturation of hiPSC-derived AO (Fig. 6d).

[0037] These results suggest that AOs derived from hiPSCs are a promising laboratory model that reflects many structural and functional aspects of distal lung epithelium. However, further investigation may be required to determine whether these cells possess full maturation and full functionality. Generating organoids with defined cellular and molecular characteristics can provide a valuable tool for elucidating the mechanisms of PF and other lung diseases. Furthermore, this model may offer new opportunities to evaluate therapeutic interventions in a controlled environment, which could facilitate the development of treatments for various lung diseases.

[0038] Morphological changes can be induced in hiPSC-derived alveolar organoids (AOs) by utilizing the fibrosis-inducing potential of TGFβ1 (Transforming Growth Factor Beta 1) (see Fig. 5a). Treatment with TGFβ1 at a concentration of 10 ng / mL resulted in a gradual and significant decrease in organoid size over 24 to 120 hours (see Fig. 5a). While slight changes were observed during the initial 24 hours, after 120 hours, the size decreased by 40% to 50%, and a significantly condensed structure appeared compared to the control group. The treated organoids exhibited a dense and condensed structure, which may contrast with the expanded, sac-like structure observed in the control group. High-resolution imaging confirmed these changes, including the loss of the thin squamous epithelial layer typically observed in healthy alveolar structures.

[0039] During the treatment period, organoid density increased and transparency decreased, which may indicate early-stage fibrotic changes. These changes can be detailed using high-resolution imaging techniques that reveal intercellular interactions and the physical condensation and changes in the extracellular matrix composition.

[0040] Histological analysis can confirm that collagen deposition in TGFβ1-treated organoids is significantly increased compared to healthy controls. Collagen accumulation is a characteristic of pulmonary fibrosis (PF) and can lead to the formation of dense fibrotic nodules that replace normal alveolar parenchyma. These changes can be confirmed through Masson's Trichome staining data, in which collagen accumulation is distinct in TGFβ1-treated organoids, whereas healthy controls maintain a thin epithelial layer and preserve alveolar structure (see Fig. 5a). Fibrotic changes can be distinguished by structural condensation and the loss of typical alveolar spaces, which may reflect extensive tissue remodeling.

[0041] In addition to histological observations, changes in fibrosis can be further verified through immunofluorescence analysis. Fluorescence microscopy images can reveal the expression of alpha-smooth muscle actin (α-SMA), a well-known marker for myofibroblasts (see Fig. 5b). In treated organoids, α-SMA expression is significantly increased, which may indicate the activation of myofibroblast formation and fibrosis. In contrast, α-SMA expression is detected only minimally in control organoids, which may highlight the specific effect of TGFβ1 exposure in inducing a myofibroblast phenotype. These images clearly demonstrate the distinct difference in α-SMA expression between the treatment and control groups and highlight the direct influence of TGFβ1 on cell behavior and the activation of fibrosis pathways.

[0042] These visual and histological analyses confirm that the structural and cellular changes induced by TGFβ1 are profound and important for understanding the mechanisms influencing fibrosis in this laboratory model. By documenting these changes through high-resolution imaging, the fibrotic process can be clearly visualized, providing valuable insights into the cellular and molecular interactions affecting the progression of pulmonary fibrosis.

[0043] Molecular evaluation can clearly elucidate cell deformation induced by TGFβ1 in AO. Real-time PCR analysis confirmed that fibrosis markers such as collagen I alpha 1 (COL1A1), fibronectin (FN), vimentin (VIM), and α-SMA (ACTA2) were significantly upregulated after TGFβ1 treatment. This expression pattern may indicate an active fibrotic remodeling process in which normal tissue structures are replaced by fibrotic tissue.

[0044] At the same time, the expression of epithelial markers such as E-cadherin (CDH1) may be significantly reduced, which may indicate a phenotypic transition from epithelium to mesenchyme at the center of fibrosis development. Quantitative expression levels may show that fibrosis gene expression in treated organoids is significantly increased compared to the control group (see Fig. 5c), and this difference may highlight the alterations indicating EMT and the effect of TGFβ1 on cell behavior.

[0045] According to these results, it can be confirmed that TGFβ1 induces significant changes in gene expression exhibiting myofibroblast activation and fibrotic characteristics. Fluorescence microscopy images and gene expression analysis can effectively visualize and quantify these changes, and the effectiveness of this model in replicating the complex disease mechanisms observed in PF in a laboratory model can be demonstrated.

[0046] The present invention may develop a deep neural network (DNN) to classify alveolar organoid (AO) images into a "control group (CTL)" and a "TGFβ1-treated group." As illustrated in FIG. 6a, during the data acquisition and preprocessing process, the image size may be adjusted from 2048 × 1536 pixels to 640 × 480 pixels, cell separation may be extracted, a region of interest (ROI) may be defined, and a procedure to label each image may be performed. To address the problem of size limitations of the initial dataset, data augmentation techniques such as horizontal and vertical flipping, rotation from 45° to 135° clockwise and counterclockwise, and horizontal and vertical distortion (ranging from -10% to +10%) may be applied. Through this, the dataset can be expanded to a total of 8,892 images, of which 3,577 are control groups and 5,315 are TGFβ1 treatment groups, providing a robust dataset for model training.

[0047] During the training process, the ROI of each image is input into the model using an augmented dataset to perform classification (see Fig. 6b). Model performance can be evaluated using various metrics such as F1 score, mean mean precision (mAP50 and mAP50-95), precision, recall, and confusion matrix. Additionally, loss functions such as box loss, segmentation loss, class loss, and distribution focus loss can also be analyzed. These evaluation results demonstrate that the model possesses high accuracy and reliability in detecting subtle morphological changes induced by TGFβ1 treatment. This can enhance the precision of pathological evaluations and serve as a valuable tool for real-time analysis of organoid models in biomedical research.

[0048] Model training was performed for 100 epochs for each dataset, and performance metrics such as average precision (mAP50-95), mAP50, precision, and recall could be continuously monitored (see Fig. 7a). The original image dataset (blue line) showed excellent performance across all metrics, with mAP50-95 values ​​reaching up to 95%. This is higher than the values ​​observed in the background-removed dataset (green line) and the combined dataset (red line), which were 87% and 84%, respectively. These results suggest that background elements in images can enhance the neural network's ability to contextualize and recognize cellular features, which may be consistent with existing research indicating that context is important for object detection and classification in complex images.

[0049] The efficiency of the classification process after training can be visually demonstrated through an overlay superimposed on the cell images, and the process by which the network identifies and classifies the processing status of the cells can be verified (see Fig. 7b). The confusion matrix can clearly measure the accuracy for each dataset, and the matrix of the original dataset demonstrates the network's ability to distinguish between the two categories by showing high true positive rates of 98% and 99% for the control group and the TGFβ1 treatment group, respectively (see Fig. 7c).

[0050] Validation loss can be evaluated across multiple dimensions, including class, box, distribution focus, and segmentation loss. The original dataset recorded the lowest value with a class loss of 0.249, which may indicate effective learning from the dataset. Box loss reflects the network's ability to accurately locate objects and can be lowest at 0.261 on the original dataset. However, segmentation loss represents a challenge in segmenting cellular features in complex backgrounds, and the original dataset showed a higher value of 1.343 compared to the modified dataset. This variability in segmentation performance highlights the importance of dataset composition in DNN training for biomedical image analysis.

[0051] In this invention, changes were induced in alveolar organoids (AOs) derived from induced pluripotent stem cells (iPSCs) by effectively utilizing TGFβ1, confirming that this allows for a close mimicry of the pathological characteristics of pulmonary fibrosis (PF). Treatment with TGFβ1 can induce structural and morphological changes similar to those observed in clinical settings, thereby verifying that this model is a powerful tool for investigating the cellular and molecular dynamics of fibrosis. Such verification is crucial for understanding disease mechanisms and developing potential anti-fibrotic therapies. Furthermore, the ability of this model to reproduce disease-specific scenarios suggests the potential to accelerate the transition of laboratory results to clinical applications.

[0052] Furthermore, the precision of observations can be significantly enhanced by integrating advanced deep neural networks to classify and analyze generated biomedical images. Through sophisticated image processing techniques, DNNs can detect and quantify subtle morphological changes induced by TGFβ1, highlighting the transformative potential of AI in the field of medical imaging. This model demonstrates high accuracy and precision across various datasets, proving its efficiency and reliability. In particular, systematic evaluation under diverse conditions can identify key factors influencing image classification model performance, such as dataset quality and the complexity of model training. These insights may be essential for optimizing DNNs for similar biomedical research.

[0053] The high accuracy and reliability of the model in distinguishing minute differences in organoid morphology demonstrate that it can serve as a valuable tool in the field. This can provide researchers with a novel approach to understanding disease mechanisms at a granular level and evaluating the efficacy of new therapies in a controlled, reproducible environment. The present invention supports the widespread use of such a model in biomedical research and indicates its potential to revolutionize disease research and approaches in the preclinical setting.

[0054] In conclusion, the present invention demonstrates that changes in iPSC-based AO induced by TGFβ1 closely mimic the pathological characteristics of pulmonary fibrosis, enabling the construction of a powerful laboratory model to elucidate disease mechanisms. The integration of advanced DNNs for image analysis can significantly enhance the ability to identify subtle morphological changes, confirming the usefulness of this model for detailed pathological evaluation. The precision and accuracy achieved in classifying these changes highlight the potential to advance the understanding of fibrotic diseases by combining high-speed imaging with AI. This model not only provides novel insights into the cellular and molecular basis of PF but also possesses the potential to accelerate the development of anti-fibrotic therapies, thereby bridging the gap between preclinical research and clinical application. By leveraging cutting-edge technology, the present invention can establish a new standard for improving disease modeling and therapeutic testing, representing a significant advance in the fields of regenerative medicine and drug discovery.

[0055] FIG. 2 illustrates elements constituting an artificial intelligence model-based classification device for the pulmonary fibrosis status of stem cell-derived alveolar organoids according to some embodiments.

[0056] Referring to FIG. 2, the classification device (100) based on an artificial intelligence model for the pulmonary fibrosis state of a stem cell-derived alveolar organoid may include a memory (110) and a processor (120). However, it is not limited thereto, and some components may be omitted from the classification device (100), or other components may be further included in the classification device (100).

[0057] The memory (110) may be configured to store various data, instructions, computer programs, software, and mobile applications processed by the classification device (100). For example, the memory (110) may be implemented as a non-volatile memory such as ROM, PROM, EPROM, EEPROM, flash memory, PRAM, MRAM, RRAM, FRAM, etc., or as a volatile memory such as DRAM, SRAM, SDRAM, PRAM, RRAM, FeRAM, etc., and may be implemented in the form of an HDD, SSD, SD, Micro-SD, etc., or a combination thereof.

[0058] The processor (120) may be configured to execute instructions, programs, applications, etc., stored in memory (110). The processor (120) may be implemented as an array of logic gates or a general-purpose microprocessor for processing various operations, and may be composed of a single processor or multiple processors. For example, the processor (120) may be implemented in at least one form of a microprocessor, CPU, GPU, and AP.

[0059] The classification device (100) may be configured to extract image features from a classification target image (10) representing an alveolar organoid derived from stem cells using a backbone network of an artificial intelligence neural network-based classification model. The classification target image (10) may be obtained through cell imaging of an alveolar organoid (AO), and the image features may represent the color, structure, shape, pattern, etc. of the alveolar organoid (AO). The backbone network of the classification model may operate as shown in FIG. 6b.

[0060] The classification device (100) may be configured to classify the state of a classification target image (10) corresponding to image features into a state of pulmonary fibrosis occurrence or a state of non-pulmonary fibrosis occurrence using a head network of a classification model. For example, when TGF-β1 is applied to an alveolar organoid (AO), symptoms of pulmonary fibrosis (PF) may occur, and the head network may classify whether symptoms of pulmonary fibrosis (PF) exist based on image features. The head network of the classification model may operate as shown in FIG. 6b.

[0061] According to an embodiment, the classification model is trained based on a training dataset, and the training dataset may include a first image dataset regarding alveolar organoids in a first state in which pulmonary fibrosis has not occurred and a second image dataset regarding alveolar organoids in a second state in which pulmonary fibrosis has occurred due to treatment with a pulmonary fibrosis-inducing factor. For example, the first image dataset may be labeled as “CTL” and the second image dataset may be labeled as “TGFβ1”. As trained through the first image dataset and the second image dataset, the classification model can dynamically and in real time detect changes in the state of alveolar organoids (AO) caused by TGFβ1 treatment.

[0062] According to the embodiment, a training dataset can be formed through cell separation, region of interest extraction, labeling for pulmonary fibrosis, and data augmentation of original images. Alveolar regions can be separated from the original images, and a region of interest can be set to include them. Labeling with “CTL” or “TGFβ1” can be performed for supervised learning of a deep learning model. Even if the number of data in the original images is insufficient, a sufficient number of training datasets required for learning can be secured through data augmentation.

[0063] According to an embodiment, data augmentation for original images may include at least one of horizontal flipping, vertical flipping, clockwise rotation, counterclockwise rotation, horizontal distortion, and vertical distortion of the original images. For example, augmented data may be obtained through horizontal and vertical flipping, rotation from 45° to 135° clockwise and counterclockwise, and horizontal and vertical distortion (-10% to +10%), and these may be divided into training data and validation data for use.

[0064] According to the embodiment, the pulmonary fibrosis-inducing factor may include transforming growth factor-beta-1 (TGF-β1), and the second image dataset may represent symptoms of pulmonary fibrosis occurring in alveolar organoids due to treatment with TGF-β1. Since TGF-β1 can be utilized as a representative pulmonary fibrosis-inducing factor and images regarding the results of treating alveolar organoids with it can be utilized as the second image dataset, the classification model can be configured to analyze the process of pulmonary fibrosis progression caused by the inducing factor in real-time and dynamically.

[0065] According to an embodiment, the classification device (100) may be configured to non-invasively perform real-time dynamic analysis of morphological changes occurring in alveolar organoids due to TGF-β1 treatment based on the results of classifying the state of the target image (10) using a classification model. Unlike conventional methods where invasive treatments such as cell staining were required for analysis, the classification device (100) can non-invasively analyze the state of the alveolar organoid (AO) through a classification model. Furthermore, since model training can be performed by setting data regarding the TGF-β1 treatment results as a second image dataset, the classification model can dynamically analyze the detailed stages of pulmonary fibrosis (PF) progression due to TGF-β1 treatment in real time.

[0066] According to an embodiment, the training dataset used for training a classification model may include a first dataset regarding the original of a first image dataset and a second image dataset, a second dataset in which the background of the alveolar region is removed from the original of the first image dataset and the second image dataset, and a third dataset combining the first dataset and the second dataset. The classification model may include a first model trained based on the first dataset, a second model trained based on the second dataset, and a third model trained based on the third dataset. The classification device (100) may be configured to non-invasively perform real-time dynamic analysis by comparing the classification result of the first model, the classification result of the second model, and the classification result of the third model for a classification target image (10). i) The first dataset may be an original including a background image, ii) The second dataset may be a background-removed version in which the background image is removed leaving only the alveolar region, and iii) The third dataset may be a case where both the first dataset and the second dataset are used as training data. By training a classification model with three different types and analyzing the target image (10) using the three types of models, three types of state classification results can be derived. For state classification using the three types of models, FIGS. 7a to 7c may be referenced. Generally, the original model based on the first dataset may exhibit high performance, but depending on the need, a model based on the second dataset or the third dataset may be utilized, taking into account learning efficiency or computational throughput. According to the embodiment, the classification device (100) may be configured to derive a final classification result by comprehensively considering the three types of state classification results in a manner such as weighted sum.

[0067] FIGS. 3a to 3c illustrate a workflow for classifying alveolar organoids (AO) based on a deep learning algorithm according to some embodiments.

[0068] Referring to Fig. 3a, the alveolar organoid (AO) initiation step may begin with generating alveolar organoids after inducing alveolar progenitor cells from human induced pluripotent stem cells (hiPSCs). To generate fibrosis organoids, the process may involve exposing the organoids to acrolein and sodium chromate for 24, 72, and 120 hours, respectively.

[0069] Referring to Fig. 3b, the evaluation of alveolar organoid viability through MTT analysis can be described as a process of treating with MTT reagent, dissolving formazan crystals with dimethyl sulfoxide after removing the reagent, and classifying organoid images through spectrophotometric measurement.

[0070] Referring to Fig. 3c, deep learning for alveolar organoid analysis can be configured to collect alveolar organoid images and process them with a deep learning algorithm to derive computational analysis results.

[0071] FIGS. 4a to 4d illustrate the formation process and molecular dynamic characteristics of an alveolar organoid (AO) derived from human induced pluripotent stem cells (hiPSC) according to some embodiments.

[0072] Referring to Figures 4a to 4d, an initial culture and differentiation protocol for hiPSCs can be described in relation to the developmental progression and molecular characterization of alveolar organoids derived from hiPSCs.

[0073] Referring to Fig. 4a, the process of culturing and differentiating hiPSCs for 25 days as described above in Fig. 1 can be illustrated. Referring to Fig. 4a, regarding the visual timeline of morphological changes, changes in iPSC morphology can be shown from Day 0, when the colony-shaped form is separated from the culture plate, to Day 14, when aggregation with a mixture of extracellular matrix proteins occurs.

[0074] Referring to Fig. 4c, regarding gene expression analysis, an increase in the expression of lung epithelial lineage markers GATA6, NKX2.1, and SOX9 and a decrease in the pluripotency marker SOX2 can be analyzed on day 15 of differentiation. Regarding the observation of morphological maturation, cell membrane formation is prominent and clear spaces are clearly observed from Day 21 to Day 25, and gene expression exhibiting the characteristics of mature alveolar epithelial cells can be analyzed along with an increase in size. Referring to Fig. 4d, regarding immunohistochemistry data, the successful differentiation and characterization of iPSC-derived alveolar organoids can be demonstrated by confirming the expression of type 1 (AEC1) alveolar epithelial cell markers RAGE and AQP5, type 2 (AEC2) marker SFTPC, and epithelial cell marker EpCAM.

[0075] FIGS. 5a through 5c illustrate pulmonary fibrosis (PF) in alveolar organoids (AO) induced by TGFβ1 treatment and subsequent molecular dynamics characteristics according to some embodiments.

[0076] Referring to Figures 5a through 5c, the induction of pulmonary fibrosis and molecular characterization in alveolar organoids via TGFβ1 treatment can be illustrated. Referring to Figure 5a, regarding changes in alveolar organoid morphology following TGFβ1 exposure, a reduction in size and structural differences in alveolar organoids upon TGFβ1 treatment can be presented, and changes can be emphasized compared to the control group. Additionally, regarding the bar graph, a significant reduction in alveolar organoid size after TGFβ1 treatment may be observed, and a difference from the control organoid may be shown. Furthermore, regarding the staining data, collagen accumulation may be observed in TGFβ1-treated alveolar organoids, which may indicate the development of fibrosis compared to the control group. While the control organoid resembles an alveolar cyst with a thin epithelial layer, such a structure may not be observed in the TGFβ1-treated organoid.

[0077] Referring to Fig. 5b, fluorescence microscopy images show that the expression of alpha-smooth muscle actin (a-SMA), a myofibroblast marker, is increased in TGFβ1-treated organoids, whereas it is barely detectable in control organoids. This may indicate signs of fibrosis. Referring to Fig. 5c, RT-PCR results show that the expression of fibrosis-related genes (COL1A1, FN, VIM, ACTA2) is increased and the expression of the epithelial marker CDH1 is decreased in TGFβ1-treated alveolar organoids. This may suggest that epithelial-mesenchymal transition (EMT) has occurred.

[0078] FIGS. 6a and 6b illustrate a process flow for training a deep neural network for image classification of an alveolar organoid (AO) according to some embodiments.

[0079] Referring to FIGS. 6a and 6b, a flowchart of the deep learning neural network training process for alveolar organoid image classification may be illustrated. Referring to FIG. 6a, regarding data collection and preprocessing, the data collection step may include collecting 'Control' (CTL) and TGFβ1-treated cell images. Cell regions may be extracted from the images and stored as labeled data. Regions of interest (ROIs) may be defined, and images may be labeled accordingly. Subsequently, data augmentation may be performed using various techniques such as flipping, rotation, cropping, and distortion to enhance the dataset. The prepared augmented images may be used as input data for the deep learning model training step. FIG. 6a may show a process in which a cell image labeled as CTL is received as input, ROI images are applied, and the model classifies the cells to highlight the identified regions in the resulting image. In the evaluation step, the original image and the result image classified after training may be compared. The left image may represent the cell image before training, and the right image may show cells successfully classified as CTL highlighted with a green overlay. The performance of the model can be quantitatively evaluated through various metrics such as F1 score, mAP50, precision, recall, box loss, segmentation loss, class loss, distribution focus loss, and confusion matrix.

[0080] Referring to Fig. 6a, regarding the structure of YOLOv8, the backbone network can extract features such as shape, color, and texture from the input image. The head network can predict the class and location of an object and increase object detection accuracy to generate a final output. Using the features extracted by the backbone, the location of an object can be accurately detected and classification can be performed. Regarding the components of the classification model, ConvModule can represent a convolution module, C2fX3 and C2fX6 can use two convolutions as a CSP bottleneck structure, SPPF can represent Spatial Pyramid Pooling-Fast, Conv2d can represent a 2D convolution layer, BatchNorm2D can represent a 2D batch normalization layer, and SiLU can represent a sigmoid linear unit.

[0081] FIGS. 7a through 7c illustrate performance indicators and visual results for cell classification through deep learning across multiple datasets according to some embodiments.

[0082] Referring to Figure 7a, regarding the performance graphs, four graphs can represent the performance shown by the model during the training and validation phases on three datasets. Each dataset can be distinguished by a unique color. The primary original image dataset can be shown in blue, the background-removed image dataset in green, and the original and background-removed image datasets in red. Metrics such as mAP50-95, mAP50, Precision, and Recall are plotted over 100 epochs, allowing the learning curve and the model's predictive stability to be shown over time.

[0083] Referring to Fig. 7b, regarding the classification result images, this set of images may represent the results derived from the model's classification task. The images may show data labeled as 'Control' (CTL) and TGFβ1-treated cells, and the model's predicted results may be displayed as a red overlay. Correctly identified regions may be highlighted in accordance with the model's prediction.

[0084] Referring to Figure 7c, regarding the confusion matrix, the three confusion matrices shown at the bottom can show the performance of the model on each dataset. Diagonal cells represent correct classifications, while off-diagonal cells represent incorrect predictions. This makes it possible to quantitatively evaluate the classification accuracy of the model.

[0085] FIG. 8 illustrates steps constituting an artificial intelligence model-based classification method for the pulmonary fibrosis status of stem cell-derived alveolar organoids according to some embodiments.

[0086] Referring to FIG. 8, an artificial intelligence model-based classification method (800) for the pulmonary fibrosis state of a stem cell-derived alveolar organoid may include steps (810) to (820). However, it is not limited thereto, some steps may be omitted or other general steps may be added, and the steps of the classification method (800) may be executed in a different order than the illustrated order.

[0087] The classification method (800) may consist of steps processed sequentially in a classification device (100) based on an artificial intelligence model for the pulmonary fibrosis state of stem cell-derived alveolar organoids. Therefore, even if the details are omitted below, the description of the classification device (100) above may be equally applicable to the classification method (800).

[0088] Steps (810) to (820) of the classification method (800) can be performed by the memory (110) and processor (120) of the classification device (100).

[0089] In step (810), the classification device (100) can perform the step of extracting image features from a classification target image representing an alveolar organoid derived from a stem cell using a backbone network of an artificial intelligence neural network-based classification model.

[0090] In step (820), the classification device (100) may perform a step of classifying the state of the image to be classified corresponding to the image features using the head network of the classification model into a state of pulmonary fibrosis occurrence or a state of non-pulmonary fibrosis occurrence.

[0091] In the classification method (800), the classification model may be trained based on a training dataset, and the training dataset may include a first image dataset regarding a first state alveolar organoid in which no pulmonary fibrosis has occurred and a second image dataset regarding a second state alveolar organoid in which pulmonary fibrosis has occurred due to the treatment of a pulmonary fibrosis-inducing factor.

[0092] According to an embodiment, the classification method (800) may be implemented in the form of a computer program stored on a computer-readable storage medium. That is, the computer program may include instructions for implementing the classification method (800), and the instructions of the program may be stored on a computer-readable storage medium. The computer program may include a mobile application.

[0093] According to an embodiment, a computer-readable storage medium may include magnetic media such as a hard disk, a floppy disk, and a magnetic tape, optical media such as a CD-ROM and a DVD, magneto-optical media such as a floptical disk, and a hardware device specifically configured to store and execute computer program instructions such as ROM, RAM, and flash memory. Computer program instructions may include machine code generated by a compiler and high-level language code that can be executed by a computer using an interpreter, etc.

[0094] Although embodiments of the present invention have been described in detail above, the scope of rights according to the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as described in the following claims should also be interpreted as being included within the scope of rights according to the present invention. Explanation of the symbols

[0095] 10: Image to be classified 20: Pulmonary fibrosis 100: Classification device 110: Memory 120: Processor

Claims

Claim 1 An artificial intelligence model-based classification device for the pulmonary fibrosis state of a stem cell-derived alveolar organoid comprises: a processor; and a memory configured to store instructions that cause the processor to perform operations when executed by the processor, wherein the operations include: an operation of extracting image features from a classification target image representing an alveolar organoid derived from a stem cell using a backbone network of an artificial intelligence neural network-based classification model; and an operation of classifying the state of the classification target image corresponding to the image features into a state of pulmonary fibrosis occurrence or a state of non-pulmonary fibrosis occurrence using a head network of the classification model, wherein the classification model is trained based on a training dataset, and the training dataset includes a first image dataset regarding an alveolar organoid in a first state in which pulmonary fibrosis has not occurred and a second image dataset regarding an alveolar organoid in a second state in which pulmonary fibrosis has occurred due to processing of a pulmonary fibrosis-inducing factor. Claim 2 In claim 1, the learning dataset is formed through cell separation, region of interest extraction, pulmonary fibrosis labeling, and data augmentation for original images, an artificial intelligence model-based classification device for the pulmonary fibrosis status of stem cell-derived alveolar organoids. Claim 3 In paragraph 2, the data augmentation for the original images comprises at least one of horizontal flipping, vertical flipping, clockwise rotation, counterclockwise rotation, horizontal distortion, and vertical distortion for the original images, an artificial intelligence model-based classification device for the pulmonary fibrosis state of stem cell-derived alveolar organoids. Claim 4 An artificial intelligence model-based classification device for the pulmonary fibrosis status of a stem cell-derived alveolar organoid, wherein the pulmonary fibrosis-inducing factor in claim 1 includes transforming growth factor-beta-1 (TGF-β1), and the second image dataset shows pulmonary fibrosis symptoms occurring in the alveolar organoid due to treatment with TGF-β1. Claim 5 An artificial intelligence model-based classification device for the pulmonary fibrosis state of a stem cell-derived alveolar organoid, wherein, in claim 4, the above operations further include an operation of non-invasively performing real-time dynamic analysis on morphological changes occurring in the alveolar organoid due to the treatment of TGF-β1 based on the result of classifying the state of the image to be classified using the classification model. Claim 6 In claim 5, the training dataset used for training the classification model comprises a first dataset relating to the original of the first image dataset and the second image dataset, a second dataset in which the alveolar region background is removed from the original of the first image dataset and the second image dataset, and a third dataset combining the first dataset and the second dataset; the classification model comprises a first model trained based on the first dataset, a second model trained based on the second dataset, and a third model trained based on the third dataset; and the operations further include an operation of non-invasively performing the real-time dynamic analysis by comparing the classification result of the first model, the classification result of the second model, and the classification result of the third model for the image to be classified. Claim 7 A method for classifying the pulmonary fibrosis state of a stem cell-derived alveolar organoid based on an artificial intelligence model, comprising: a step of extracting image features from a target image representing an alveolar organoid derived from a stem cell using a backbone network of an artificial intelligence neural network-based classification model; and a step of classifying the state of the target image corresponding to the image features into a state of pulmonary fibrosis occurrence or a state of non-pulmonary fibrosis occurrence using a head network of the classification model; wherein the classification model is trained based on a training dataset, and the training dataset includes a first image dataset regarding an alveolar organoid in a first state in which pulmonary fibrosis has not occurred and a second image dataset regarding an alveolar organoid in a second state in which pulmonary fibrosis has occurred due to the treatment of a pulmonary fibrosis-inducing factor. Claim 8 In claim 7, the above-mentioned learning dataset is formed through cell separation, region of interest extraction, pulmonary fibrosis labeling, and data augmentation of original images, an artificial intelligence model-based classification method for the pulmonary fibrosis status of stem cell-derived alveolar organoids. Claim 9 In claim 8, the data augmentation for the original images comprises at least one of horizontal flipping, vertical flipping, clockwise rotation, counterclockwise rotation, horizontal distortion, and vertical distortion for the original images, in an artificial intelligence model-based classification method for the pulmonary fibrosis state of a stem cell-derived alveolar organoid. Claim 10 In claim 7, the pulmonary fibrosis-inducing factor comprises transforming growth factor-beta-1 (TGF-β1), and the second image dataset represents the pulmonary fibrosis symptoms occurring in the alveolar organoid due to treatment with TGF-β1, an artificial intelligence model-based classification method for the pulmonary fibrosis status of a stem cell-derived alveolar organoid. Claim 11 In claim 10, the classification method further comprises the step of non-invasively performing real-time dynamic analysis on morphological changes occurring in the alveolar organoid due to treatment with TGF-β1 based on the result of classifying the state of the image to be classified using the classification model; an artificial intelligence model-based classification method for the pulmonary fibrosis state of a stem cell-derived alveolar organoid. Claim 12 In claim 11, the training dataset used for training the classification model comprises a first dataset relating to the original of the first image dataset and the second image dataset, a second dataset in which the background of the alveolar region is removed from the original of the first image dataset and the second image dataset, and a third dataset combining the first dataset and the second dataset; the classification model comprises a first model trained based on the first dataset, a second model trained based on the second dataset, and a third model trained based on the third dataset; and the classification method further comprises the step of non-invasively performing the real-time dynamic analysis by comparing the classification result of the first model, the classification result of the second model, and the classification result of the third model for the image to be classified.

Images