Method for predicting and determining organoid production using artificial neural network model

An artificial neural network-based method predicts and determines organoid tissue module formation, addressing subjective classification issues and enhancing manufacturing efficiency by automating the production process.

KR103046190B1Active Publication Date: 2026-11-10SEOUL NATIONAL UNIVERSITY R&DB FOUNDATION
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
KR1020240036311
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2024-03-15
Publication Date
2026-11-10
Estimated Expiration
2044-03-15

AI Technical Summary

Technical Problem

Current methods for identifying and characterizing organoid tissue modules are subjective and labor-intensive, leading to variations in classification results based on researcher proficiency, and there is a need for an objective and efficient approach to predict organoid formation during fabrication.

Method used

A method using a trained artificial neural network, specifically a convolutional neural network, to predict the formation of cup-shaped organoid tissue modules by analyzing image data during the fabrication process, minimizing classification variations and optimizing manufacturing conditions.

Benefits of technology

The method enables accurate prediction of organoid tissue module formation, reducing defect rates and process losses by automating the determination of successful production, thereby increasing process efficiency and consistency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112024029256475-PAT00002_ABST
    Figure 112024029256475-PAT00002_ABST
Patent Text Reader

Abstract

The present invention relates to a technology for predicting whether a specific type of organoid tissue module can be produced at an early stage of production and determining the success of the organoid tissue module production by identifying completion conditions. More specifically, the invention relates to a method for predicting whether an organoid tissue module can be produced at an early stage of production using image analysis based on artificial intelligence technology and determining whether the production of the completed organoid tissue module is successful. By providing this invention, it is possible to predict from the initial image state whether oxygen and nutrients are sufficiently supplied during the organoid tissue module production process to ensure successful organoid tissue module formation, thereby enabling the production of cup-shaped organoid tissue modules with a low defect rate. Meanwhile, at the final stage of organoid tissue module production, the success of the production can be determined solely by the judgment of artificial intelligence, thereby enabling automation for mass production.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to a method for predicting or determining the formation of a desired organoid tissue module by analyzing image data during the fabrication of an organoid tissue module. Specifically, the present invention relates to a method for predicting early whether an organoid tissue module will be formed during the initial stage of fabrication using a trained artificial neural network model, a method for determining whether the organoid tissue module has been successfully formed after fabrication is completed using a trained artificial neural network model, and a system for implementing said method. Background Technology

[0003] The development of organoid production systems allows for the creation of complex structures of tissues and organs in vitro ( In vitro This has enabled tissue development, disease modeling, drug discovery, and regenerative medicine research by making it possible to implement them (Clevers, H., 2016; Lee, S.-Y., et al., 2023). In particular, organoids derived from adult stem cells, embryonic stem cells, and induced pluripotent stem cells have the potential to be regenerative therapies and aim to restore or treat the intrinsic functions of tissues (Grassi, L., et al., 2019; Kim, J., et al, 2020). To develop such regenerative therapies, research on organoids capable of mimicking human organ tissues is attracting attention as a very promising field.

[0004] Organoids, which are three-dimensional cell aggregates, can be created by utilizing the self-renewal and self-organization capabilities of stem cells (Kaushik et al, 2018). Since organoids can mimic the structural and functional characteristics of biological tissues, models physiologically similar to complex human organs can be constructed in vitro (Fatehullah et al., 2016). Furthermore, they are utilized not only for regenerative therapies but also as miniaturized organ models for patient-specific disease modeling or drug screening systems (Kaushik et al., 2018). As described above, organoid therapies are attracting attention as a promising technology in the field of regenerative medicine, but there are challenges that must be overcome for further development. Organoid fabrication methods are complex and time-consuming, and current methods for identifying and characterizing them are subjective and labor-intensive. Therefore, an objective and efficient approach is required to identify and predict organoids, which are three-dimensional cultures.

[0005] The inventors of the present invention overcame the aforementioned problems by developing a method to manufacture a cup-shaped organoid tissue module in which oxygen and nutrients are sufficiently supplied to the center through the self-assembly and self-structing ability of stem cells without the aid of an artificial scaffold in the invention of application number 10-2022-0130869 (method for manufacturing an organoid using the self-structing ability of stem cells and an organoid manufactured by said method), but discovered a phenomenon in which the results of manufacturing the organoid tissue module differ even when starting with the same culture conditions when manufacturing the organoid tissue module.

[0006] In addition, it was confirmed that verification by a skilled researcher is required to determine whether the manufactured organoid tissue module is in the form of a cup with a concave center, and that variations in classification results may occur depending on the researcher's proficiency.

[0007] Meanwhile, deep learning is a type of machine learning that involves a computer building a high-level conceptual model using large amounts of data based on a multi-layered neural network. It is characterized by enabling the computer to learn and understand concepts just like a human. Unlike existing machine learning algorithms that explicitly extract features through predefined rules, deep learning has the advantage of being able to learn how to extract features according to the task using raw data. Furthermore, as advancements in hardware technology have made it possible to significantly increase the number of hidden layers, the industrial utility of deep learning is increasing dramatically.

[0008] A Convolutional Neural Network (CNN) is a method designed to process two-dimensional data, such as images, more effectively by applying filtering techniques to an artificial neural network. Unlike processing images using fixed filtering techniques, a CNN allows each element of a filter, represented as a matrix, to be automatically learned to be suitable for data processing. CNNs were first introduced by LeCun et al. in 1989 to process images more effectively by applying filtering techniques to artificial neural networks, and subsequently, the form of CNN currently used in deep learning was proposed by LeCun et al. in 1998. The structure of a convolutional neural network generally consists of a convolutional layer, a pooling layer, and a fully-connected layer, where the fully-connected layer derives predicted values ​​of the input data using an artificial neural network.

[0009] The inventors of the present invention completed the invention by developing a method to predict whether a cup-shaped organoid tissue module is formed at the early stages of culture using artificial intelligence, specifically a convolutional neural network, in order to solve the problem that verification by a skilled researcher is required to determine whether a manufactured organoid tissue module is in the form of a cup with a concave center and that variations in classification results occur depending on the researcher's proficiency. They also developed a method to minimize classification variations of the manufactured organoid tissue module. The problem to be solved

[0011] The present invention utilizes artificial intelligence technology to predict the success of producing a specific type of organoid tissue module based on initial conditions, and to determine whether the organoid tissue module has been successfully manufactured after production is complete. Through this, optimal conditions for producing the organoid tissue module can be secured during the manufacturing process, thereby reducing process losses and lowering the defect rate, and furthermore, the optimal process for producing the organoid tissue module can be identified. means of solving the problem

[0013] To achieve the above-described objective, the present invention provides a method for determining organoid tissue module formation using deep learning.

[0014] The above method for determining formation may include: 1) collecting image data at any point in time during the culture process of a microblock for manufacturing an organoid tissue module; 2) processing or augmenting the image data; 3) training an artificial neural network using the processed and augmented data; and 4) determining the formation of the organoid tissue module using the trained artificial neural network model.

[0015] In one embodiment, the image data may be an image at the start of the microblock culture process.

[0016] In one embodiment, the image data may be an image at the end of the microblock culture process.

[0017] In one embodiment, the image data may be images taken at one-hour intervals from the start to the end of the microblock culture process.

[0018] In one embodiment, the processing of the image data may involve adjusting the image to a set size or extracting the portion of the image where the organoid tissue module exists.

[0019] In one embodiment, the enhancement of the image data may be to rotate the image at a random angle.

[0020] In one embodiment, the enhancement of the image data may be to brighten or darken the image.

[0021] In one embodiment, the enhancement of the image data may make the brightness contrast of the image stronger or weaker.

[0022] In one embodiment, the enhancement of the image data may be to change the image to be sharp or smooth.

[0023] In one embodiment, the augmentation of the image data may be a deformation of the image by distorting the plane on which the image is drawn.

[0024] In one embodiment, the plane on which the image is drawn may be a rectangular frame.

[0025] In one embodiment, the culture of the microblock may be carried out for 24 to 240 hours, preferably for 24 to 120 hours, and more preferably for 72 hours.

[0026] In one embodiment, the learned artificial neural network model may be produced by the steps of: 1) obtaining an image at the start of culture and an image at the end of culture of a microblock; 2) determining the image at the end of culture according to a classification criterion; 3) labeling the determination result of the image at the end of culture to the image at the start of culture or the image at the end of culture; and 4) training the artificial neural network with the labeled image.

[0027] In one embodiment, the determination of step 2) may be based on whether one or more characteristics selected from the group consisting of assembly, aggregation, lifting, folding, and densification exist on the image.

[0028] In one embodiment, the determination of step 2) may be based on whether there is a characteristic of being assembled and aggregated as an organoid tissue module on the image.

[0029] In one embodiment, the determination of step 2) may be based on whether there is a characteristic in the image where the outer edge of the organoid tissue module is lifted and the outer edge of the organoid tissue module is folded inward.

[0030] In one embodiment, the determination of step 2) may be based on whether there is a characteristic of the organoid tissue module being densified in the image.

[0031] In one embodiment, the learned artificial neural network model may be learned by a convolutional neural network (CNN).

[0032] In one embodiment, the formed organoid tissue module may be cup-shaped.

[0033] In one embodiment, the learned artificial neural network model may be trained with a virtual image produced by an adversarial generative neural network.

[0034] The present invention provides a system for determining the formation of an organoid tissue module, comprising: a shooting unit for collecting image data during the culture process of a microblock for manufacturing an organoid tissue module; and a judgment unit for determining the shape of the organoid tissue module by applying a deep learning or convolutional neural network (CNN) algorithm using the image data.

[0035] In one embodiment, the imaging unit may include an optical microscope that magnifies an image of a microblock and a camera that captures the magnified image.

[0036] In one embodiment, the judgment unit may include a deep learning unit that learns whether to determine whether to form an organoid tissue module based on the image; and a judgment unit that determines whether to produce an organoid tissue module using an artificial neural network learned by the deep learning unit.

[0037] In one embodiment, the system may additionally include a processing unit that processes or enhances image data.

[0038] In one embodiment, the shooting unit and the judgment unit may be attached or exist separately.

[0039] In one embodiment, the system may be a device composed of one or more parts.

[0040] The above system may be implemented as a hardware component, a software component, or a combination thereof. For example, it may be implemented by a general-purpose computer including a camera, and may be performed through an operating system of the general-purpose computer and one or more software running on said operating system.

[0041] The present invention provides a method for detecting any one of the location, size, and type of a microblock or a combination thereof, comprising: 1) a step of collecting image data at any point in time during the culture process of a microblock for manufacturing an organoid tissue module; 2) a step of processing or augmenting the image data; 3) a step of training an artificial neural network using the data from the second step; and 4) a fourth step of using the trained artificial neural network model.

[0042] In one embodiment, the image data may include an image at the start of the culture process of the microblock.

[0043] In one embodiment, the processing of the image data may involve adjusting the image to a set size or extracting the portions where microblocks exist from the image.

[0044] In one embodiment, the learned artificial neural network model may be produced by a step comprising: 1) obtaining an image at the start of the culture process of a microblock for manufacturing an organoid tissue module; 2) labeling one or more selected from a group consisting of the location, size, and type of the microblock on the image at the start of the culture; and 3) training the artificial neural network with the labeled image.

[0045] In one embodiment, the learned artificial neural network model may be an object detection artificial neural network.

[0046] The present invention provides a system for detecting microblocks for manufacturing an organoid tissue module, comprising: a shooting unit for collecting image data during the culture process of microblocks for manufacturing an organoid tissue module; and a detection unit for detecting any one of the location, size, and type of the microblocks or a combination thereof by applying an object detection artificial neural network algorithm using the image data.

[0047] In one embodiment, the imaging unit may include an optical microscope that magnifies an image of a microblock and a camera that captures the magnified image.

[0048] In one embodiment, the detection unit may include a deep learning unit that learns the location, size, and type of microblocks from the image; and a determination unit that determines the location, size, and type of microblocks using an artificial neural network learned by the deep learning unit.

[0049] In one embodiment, the system may additionally include a processing unit that processes or enhances image data.

[0050] In one embodiment, the imaging unit and the detection unit may be attached or exist separately.

[0051] In one embodiment, the system may be a device composed of one or more parts.

[0052] The above system may be implemented as a hardware component, a software component, or a combination thereof. For example, it may be implemented by a general-purpose computer including a camera, and may be performed through an operating system of the general-purpose computer and one or more software running on said operating system. Effects of the invention

[0054] The present invention provides a prediction method for predicting whether the fabrication process will succeed or fail during the initial stage of organoid tissue module fabrication. Furthermore, the present invention provides a method for determining whether the organoid tissue module formation will succeed during the final stage of organoid tissue module fabrication, and provides a system for performing the prediction and determination methods.

[0055] By providing the present invention, it is possible to predict from the initial image whether a desired shape of organoid tissue module can be produced by controlling conditions such as cell count and size during the organoid tissue module manufacturing process. As a result, process efficiency can be increased, and by minimizing the classification deviation of the manufactured organoid tissue module, it is possible to produce a cup-shaped organoid tissue module with a low defect rate.

[0056] Meanwhile, in the final stage of organoid tissue module production, the success of production can be determined solely by the judgment of artificial intelligence, thereby enabling automation for mass production. Brief explanation of the drawing

[0058] Figure 1 is a diagram illustrating the overall method for determining / evaluating the fabrication of organoid tissue modules using deep learning. Figure 2 is a diagram explaining the method for fabricating organoid tissue modules to be used in deep learning. Figure 3 is a diagram illustrating the criteria for true labeling of collected organoid tissue modules. Figure 4 is a diagram showing the true labeling results for actual data using established success / failure classification criteria for organoid tissue modules. Figure 5 is a graph showing the true labeling results for actual data using established success / failure classification criteria for organoid tissue modules. Figure 6 is a diagram illustrating an algorithm for analyzing and learning organoid tissue modules based on a CNN (Convolutional Neural Network). Figure 7 is a diagram illustrating a learning plan for developing an organoid tissue module determination algorithm. Figure 8 is a diagram illustrating the actual construction method for the organoid tissue module determination algorithm. Figure 9 is a graph showing the discrimination precision and recall values ​​for each organoid tissue module judgment algorithm model. Figure 10 is a graph showing the determination precision and recall values ​​for success / failure results for each organoid tissue module determination algorithm model. Figure 11 is a graph showing the prediction rate and judgment rate of each organoid tissue module judgment algorithm model as classification performance indicators. Figure 12 is a photograph showing the feature parts of an image for each algorithm model using Grad-CAM, VisualBackProp, and LRP visualization algorithms. Figure 13 is a diagram illustrating an algorithm for analyzing and learning organoid tissue modules based on a Generative Adversarial Network (GAN). Figure 14 is a diagram illustrating the actual construction method for the organoid tissue module production prediction algorithm. Figure 15 is a diagram illustrating the method and results of generating organoid tissue module images using a GAN (Generative Adversarial Network). Figure 16 is a diagram illustrating an algorithm for analyzing and learning organoid tissue modules based on an object detection algorithm. Figure 17 is a diagram illustrating the results of counting microblocks within an organoid tissue module using an object detection algorithm. Figure 18 is a graph illustrating the results of the number of microblocks and the content ratio within an organoid tissue module using an object detection algorithm. Specific details for implementing the invention

[0059] Hereinafter, embodiments and examples of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. However, the present invention may be embodied in various forms and is not limited to the embodiments and examples described herein.

[0060] The term "microblock (MiB)" used in the present invention refers to unit spheroids of various sizes for producing organoid tissue modules.

[0061] The term "organoid" used in the present invention refers to a tissue mimic that is produced by three-dimensionally culturing stem cells and takes on a form similar to actual human tissues and organs through self-structuring.

[0062] The terms "organoid tissue module" or "organoid-tissue module" as used in this invention refer to an organoid that has not differentiated into a specific organ tissue, produced through the self-structuring of stem cells following the self-assembly of microblocks.

[0063] The term "cup shape" as used in this invention refers to a shape having a concave groove in the middle.

[0064] The term "Convolutional Neural Network (CNN)" used in this invention refers to an artificial neural network designed to effectively process image data, etc., by applying filtering techniques to the artificial neural network.

[0065] The term "Generative adversarial network (GAN)" used in the present invention refers to an artificial neural network that performs competitive learning through two competing artificial neural networks.

[0066] The term "object detection artificial neural network" used in the present invention refers to an artificial neural network capable of detecting major objects and learning to distinguish objects by displaying a bounding box or bounding polygon centered on the detected objects.

[0067] In the present invention, "stem cells" include adult stem cells, preferably "adult mesenchymal stem cells," and more preferably "human adipose-derived mesenchymal stem cells (hADMSC)."

[0068] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0069] The present invention is to be explained in more detail through the following examples, but the following examples are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0071] [Example 1]

[0072] Production of organoid tissue modules

[0074] Mesenchymal stem cells were extracted from adipose tissue to construct organoid tissue modules. As shown in Figure 2, mesenchymal stem cells were subcultured starting from a confluency of 70–90%, at a density of 2,500 cells / cm². 2 Mesenchymal stem cells were dispensed into culture vessels. Mesenchymal stem cells were cultured in culture vessels for 3 to 5 days in Dulbecco’s modified Eagles medium (DMEM) containing 10% FBS.

[0075] Cultured mesenchymal stem cells were recovered and placed in Aggrewell 400 / 800 at 0.6 × 10⁶ 6 and 0.9 × 10 6 Dispense cells / well and L-proline, Sodium pyruvate, L-Ascorbic acid 2-phosphate, insulin transferrin selenium, and TGF- Two types of microblocks were prepared as shown in Figure 2 by culturing for 72 hours in DMEM medium containing triglycerides. The two types of microblocks were mixed in a 12:1 ratio in an ultra-low attachment 96-well plate, resulting in a total cell count of 0.9 × 10⁶ 6 Organoid tissue modules were fabricated to form cells / well. Organoid tissue modules were fabricated by culturing for 72 hours in the medium used to fabricate the microblocks and inducing self-assembly.

[0077] [Example 2]

[0078] Collection of data for deep learning training

[0080] In the above organoid tissue module culture process, training data was constructed by acquiring images using high-resolution imaging equipment such as JuLi stage and CQ1 at 1-hour intervals from the start to the end of culture for 72 hours starting from the time the microblocks were mixed. Alternatively, training data was constructed by independently acquiring images at the start and end times of culture for each organoid tissue module and used for specific deep learning training.

[0081] To acquire high-resolution images, imaging equipment such as inverted or upright optical microscopes, fluorescence microscopes, and confocal fluorescence microscopes can be used. A single organoid tissue module was acquired as multiple separate images, which were then combined or stitched into a single image using image analysis software such as ImageJ and collected as training data.

[0083] [Example 3]

[0084] Classification criteria for the success or failure of organoid tissue module formation and data labeling

[0086] The success and failure of organoid tissue module fabrication were determined through image analysis from the time the microblocks were mixed until the completion of culture 72 hours later. Specifically, organoid tissue modules that met all the criteria presented in Figure 3 were classified as successful.

[0087] Success and failure of organoid tissue module fabrication were determined by analyzing images in three categories: assembly, condensation, lifting, folding, and tightening. Success was determined when 1) microblocks were evenly aggregated around the wells to form a single tissue body, 2) the outer edge of the organoid tissue module was lifted in a balanced manner and the edges were folded inward to form a concave space, and 3) the microblocks constituting the organoid tissue module were densely self-assembled so that there were no gaps inside the organoid tissue module, the surface was smooth, and it was tightened without any traces of cell death.

[0088] Labeling of organoid tissue module images was performed to use the images of organoid tissue modules as training data for an artificial neural network model. As shown in Fig. 3, two experimenters classified each organoid tissue module image into success and failure groups, determining success if all three criteria were met and failure if even one of the three criteria was not met. Classification was completed for a total of 911 organoid tissue module images, and the results of the classification are shown in Figs. 4 and 5. At this time, organoid tissue module images classified as success or failure by both experts were labeled as O or X, and if the classifications of the two experts did not match, true labeling was performed as △. The results of the true labeling for the 911 organoid tissue module images are shown in Figs. 4 and 5.

[0090] [Example 4]

[0091] Data Processing and Augmentation

[0093] The data obtained above was processed as follows to convert it into an image suitable for training an artificial neural network model. To prevent interference from the background image during organoid tissue module image recognition, the well portion of the plate in the image was separated from the rest of the background using the Hough Circle Transform provided by the OpenCV library in Python.

[0094] The size of the extracted organoid tissue module well images was adjusted to a preset size. Subsequently, image thresholding was applied to distinguish between areas within the wells where actual organoid tissue modules were present and those where they were not. For model training requiring data augmentation, the data was augmented by rotating the images by an arbitrary angle. Additionally, further image processing such as adjusting brightness, contrast, resizing, cropping, and distorting the image plane (rectangular frame) can be considered for the organoid tissue module well images.

[0096] [Example 5]

[0097] Designing an Artificial Neural Network Model Training Plan

[0099] When classifying the success and failure images of the above organoid tissue module, actual value labels that did not match due to the experimenter's subjective judgment accounted for approximately 16%. Since the proportion of △s where the opinions of two experts differed was high, it is necessary to decide whether to classify the above images as success or failure and to train them.

[0100] Learning was conducted using four learning methods, assuming that the judgment ability of two experts is the same, classifying O, X, and △ into separate classes and learning classification (Learning Plan 1); classifying only O and X classes excluding △ and learning classification (Learning Plan 2); classifying △ as success and classifying into two classes and learning classification (Learning Plan 3); and classifying △ as failure and classifying into two classes and learning classification (Learning Plan 4).

[0101] In addition, although the success and failure groups were classified based on the opinions of two experts in this embodiment, three or more experts may participate to classify them into success and failure groups. In this case, if opinions regarding success and failure do not coincide, they may be excluded from learning. However, if the training data is insufficient, artificial intelligence learning may proceed by classifying into one of the success or failure groups according to the majority rule principle. Additionally, if the number of expert opinions is equal for success and failure, the process may proceed by classifying it as either success or failure.

[0102] In the present invention, if the ratio of the amount of training data classified into success or failure groups is not similar, there may be differences in performance or lead to overfitting, and the training performance may be poor. To overcome this, the number of success and failure training data is configured to be similar to improve the training performance.

[0104] [Example 6]

[0105] Setting up an artificial neural network model through pre-training

[0107] Training was conducted based on artificial intelligence training frameworks such as PyTorch. CNN models including ResNet, RexNet, MobileNet, and EfficientNet were selected, and pre-training was performed using datasets released by ImageNet. Appropriate training frameworks or CNN models can be selected as needed, and if higher-performance CNN models or more convenient frameworks become available later, they can be utilized. At this stage, the number of categories into which the model would classify images was determined, and the final classifier section of the acquired model was modified accordingly. Since the model will ultimately classify three classes—○, X, and △—the number of final classifier output stages was adjusted to fit this.

[0108] For pre-training, two image datasets were used from images taken during the 72 hours from the time the microblocks were mixed, specifically at the start and end of the culture. The images at the start of the culture were used as training data to predict whether an organoid tissue module would be produced through successful self-assembly of the microblocks, and the images at the end of the production were used as training data to determine whether an organoid tissue module of the desired shape had been formed.

[0109] Datasets were randomly selected from 911 datasets and divided into training, validation, and test datasets. The training dataset is the data used for the model's actual learning. The validation dataset was used to evaluate the model's performance at the end of each training epoch. The test dataset was used to assess whether the model could generalize after training was complete. When loading the training dataset during training, the data was augmented by rotating the images by a random angle.

[0110] When training the model, a training framework such as PyTorch was utilized, and early stopping was introduced so that training was halted if the validation loss calculated at the end of each training cycle did not improve for a preset number of iterations. If not halted, training continued for a preset maximum number of iterations. The artificial neural network model was evaluated at the end of each training session. Model performance was assessed by calculating precision, recall, accuracy, and the F1 score for each classification. Precision is the ratio of actual occurrences to those classified by the model as a specific category. Recall is the ratio of actual occurrences classified by the model as a specific category. Accuracy is the ratio of correctly classified occurrences to the total. The F1 score is the harmonic mean of precision and accuracy.

[0111] In addition, Grad-CAM, VisualBackProp, and LRP visualization algorithms were used to verify which features of the image the artificial neural network model used for classification.

[0112] As shown in FIG. 8, an established artificial neural network model system for predicting and determining organoid tissue module formation was databased, comprising a first artificial neural network model that predicts whether an organoid tissue module will be formed by analyzing an initial production image and a second artificial neural network model system that determines whether the organoid tissue module was successfully produced by analyzing a final production image. By accessing the database of the present invention and loading a determination algorithm, the success and failure of the newly produced organoid tissue module were determined and predicted, and the results were organized into a dataset and utilized to enhance the artificial neural network model.

[0114] [Example 7]

[0115] Determination result of the organoid tissue module in the artificial neural network model

[0117] After randomly shuffling the samples, the precision and recall of the classification performance indicators derived by repeating the artificial neural network dataset 10 times were checked, and the results are shown in Fig. 9. In the 10 evaluations, the classification performance indicators for O and X were checked respectively, and the results are shown in Fig. 10. It was confirmed that no overfitting occurred by the fact that the classification performance indicator values ​​for O and X were not different. In the 10 evaluations, the prediction rate was compared with various classification performance indicators, and the results are shown in Fig. 11.

[0118] Each artificial neural network model identified the features of the image using Grad-CAM, VisualBackProp, and LRP visualization algorithms, and the results are shown in Fig. 12. As shown in Fig. 12, the artificial neural network model used Grad-CAM, VisualBackProp, and LRP visualization algorithms to determine which part of the image the model used as a reference for judgment.

[0120] [Example 8]

[0121] Method for applying Type 1 artificial neural network model algorithms

[0123] The Type 1 artificial neural network model performed a prediction regarding the start time of organoid tissue module fabrication. First, during organoid tissue module fabrication, an image corresponding to the time immediately after mixing the microblocks (hour 0) was secured using high-resolution imaging equipment. Subsequently, the initial image of organoid tissue module fabrication corresponding to hour 0 was analyzed using the trained Type 1 artificial neural network model obtained in Example 6. At this time, the characteristics of the organoid tissue module were identified through image filtering using the pre-trained artificial neural network model. Finally, the prediction value of the Type 1 artificial neural network model was verified to predict whether the organoid tissue module fabrication would succeed or fail. Based on the prediction value of the Type 1 artificial neural network model, if success was predicted, culture was continued for 72 hours to complete the organoid tissue module. If failure was predicted, the microblocks were recovered and resuspended to re-examine the screening process or were discarded.

[0125] [Example 9]

[0126] Method for applying Type 2 artificial neural network model algorithms

[0128] The second type of artificial neural network model performed a determination regarding the completion time of the organoid tissue module production. First, during the production of the organoid tissue module, an image corresponding to the completion time was obtained using high-resolution image capture equipment. Subsequently, the organoid tissue module image corresponding to the completion time was analyzed using the second artificial neural network model obtained in Example 6. At this time, the characteristics of the organoid tissue module were identified through image filtering using the pre-trained artificial neural network model. Finally, the judgment value of the second type of artificial neural network model was checked to determine whether the production of the organoid tissue module was successful or failed. Based on the judgment value of the second artificial neural network, if selected as successful, the organoid tissue module production was determined to be complete. If selected as a failure, it was discarded.

[0130] [Example 10]

[0131] Algorithm learning method for the advancement of organoid tissue module diagnosis models

[0133] High-resolution images of true-labeled organoid tissue modules were collected and preprocessed as in Example 6 above. Subsequently, images were created in which images from the beginning and end of each organoid tissue module production were juxtaposed for training an artificial neural network model.

[0134] An adversarial generative neural network consists of a generator that creates data and a discriminator that distinguishes whether the given data is actual data or created data. The adversarial generative neural network used in the present invention is a conditional adversarial generative neural network, in which the generator receives condition information and a random noise vector as input to generate data. As shown in FIGS. 13 and 14, the first adversarial generative neural network is trained to generate a final time point image when an initial time point image of a case where the organoid tissue module production is successful is input as condition information, and the second adversarial generative neural network is trained to generate a final time point image when an initial time point image of a case where production fails is input as condition information.

[0135] As shown in Fig. 15, trained adversarial generative neural networks are stored in a database so that, as needed, the first adversarial generative neural network or the second adversarial generative neural network can be retrieved to infer the expected end point image when the initial image of the organoid tissue module is input.

[0137] [Example 11]

[0138] Method for applying a microblock object detection model for evaluating organoid tissue module judgment models

[0140] As shown in Figures 16 and 17, the results of predicting production success and failure using a first type of artificial neural network were analyzed using a microblock object detection model. By classifying production success / failure images at the initial stage of production, bbox label data indicating the location and type of microblocks constituting the organoid tissue module was collected. The model was trained to detect microblocks constituting the organoid tissue module, distinguish between two types of microblocks, and quantify their counts.

[0141] Using a trained microblock object detection model, information on the number of microblocks according to production success and failure was compared, and the results are shown in Fig. 18. When applying the first type of artificial neural network, the microblock object detection model was utilized as needed to infer the expected number of microblocks.

Claims

Claim 1 A method for determining the formation of an organoid tissue module, performed by a system for determining the formation of an organoid tissue module, comprising: a first step in which the system collects image data at any point in time during the culture process of a microblock for manufacturing an organoid tissue module; a second step in which the system processes or augments the image data; a third step in which the system trains an artificial neural network using the data from the second step; and a fourth step in which the system determines the formation of an organoid tissue module using the trained artificial neural network model, wherein the third step includes obtaining an image at the start of culture and an image at the end of culture of the microblock, determining the image at the end of culture according to classification criteria, and training the artificial neural network model using an image labeled with the determination result of the image at the start of culture, and wherein in the fourth step, the trained artificial neural network model determines whether an organoid tissue module is formed after the end of culture based on the image at the start of culture. Claim 2 delete Claim 3 delete Claim 4 A method for determining organoid tissue module formation according to claim 1, characterized in that the image data further includes images taken at one-hour intervals from the start to the end of the microblock culture process. Claim 5 A method for determining the formation of an organoid tissue module according to claim 1, wherein the processing of the image data includes adjusting the image to a set size or extracting a portion of the image in which an organoid tissue module exists. Claim 6 A method for determining organoid tissue module formation according to claim 1, wherein the enhancement of the image data includes rotating the image at a random angle. Claim 7 A method for determining organoid tissue module formation according to claim 1, wherein the enhancement of the image data includes brightening or darkening the image. Claim 8 A method for determining the formation of an organoid tissue module according to claim 1, wherein the enhancement of the image data includes making the brightness and contrast of the image stronger or weaker. Claim 9 A method for determining organoid tissue module formation according to claim 1, wherein the enhancement of the image data includes changing the image to be sharp or smooth. Claim 10 A method for determining the formation of an organoid tissue module according to claim 1, wherein the augmentation of the image data includes distorting the plane on which the image is drawn to deform the image. Claim 11 A method for determining organoid tissue module formation according to claim 1, characterized in that the culture of the microblock is carried out for 24 to 120 hours. Claim 12 A method for determining organoid tissue module formation according to claim 11, characterized in that the culture of the microblock is carried out for 72 hours. Claim 13 delete Claim 14 A method for determining organoid tissue module formation according to claim 1, wherein in the third step, the determination of the image at the end of the culture is based on whether one or more characteristics selected from the group consisting of assembly, aggregation, lifting, folding, and densification exist on the image. Claim 15 A method for determining organoid tissue module formation according to claim 1, wherein the determination of the image at the end of the culture in the third step is based on whether there is a characteristic of being assembled and aggregated as an organoid tissue module on the image. Claim 16 A method for determining the formation of an organoid tissue module according to claim 1, wherein in the third step, the determination of the image at the end of the culture is based on whether there is a characteristic in the image where the outer edge of the organoid tissue module is lifted and the outer edge of the organoid tissue module is folded inward. Claim 17 A method for determining organoid tissue module formation according to claim 1, wherein the determination of the image at the end of the culture in the third step is based on whether there is a characteristic of the organoid tissue module being densified on the image. Claim 18 A method for determining organoid tissue module formation according to claim 1, characterized in that the learned artificial neural network model is learned by a convolutional neural network (CNN). Claim 19 A method for determining the formation of an organoid tissue module according to claim 1, characterized in that the formed organoid tissue module is cup-shaped. Claim 20 A method for determining organoid tissue module formation according to claim 1, characterized in that the learned artificial neural network model is trained with a virtual image produced by an adversarial generative neural network. Claim 21 A system for determining the formation of an organoid tissue module, comprising: a shooting unit for collecting image data during the culture process of a microblock for manufacturing an organoid tissue module; and a judgment unit for determining the formation of an organoid tissue module by applying a deep learning or convolutional neural network (CNN) algorithm using the image data, wherein the judgment unit is equipped with an artificial neural network model learned by labeling the judgment result of the image at the end of culture to the image at the start of culture, and configured to determine whether an organoid tissue module is formed after the end of culture from the image at the start of culture using the artificial neural network model. Claim 22 A system for determining organoid tissue module formation, characterized in that, in claim 21, it further includes a processing unit for processing or augmenting image data. Claim 23 delete Claim 24 delete Claim 25 delete Claim 26 delete Claim 27 delete Claim 28 delete Claim 29 delete

Citation Information

Patent Citations

  • Organ-like activity identification method and system based on artificial intelligence

    CN116012838A

  • Organ-like activity evaluation system based on CBAM-YOLOv3 integration algorithm and application

    CN116778480A

  • Method and system for capturing fluorescence images of living cell biological samples

    KR1020230040306A

  • Apparatus and method for cell selection

    KR1020230126264A