A method and system for intelligent identification of organoid types based on microscopic images

Through the intelligent recognition method of alveolar organoid type based on microscopic images, image recognition is used using VGG, ResNet and GoogleNet models to solve the problem of alveolar organoid classification in the existing technology, realize automatic recognition and high-throughput operations, and provide Unified discrimination criteria and detailed typing ratio data are suitable for library construction and research.

CN114463290BActive Publication Date: 2024-11-01ACCURATE INT BIOTECHNOLOGY (GUANGZHOU) CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210066327.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-20
Publication Date
2024-11-01
Estimated Expiration
2042-01-20

AI Technical Summary

Technical Problem

The existing technology lacks an intelligent identification system for alveolar organoid classification. Types and classification mainly rely on artificial judgments, lack of recognized standards and parameters that can be introduced into AI recognition, making it difficult to effectively distinguish and count the types and maturity stages of lung organoids.

Method used

Using the intelligent recognition method of alveolar organoid type based on microscopic images, by acquiring microscopic images, using VGG, ResNet and GoogleNet models for image recognition, combined with minimal loss optimization and F1 value evaluation, the optimal model is selected for alveolar organoid type identification and judgment.

Benefits of technology

It realizes the automated identification of alveolar organoid types, provides unified discrimination standards, is fast and intuitive, and can export detailed classification proportional data, is suitable for library construction and quality control, supports high-throughput operations, and is suitable for research on cytokine-induced differentiation efficiency .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114463290B_ABST
    Figure CN114463290B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for intelligent identification of organoid types based on microscopic images. The method comprises: obtaining alveolar organoid microscopic images; selecting each alveolar organoid and extracting it separately, and using xi to represent one of the alveolar organoid images; inputting each alveolar organoid image into three types of intelligent judgment and recognition models to obtain three prediction results; after all alveolar organoid images are processed, the precision and recall results of the three types of intelligent judgment and recognition models are statistically analyzed, and the three models are evaluated using these results, and the model with the best evaluation result is selected as the final sorting model; and the alveolar organoid type is identified and judged using it. The present invention uses alveolar organoid images as data input, and can directly derive detailed typing ratio data. It has a unified discrimination standard and is fast and intuitive, and can perform high-throughput operations. The typing ratio data can be used as a library construction and quality control standard, and can also be used as a research parameter.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of biomedicine technology, and specifically relates to a method and system for intelligently identifying organoid types based on microscopic images. Background Art

[0002] Lung tissue development is a complex process of strict coordination of cell growth and differentiation. Mouse lung tissue occurs on embryonic day 9.5 (E9.5); subsequently, the development of lung tissue usually goes through pseudoglandular stage, tubular stage, cystic stage, primitive alveolar stage and alveolar stage; epithelial tissue continues to grow and repeatedly branch to form trachea, bronchi and alveoli at all levels, among which alveolarization is a process coordinated by multiple cells and biological molecules, covering the development from primitive alveolar stage to alveolar stage, with thinning of alveolar wall and occurrence and growth of secondary septa as the main morphological characteristics. Lung development is a process from proximal to distal, starting from trachea and developing to alveoli, forming a delicate alveolar system at the distal end of the lung, containing alveolar type II (AT2) cells that produce surfactant and alveolar type I (AT1) cells with gas exchange function, which perform the main functions of lung tissue.

[0003] At present, there is no intelligent recognition system for alveolar organoid typing. The morphology and typing are all manually judged, lacking recognized standards and parameters that can be imported into AI recognition. For example, three morphologically different colony types are produced based on lung epithelial organoids. One is a large and round cavity with thin lumen and wall, called type A; one is more irregular in shape, with loose or cavity visible inside, called type B; the other is a solid round or oval shape with no cavity or a smaller cavity, called type C. The three types correspond to different stages of maturity of lung organoids, and the immune markers are also different; distinguishing the types of lung organoids and counting the proportions can effectively understand the growth status and maturity stage of organoids, which is helpful for library construction and quality control. Therefore, it is necessary to develop an intelligent identification method and system for organoid types based on microscopic images to solve the above problems. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention aims to provide a method and system for intelligently identifying organoid types based on microscopic images, which are specifically targeted at alveolar organoids. The technical solution of the present invention is:

[0005] In a first aspect, the present invention provides a method for intelligently identifying alveolar organoid types based on microscopic images, comprising:

[0006] Step 1, obtaining a series of microscopic images of alveolar organoids and performing image synthesis;

[0007] Step 2, select each alveolar organoid in the synthesized microscopic image, and extract each selected alveolar organoid image one by one, and use xi to represent one of the alveolar organoid images, where i is an integer;

[0008] Step 3: Input each alveolar organoid image into three types of intelligent judgment and recognition models: VGG, ResNet, and GoogleNet, and obtain three prediction results of the alveolar organoid type in the image. If there are manual annotation results of the alveolar organoid in the image, the three prediction results are compared with the manual annotation results, and the three types of intelligent judgment and recognition models are optimized by minimizing the loss; otherwise, the three prediction results are compared. If at least two prediction results are consistent, the consistent results are used as the pseudo-label of the alveolar organoid type of the image, and then the model corresponding to the remaining prediction result is optimized by minimizing the loss;

[0009] Step 4: After all alveolar organoid images are processed, the precision and recall results of the three types of intelligent judgment and recognition models are statistically analyzed, and the three models are evaluated using the model evaluation index F1 value using these results. The model with the best evaluation result is selected as the final sorting model.

[0010] Step 5: Use the sorting model to identify and determine the type of alveolar organoids.

[0011] Furthermore, the formula used for the minimization loss optimization is:

[0012]

[0013] In Formula I, represents the prediction result of the organoid in the xith image, j represents the model number: VGG model, j = 1; ResNet model, j = 2; GoogleNet model, j = 3; Indicates the manual annotation results.

[0014] Furthermore, the calculation formula of the F1 value is:

[0015]

[0016] In Formula II, Precision represents the precision rate, Recall represents the recall rate, P / N represents the positive / negative judgment result of the model, T / F represents the true / false judgment result of the model; FP represents false positive examples, FN represents false negative examples, TP represents true positive examples, TN represents true negative examples, and:

[0017] a) precision = TP / (TP+FP) The proportion of positive examples (TP) correctly identified by the model to all positive examples in the data set;

[0018] b) recall = TP / (TP+FN): the proportion of positive examples (TP) correctly identified by the model to all actual positive examples in the data set;

[0019] c) accuracy = (TP+TN) / (TP+FP+TN+FN): the proportion of data (TP+TN) correctly judged by the model to the total data.

[0020] In a second aspect, the present invention provides an intelligent identification system for alveolar organoid types based on microscopic images, comprising:

[0021] A microscopic image acquisition device, used for acquiring a microscopic image of the alveolar organoid;

[0022] an organoid type determination and identification device connected to the microscopic image acquisition device, the organoid type determination and identification device comprising an organoid detector and an organoid classifier, the organoid detector being used to extract each alveolar organoid in the microscopic image; the alveolar organoid classifier being used to determine and identify the type of the extracted alveolar organoid;

[0023] A communication device is used to connect the microscopic image acquisition device, the organoid detector, and the organoid classifier in communication.

[0024] The third aspect is the application of the above-mentioned microscopic image-based intelligent identification method and system of alveolar organoid types in alveolar organoid culture or alveolar organoid library construction.

[0025] In a fourth aspect, the present invention provides a method for culturing alveolar organoids, comprising the following steps:

[0026] Obtaining tissue to be cultured;

[0027] Cultivation of alveolar organoids;

[0028] During the culture process, the above-mentioned intelligent judgment and identification method and system of alveolar organoid type are used to judge and identify the organoid type.

[0029] In a fifth aspect, the present invention provides a method for building a library of alveolar organoids, comprising the following steps:

[0030] The alveolar organoids obtained by culture are judged and identified by using the above-mentioned intelligent judgment and identification method and system for the type of alveolar organoids;

[0031] Organoids that meet the requirements are assembled into a sample library.

[0032] The beneficial effects of the present invention are as follows: the present invention uses alveolar organoid images as data input, directly derives detailed typing ratio data, has a unified judgment standard and is fast and intuitive, can perform high-throughput operations, and the typing ratio data has multiple uses: 1. It can be used as a library construction and quality control standard, and only organoid group samples that reach a certain ratio can be built into the library or judged as qualified products; 2. It can be used as a research parameter, such as the study of cytokine-induced differentiation efficiency, which requires a comparison of the changes in the ratio before and after differentiation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is one of the alveolar organoid image data obtained in Example 1 of the present invention.

[0034] Figure 2 This is another image data of alveolar organoid obtained in Example 1 of the present invention.

[0035] Figure 3 This is one of the alveolar organoid image data that is automatically calibrated and classified in Example 1 of the present invention.

[0036] Figure 4 This is another image data of alveolar organoids that is automatically calibrated and classified in Example 1 of the present invention.

[0037] Figure 5 This is one of the alveolar organoid image data obtained in Example 2 of the present invention.

[0038] Figure 6 This is another image data of alveolar organoid obtained in Example 2 of the present invention.

[0039] Figure 7 This is one of the alveolar organoid image data that is automatically calibrated and classified in Example 2 of the present invention.

[0040] Figure 8 This is another image data of alveolar organoid that is automatically calibrated and classified in Example 2 of the present invention.

[0041] Fig. 9 This is one of the alveolar organoid image data obtained in Example 3 of the present invention.

[0042] Fig.10 This is another image data of alveolar organoid obtained in Example 3 of the present invention.

[0043] Fig.11 This is one of the alveolar organoid image data that is automatically calibrated and classified in Example 3 of the present invention.

[0044] Fig.12 This is another image data of alveolar organoids that is automatically calibrated and classified in Example 3 of the present invention. DETAILED DESCRIPTION

[0045] In the description of the present invention, it should be noted that, if the specific conditions are not specified in the examples, the experiments were carried out according to conventional conditions or the conditions recommended by the manufacturer. The reagents or instruments used, if the manufacturer is not specified, are all conventional products that can be purchased commercially.

[0046] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments, which are intended to explain the present invention rather than to limit it.

[0047] A specific embodiment of the present invention provides a method for intelligently identifying alveolar organoid types based on microscopic images, comprising:

[0048] Step 1: Acquire a series of microscopic images of alveolar organoids and perform image synthesis, that is, stitch single images into a whole image.

[0049] Step 2, select each alveolar organoid in the synthesized microscopic image, and extract each selected alveolar organoid image one by one, and use xi to represent one of the alveolar organoid images, where i is an integer.

[0050] Step 3, input each alveolar organoid image into three types of intelligent judgment and recognition models: VGG, ResNet and GoogleNet, and obtain three prediction results of the alveolar organoid type in the image. If there are manual annotation results of the alveolar organoid in the image, the three prediction results obtained are compared with the manual annotation results, and the three types of intelligent judgment and recognition models are optimized by minimizing loss; otherwise, the three prediction results are compared. If at least two prediction results are consistent, the consistent results are used as the pseudo-label of the alveolar organoid type of the image, and then the model corresponding to the remaining prediction result is optimized by minimizing loss; the formula used for minimizing loss optimization is:

[0051]

[0052] In Formula I, represents the prediction result of alveolar organoids in the xith image, j represents the model number: VGG model, j = 1; ResNet model, j = 2; GoogleNet model, j = 3; Indicates the manual annotation results.

[0053] Step 4: After all alveolar organoid images are processed, the precision and recall results of the three types of intelligent judgment and recognition models are counted, and these results are used to evaluate the F1 value of the three models, and the model with the best evaluation result is selected as the final sorting model; the calculation formula of the F1 value is:

[0054]

[0055] In Formula II, Precision represents the precision rate, Recall represents the recall rate, P / N represents the positive / negative judgment result of the model, T / F represents the true / false judgment result of the model; FP represents false positive examples, FN represents false negative examples, TP represents true positive examples, TN represents true negative examples, and:

[0056] a) precision = TP / (TP+FP) The proportion of positive examples (TP) correctly identified by the model to all positive examples in the data set;

[0057] b) recall = TP / (TP+FN): the proportion of positive examples (TP) correctly identified by the model to all actual positive examples in the data set;

[0058] c) accuracy = (TP+TN) / (TP+FP+TN+FN): the proportion of data (TP+TN) correctly judged by the model to the total data.

[0059] The F1 values ​​of the three models finally obtained are shown in Table 1.

[0060] Table 1

[0061] Model Name VGG ResNet GoogleNet F1 value 0.82 0.76 0.91

[0062] Step 5, finally selecting Googlenet as the sorting model of the present invention, and using the sorting model to identify and judge the type of alveolar organoids.

[0063] A specific embodiment of the present invention further provides an intelligent identification system for alveolar organoid types based on microscopic images, comprising:

[0064] The microscopic image acquisition device is used to acquire the microscopic image of the organoid. In the specific embodiment of the present invention, an optical microscope, model CYTATION5, is used. In addition, other types of optical microscopes and electron microscopes can also be used as the microscopic image acquisition device of the present invention. The images are spliced ​​using image synthesis adjustment software.

[0065] The organoid type determination and identification device is connected to the microscopic image acquisition device, and the organoid type determination and identification device includes an organoid detector and an organoid classifier, wherein the organoid detector is used to extract each alveolar organoid in the microscopic image; and the organoid classifier is used to determine and identify the type of the extracted alveolar organoid. The microscopic image acquisition device used in the specific embodiment of the present invention is Biotek Cytation5, which can support all conventional microplate detectors, and the image synthesis adjustment software model is GEN5 3.09.

[0066] The communication device is used to connect the microscopic image acquisition device, the organoid detector, and the organoid classifier to each other in a communication manner. In this embodiment, a wired connection is specifically adopted.

[0067] The above-mentioned alveolar organoid type intelligent judgment and identification method and system can be applied to alveolar organoid culture or alveolar organoid library construction. And the alveolar organoid culture method involved includes the following steps:

[0068] Obtaining tissue to be cultured;

[0069] Cultivating alveolar organoids; the method for culturing alveolar organoids can refer to CN202010843695.5;

[0070] During the culture process, the above-mentioned microscopic image-based intelligent identification method and system for alveolar organoid types are used to judge and identify the alveolar organoid types.

[0071] The method for building a library of alveolar organoids comprises the following steps:

[0072] The alveolar organoids obtained by culture are judged and identified by using the above-mentioned intelligent identification method and system for alveolar organoid types based on microscopic images;

[0073] Organoids that meet the requirements are assembled into a sample library.

[0074] Example 1

[0075] This embodiment provides a method for intelligently determining and identifying the type of human alveolar organoids, comprising the following steps:

[0076] (1) Obtain imaging data of human alveolar organoids. The specific method is: use an optical microscope to scan and photograph the well plate (or culture dish) of human alveolar organoids to be counted. The microscopic image needs to contain three-dimensional information (i.e., layer superposition), such as Figure 1 and Figure 2 The data for two of the images.

[0077] (2) Each organoid in the series of microscopic images of alveolar organoids is framed and selected, and each framed alveolar organoid image is extracted one by one, ignoring unnecessary organoids (those with a diameter less than 30 μm, loose outer walls, collapsed shapes, poor light transmittance, and internal necrosis); Figure 3 and Figure 4 shown.

[0078] (3) The data is loaded into the selected sorting model (organoid type determination and identification device) GoogleNet to identify and determine the organoid type.

[0079] (4) Statistic all judgment results and export the EXCEL data table, as shown in Table 1. Figure 3 and Figure 4 In the figure, dark gray represents type A, light gray represents type B, and white represents type C.

[0080] Table 1 Automatic typing results of human alveolar organoids

[0081] Classification A B C Figure 1 30 5 8 Figure 2 31 2 3

[0082] The results in Table 1 show that Figure 1 There are 30 type A human alveolar organoids, 5 type B human alveolar organoids, and 8 type C human alveolar organoids; Figure 2 There are 31 type A human alveolar organoids, 2 type B human alveolar organoids, and 3 type C human alveolar organoids in the dataset. The results are the same as those obtained by manual annotation.

[0083] Example 2

[0084] This embodiment provides a method for intelligently judging and identifying the type of mouse expanded alveolar organoids, comprising the following steps:

[0085] (1) Obtain imaging data of mouse expanded alveolar organoids. The specific method is: use an optical microscope to scan and photograph the mouse expanded alveolar organoid well plate (or culture dish) to be counted. The microscopic image needs to contain three-dimensional information (i.e., layer superposition), such as Figure 5 and Figure 6 The data for two of the images.

[0086] (2) Each organoid in a series of acquired microscopic images of mouse expanded alveolar organoids is framed, and each framed mouse alveolar organoid image is extracted one by one, ignoring unnecessary organoids (those with a diameter less than 30 μm, loose outer walls, collapsed shapes, poor light transmittance, and internal necrosis); Figure 7 and Figure 8 shown.

[0087] (3) The data is loaded into the selected sorting model (organoid type determination and identification device) GoogleNet to identify and determine the organoid type.

[0088] (4) Statistic all judgment results and export the EXCEL data table, as shown in Table 2. Figure 7 and Figure 8 In the figure, dark gray represents type A, light gray represents type B, and white represents type C.

[0089] Table 2 Automatic typing results of mouse expanded alveolar organoids

[0090] Classification A B C Figure 5 78 63 63 Figure 6 76 49 73

[0091] The results in Table 2 show that Figure 5 Among them, 78 type A mice expanded alveolar organoids, 63 type B mice expanded alveolar organoids, and 63 type C mice expanded alveolar organoids; Figure 6 Among them, 76 A-type mice expanded alveolar organoids, 49 B-type mice expanded alveolar organoids, and 73 C-type mice expanded alveolar organoids. This result is the same as the manual annotation result.

[0092] Example 3

[0093] This embodiment provides a method for intelligently judging and identifying the type of differentiated alveolar organoids in mice, comprising the following steps:

[0094] (1) Obtain imaging data of mouse differentiated alveolar organoids. The specific method is: use an optical microscope to scan and photograph the mouse differentiated alveolar organoid well plate (or culture dish) to be counted. The microscopic image needs to contain three-dimensional information (i.e., layer superposition), such as Fig. 9 and Fig.10 The data for two of the images.

[0095] (2) Each organoid in a series of acquired microscopic images of differentiated mouse alveolar organoids was framed and each framed mouse alveolar organoid image was extracted one by one, ignoring unnecessary organoids (those with a diameter less than 30 μm, loose outer walls, collapsed shapes, poor light transmittance, and internal necrosis); Fig.11 and Fig.12 shown.

[0096] (3) The data is loaded into the selected sorting model (organoid type determination and identification device) GoogleNet to identify and determine the organoid type.

[0097] (4) Statistic all judgment results and export the EXCEL data table, as shown in Table 3. Fig.11 and Fig.12 In the figure, dark gray represents type A, light gray represents type B, and white represents type C.

[0098] Table 3 Automatic typing results of mouse differentiated alveolar organoids

[0099] Classification A B C Fig. 9 85 5 6 Fig.10 81 5 7

[0100] The results in Table 3 show that Fig. 9 There were 85 type A mouse differentiated alveolar organoids, 5 type B mouse differentiated alveolar organoids, and 6 type C mouse differentiated alveolar organoids; Fig.10 There were 81 type A mouse differentiated alveolar organoids, 5 type B mouse differentiated alveolar organoids, and 7 type C mouse differentiated alveolar organoids. This result is the same as the manual annotation result.

[0101] In summary, the method of the present invention uses alveolar organoid images as data input and directly derives detailed typing ratio data. It has a unified judgment standard and is fast and intuitive, and can perform high-throughput operations. The typing ratio data has multiple uses: 1. It can be used as a library construction and quality control standard. Only alveolar organoid group samples that reach a certain ratio can be built into the library or judged as qualified products; 2. It can be used as a research parameter, such as the study of cytokine-induced differentiation efficiency, which requires a comparison of the changes in the ratio before and after differentiation.

[0102] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for intelligent identification of organoid types based on microscopic images, characterized in that: include: Step 1, obtaining a series of microscopic images of alveolar organoids and performing image synthesis; Step 2, select each alveolar organoid in the synthesized microscopic image, and extract each selected alveolar organoid image one by one, and use xi to represent one of the alveolar organoid images, where i is an integer; Step 3: Input each alveolar organoid image into three types of intelligent judgment and recognition models: VGG, ResNet, and GoogleNet, and obtain three prediction results of the alveolar organoid type in the image. If there are manual annotation results of the alveolar organoid in the image, the three prediction results are compared with the manual annotation results, and the three types of intelligent judgment and recognition models are optimized by minimizing the loss; otherwise, the three prediction results are compared. If at least two prediction results are consistent, the consistent results are used as the pseudo-label of the alveolar organoid type of the image, and then the model corresponding to the remaining prediction result is optimized by minimizing the loss; Step 4: After all alveolar organoid images are processed, the precision and recall results of the three types of intelligent judgment and recognition models are statistically analyzed, and the three models are evaluated using the model evaluation index F1 value using these results. The model with the best evaluation result is selected as the final sorting model. Step 5: Use the sorting model to identify and determine the type of alveolar organoids.

2. The method for intelligently identifying organoid types based on microscopic images according to claim 1, characterized in that: The formula used for minimizing loss optimization is: , in formula I, y i j Indicates xi Prediction results of organoids in images, j Indicates the model number: VGG model, j =1; ResNet model, j =2; GoogleNet model, j =3; Indicates the manual annotation results.

3. The method for intelligently identifying organoid types based on microscopic images according to claim 1, characterized in that: The calculation formula of the F1 value is: , II In formula II, Precision represents the precision rate, Recall represents the recall rate, P / N represents the positive / negative judgment result of the model, T / F represents the true / false judgment result of the model; FP represents false positive examples, FN represents false negative examples, TP represents true positive examples, TN represents true negative examples, and: a) precision = TP / (TP+FP) The proportion of positive examples (TP) correctly identified by the model to all positive examples in the data set; b) recall = TP / (TP+FN) The proportion of positive examples (TP) correctly identified by the model to all actual positive examples in the data set; c)accuracy = (TP+TN) / (TP+FP+TN+FN) The proportion of data (TP+TN) judged correctly by the model to the total data.

4. An intelligent identification system for alveolar organoid types based on microscopic images, characterized in that: For implementing the method described in any one of claims 1 to 3, the system comprises: A microscopic image acquisition device, used for acquiring a microscopic image of the alveolar organoid; an organoid type determination and identification device connected to the microscopic image acquisition device, the organoid type determination and identification device comprising an organoid detector and an organoid classifier, the organoid detector being used to extract each alveolar organoid in the microscopic image; the alveolar organoid classifier being used to determine and identify the type of the extracted alveolar organoid; A communication device is used to connect the microscopic image acquisition device, the organoid detector, and the organoid classifier in communication.

5. A method for using the method described in any one of claims 1 to 3 or the system described in claim 4 in alveolar organoid culture or alveolar organoid banking.

6. A method for culturing alveolar organoids, characterized in that: The following steps are involved: Obtaining tissue to be cultured; Cultivation of alveolar organoids; During the culture process, the method described in any one of claims 1 to 3 or the system described in claim 4 is used to determine and identify the organoid type.

7. A method for building a library of alveolar organoids, characterized in that: The following steps are involved: Determine and identify the type of the alveolar organoids obtained by culture using the method described in any one of claims 1 to 3 or the system described in claim 4; Organoids that meet the requirements are assembled into a sample library.