Methods and apparatuses for determining organoid characteristics, analysis devices, storage media

Abstract

The application relates to the technical field of organoid recognition, and discloses a method and device for determining organoid features, an analysis equipment and a storage medium. The method comprises the following steps: acquiring a plurality of two-dimensional organoid images; each two-dimensional organoid image is an image obtained by scanning an organoid at different scanning heights; stacking each two-dimensional organoid image to obtain a three-dimensional organoid image; and identifying the three-dimensional organoid image by using a preset feature recognition model to obtain organoid features. In this way, the two-dimensional organoid image is converted into a three-dimensional organoid image, and then the feature recognition model is used to identify the organoid features from the three-dimensional organoid image. Since the three-dimensional organoid image can restore the three-dimensional morphology of the organoid, the cells that are hidden due to stacking in the two-dimensional organoid image can be identified, so that complete geometric information of the organoid can be provided, and the accuracy of the determined organoid features can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of organoid recognition, and in particular to a method and device for determining organoid characteristics, an analysis apparatus, and a storage medium. BACKGROUND

[0002] An organoid is a small in-vitro organ that highly simulates the characteristics of a real organ and is developed from tumor tissue or stem cells extracted from a living being in a specific in-vitro microenvironment. Usually, an experimenter will culture an organoid in a culture device, and after the organoid is cultured to a certain morphology, the number of cells of the organoid in the culture device will be counted. If the number of cells of the organoid reaches a certain quantity, drug sensitivity detection will be performed, otherwise the culture will be continued.

[0003] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0004] To provide a simple summary of some aspects of the disclosed embodiments, the following is given. The summary is not an extensive overview of the application, nor is it intended to identify key / critical elements of the embodiments or to delineate the scope of the embodiments' scope, but to serve as an introduction to the detailed description below.

[0005] The embodiments of the present application provide a method for determining organoid characteristics, so as to improve the accuracy of the determined organoid characteristics.

[0006] The embodiments of the present application provide a method for determining organoid characteristics, comprising: obtaining a plurality of two-dimensional organoid images; each of the two-dimensional organoid images is an image obtained by scanning an organoid at different scanning heights; stacking each of the two-dimensional organoid images to obtain a three-dimensional organoid image; and using a pre-set feature recognition model to recognize the three-dimensional organoid image to obtain organoid characteristics.

[0007] In the above embodiment, the two-dimensional organoid image is converted into a three-dimensional organoid image, and then a feature recognition model is used to recognize the organoid characteristics from the three-dimensional organoid image. Since the three-dimensional organoid image can restore the three-dimensional morphology of the organoid, the cells that are blocked in the two-dimensional organoid image due to stacking can be recognized, so that more comprehensive cell information can be provided, and the accuracy of the determined organoid characteristics can be improved.

[0008] Further, the two-dimensional organoid images are stacked to obtain a three-dimensional organoid image, including: screening to-be-processed organoid images from the two-dimensional organoid images according to a preset height range and a preset number; and combining the to-be-processed organoid images in a depth direction according to a preset height order to obtain the three-dimensional organoid image.

[0009] In the above embodiment, the more two-dimensional organoid images obtained, the easier it is to retain the characteristics of the organoids, so that the restored organoids are more complete. By obtaining multiple two-dimensional organoid images and screening the two-dimensional organoid images, different combinations of two-dimensional organoid images can be obtained, and then different three-dimensional organoid images can be formed according to different combinations of two-dimensional organoid images. This facilitates subsequent multi-angle prediction of organoid characteristics according to different three-dimensional organoid images, so as to improve the accuracy of the organoid characteristics.

[0010] Further, the preset height range includes a first range, a second range, and a third range; the preset number includes a first number, a second number, and a third number; the screening of the to-be-processed organoid images from the two-dimensional organoid images according to the preset height range and the preset number includes: randomly selecting the first number of the two-dimensional organoid images as first candidate organoid images from the two-dimensional organoid images with a scanning height conforming to the first range; randomly selecting the second number of the two-dimensional organoid images as second candidate organoid images from the two-dimensional organoid images with a scanning height conforming to the second range; and randomly selecting the third number of the two-dimensional organoid images as third candidate organoid images from the two-dimensional organoid images with a scanning height conforming to the third range; and the first candidate organoid images, the second candidate organoid images, and the third candidate organoid images are used as the to-be-processed organoid images.

[0011] In the above embodiment, if only random screening is used to obtain to-be-processed organoid images, the scanning depths of the randomly screened two-dimensional organoid images may be concentrated in a certain specific depth range, which may result in a large difference between the three-dimensional organoid image restored according to the two-dimensional organoid images and the actual organoid structure. By dividing the preset height range into multiple ranges, two-dimensional organoid images at different scanning depths can be retained, thereby retaining information of the organoids at different depths, so that the three-dimensional organoid image determined according to the to-be-processed organoid images can better restore a three-dimensional organoid, and thus the organoid characteristics identified according to the three-dimensional organoid image are more accurate.

[0012] Further, the three-dimensional organoid image is identified by using a preset feature recognition model to obtain organoid features, including: determining a first feature vector representing spatial structure information of the three-dimensional organoid image; inputting the first feature vector into a preset feature recognition model to obtain organoid features.

[0013] In the above embodiments, the model can learn the hidden patterns and rules in the data through a large amount of data training, and make accurate predictions accordingly. Therefore, by predicting the three-dimensional organoid image by using the pre-trained feature recognition model, the organoid features can be accurately obtained.

[0014] Further, the first feature vector representing the spatial structure information of the three-dimensional organoid image is determined, including: dividing the three-dimensional organoid image into a preset number of sequence graph small blocks; wherein the size of each sequence graph small block is a preset size; respectively determining a second feature vector representing the spatial structure information of each sequence graph small block; and integrating each second feature vector to obtain the first feature vector representing the spatial structure information of the three-dimensional organoid image.

[0015] In the above embodiments, the three-dimensional organoid image is divided into a preset number of sequence graph small blocks, and a second feature vector representing the spatial structure information of each sequence graph small block is determined. Since the data amount of a single sequence graph small block is less than that of the three-dimensional organoid image, the second feature vector of the single sequence graph small block is determined, and then the second feature vectors are integrated into the first feature vector. Compared with directly extracting the first feature vector of the three-dimensional organoid image, the computational load can be reduced, thereby reducing the hardware cost of the analysis equipment for determining the organoid features.

[0016] Further, the three-dimensional organoid image is divided into a preset number of sequence graph small blocks, including: using a preset sliding window to slide on the three-dimensional organoid image to extract data in the sliding window as a sequence graph small block.

[0017] In the above embodiments, the sliding window can adapt to targets of different scales, has certain stability and robustness, and can conveniently and stably obtain the sequence graph small block.

[0018] Further, the second feature vector representing the spatial structure information of each sequence graph small block is determined, including: inputting each sequence graph small block into a preset three-dimensional structure extraction model to obtain a second feature vector corresponding to each sequence graph small block.

[0019] In the above embodiment, since the model can learn complex and fine feature information in the three-dimensional image, including shape, structure, texture, etc. And the feature vector extracted by the model can comprehensively and accurately describe the characteristics of the three-dimensional image, so that the pre-trained three-dimensional structure extraction model can accurately and conveniently extract the second feature vector of the sequence image small block.

[0020] The embodiment of the present application provides a device for determining organoid characteristics, comprising: an acquisition module, configured to acquire a plurality of two-dimensional organoid images; each of the two-dimensional organoid images is an image obtained by scanning an organoid at different scanning heights; a three-dimensional conversion module, configured to stack each of the two-dimensional organoid images to obtain a three-dimensional organoid image; and a quantity determination module, configured to identify the three-dimensional organoid image by using a preset feature recognition model to obtain organoid characteristics.

[0021] The embodiment of the present application provides an analysis device, comprising a processor and a memory, the memory stores computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the above-mentioned method for determining organoid characteristics.

[0022] In some embodiments, the computer storage medium stores computer executable instructions, when the computer executable instructions are called and executed by the processor, the computer executable instructions enable the processor to implement the above-mentioned method for determining organoid characteristics.

[0023] The foregoing general description and the following description are only exemplary and explanatory, and are not used to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0024] One or more embodiments are exemplarily illustrated by corresponding drawings, which do not constitute limitation on the embodiments, elements with the same reference numerals in the drawings are shown as similar elements, the drawings do not constitute proportional limitation, and wherein:

[0025] Figure 1 is a schematic diagram of a method for determining organoid characteristics provided by the embodiment of the present application;

[0026] Figure 2 is a schematic diagram of a flow for determining organoid characteristics provided by the embodiment of the present application;

[0027] Figure 3 is a schematic diagram of a device for determining organoid characteristics provided by the embodiment of the present application;

[0028] Figure 4 is a schematic diagram of an analysis device provided by the embodiment of the present application.

[0029] Reference signs:

[0030] 1: acquisition module; 2: three-dimensional conversion module; 3: quantity determination module; 4: memory; 5: communication interface; 6: bus; 7: processor. DETAILED DESCRIPTION

[0031] In order to enable a more detailed understanding of the features and technical content of the embodiments of the present application, the implementation of the embodiments of the present application will be described in detail below, and the attached drawings are only used for reference and are not used to limit the embodiments of the present application. In the following technical description, in order to facilitate explanation, through multiple details, a sufficient understanding of the disclosed embodiments is provided. However, one or more embodiments can still be implemented without these details. In other cases, in order to simplify the drawings, well-known structures and devices can be simplified.

[0032] The terms "first", "second", and the like in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the embodiments of the present application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.

[0033] Unless otherwise specified, the term "multiple" means two or more.

[0034] The term "corresponding" can refer to an association or binding relationship. A and B correspond to each other means that there is an association or binding relationship between A and B.

[0035] A method for determining organoid characteristics is provided in the embodiments of the present application. Referring to Figure 1 as shown, Figure 1 The basic flowchart of the method for determining organoid characteristics provided in the embodiments of the present application includes:

[0036] Step S101, acquiring a plurality of two-dimensional organoid images.

[0037] Each two-dimensional organoid image is an image obtained by scanning a single organoid at different scanning heights.

[0038] For example, assuming that the height range of the dispersed cells of the organoid is 0 microns to 15 microns, the organoid can be scanned at a scanning height of 5 microns to obtain a two-dimensional organoid image, and then the organoid can be scanned at a scanning height of 10 microns to obtain a two-dimensional organoid image. In this way, two-dimensional organoid images at different scanning heights can be obtained. To facilitate the description of the height, a three-dimensional coordinate system can be established according to the bottom of the culture dish in which the organoid is cultured, and the Z axis of the three-dimensional coordinate system is used to measure the scanning height.

[0039] In the embodiments of the present application, the two-dimensional organoid images can be bright field images. That is, different two-dimensional organoid images can be bright field images obtained by scanning a single organoid at different scanning heights.

[0040] In some embodiments, the imaging device can be used to perform hierarchical scanning on the organoid, thereby obtaining a plurality of two-dimensional organoid images. The imaging device can also be used to perform hierarchical scanning on the culture dish in which the organoid is cultured, thereby obtaining a plurality of two-dimensional organoid images for the organoid in the culture dish. The imaging device can be an existing microscope observation device capable of capturing bright field images of the organoid, which is not limited herein.

[0041] Example 1: The imaging device is used to perform hierarchical scanning on the organoid, thereby obtaining a plurality of two-dimensional organoid images. The imaging device is further used to send the plurality of two-dimensional organoid images to the analysis device, and the analysis device receives the plurality of two-dimensional organoid images.

[0042] Example 2: The analysis device is integrated with an imaging device, and the imaging device of the analysis device is used to perform hierarchical scanning on the organoid, thereby obtaining a plurality of two-dimensional organoid images.

[0043] It can be understood that, in the embodiments of the present application, one organoid can be included in the two-dimensional organoid image. The specific number of two-dimensional organoid images obtained can be set according to the experience of engineers, for example, 15, 50, etc.

[0044] Step S102: Stacking each two-dimensional organoid image to obtain a three-dimensional organoid image.

[0045] In some embodiments, stacking each two-dimensional organoid image to obtain a three-dimensional organoid image can be: selecting a to-be-processed organoid image from each two-dimensional organoid image according to a preset height range and a preset number. The three-dimensional organoid image is determined according to each to-be-processed organoid image.

[0046] In an optional implementation of the above embodiment, the preset height range includes a first range, a second range, and a third range; the preset quantity includes a first quantity, a second quantity, and a third quantity; and the screening of the to-be-processed organoid images from the two-dimensional organoid images according to the preset height range and the preset quantity can be: randomly selecting a first quantity of two-dimensional organoid images with a scanning height conforming to the first range as first candidate organoid images; randomly selecting a second quantity of two-dimensional organoid images with a scanning height conforming to the second range as second candidate organoid images; randomly selecting a third quantity of two-dimensional organoid images with a scanning height conforming to the third range as third candidate organoid images; and taking the first candidate organoid images, the second candidate organoid images, and the third candidate organoid images as the to-be-processed organoid images. The maximum value of the first range is less than the minimum value of the second range; and the maximum value of the second range is less than the minimum value of the third range.

[0047] In the above implementation, the first quantity can be equal to the second quantity, and the second quantity can be equal to the third quantity.

[0048] For example, a plurality of two-dimensional organoid images are acquired. It is assumed that the cells of the organoids are distributed in a space range with a height from A to B. The height from A to B is divided into three continuous height intervals, for example: a height range a1, a height range a2, and a height range a3. That is, the sum of the height range a1, the height range a2, and the height range a3 is equal to the height from A to B. The first range is set as the height range a1, the second range is set as the height range a2, and the third range is set as the height range a3. A first quantity of two-dimensional organoid images with a scanning height conforming to the height range a1 are randomly selected as first candidate organoid images. A second quantity of two-dimensional organoid images with a scanning height conforming to the height range a2 are randomly selected as second candidate organoid images. A third quantity of two-dimensional organoid images with a scanning height conforming to the height range a3 are randomly selected as third candidate organoid images. Then, the first candidate organoid images, the second candidate organoid images, and the third candidate organoid images are taken together as to-be-processed organoid images.

[0049] It can be understood that the preset height range can be set according to the experience of engineers, and the preset height range can include multiple ranges, for example, the preset height range includes a first range and a second range. Or, the preset height range includes a first range, a second range, a third range and a fourth range. Not limited here. Correspondingly, the preset number can be set according to the experience of engineers, and the preset number can include multiple numbers, for example, the preset number includes a first number and a second number. Or, the preset number includes a first number, a second number, a third number and a fourth number. It is worth noting that each range in the preset height range corresponds to a number. In this way, engineers can adaptively adjust the preset height range and the preset number according to the analysis of the computing power of the device and the experimental effect of forming a three-dimensional organ image from a two-dimensional organ image.

[0050] It is worth noting that the multiple ranges included in the preset height range can also not be continuous height intervals. Assuming that the cells of the organoid are distributed in a spatial range of height C to D, for example, 3 height intervals are selected from the height range from C to D, for example, height range c1, height range c2 and height range c3. The range interval formed by the height range c1, the height range c2 and the height range c3 is less than the spatial range of height C to D.

[0051] In another optional implementation of the above embodiment, the filtering of the to-be-processed organoid images from the two-dimensional organoid images according to the preset height range and the preset number can be: directly randomly selecting a preset number of two-dimensional organoid images as to-be-processed organoid images from the two-dimensional organoid images whose scanning heights conform to the preset height range.

[0052] Illustratively, a plurality of two-dimensional organoid images are obtained. Assuming that the cells of the organoid are distributed in a spatial range of height E to D, and the scanning heights of the two-dimensional organoid images are all located in the spatial range of height E to D. Assuming that the preset height range is E to D, and the preset number is 9. Directly randomly select 9 two-dimensional organoid images from the plurality of two-dimensional organoid images as to-be-processed organoid images.

[0053] Alternatively, in the above embodiment, determining the three-dimensional organoid image according to each to-be-processed organoid image can be: stacking each to-be-processed organoid image in the depth direction according to the preset height order to obtain a three-dimensional organoid image.

[0054] Exemplarily, in a two-dimensional image, data is usually represented as three dimensions: the number of channels (C), height (H) and width (W). Such a representation is [C, H, W], where C represents the color channel of the image (for example, an RGB image has three channels), H represents the height of the image, and W represents the width of the image. In this application, since the two-dimensional organoid image is a bright field image, that is, a grayscale image, the number of channels is 1. In order to extend the two-dimensional image representation to three dimensions, an additional dimension, "depth", is needed. When processing a three-dimensional image, the depth dimension usually represents a sequence of consecutive layers or a plurality of consecutive image frames. Therefore, for a three-dimensional image, data will have four dimensions: the number of channels, depth, height and width, represented as [C, Depth, H, W]. Thus, by stacking each to-be-processed organoid image in the depth direction according to a preset height order, an additional depth dimension can be added to the two-dimensional image, thereby forming a three-dimensional organoid image. That is, there is not only one two-dimensional image under each channel, but a three-dimensional volume composed of a plurality of two-dimensional images, where each image is a "slice" or "layer" in the volume. It can be understood that this method is similar to the way of layer-by-layer stacking images in CT (Computed Tomography, Computed Tomography) scanning, where each layer image is a two-dimensional image, but after merging, a three-dimensional spatial structure is displayed.

[0055] Alternatively, in the above embodiment, determining the three-dimensional organoid image according to the to-be-processed organoid images can be: inputting the to-be-processed organoid images arranged in the preset height order into a preset 3D construction model to obtain the three-dimensional organoid image.

[0056] Preferably, the preset height order can be set according to the experience of engineers, for example: the preset height order can be from low to high in scanning depth, or from high to low in scanning depth.

[0057] Notably, the user can set the number of three-dimensional organoid images formed, and repeat step S102 to obtain a set number of three-dimensional organoid images.

[0058] Step S103, according to the three-dimensional organoid image, using a preset feature recognition model for recognition, obtaining an organoid feature.

[0059] In some embodiments, according to the three-dimensional organoid image, using a preset feature recognition model for recognition, obtaining an organoid feature can be: determining a first feature vector representing the spatial structure information of the three-dimensional organoid image; inputting the first feature vector into a preset feature recognition model to obtain the organoid feature.

[0060] The organoid feature can be a morphological category of the organoid, a cell quantity of the organoid, an area of a cell of the organoid, a volume of the cell of the organoid, or the like. For example, the morphological category of the organoid includes apoptosis, fusion, normal, and the like. It can be understood that the area of the cell can be an average area of each cell in the organoid, and the volume of the cell can be an average volume of each cell in the organoid.

[0061] It is worth noting that in the case of stacking each two-dimensional organoid image to obtain a plurality of three-dimensional organoid images, a first feature vector representing the spatial structure information of each three-dimensional organoid image needs to be determined, and then each first feature vector is input into a preset feature recognition model to obtain a plurality of candidate organoid features, and then an average organoid feature of each candidate organoid feature is taken as the final organoid feature.

[0062] In an optional manner of the above embodiment, the first feature vector representing the spatial structure information of the three-dimensional organoid image can be: inputting the three-dimensional organoid image into a preset three-dimensional structure extraction model to obtain the first feature vector representing the spatial structure information of the three-dimensional organoid image. In this way, by directly extracting features from the entire three-dimensional organoid image, more global information can be retained, so that the organoid feature recognized by using the first feature vector is more accurate.

[0063] In another optional manner of the above embodiment, the first feature vector representing the spatial structure information of the three-dimensional organoid image can be: dividing the three-dimensional organoid image into a preset number of sequence graph small blocks. A second feature vector representing the spatial structure information of each sequence graph small block is determined. The second feature vectors are integrated to obtain the first feature vector representing the spatial structure information of the three-dimensional organoid image. In this way, since the features are directly extracted from the whole three-dimensional organoid image, due to the high resolution and large size of the image, it may cause large amount of calculation and high memory consumption. Dividing the three-dimensional organoid image into a preset number of sequence graph small blocks can reduce the amount of calculation, so that the first feature vector can be obtained more efficiently.

[0064] Each sequence graph small block has a preset size. It can be understood that the preset size can be set according to the experience of engineers, for example, the preset size is 224x224 pixels, or the preset size is 512x512 pixels.

[0065] In the above optional manner, a preset sliding window can be used to slide on the three-dimensional organoid image, and the data in the sliding window is extracted as a sequence graph small block.

[0066] In the optional manner, the second feature vector representing the spatial structure information of each sequence image small block can be: inputting each sequence image small block into a preset three-dimensional structure extraction model to obtain a corresponding second feature vector of each sequence image small block.

[0067] The preset three-dimensional structure extraction model can be a model capable of extracting spatial structure information of a three-dimensional image in the prior art, for example, 3D ResNet (3D residual neural network), 3D U-Net, etc., which is not limited herein. Since the 3D ResNet network is specially designed for extracting deep features from three-dimensional data, it can effectively capture the spatial structure information of organoid cells. Since the 3D U-Net not only extracts features but also maintains the spatial hierarchy of the image, it is suitable for processing 3D images with obvious hierarchy and structure. Therefore, in the embodiments of the present application, the 3D ResNet or 3D U-Net (3D semantic segmentation network) model is selected to extract the first feature vector or the second feature vector.

[0068] Optionally, in the optional manner, the second feature vector is integrated to obtain the first feature vector representing the spatial structure information of the three-dimensional organoid image can be: inputting each second feature vector into a preset integration model to obtain the first feature vector.

[0069] The preset integration model can be a model capable of integrating feature vectors in the prior art, for example, Transformer Encoder (encoder), convolutional neural network model, recurrent neural network model, etc., which is not limited herein. By introducing an advanced encoder mechanism to process these small blocks, the global coherence of the image can be maintained and strengthened while extracting the key local details of the sequence image small blocks. Therefore, in the embodiments of the present application, the Transformer Encoder model is selected to integrate the second feature vectors.

[0070] Alternatively, in the optional manner, the second feature vector is integrated to obtain the first feature vector representing the spatial structure information of the three-dimensional organoid image can be: splicing each second feature vector to obtain the first feature vector.

[0071] The second feature vectors are spliced to obtain the first feature vector can be: splicing each second feature vector according to the sliding order of the sliding window.

[0072] Exemplarily, the sliding window slides according to a preset sliding order to extract the sequence map patches. In this case, the data in the sliding window is extracted as a sequence map patch each time the sliding window slides, and the sequence map patch is assigned a sequence number representing the sliding order of the sliding window. The second feature vector representing the spatial structure information of each sequence map patch is determined, and the second feature vectors corresponding to the sequence map patches are spliced according to the sequence numbers of the sequence map patches.

[0073] In this case, the second feature vectors can also be spliced to obtain the first feature vector according to the positions of the sequence map patches on the three-dimensional organoid image according to a preset position order.

[0074] Exemplarily, the three-dimensional organoid image is divided into a preset number of sequence map patches, and the coordinates of the geometric center of the sequence map patch in the three-dimensional organoid image are determined as the position of the sequence map patch on the three-dimensional organoid image. The second feature vectors corresponding to the positions are spliced according to a preset position order. In this case, the preset position order can be a coordinate order.

[0075] Alternatively, in the above optional manner, the first feature vector representing the spatial structure information of the three-dimensional organoid image can also be obtained by averaging and pooling the second feature vectors.

[0076] Alternatively, in the above embodiment, the preset feature recognition model is obtained by inputting the to-be-tested feature vector with the organoid feature label into a preset multi-layer perceptron network model for training to obtain the preset feature recognition model.

[0077] It can be understood that when there are multiple organoid features, a corresponding feature recognition model can be trained for each organoid feature. Exemplarily, when the organoid feature is the number of cells, the to-be-tested feature vector with the number of cells is input into a preset multi-layer perceptron network model for training to obtain the preset feature recognition model. When the organoid feature is the cell area, the to-be-tested feature vector with the cell area is input into a preset multi-layer perceptron network model for training to obtain the preset feature recognition model.

[0078] It can be understood that the multi-layer perceptron network model (MLP) can also be replaced by other models capable of predicting organoid features through training in the prior art, such as a convolutional neural network (CNN) and a deep belief network (DBN), without limitation.

[0079] Exemplarily, the organoid feature is cell number. The technician processes the test organoid, separates and uniformly disperses the cells of the test organoid by digestion and dispersion to avoid cell aggregation or overlap to affect the accuracy of counting. Then, the technician adds CTG (CellTiter-Glo® Luminescent Cell Viability Assay) reagent to count the cell number of the organoid. The CTG reagent is a high-sensitivity bioluminescence detection technology-based reagent that measures the number of viable cells and cell viability in culture by quantifying adenosine triphosphate. Then, the test organoid is scanned by an imaging device to obtain a sample two-dimensional organoid image. The sample two-dimensional organoid images are stacked to obtain a sample three-dimensional organoid image, the sample three-dimensional organoid image is processed to obtain a test feature vector, and the test feature vector is labeled with the cell number, thereby obtaining a test feature vector labeled with the cell number.

[0080] Exemplarily, the organoid feature is cell area. The technician processes the test organoid, separates and uniformly disperses the cells of the test organoid by digestion and dispersion to avoid cell aggregation or overlap to affect the accuracy of counting. Then, the technician counts the area of each cell and obtains the average area of each cell as the cell area. Then, the test organoid is scanned by an imaging device to obtain a sample two-dimensional organoid image. The sample two-dimensional organoid images are stacked to obtain a sample three-dimensional organoid image, the sample three-dimensional organoid image is processed to obtain a test feature vector, and the test feature vector is labeled with the cell area, thereby obtaining a test feature vector labeled with the cell area. Similarly, a test feature vector labeled with other organoid features can also be obtained by a similar method.

[0081] Notably, the sample three-dimensional organoid image is processed to obtain the test feature vector in the same way as the first feature vector of the three-dimensional organoid image is determined.

[0082] Notably, to improve the accuracy of prediction and the generalization ability of the model, a cross-validation method can be used to evaluate the performance of the model, and the model can be optimized by adjusting the model parameters (such as learning rate, number of layers, number of hidden units, etc.), and the optimized model is used for operation in the present application. At the same time, a series of indicators (such as accuracy, recall rate, F1 score, etc.) can be used to evaluate the performance of the model on the test set, and the model that meets the indicators is used for operation in the present application.

[0083] Exemplarily, in combination with the above-mentioned method for determining the first feature vector of the three-dimensional organoid image, the method for determining the second feature vector of the three-dimensional organoid image comprises the following steps: Figure 2As shown, the organoids are subjected to hierarchical scanning by using the imaging device of the analysis device, 15 two-dimensional organoid images with different scanning heights are obtained, and the size of each two-dimensional organoid image is 6400x6400 pixels. Assuming that the scanning heights corresponding to the 15 two-dimensional organoid images are all located in the preset height range, 9 two-dimensional organoid images are randomly selected from the 15 two-dimensional organoid images, and the 9 two-dimensional organoid images are arranged and combined in the preset height order to construct a three-dimensional organoid image with depth information, and at this time the format of the three-dimensional organoid image is 6400x6400x9. The process of converting the two-dimensional organoid image into the three-dimensional organoid image aims to simulate the three-dimensional structure of the organoid, so as to more accurately capture the spatial distribution of the cells. Then, the three-dimensional organoid image with a depth of 9 obtained is divided into 16 sequence image patches with a size of 400x400, i.e., patches1, patches2, patches3, …, patches14, patches15 and patches16. By dividing the three-dimensional organoid image into sequence image patches, the computational load of the model in extracting feature vectors can be reduced. Then, the divided patches are input into the 3D ResNet model respectively to obtain the second feature vectors corresponding to each sequence image patch respectively. Then, the second feature vectors are input into the transformer model to obtain the first feature vector representing the spatial structure information of the three-dimensional organoid image. The first feature vector is input into the MLP model for identification to obtain the organoid feature.

[0084] In some embodiments, after obtaining the organoid feature, it can further include: determining whether the organoid can be subcultured according to the organoid feature.

[0085] In the case where the organoid feature is the number of cells, determining whether the organoid can be subcultured according to the organoid feature can be determining that the organoid can be subcultured in the case where the number of cells is greater than a preset number. In the case where the organoid feature is the cell area, determining whether the organoid can be subcultured according to the organoid feature can be determining that the organoid can be subcultured in the case where the cell area is greater than a preset area. In this way, since the three-dimensional organoid image can restore the three-dimensional morphology of the organoid, the cells that are blocked in the two-dimensional organoid image due to stacking can be identified, so that complete geometric information of the organoid can be provided, thereby improving the accuracy of the determined organoid feature, and further enabling the determination of whether the organoid can be subcultured to be more accurate.

[0086] Embodiment Two

[0087] In combination Figure 3As shown, the embodiment of the present application provides a device for determining organoid features, comprising: an acquisition module 1, a three-dimensional conversion module 2 and a quantity determination module 3. The acquisition module 1 is configured to acquire a plurality of two-dimensional organoid images; each two-dimensional organoid image is an image obtained by scanning an organoid at a different scanning height; the three-dimensional conversion module 2 is configured to stack each two-dimensional organoid image to obtain a three-dimensional organoid image; and the quantity determination module 3 is configured to identify the three-dimensional organoid image using a preset feature recognition model to obtain organoid features.

[0088] Optionally, the three-dimensional conversion module 2 is configured to stack each two-dimensional organoid image to obtain a three-dimensional organoid image by: screening a to-be-processed organoid image from each two-dimensional organoid image according to a preset height range and a preset number; and combining each to-be-processed organoid image in a depth direction according to a preset height order to obtain a three-dimensional organoid image.

[0089] Optionally, the preset height range comprises a first range, a second range and a third range; the preset number comprises a first number, a second number and a third number; and the three-dimensional conversion module 2 is configured to screen a to-be-processed organoid image from each two-dimensional organoid image according to a preset height range and a preset number by: randomly selecting a first number of two-dimensional organoid images from two-dimensional organoid images with scanning heights conforming to the first range as first candidate organoid images; randomly selecting a second number of two-dimensional organoid images from two-dimensional organoid images with scanning heights conforming to the second range as second candidate organoid images; randomly selecting a third number of two-dimensional organoid images from two-dimensional organoid images with scanning heights conforming to the third range as third candidate organoid images; and taking the first candidate organoid images, the second candidate organoid images and the third candidate organoid images as the to-be-processed organoid images.

[0090] Optionally, the quantity determination module 3 is configured to identify the three-dimensional organoid image using a preset feature recognition model to obtain organoid features by: determining a first feature vector representing spatial structure information of the three-dimensional organoid image; and inputting the first feature vector into the preset feature recognition model to obtain the organoid features.

[0091] Optionally, the quantity determination module 3 is configured to determine a first feature vector representing spatial structure information of the three-dimensional organoid image by: dividing the three-dimensional organoid image into a preset number of sequence map small blocks; wherein each sequence map small block has a preset size; determining a second feature vector representing spatial structure information of each sequence map small block; and integrating each second feature vector to obtain the first feature vector representing spatial structure information of the three-dimensional organoid image.

[0092] Optionally, the quantity determination module 3 is configured to segment the three-dimensional organoid image into a preset number of sequence graph patches by sliding a preset sliding window on the three-dimensional organoid image and extracting data in the sliding window as the sequence graph patches.

[0093] Optionally, the quantity determination module 3 is configured to determine the second feature vector representing the spatial structure information of each sequence graph patch by inputting each sequence graph patch into a preset three-dimensional structure extraction model to obtain a second feature vector corresponding to each sequence graph patch.

[0094] It can be understood that the embodiments described in Embodiment One are also applicable in Embodiment Two without conflict. For the sake of brevity, the description is not repeated here.

[0095] Embodiment Three

[0096] In combination with Figure 4 The embodiments of the present application provide an analysis device, which includes a processor 7 and a memory 4. Optionally, the device can also include a communication interface 5 and a bus 6. The processor 7, the communication interface 5, and the memory 4 can communicate with each other through the bus 6. The communication interface 5 can be used for information transmission. The processor 7 can invoke the logical instructions in the memory 4 to execute the method for determining organoid features in the above-described embodiments.

[0097] In addition, the logical instructions in the memory 4 described above can be implemented in the form of a software functional unit and sold or used as an independent product. In this case, the software functional unit can be stored in a computer readable storage medium.

[0098] The memory 4 is a computer readable storage medium, which can be used to store software programs, computer executable programs, such as program instructions / modules corresponding to the method in the embodiments of the present application. The processor 7 executes the program instructions / modules stored in the memory 4 to perform functional applications and data processing, that is, to implement the method for determining organoid features in the above-described embodiments.

[0099] The memory 4 can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required by a function. The data storage area can store data created according to the use of the terminal device. In addition, the memory 4 can include a high-speed random access memory and can also include a non-volatile memory.

[0100] By using the analysis device provided in the embodiments of the present application, the two-dimensional organoid image is converted into a three-dimensional organoid image, and then the organoid features are recognized from the three-dimensional organoid image by using the feature recognition model. Since the three-dimensional organoid image can restore the three-dimensional morphology of the organoid, the cells that are blocked due to stacking in the two-dimensional organoid image can be recognized, thereby improving the accuracy of the determined organoid features.

[0101] The embodiments of the present application provide a computer storage medium, which stores computer executable instructions, and the computer executable instructions are configured to execute the method for determining organoid features.

[0102] The embodiments of the present application provide a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method for determining organoid features.

[0103] The computer readable storage medium described above can be a transitory computer readable storage medium or a non-transitory computer readable storage medium.

[0104] The technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes one or more instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method of the embodiments of the present application. The storage medium described above can be a non-transitory storage medium, including a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes, or can be a transitory storage medium.

[0105] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic, for example, the division of units is only a logical function division, and in actual implementation, another division mode can be used, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0106] The above is only an embodiment of the present application, and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. Meanwhile, the above embodiments can be combined with each other without conflict to form new embodiments.

Claims

1. A method of determining organoid characteristics, characterized in that, The method comprises the following steps: acquiring a plurality of two-dimensional organoid images; each of the two-dimensional organoid images is a bright field image obtained by scanning an organoid at different scanning heights; stacking each of the two-dimensional organoid images to obtain a three-dimensional organoid image; performing identification on the three-dimensional organoid image by using a preset feature recognition model to obtain an organoid feature; the organoid feature is a morphological category of the organoid, a cell number of the organoid, an area of a cell of the organoid, a volume of the cell of the organoid, and the morphological category of the organoid includes apoptosis, fusion, and normal; training a corresponding feature recognition model for each organoid feature, and the preset feature recognition model is obtained by the following method: inputting a to-be-measured feature vector with an organoid feature label into a preset multi-layer perceptron network model to obtain the preset feature recognition model; performing identification on the three-dimensional organoid image by using the preset feature recognition model to obtain the organoid feature, comprising: determining a first feature vector representing spatial structure information of the three-dimensional organoid image; inputting the first feature vector into the preset feature recognition model to obtain the organoid feature; determining the first feature vector representing the spatial structure information of the three-dimensional organoid image, comprising: dividing the three-dimensional organoid image into a preset number of sequence graph small blocks; wherein the size of each sequence graph small block is a preset size; inputting each sequence graph small block into a preset three-dimensional structure extraction model to obtain a second feature vector corresponding to each sequence graph small block; integrating each second feature vector to obtain the first feature vector representing the spatial structure information of the three-dimensional organoid image; the preset three-dimensional structure extraction model is 3D ResNet or 3D U-Net.

2. The method of claim 1, wherein, stacking each of the two-dimensional organoid images to obtain a three-dimensional organoid image, comprising: screening to-be-processed organoid images from each of the two-dimensional organoid images according to a preset height range and a preset number; determining a three-dimensional organoid image according to each of the to-be-processed organoid images.

3. The method of claim 2, wherein, The preset height range comprises a first range, a second range, and a third range; and the preset number comprises a first number, a second number, and a third number. Screening to-be-processed organoid images from each of the two-dimensional organoid images according to a preset height range and a preset number, comprising: randomly selecting the first number of the two-dimensional organoid images whose scanning heights conform to the first range from the two-dimensional organoid images as first candidate organoid images; randomly selecting the second number of the two-dimensional organoid images whose scanning heights conform to the second range from the two-dimensional organoid images as second candidate organoid images; randomly selecting the third number of the two-dimensional organoid images whose scanning heights conform to the third range from the two-dimensional organoid images as third candidate organoid images; taking the first candidate organoid images, the second candidate organoid images, and the third candidate organoid images as the to-be-processed organoid images.

4. The method of claim 1, wherein, Dividing the three-dimensional organoid image into a preset number of sequence graph small blocks, comprising: The preset sliding window is used to slide on the three-dimensional organoid image, and data in the sliding window is extracted as a sequence graph small block.

5. The method of claim 1, wherein, Second feature vectors respectively representing spatial structure information of each sequence graph small block are determined. Each sequence graph small block is input into a preset three-dimensional structure extraction model to obtain a second feature vector corresponding to each sequence graph small block.

6. An apparatus for determining organoid characteristics, characterized in that, The method comprises the following steps: The acquisition module is configured to acquire a plurality of two-dimensional organoid images; each two-dimensional organoid image is a bright field image obtained by scanning an organoid at different scanning heights; The three-dimensional conversion module is configured to stack each two-dimensional organoid image to obtain a three-dimensional organoid image; The number determination module is configured to identify the three-dimensional organoid image by using a preset feature recognition model to obtain organoid features; The organoid features include the morphological category of the organoid, the number of cells in the organoid, the area of the cells in the organoid, and the volume of the cells in the organoid; the morphological category of the organoid includes apoptosis, fusion, and normality. Each feature recognition model corresponding to each organoid feature is trained, and the preset feature recognition model is obtained by the following method: a feature vector to be measured with an organoid feature label is input into a preset multi-layer perceptron network model for training to obtain the preset feature recognition model; The three-dimensional organoid image is identified by using the preset feature recognition model to obtain organoid features, which comprises the following steps: determining a first feature vector representing spatial structure information of the three-dimensional organoid image; inputting the first feature vector into the preset feature recognition model to obtain organoid features; The first feature vector representing spatial structure information of the three-dimensional organoid image is determined, which comprises the following steps: dividing the three-dimensional organoid image into a preset number of sequence graph small blocks; the size of each sequence graph small block is a preset size; each sequence graph small block is input into a preset three-dimensional structure extraction model to obtain a second feature vector corresponding to each sequence graph small block; the second feature vectors are integrated to obtain the first feature vector representing spatial structure information of the three-dimensional organoid image; the preset three-dimensional structure extraction model is a 3D ResNet or a 3D U-Net.

7. An analysis device, characterized by The processor and the memory are included, the memory stores computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the method for determining organoid features according to any one of claims 1 to 5.

8. A computer storage medium, characterized in that The computer storage medium stores computer executable instructions, and when the computer executable instructions are called and executed by the processor, the computer executable instructions enable the processor to implement the method for determining organoid features according to any one of claims 1 to 5.

Images