Fusion degree evaluation method and equipment for stem cell multiplication culture

Abstract

The invention relates to the technical field of stem cell culture, and provides a fusion degree evaluation method and equipment for stem cell multiplication culture, and the method comprises the following steps: calculating microenvironment characteristics by using microenvironment parameters and stem cell fusion degree parameters; performing time perception based on the two-dimensional image to obtain a time perception label, and performing fusion degree classification to obtain a fusion degree prediction label; based on the time perception label and the fusion degree prediction label, fusing the two-dimensional image and the three-dimensional image to obtain an enhanced stem cell image; performing time perception on the enhanced stem cell image to obtain an enhanced time perception label, and performing fusion degree evaluation to obtain an enhanced fusion degree prediction label; and based on the microenvironment features, the time perception label, the fusion degree prediction label, the enhanced time perception label and the enhanced fusion degree prediction label, performing fusion degree evaluation by using the fusion degree evaluation model to obtain a fusion degree evaluation value. According to the method, the accuracy of fusion degree evaluation can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of stem cell culture, and in particular to a method and device for evaluating the degree of fusion of stem cell expansion culture. Background Art

[0002] In today's biomedical field, cell therapy, especially stem cell therapy, has shown great potential, and vigorously developing cell therapy has become one of the key industries that all countries are paying attention to. Stem cells have unique self-renewal and multidirectional differentiation capabilities, which makes them have broad application prospects in many important aspects such as tissue engineering, regenerative medicine and drug screening.

[0003] The degree of stem cell confluence (i.e., the ratio of the area covered by cells on the surface of the culture dish to the surface area of ​​the culture dish) directly affects the metabolic activity of stem cells. Evaluating the degree of confluence of stem cell expansion culture helps maintain the healthy expansion state of stem cells and avoid differentiation and cell death. When cells are at a low degree of confluence, the metabolic level of the cells is low, the consumption of nutrients in the culture medium is low, and the culture medium does not need to be replaced frequently; when the cells reach a medium degree of confluence, the metabolic level increases, the cells need more nutrients, and the accumulation rate of metabolic waste also accelerates; when the cells reach a high degree of confluence, the cell density is large, the nutritional demand increases, and the accumulation of metabolic waste will be faster, so the culture medium needs to be replaced more frequently. How to effectively evaluate the degree of confluence of stem cell expansion culture is an important challenge.

[0004] In addition to the methods that rely on microscopic observation and manual analysis, image recognition technology occupies an important position in the existing methods. These image recognition technologies based on machine learning make up for the shortcomings of traditional manual methods, such as strong subjectivity, significant time consumption, and difficulty in coping with large-scale data processing requirements, but there are still some shortcomings: First, the existing methods do not consider the microenvironmental changes caused by fluid replacement. This change in the microenvironment usually affects the short-term growth trend of cell fusion, which is of great reference significance for the evaluation of cell fusion. Second, the existing methods generally use the entire two-dimensional (2D) stem cell image for analysis, without considering the topological structure and block information aggregation between image blocks, resulting in the lack of image block features, information aggregation in the time perception task and cell fusion evaluation task. The connection and constraint, which leads to incomplete or biased evaluation results. Third, the existing methods often perform image feature analysis based on 2D images, with a single feature dimension, ignoring the interactivity between 2D images and a small number of precious three-dimensional (3D) images. It can be seen that the current stem cell fusion evaluation method has the problem of low accuracy of fusion evaluation. Summary of the invention

[0005] The present application provides a method and device for evaluating the degree of fusion of stem cell expansion culture, which can solve the problem of low accuracy of fusion evaluation.

[0006] In a first aspect, the present application provides a method for evaluating the degree of confluence of stem cell expansion culture, the method comprising:

[0007] Obtaining two-dimensional images, three-dimensional images, microenvironment parameters, and stem cell fusion parameters of the environment for stem cell expansion and culture at multiple times;

[0008] The microenvironmental parameters and stem cell confluence parameters at all times are used to calculate the microenvironmental characteristics at each time;

[0009] Perform time perception based on the two-dimensional images at all times to obtain the time perception label of each moment, and perform fusion degree classification on the two-dimensional images at each moment to obtain the fusion degree prediction label of each moment;

[0010] For each moment, based on the time perception label and fusion degree prediction label corresponding to the moment, the two-dimensional image and the three-dimensional image of the moment are fused to obtain an enhanced stem cell image;

[0011] For each enhanced stem cell image, time perception is performed on the enhanced stem cell image to obtain an enhanced time perception label, and fusion evaluation is performed on the enhanced stem cell image to obtain an enhanced fusion prediction label;

[0012] Based on all microenvironment features, all time perception labels, all fusion degree prediction labels, all enhanced time perception labels, and all enhanced fusion degree prediction labels, a fusion degree evaluation model is used to perform fusion degree evaluation to obtain a fusion degree evaluation value at each moment;

[0013] The fusion evaluation model is trained using all fusion evaluation values ​​to obtain a trained fusion evaluation model, and the trained fusion evaluation model is used to perform fusion evaluation on the environment at the time to be evaluated to obtain a final fusion evaluation value.

[0014] Optionally, the microenvironmental parameters include environmental parameters, nutrient and physicochemical parameters, and genetic material enrichment parameters;

[0015] The microenvironment parameters and stem cell fusion parameters at all times are used to calculate the microenvironment characteristics at each time, including:

[0016] By formula:

[0017] Data mv (t)=Mantel_test(P1(t)∪P2(t)∪P3(t),D(t-Δt)∪D(t)∪D(t+Δt), stp≤p 阈值2 )

[0018] Calculate the microenvironment characteristics Data at the tth momentmv (t);

[0019] Wherein, Mantel_test() represents Mantel test, P1(t) represents environmental parameters at the tth moment, P2(t) represents nutrients and physicochemical parameters at the tth moment, P3(t) represents genetic material enrichment parameters at the tth moment, D(t) represents stem cell fusion parameter at the tth moment, D(t-Δt) represents stem cell fusion parameter at the t-Δtth moment, D(t+Δt) represents stem cell fusion parameter at the t+Δtth moment, t∈{1,2,,...,T}, T represents the last moment in the cell expansion culture cycle, st represents constraint, p represents significance, p represents 阈值2 Represents the significance threshold.

[0020] Optionally, time perception is performed based on the two-dimensional images at all times to obtain a time-aware label for each time, including:

[0021] For each moment, perform the following steps:

[0022] Cut the two-dimensional image at the moment into multiple image blocks;

[0023] The time-aware model is used to extract features from each image block to obtain the image features of each image block. The image block time-aware label of each image block is calculated based on all image features. All image block time-aware labels are aggregated to obtain the initial time-aware label of the moment.

[0024] A time-aware loss function is constructed according to the initial time-aware label, and the time-aware loss function is minimized based on the time-aware model to obtain the time-aware label of the moment.

[0025] Optionally, all image block time-aware labels are aggregated to obtain the initial time-aware label of the moment, including:

[0026] By formula:

[0027]

[0028] Calculate the initial time-aware label T′(t) at the tth moment;

[0029] Among them, Pooling_aggr() represents pooling aggregation, K represents the number of hidden layers, represents the weight coefficient of the lth hidden layer at the tth moment, represents the feature representation of the lth hidden layer corresponding to the time-aware labels of all image blocks at the tth moment, represents the message construction function of the lth hidden layer at the tth time, represents the permutation-invariant aggregation function of the lth hidden layer at the tth time, represents the update function of the lth hidden layer at the tth time, represents the embedded feature representation after pooling the node features of the last hidden layer at the tth moment, R 1×1 Represents dimension;

[0030] The time-aware loss function is:

[0031] min(L whole-time-aware (t))=min(λ·L patch-time-aware (t)+(1-λ)·L patchs-time-aggr (t))

[0032] Among them, L whole-time-aware (t) represents the value of the time-aware loss function, λ represents the weight coefficient, and L patch-time-aware (t) represents the feature loss function, L patchs-time-aggr (t) represents the aggregation loss function:

[0033] L patch-time-aware (t) = α·L features (p (t,t±Δt) )+β·L remove_corr (s (t,t±Δt) )+γ·L errors (y (t,t±Δt) ≠y ( ' t,t±Δt) )

[0034]

[0035] L patchs-time-aggr (t)=Probability(T(t)≠T'(t))

[0036] Among them, α, β, and γ all represent weight coefficients, L features () represents the feature loss function, p (t,t±Δt) represents all image blocks corresponding to the t-th moment, the t-Δt-th moment, and the t+Δt-th moment, L remove_corr () represents the loss function of stacking features to remove correlation, s (t,t±Δt) represents all image features corresponding to the t-th moment, t-Δt-th moment, and t+Δt-th moment, L errors () represents the error rate loss function, y (t,t±Δt) represents the time-perceptual labels of all image blocks corresponding to the t-th moment, t-Δt-th moment, and t+Δt-th moment, y′ (t,t±Δt) represents the temporal perception true labels of all image blocks corresponding to the t-th moment, t-Δt-th moment, and t+Δt-th moment, δ represents the weight coefficient, and C(i,i) represents the i-th row and i-th column element in the correlation matrix, C (i,j) represents the i-th row and j-th column element in the correlation matrix, Probability() represents the probability, T(t) represents the time-aware true label at the t-th moment, t∈{1,2,,...,T}, and T represents the last moment in the cell expansion culture cycle.

[0037] T(t) represents the time-aware true label at the t-th moment, t∈{1,2,,...,T}, and T represents the last moment in the cell expansion culture cycle.

[0038] Optionally, the fusion degree of the two-dimensional image at each moment is classified to obtain a fusion degree prediction label at each moment, including:

[0039] For each moment, perform the following steps:

[0040] Cut the two-dimensional image at a moment into multiple image blocks;

[0041] The fusion degree classification model is used to classify the fusion degree of each image block to obtain the image block fusion degree prediction label, and the fusion degree prediction labels of all image blocks are aggregated to obtain the initial fusion degree prediction label at the moment;

[0042] Construct a fusion loss function based on the initial fusion prediction label;

[0043] Based on the fusion degree classification model, the fusion degree loss function is minimized and solved to obtain the fusion degree prediction label at the moment.

[0044] Optionally, the fusion loss function is:

[0045] min(L whole-2D-degree (t)) = min(ε·{L errors (d (t) ≠d′ (t) )|stL features (p (t) )}+(1-ε)·L patchs-degree-aggr (t))

[0046] Among them, L whole-2D-degree (t) represents the value of the fusion loss function, ε represents the weight coefficient, L errors () represents the error rate loss function, d (t) Represents the true label of the fusion degree of all image blocks, d′ (t) represents the predicted labels of the fusion degree of all image blocks, st represents the constraint, and L features () represents the feature loss function, p (t) represents all image blocks at the tth moment, Lpatchs-degree-aggr (t) represents the fusion aggregation loss function:

[0047] L patchs-degree-aggr (t)=Probability(D(t)≠D'(t))

[0048] Among them, Probability() represents probability, D (t) Denotes the true label of the fusion degree, D′ (t) Represents the initial fusion prediction label.

[0049] Optionally, based on the time perception label and the fusion degree prediction label corresponding to the moment, the two-dimensional image and the three-dimensional image at the moment are fused to obtain an enhanced stem cell image, including:

[0050] By formula:

[0051]

[0052] Acquiring enhanced images of stem cells

[0053] in, represents the enhanced two-dimensional image, Represents a two-dimensional projection image:

[0054]

[0055]

[0056]

[0057] Among them, w 21 、w 22 、w 23 、w 24 All represent weights, represents the image obtained by geometric transformation of a two-dimensional image, It means that the time-aware label and the fusion degree prediction label are used as a priori, and the image is enhanced by variational autoencoding on the two-dimensional image. It represents the image obtained by using the time-aware label and the fusion degree prediction label as a priori and performing generative adversarial network enhancement on the two-dimensional image. It represents the image obtained by performing a two-dimensional diffusion model enhancement on a two-dimensional image by taking the time perception label and the fusion degree prediction label as a priori. Views represents the viewing angle of the image enhancement. Views represents the number of multi-views preset for image enhancement. Projectionoperator represents the projection operator. Represents a three-dimensional image, represents the three-dimensional diffusion model, represents the three-dimensional diffusion model generating function, represents a two-dimensional image, and st represents a constraint.

[0058] Optionally, based on all microenvironment features, all time perception labels, all fusion degree prediction labels, all enhanced time perception labels, and all enhanced fusion degree prediction labels, a fusion degree evaluation model is used to perform fusion degree evaluation to obtain a fusion degree evaluation value at each moment, including:

[0059] For each moment, perform the following steps:

[0060] Calculate multi-scale feature fusion data based on micro-environment features corresponding to the moment, time-aware labels, fusion degree prediction labels, enhanced time-aware labels, and enhanced fusion degree prediction labels;

[0061] The fusion degree evaluation model is used to evaluate the fusion degree of multi-scale feature fusion data and obtain the fusion degree evaluation value at the moment.

[0062] Optionally, multi-scale feature fusion data is calculated based on micro-environment features corresponding to the moment, time-aware labels, fusion degree prediction labels, enhanced time-aware labels, and enhanced fusion degree prediction labels, including:

[0063] By formula:

[0064] Data fusion (t)=[Data mv (t),Data ota (t)∪Data ata (t),Data od (t)∪Data ad (t)]

[0065] Calculate multi-scale feature fusion data fusion (t);

[0066] Among them, Data mv (t) represents the microenvironmental characteristics corresponding to the tth moment, Data ota (t) represents the time-aware label corresponding to the tth moment, Data ata (t) represents the enhanced time-aware label corresponding to the t-th moment, Data od (t) represents the fusion prediction label corresponding to the tth moment, Data ad (t) represents the enhanced fusion prediction label corresponding to the t-th moment, t∈{1,2,,...,T}, and T represents the last moment in the cell expansion culture cycle.

[0067] In a second aspect, the present application provides a device for evaluating the degree of fusion of stem cell expansion culture, comprising:

[0068] An acquisition module, which acquires two-dimensional images, three-dimensional images, microenvironment parameters, and stem cell fusion degree parameters of the environment for stem cell expansion and culture at multiple times;

[0069] A calculation module calculates the microenvironment characteristics at each moment using the microenvironment parameters and stem cell fusion parameters at all moments;

[0070] The first time perception module performs time perception based on the two-dimensional images at all times to obtain the time perception label of each time, and classifies the fusion degree of the two-dimensional images at each time to obtain the fusion degree prediction label of each time;

[0071] The fusion module fuses the two-dimensional image and the three-dimensional image at each moment based on the time perception label and the fusion degree prediction label corresponding to the moment to obtain an enhanced stem cell image;

[0072] The second time perception module performs time perception on each enhanced stem cell image to obtain an enhanced time perception label, and performs fusion evaluation on the enhanced stem cell image to obtain an enhanced fusion prediction label;

[0073] The fusion evaluation module uses the fusion evaluation model to evaluate the fusion degree based on all microenvironment features, all time perception labels, all fusion degree prediction labels, all enhanced time perception labels, and all enhanced fusion degree prediction labels to obtain the fusion degree evaluation value at each moment;

[0074] The training module uses all fusion evaluation values ​​to train the fusion evaluation model to obtain the trained fusion evaluation model, and uses the trained fusion evaluation model to perform fusion evaluation on the environment at the time to be evaluated to obtain the final fusion evaluation value.

[0075] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for evaluating the degree of fusion of stem cell expansion culture when executing the above-mentioned computer program.

[0076] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for evaluating the degree of fusion of stem cell expansion culture.

[0077] The above solution of the present application has the following beneficial effects:

[0078] In an embodiment of the present application, the microenvironment characteristics of each moment are calculated by using the microenvironment parameters and stem cell fusion parameters of all moments, and then time perception is performed based on the two-dimensional images of all moments to obtain the time perception label of each moment, and the fusion degree of the two-dimensional image at each moment is classified to obtain the fusion degree prediction label of each moment, and then for each moment, based on the time perception label and fusion degree prediction label corresponding to the moment, the two-dimensional image and the three-dimensional image of the moment are fused to obtain an enhanced stem cell image, and then for each enhanced stem cell image, the enhanced stem cell image is time-perceived to obtain an enhanced time perception label, and the enhanced stem cell image is evaluated for fusion to obtain an enhanced fusion degree prediction label, and then based on all microenvironment characteristics, all time perception labels, all fusion degree prediction labels, all enhanced time perception labels, and all enhanced fusion degree prediction labels, a fusion degree evaluation model is used to perform fusion degree evaluation to obtain a fusion degree evaluation value for each moment, and finally all fusion degree evaluation values ​​are used to train the fusion degree evaluation model to obtain a trained fusion degree evaluation model, and the trained fusion degree evaluation model is used to perform fusion degree evaluation on the environment at the moment to be evaluated to obtain a final fusion degree evaluation value. Among them, the enhanced stem cell image is obtained by fusing the two-dimensional image and the three-dimensional image based on the time perception label and the fusion degree prediction label, which can improve the information richness and accuracy of the enhanced stem cell image. The fusion degree is evaluated by using the highly accurate enhanced stem cell image in combination with the microenvironment characteristics, which improves the information richness of the fusion degree evaluation and effectively improves the accuracy of the fusion degree evaluation.

[0079] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0081] Figure 1 A flow chart of a method for evaluating the degree of confluence of stem cell expansion culture provided in one embodiment of the present application;

[0082] Figure 2 A schematic diagram of the structure of a device for evaluating the degree of fusion of stem cells for expansion and culture provided in one embodiment of the present application;

[0083] Figure 3 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0084] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0085] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0086] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0087] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0088] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0089] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0090] In order to solve the problem of low accuracy of existing fusion evaluation, an embodiment of the present application provides a fusion evaluation method for stem cell expansion culture, which calculates the microenvironment characteristics of each moment by using the microenvironment parameters and stem cell fusion parameters at all moments, and then performs time perception based on the two-dimensional images of all moments to obtain the time perception label of each moment, and performs fusion degree classification on the two-dimensional images of each moment to obtain the fusion degree prediction label of each moment, and then respectively for each moment, based on the time perception label and fusion degree prediction label corresponding to the moment, fuses the two-dimensional image and the three-dimensional image at the moment to obtain an enhanced stem cell image, and then respectively for each enhanced moment, Stem cell image, time perception is performed on the enhanced stem cell image to obtain enhanced time perception labels, and fusion degree evaluation is performed on the enhanced stem cell image to obtain enhanced fusion degree prediction labels, and then based on all microenvironment features, all time perception labels, all fusion degree prediction labels, all enhanced time perception labels, and all enhanced fusion degree prediction labels, a fusion degree evaluation model is used to perform fusion degree evaluation to obtain a fusion degree evaluation value at each moment, and finally all fusion degree evaluation values ​​are used to train the fusion degree evaluation model to obtain a trained fusion degree evaluation model, and the trained fusion degree evaluation model is used to perform fusion degree evaluation on the environment at the time to be evaluated to obtain a final fusion degree evaluation value. Among them, the enhanced stem cell image is obtained by fusing the two-dimensional image and the three-dimensional image based on the time perception label and the fusion degree prediction label, which can improve the information richness and accuracy of the enhanced stem cell image, and the fusion degree evaluation is performed by using the enhanced stem cell image with high accuracy and combining the microenvironment features, which improves the information richness of the fusion degree evaluation and effectively improves the accuracy of the fusion degree evaluation.

[0091] Next, the method for evaluating the degree of fusion of stem cell expansion culture provided in the present application is exemplified.

[0092] like Figure 1 As shown, the method for evaluating the degree of fusion of stem cell expansion culture provided in the present application comprises the following steps:

[0093] Step 11, obtaining two-dimensional images, three-dimensional images, microenvironment parameters, and stem cell fusion degree parameters of the environment for stem cell expansion and culture at multiple times.

[0094] The above environment is a culture environment in which stem cell expansion and culture is being carried out, such as a culture dish, etc. The above microenvironment parameters include parameters related to stem cell culture, such as temperature, humidity, hormone concentration, etc. The stem cell fusion parameter is the ratio of the coverage area of ​​the stem cells on the surface of the culture environment to the surface area of ​​the culture environment. The multiple moments are multiple historical moments with real stem cell fusion parameters.

[0095] In some embodiments of the present application, a two-dimensional image may be acquired using a camera or other device, a three-dimensional image may be acquired using a computer's three-dimensional modeling software, and microenvironment parameters may be acquired using a sensor or other device.

[0096] Step 12, using the microenvironment parameters and stem cell fusion parameters at all times to calculate the microenvironment characteristics at each time.

[0097] The above-mentioned microenvironment parameters include environmental parameters (such as temperature, carbon dioxide concentration, humidity, oxygen concentration, pressure, light, oscillation / shaking frequency, etc.), nutrients and physicochemical parameters (such as amino acids, glucose, growth factors, vitamins, hormones, antibiotics, salts, minerals, as well as pH value, osmotic pressure, etc.), and genetic material enrichment parameters (such as gene annotation enrichment, metabolic pathway enrichment, protein interaction enrichment, etc.).

[0098] Specifically, through the formula:

[0099] Data mv (t)=Mantel_test(P1(t)∪P2(t)∪P3(t),D(t-Δt)∪D(t)∪D(t+Δt), stp≤p 阈值2 )

[0100] Calculate the microenvironment characteristics Data at the tth moment mv (t).

[0101] Wherein, Mantel_test() represents Mantel test, P1(t) represents environmental parameters at the tth moment, P2(t) represents nutrients and physicochemical parameters at the tth moment, P3(t) represents genetic material enrichment parameters at the tth moment, D(t) represents stem cell fusion parameter at the tth moment, D(t-Δt) represents stem cell fusion parameter at the t-Δtth moment, D(t+Δt) represents stem cell fusion parameter at the t+Δtth moment, t∈{1,2,,...,T}, T represents the last moment in the cell expansion culture cycle, st represents constraint, p represents significance, p represents 阈值2 Represents the significance threshold.

[0102] It should be noted that the calculation formula for the genetic material enrichment parameter is:

[0103]

[0104] Among them, E(t), E GO (t), E KEGG (t) and E PPI(t) are the overall enrichment, gene ontology (GO) enrichment, Kyoto Encyclopedia of Genes and Genomes (KEGG) metabolic pathway enrichment and protein-protein interaction (PPI) enrichment results at the tth moment; st is the constraint condition, p.adjust is the adjusted p value after Bonferroni / Benjamini / false positive rate correction method, p 阈值1 is the preset significance threshold, N GO (t), N KEGG (t), N PPI (t) are the enriched entries of GO, KEGG and PPI at the tth moment; T is the entire cell expansion culture cycle; ∪ is the union symbol; term is the union of the enriched entries at all moments; BP, CC and MF are the biological processes, cellular components and molecular functions enriched in GO, respectively; KEGG_passway is the KEGG enriched pathway, including metabolism, genetic information processing, environmental information processing, cellular process, organism system, human disease and drug development; PPI_passway is the PPI enriched pathway, including physical binding, functional synergy, protein complex construction, regulatory network, pathway-level interaction, etc.

[0105] The genetic material enrichment parameters are the enrichment multiple (Fold Enrichment) that meets the above conditions, the number of differential genes in the enriched entries / the total number of differential genes (GeneRAatio), the total number of genes in the entry / the total number of annotated genes in human stem cells (BgRatio), the corrected p-value (p.adjust), and the number of differential genes enriched in the entry (Count).

[0106] Step 13, performing time perception based on the two-dimensional images at all moments to obtain the time perception label of each moment, and performing fusion degree classification on the two-dimensional images at each moment to obtain the fusion degree prediction label of each moment.

[0107] The above-mentioned time perception label is used to describe the time position of a moment in all moments, and the fusion degree prediction label is used to describe the fusion degree of stem cells in the moment environment corresponding to the two-dimensional image (such as high fusion degree, medium fusion degree, low fusion degree, etc.).

[0108] In some embodiments of the present application, the steps of performing time perception based on the two-dimensional images at all times, obtaining the time perception label at each time, and performing fusion degree classification on the two-dimensional images at each time, and obtaining the fusion degree prediction label at each time are specifically as follows:

[0109] The first step is to perform time perception based on the two-dimensional images at all times to obtain the time-aware label for each moment.

[0110] For each moment, perform the following steps:

[0111] First, the 2D image at the moment is cropped into multiple image blocks.

[0112] Then, the time-aware model is used to extract features of each image block to obtain the image features of each image block, and the image block time-aware label of each image block is calculated based on all image features. All image block time-aware labels are aggregated to obtain the initial time-aware label of the moment.

[0113] It should be noted that the image blocks can be feature extracted using dual attention algorithms based on space and channels to obtain discriminative image features. The calculation formula is as follows:

[0114]

[0115] in, is the comprehensive attention weight, It is a spatial information extraction function implemented by convolution operation and global pooling. It is a channel information extraction function implemented by global pooling and multi-layer perceptron; patch whole is the overall patch image, patch seg is the patch image for image segmentation, L features () represents the feature loss function, Represents feature maps of different depths and scales, depth represents the number of layers of the feature map, and scale represents the scale. is the comprehensive attention weight, Loss represents the loss, and sigmoid represents the activation function.

[0116] The graph neural network can be used to calculate the image block time-aware label of each image block. The update formula for each hidden layer in the graph neural network is:

[0117]

[0118] Among them, (X t , A t ) is the constructed graph, X t ∈R m×1 is the feature matrix of the image features, m is the number of GCN nodes, and its m value is the number of patches; 1 represents the time-aware feature dimension; A tis the adjacency matrix. When there is an edge between two nodes (multiple nodes correspond to multiple image blocks, and the edge is determined according to the geometric position, pixel features and topological relationship between the corresponding two image blocks. For example, if two image blocks are directly adjacent in space, it is considered that there is an edge between the two nodes they correspond to), the value at the corresponding position of the matrix is ​​1, otherwise it is 0; l is the lth hidden layer of GCN, Con is the message construction function; Per is the permutation-invariant aggregation function, which realizes the aggregation of all messages transmitted to a certain node; Up is the update function for updating the features of existing nodes; V is a node in the graph; Mes (t,v) It is the aggregation of all messages at node V.

[0119] The above-mentioned image block time-aware label of each image block is calculated based on all image features, and all image block time-aware labels are aggregated to obtain the initial time-aware label at the moment:

[0120] By formula:

[0121]

[0122] Calculate the initial time-aware label T′(t) at the tth moment;

[0123] Among them, Pooling_aggr() represents pooling aggregation, K represents the number of hidden layers, represents the weight coefficient of the lth hidden layer, represents the feature representation of the lth hidden layer corresponding to the time-aware labels of all image blocks at the tth moment, represents the message construction function of the lth hidden layer at the tth time, represents the permutation-invariant aggregation function of the lth hidden layer at the tth time, represents the update function of the lth hidden layer at the tth time, represents the embedded feature representation after pooling the node features of the last hidden layer at the tth moment, R 1×1 Represents dimension.

[0124] It can be understood that the operations in the time-aware model are the above-mentioned dual attention algorithm operations based on space and channels, graph neural network operations, and the above-mentioned operations for calculating the initial time-aware labels.

[0125] Finally, a time-aware loss function is constructed based on the initial time-aware label, and the time-aware loss function is minimized based on the time-aware model to obtain the time-aware label of the moment.

[0126] Specifically, the time-aware loss function is:

[0127] min(L whole-time-aware (t))=min(λ·Lpatch-time-aware (t)+(1-λ)·L patchs-time-aggr (t))

[0128] Among them, L whole-time-aware (t) represents the value of the time-aware loss function, λ represents the weight coefficient, and L patch-time-aware (t) represents the feature loss function, L patchs-time-aggr (t) represents the aggregation loss function:

[0129] L patch-time-aware (t) = α·L features (p (t,t±Δt) )+β·L remove_corr (s (t,t±Δt) )+γ·L errors (y (t,t±Δt) ≠y ( ' t,t±Δt) )

[0130]

[0131] L patchs-time-aggr (t)=Probability(T(t)≠T'(t))

[0132] Among them, α, β, and γ all represent weight coefficients, L features () represents the feature loss function, p (t,t±Δt) represents all image blocks corresponding to the t-th moment, the t-Δt-th moment, and the t+Δt-th moment, L remove_corr () represents the loss function of stacking features to remove correlation, s (t,t±Δt) represents all image features corresponding to the t-th moment, t-Δt-th moment, and t+Δt-th moment, L errors () represents the error rate loss function, y (t,t±Δt) represents the time-perceptual labels of all image blocks corresponding to the t-th moment, t-Δt-th moment, and t+Δt-th moment, y′ (t,t±Δt) represents the temporal perception true labels of all image blocks corresponding to the t-th moment, t-Δt-th moment, and t+Δt-th moment, δ represents the weight coefficient, and C (i,i) represents the i-th row and i-th column element in the correlation matrix, C (i,j) represents the i-th row and j-th column element in the correlation matrix, Probability() represents the probability, T(t) represents the time-aware true label at the t-th moment, t∈{1,2,,...,T}, and T represents the last moment in the cell expansion culture cycle.

[0133] Exemplarily, the gradient descent method or other algorithms can be used to perform minimization solution. When the value of the time-aware loss function is not minimized, the parameters of the time-aware model are adjusted, and the step of using the time-aware model to extract features from each image block to obtain image features of each image block is returned, and the image block time-aware label of each image block is calculated based on all image features, and all image block time-aware labels are aggregated to obtain the initial time-aware label at the moment, until the loss function is minimized, and the time-aware model at this time is used to extract features from each image block to obtain image features of each image block, and the image block time-aware label of each image block is calculated based on all image features, and all image block time-aware labels are aggregated to obtain the time-aware label at the moment.

[0134] In the second step, the fusion degree of the two-dimensional image at each moment is classified to obtain the fusion degree prediction label at each moment.

[0135] For each moment, perform the following steps:

[0136] First, the 2D image at the moment is cropped into multiple image blocks.

[0137] Then, the fusion degree classification model is used to classify the fusion degree of each image block to obtain the image block fusion degree prediction label, and the fusion degree prediction labels of all image blocks are aggregated to obtain the initial fusion degree prediction label at the moment.

[0138] It should be noted that the dual attention algorithm based on space and channel mentioned above can be used to extract features from each image block, and then the graph convolutional network can be used to classify the fusion degree of each image block to obtain the image block fusion degree prediction label, and finally the global attention can be used to aggregate the fusion degree prediction labels of all image blocks to obtain the initial fusion degree prediction label at the moment.

[0139] It can be understood that the operations of the fusion classification model are the dual attention algorithm based on space and channel, graph convolutional neural network and global attention in sequence.

[0140] Then, a fusion loss function is constructed based on the initial fusion prediction labels.

[0141] The above fusion loss function is:

[0142] min(L whole-2D-degree (t))=min(ε·{L errors (d (t) ≠d′ (t) )|stL features (p (t) )}+(1-ε)·L patchs-degree-aggr (t))

[0143] Among them, L whole-2D-degree (t) represents the value of the fusion loss function, ε represents the weight coefficient, L errors () represents the error rate loss function, d (t) Represents the true label of the fusion degree of all image blocks, d′ (t) represents the predicted labels of the fusion degree of all image blocks, st represents the constraint, and L features () represents the feature loss function, p (t) represents all image blocks at the tth moment, L patchs-degree-aggr (t) represents the fusion aggregation loss function:

[0144] L patchs-degree-aggr (t)=Probability(D(t)≠D'(t))

[0145] Among them, D (t) Denotes the true label of the fusion degree, D′ (t) Represents the initial fusion prediction label.

[0146] Finally, based on the fusion degree classification model, the fusion degree loss function is minimized to obtain the fusion degree prediction label at the moment.

[0147] Exemplarily, the gradient descent method or other algorithms can be used for minimization solution. When the value of the fusion loss function is not minimized, the parameters of the fusion classification model are adjusted, and the step of using the fusion classification model to classify the fusion degree of each image block to obtain the image block fusion degree prediction label, and aggregate all the image block fusion degree prediction labels to obtain the initial fusion degree prediction label at the moment, until the value of the fusion loss function is minimized, and the fusion classification model at this time is used to classify the fusion degree of each image block to obtain the image block fusion degree prediction label, and aggregate all the image block fusion degree prediction labels to obtain the fusion degree prediction label at the moment.

[0148] It should be noted that since both the time perception model and the fusion degree classification model have dual attention algorithms based on space and channels for feature extraction of image blocks, the parameters of the dual attention algorithms based on space and channels in the time perception model have been trained in the relevant steps of the time perception model. In this step, the parameters of the dual attention algorithms based on space and channels in the fusion degree classification model do not need to be trained, and the parameters of the dual attention algorithms based on space and channels in the trained time perception model can be directly used to improve the model training efficiency.

[0149] Step 14, for each moment, based on the time perception label and fusion degree prediction label corresponding to the moment, the two-dimensional image and the three-dimensional image of the moment are fused to obtain an enhanced stem cell image.

[0150] Specifically, through the formula:

[0151]

[0152] Acquiring enhanced images of stem cells

[0153] in, represents the enhanced two-dimensional image, Represents a two-dimensional projection image:

[0154]

[0155]

[0156]

[0157] Among them, w 21 、w 22 、w 23 、w 24 All represent weights, represents the image obtained by geometric transformation of a two-dimensional image, It means that the time-aware label and the fusion degree prediction label are used as a priori, and the image is enhanced by variational autoencoding on the two-dimensional image. It represents the image obtained by using the time-aware label and the fusion degree prediction label as a priori and performing generative adversarial network enhancement on the two-dimensional image. It represents the image obtained by performing a two-dimensional diffusion model enhancement on a two-dimensional image by taking the time perception label and the fusion degree prediction label as a priori. Views represents the viewing angle of the image enhancement. Views represents the number of multi-views preset for image enhancement. Projectionoperator represents the projection operator. Represents a three-dimensional image, represents the three-dimensional diffusion model, represents the three-dimensional diffusion model generating function, represents a two-dimensional image, and st represents a constraint.

[0158] Step 15, for each enhanced stem cell image, time perception is performed on the enhanced stem cell image to obtain an enhanced time perception label, and fusion evaluation is performed on the enhanced stem cell image to obtain an enhanced fusion prediction label.

[0159] Specifically, the step of performing time perception on the enhanced stem cell image to obtain the enhanced time perception label is the same as the step of performing time perception based on the two-dimensional image at all moments to obtain the time perception label at each moment in step 13. Since the parameters of the time perception model have been trained in step 13, the trained time perception model can be directly used in this step to calculate the enhanced time perception label, that is, the enhanced stem cell image is cut into multiple image blocks; the trained time perception model is used to extract features of each image block to obtain the image features of each image block, the image block time perception label of each image block is calculated based on all image features, and all image block time perception labels are aggregated to obtain the enhanced time perception label.

[0160] The step of evaluating the fusion degree of the enhanced stem cell image to obtain the enhanced fusion degree prediction label is the same as the step of performing time perception based on the two-dimensional images at all moments in step 13 to obtain the time perception label at each moment. Since the fusion degree classification model has been parameter trained in step 13, the trained fusion degree classification model can be directly used in this step to calculate the enhanced fusion degree prediction label, that is, the enhanced stem cell image is cut into multiple image blocks; the trained fusion degree classification model is used to perform fusion degree classification on each image block to obtain the image block fusion degree prediction label, and the fusion degree prediction labels of all image blocks are aggregated to obtain the enhanced fusion degree prediction label.

[0161] Step 16, based on all microenvironment features, all time perception labels, all fusion degree prediction labels, all enhanced time perception labels, and all enhanced fusion degree prediction labels, a fusion degree evaluation model is used to perform fusion degree evaluation to obtain a fusion degree evaluation value at each moment.

[0162] The above fusion evaluation value is a stem cell fusion parameter evaluated using a fusion evaluation model.

[0163] For each moment, perform the following steps:

[0164] The first step is to calculate the multi-scale feature fusion data based on the micro-environment features corresponding to the moment, time-aware labels, fusion degree prediction labels, enhanced time-aware labels, and enhanced fusion degree prediction labels.

[0165] Specifically, through the formula:

[0166] Data fusion (t)=[Data mv (t),Data ota (t)∪Data ata (t),Data od (t)∪Data ad (t)]

[0167] Calculate multi-scale feature fusion data fusion (t).

[0168] Among them, Data mv (t) represents the microenvironmental characteristics corresponding to the tth moment, Data ota (t) represents the time-aware label corresponding to the tth moment, Data ata (t) represents the enhanced time-aware label corresponding to the t-th moment, Data od (t) represents the fusion prediction label corresponding to the tth moment, Data ad (t) represents the enhanced fusion prediction label corresponding to the t-th moment, t∈{1,2,,...,T}, and T represents the last moment in the cell expansion culture cycle.

[0169] In the second step, the fusion degree evaluation model is used to evaluate the fusion degree of the multi-scale feature fusion data to obtain the fusion degree evaluation value at the moment.

[0170] Specifically, the multi-scale feature fusion data is input into the fusion degree evaluation model to obtain a fusion degree evaluation value.

[0171] It should be noted that the above-mentioned fusion degree evaluation model can be a model such as a multi-layer perceptron.

[0172] Step 17, using all fusion degree evaluation values ​​to train the fusion degree evaluation model to obtain a trained fusion degree evaluation model, and using the trained fusion degree evaluation model to perform fusion degree evaluation on the environment at the time to be evaluated to obtain a final fusion degree evaluation value.

[0173] The above-mentioned final fusion evaluation value is a stem cell fusion parameter obtained by evaluating the trained fusion evaluation model.

[0174] It should be noted that the fusion evaluation values ​​and stem cell fusion parameters corresponding to all two-dimensional images are used as training data, and the adaptive quantum evolution optimization algorithm can be used to perform local and global searches on the model parameters of the fusion evaluation model to obtain the optimal model parameter θ * ,θ * The update formula is:

[0175]

[0176] Among them, Loss represents the loss function (constructed according to the fusion evaluation value and stem cell fusion parameter), the initial parameter of the model is θ, It represents the fusion evaluation value calculated based on multi-scale feature fusion data, and argmin represents average minimization.

[0177] Exemplarily, the steps of using the adaptive quantum evolution optimization algorithm to perform local and global searches on model parameters are:

[0178] Initialize a group of quantum bit populations through the quantum evolution algorithm, and apply quantum rotating gate operation to the quantum bit population, traversing different parameter combinations by adjusting the rotation angle;

[0179] After multiple iterations, the quantum evolutionary algorithm searches for a relatively good parameter population, selects a parameter combination with a higher fitness from the parameter population, and uses it as the initial point of the adaptive gradient descent algorithm;

[0180] Calculate the gradient of the loss function to the model parameters, and dynamically adjust the learning rate based on the gradient size and historical information;

[0181] Update the model parameters using the adjusted learning rate and calculated gradients;

[0182] Repeat the above steps and perform multiple iterations of local optimization until the stopping condition is met.

[0183] After the training is completed, the output layer expression of the trained fusion evaluation model is:

[0184]

[0185] Among them, Activation is an activation function, the time sliding window is set to [t-Δt, t+Δt], η t is the attention weight, is the quantum hidden state of the multi-layer quantum long short-term memory network at the tth moment, which is updated through quantum gate operation (obtained by calculating the multi-scale feature fusion data, and the expression is in, for quantum gate operations); and are the output weight matrix and bias term at the tth moment respectively.

[0186] The obtained optimal model parameters are substituted into the fusion degree evaluation model to obtain the trained fusion degree evaluation model, and then based on the two-dimensional images, three-dimensional images and microenvironment parameters of the environment at the moment to be evaluated, as well as Δt moments before the moment to be evaluated and Δt moments after the moment to be evaluated, according to the above steps, multiple microenvironment features, multiple time perception labels, multiple fusion degree prediction labels, multiple enhanced time perception labels, and multiple enhanced fusion degree prediction labels are obtained, and the trained fusion degree evaluation model is used to perform fusion degree evaluation on the above data to obtain the final fusion degree evaluation value at the moment to be evaluated.

[0187] It should be noted that in this step, since there is no corresponding stem cell fusion parameter at the time to be evaluated, blank data will be used to replace the stem cell fusion parameter in the original process when calculating the corresponding data, and the trained time perception model and fusion classification model will be directly used when performing time perception and fusion evaluation.

[0188] It is worth mentioning that the enhanced stem cell image is obtained by fusing the two-dimensional image and the three-dimensional image based on the time-aware label and the fusion degree prediction label, which can improve the information richness and accuracy of the enhanced stem cell image. The fusion degree is evaluated by using the highly accurate enhanced stem cell image in combination with the microenvironment characteristics, which improves the information richness of the fusion degree evaluation and effectively improves the accuracy of the fusion degree evaluation.

[0189] The following is an exemplary description of the device for evaluating the degree of fusion of stem cell expansion culture provided in the present application.

[0190] like Figure 2 As shown, the embodiment of the present application provides a device for evaluating the degree of fusion of stem cell expansion culture. The device 200 for evaluating the degree of fusion of stem cell expansion culture includes:

[0191] An acquisition module 201 is used to acquire two-dimensional images, three-dimensional images, microenvironment parameters, and stem cell fusion degree parameters of the environment for stem cell expansion and culture at multiple times;

[0192] A calculation module 202 calculates the microenvironment characteristics at each moment using the microenvironment parameters and stem cell fusion degree parameters at all moments;

[0193] The first time perception module 203 performs time perception based on the two-dimensional images at all moments to obtain a time perception label at each moment, and performs fusion degree classification on the two-dimensional images at each moment to obtain a fusion degree prediction label at each moment;

[0194] A fusion module 204, for each moment, fuses the two-dimensional image and the three-dimensional image at the moment based on the time perception label and the fusion degree prediction label corresponding to the moment, to obtain an enhanced stem cell image;

[0195] The second time perception module 205 performs time perception on each enhanced stem cell image to obtain an enhanced time perception label, and performs fusion evaluation on the enhanced stem cell image to obtain an enhanced fusion prediction label;

[0196] The fusion evaluation module 206 performs fusion evaluation based on all microenvironment features, all time perception labels, all fusion prediction labels, all enhanced time perception labels, and all enhanced fusion prediction labels using a fusion evaluation model to obtain a fusion evaluation value at each moment;

[0197] The training module 207 uses all fusion evaluation values ​​to train the fusion evaluation model to obtain a trained fusion evaluation model, and uses the trained fusion evaluation model to perform fusion evaluation on the environment at the time to be evaluated to obtain a final fusion evaluation value.

[0198] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0199] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0200] like Figure 3 As shown, an embodiment of the present application provides a terminal device, and the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 3 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 implements the steps of any of the above-mentioned method embodiments when executing the computer program D102.

[0201] Specifically, when the processor D100 executes the computer program D102, the microenvironment characteristics of each moment are calculated by using the microenvironment parameters and stem cell fusion degree parameters at all moments, and then time perception is performed based on the two-dimensional images at all moments to obtain a time perception label for each moment, and the fusion degree of the two-dimensional image at each moment is classified to obtain a fusion degree prediction label for each moment, and then, for each moment, based on the time perception label and fusion degree prediction label corresponding to the moment, the two-dimensional image and the three-dimensional image of the moment are fused to obtain an enhanced stem cell image, and then, for each enhanced stem cell image, the enhanced stem cell image is classified. The method is as follows: time perception is performed to obtain enhanced time perception labels, and the fusion degree of the enhanced stem cell image is evaluated to obtain enhanced fusion degree prediction labels, and then based on all microenvironment features, all time perception labels, all fusion degree prediction labels, all enhanced time perception labels, and all enhanced fusion degree prediction labels, the fusion degree evaluation model is used to evaluate the fusion degree, and the fusion degree evaluation value at each moment is obtained. Finally, all fusion degree evaluation values ​​are used to train the fusion degree evaluation model to obtain the trained fusion degree evaluation model, and the trained fusion degree evaluation model is used to evaluate the fusion degree of the environment at the time to be evaluated, and the final fusion degree evaluation value is obtained. Among them, the enhanced stem cell image is obtained by fusing the two-dimensional image and the three-dimensional image based on the time perception label and the fusion degree prediction label, which can improve the information richness and accuracy of the enhanced stem cell image, and the fusion degree evaluation is performed by using the enhanced stem cell image with high accuracy and combining the microenvironment features, which improves the information richness of the fusion degree evaluation and effectively improves the accuracy of the fusion degree evaluation.

[0202] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0203] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0204] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0205] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0206] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code to the fusion degree assessment method device / terminal device of stem cell expansion culture. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0207] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0208] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0209] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for evaluating the degree of confluence of stem cell expansion culture, characterized in that: include: Obtaining two-dimensional images, three-dimensional images, microenvironment parameters, and stem cell fusion parameters of the environment for stem cell expansion and culture at multiple times; Calculating the microenvironment characteristics at each moment using the microenvironment parameters and stem cell fusion parameters at all moments; Performing time perception based on the two-dimensional images at all moments to obtain a time perception label for each moment, and performing fusion degree classification on the two-dimensional images at each moment to obtain a fusion degree prediction label for each moment; For each of the moments, based on the time perception label and the fusion degree prediction label corresponding to the moment, the two-dimensional image and the three-dimensional image at the moment are fused to obtain an enhanced stem cell image; For each of the enhanced stem cell images, respectively, time perception is performed on the enhanced stem cell image to obtain an enhanced time perception label, and fusion evaluation is performed on the enhanced stem cell image to obtain an enhanced fusion degree prediction label; Based on all microenvironment features, all time perception labels, all fusion degree prediction labels, all enhanced time perception labels, and all enhanced fusion degree prediction labels, a fusion degree evaluation model is used to perform fusion degree evaluation to obtain a fusion degree evaluation value at each of the moments; The fusion degree evaluation model is trained using all fusion degree evaluation values ​​to obtain a trained fusion degree evaluation model, and the trained fusion degree evaluation model is used to perform fusion degree evaluation on the environment at the time to be evaluated to obtain a final fusion degree evaluation value.

2. The fusion degree evaluation method according to claim 1, characterized in that: The microenvironment parameters include environmental parameters, nutrients and physical and chemical parameters, and genetic material enrichment parameters; The method of calculating the microenvironmental characteristics at each moment using the microenvironmental parameters and stem cell fusion degree parameters at all moments includes: By formula: Data mv (t)=Mantel_test(P1(t)∪P2(t)∪P3(t),D(t-Δt)∪D(t)∪D(t+Δt),|s.t.p≤p 阈值2 ) Calculate the microenvironment characteristics Data at the tth moment mv (t); Wherein, Mantel_test( ) represents Mantel test, P1(t) represents environmental parameters at the t-th moment, P2(t) represents nutrients and physicochemical parameters at the t-th moment, P3(t) represents genetic material enrichment parameters at the t-th moment, D(t) represents stem cell fusion parameter at the t-th moment, D(t-Δt) represents stem cell fusion parameter at the t-Δt-th moment, D(t+Δt) represents stem cell fusion parameter at the t+Δt-th moment, t∈{1,2,,...,T}, T represents the last moment in the cell expansion culture cycle, st represents constraint, p represents significance, p represents 阈值2 Represents the significance threshold.

3. The fusion degree evaluation method according to claim 1, characterized in that: The performing time perception based on the two-dimensional images at all moments to obtain a time perception label for each moment includes: For each of the moments, perform the following steps: Cutting the two-dimensional image at the moment into a plurality of image blocks; Extract features from each image block using a time-aware model to obtain image features of each image block, calculate an image block time-aware label for each image block based on all image features, aggregate all image block time-aware labels to obtain an initial time-aware label at the moment; A time-aware loss function is constructed according to the initial time-aware label, and the time-aware loss function is minimized based on the time-aware model to obtain the time-aware label at the moment.

4. The fusion degree evaluation method according to claim 3 is characterized in that: The aggregating the time-aware labels of all image blocks to obtain the initial time-aware label of the moment includes: By formula: Calculate the initial time-aware label T′(t) at the tth moment; Among them, Pooling_aggr() represents pooling aggregation, K represents the number of hidden layers, represents the weight coefficient of the lth hidden layer at the tth moment, represents the feature representation of the lth hidden layer corresponding to the time-aware labels of all image blocks at the tth moment, represents the message construction function of the lth hidden layer at the tth moment, represents the permutation-invariant aggregation function of the lth hidden layer at the tth time, represents the update function of the lth hidden layer at the tth moment, represents the embedded feature representation after pooling the node features of the last hidden layer at the tth moment, R 1×1 Represents dimension; The time-aware loss function is: min(L whole-time-aware (t))=min(λ·L patch-time-aware (t)+(1-λ)·L patchs-time-aggr (t)) Among them, L whole-time-aware (t) represents the value of the time-aware loss function, λ represents the weight coefficient, and L patch-time-aware (t) represents the feature loss function, L patchs-time-aggr (t) represents the aggregation loss function: L patch-time-aware (t)=α·L features (p (t,t±Δt) )+β·L remove_corr (s (t,t±Δt) )+γ·L errors (y (t,t±Δt) ≠y ( ' t,t±Δt) ) L patchs-time-aggr (t)=Probability(T(t)≠T'(t)) Among them, α, β, and γ all represent weight coefficients, L features ( ) represents the feature loss function, p (t,t±Δt) represents all image blocks corresponding to the t-th moment, the t-Δt-th moment, and the t+Δt-th moment, L remove_corr () represents the loss function of stacking features to remove correlation, s (t,t±Δt) represents all image features corresponding to the t-th moment, t-Δt-th moment, and t+Δt-th moment, L errors ( ) represents the error rate loss function, y (t,t±Δt) represents the time-perceptual labels of all image blocks corresponding to the t-th moment, t-Δt-th moment, and t+Δt-th moment, y′ (t,t±Δt) represents the temporal perception true labels of all image blocks corresponding to the t-th moment, t-Δt-th moment, and t+Δt-th moment, δ represents the weight coefficient, and C (i,i) represents the i-th row and i-th column element in the correlation matrix, C (i,j) represents the i-th row and j-th column element in the correlation matrix, Probability( ) represents the probability, T(t) represents the time-aware true label at the t-th moment, t∈{1,2,,...,T}, and T represents the last moment in the cell expansion culture cycle.

5. The fusion degree evaluation method according to claim 1, characterized in that: The step of classifying the fusion degree of the two-dimensional image at each moment to obtain a fusion degree prediction label at each moment includes: For each of the moments, perform the following steps: Cutting the two-dimensional image at the moment into a plurality of image blocks; Using a fusion degree classification model to classify the fusion degree of each image block, obtain an image block fusion degree prediction label, and aggregate all image block fusion degree prediction labels to obtain an initial fusion degree prediction label at the moment; Constructing a fusion loss function according to the initial fusion prediction label; Based on the fusion degree classification model, the fusion degree loss function is minimized to obtain a fusion degree prediction label at the moment.

6. The fusion degree evaluation method according to claim 5, characterized in that: The fusion loss function is: min(L whole-2D-degree (t))=min(ε·{L errors (d (t) ≠d ( ' t) )stL features (p (t) )}+(1-ε)·L patchs-degree-aggr (t)) Among them, L whole-2D-degree (t) represents the value of the fusion loss function, ε represents the weight coefficient, L errors ( ) represents the error rate loss function, d (t) Represents the true label of the fusion degree of all image blocks, d′ (t) represents the predicted labels of the fusion degree of all image blocks, st represents the constraint, and L features ( ) represents the feature loss function, p (t) represents all image blocks at the tth moment, L patchs-degree-aggr (t) represents the fusion aggregation loss function: L patchs-degree-aggr (t)=Probability(D(t)≠D'(t)) Among them, Probability ( ) represents probability, D (t) Denotes the true label of the fusion degree, D′ (t) Represents the initial fusion prediction label.

7. The fusion degree evaluation method according to claim 1, characterized in that: The step of fusing the two-dimensional image and the three-dimensional image at the moment based on the time perception label and the fusion degree prediction label corresponding to the moment to obtain an enhanced stem cell image includes: By formula: Acquiring enhanced images of stem cells in, represents the enhanced two-dimensional image, Represents a two-dimensional projection image: Among them, w 21 、w 22 、w 23 、w 24 Both represent weights, represents the image obtained by geometric transformation of a two-dimensional image, It represents an image obtained by performing variational autoencoding enhancement on the two-dimensional image by taking the time perception label and the fusion degree prediction label as a priori. represents an image obtained by performing generative adversarial network enhancement on the two-dimensional image by taking the time perception label and the fusion degree prediction label as a priori, It indicates that the time perception label and the fusion degree prediction label are used as a priori, and the image is enhanced by the two-dimensional diffusion model on the two-dimensional image. Views indicates the viewing angle of the image enhancement. Views indicates the number of multi-views preset for image enhancement. Projection operator indicates the projection operator. Represents a three-dimensional image, represents the three-dimensional diffusion model, represents the three-dimensional diffusion model generating function, represents a two-dimensional image, and st represents a constraint.

8. The fusion degree evaluation method according to claim 1, characterized in that: The method of performing fusion evaluation based on all microenvironment features, all time perception labels, all fusion prediction labels, all enhanced time perception labels, and all enhanced fusion prediction labels using a fusion evaluation model to obtain a fusion evaluation value at each moment includes: For each of the moments, perform the following steps: Calculate multi-scale feature fusion data based on the microenvironment features corresponding to the moment, the time perception label, the fusion degree prediction label, the enhanced time perception label, and the enhanced fusion degree prediction label; The fusion degree evaluation model is used to evaluate the fusion degree of the multi-scale feature fusion data to obtain a fusion degree evaluation value at the moment.

9. The method for evaluating fusion degree according to claim 8, characterized in that: The calculating of multi-scale feature fusion data based on the microenvironment features corresponding to the moment, the time perception label, the fusion degree prediction label, the enhanced time perception label, and the enhanced fusion degree prediction label includes: By formula: Data fusion (t)=[Data mv (t),Data ota (t)∪Data ata (t),Data od (t)∪Data ad (t)] Calculate multi-scale feature fusion data fusion (t); Among them, Data mv (t) represents the microenvironmental characteristics corresponding to the tth moment, Data ota (t) represents the time-aware label corresponding to the tth moment, Data ata (t) represents the enhanced time-aware label corresponding to the t-th moment, Data od (t) represents the fusion prediction label corresponding to the tth moment, Data ad (t) represents the enhanced fusion prediction label corresponding to the t-th moment, t∈{1,2,,...,T}, and T represents the last moment in the cell expansion culture cycle.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for evaluating the degree of confluence of stem cell expansion culture according to any one of claims 1 to 9 is implemented.

Images