A model training, cell segmentation system, method and storage medium

Abstract

This disclosure provides a model training and cell segmentation system, method, and storage medium. The model training system includes a first processor configured to: acquire a first dataset and a self-supervised training model; augment the first dataset to obtain first augmented data and second augmented data, inputting the first augmented data into a first encoder to obtain first positive sample features, and inputting the second augmented data into a second encoder to obtain second positive sample features; inputting the first positive sample features, the second positive sample features, and negative sample features constructed based on the second positive sample features into a contrastive learning module to obtain a contrastive learning result; and adjusting the model parameters of the self-supervised training model based on the contrastive learning result to obtain a feature extraction model, wherein the feature extraction model is used to implement the segmentation process of at least two cells. The technical solution of this disclosure can accurately segment cells.

Description

Technical Field

[0001] This disclosure relates to the field of machine learning, and more particularly to a model training, cell segmentation system, method, and storage medium. Background Technology

[0002] Space omics technology can simultaneously measure spatial context information and molecular-level information (such as proteins), which helps to understand and characterize molecular differences in complex tissues.

[0003] Based on this, spatial omics data obtained by spatial omics technology is used to segment the spatial microenvironment, that is, to achieve the segmentation process of at least two cells corresponding to the spatial omics data. This helps to discover the structure of the tumor microenvironment, thereby understanding the tumor-immune interaction mechanism and providing assistance for clinical and translational medicine.

[0004] However, the segmentation accuracy achieved by the current cell segmentation schemes needs to be improved. Summary of the Invention

[0005] This disclosure provides a model training, cell segmentation system, method, and storage medium to improve the accuracy of cell segmentation schemes based on a first dataset generated using spatial omics principles.

[0006] According to one aspect of this disclosure, a model training system is provided, which may include:

[0007] The first processor is configured as follows:

[0008] Obtain the first dataset and the self-supervised training model to be trained. The first dataset includes data generated using spatial omics principles for each cell in at least two cells. The self-supervised training model includes a first encoder, a second encoder, and a contrastive learning module.

[0009] The first dataset is augmented to obtain first augmented data and second augmented data. The first augmented data is input into the first encoder to obtain the first positive sample feature, and the second augmented data is input into the second encoder to obtain the second positive sample feature.

[0010] The first positive sample features, the second positive sample features, and the negative sample features constructed based on the second positive sample features are input into the contrastive learning module to obtain the contrastive learning results.

[0011] The model parameters of the self-supervised training model are adjusted based on the results of the contrastive learning to obtain a feature extraction model, which is used to achieve the segmentation process of at least two cells.

[0012] According to another aspect of this disclosure, a cell segmentation system is provided, which may include:

[0013] The second processor is configured as follows:

[0014] Obtain a feature extraction model trained by the model training system provided in any embodiment of this disclosure, and a first dataset used in the training process of the feature extraction model, wherein the first dataset includes data generated using spatial omics principles for each cell in at least two cells;

[0015] The first dataset is input into the feature extraction model to obtain the cell features of each cell, and the cell features of each cell are clustered to obtain the segmentation results of at least two cells.

[0016] According to another aspect of this disclosure, a model training method is provided, which may include:

[0017] Obtain the first dataset and the self-supervised training model to be trained. The first dataset includes data generated using spatial omics principles for each cell in at least two cells. The self-supervised training model includes a first encoder, a second encoder, and a contrastive learning module.

[0018] The first dataset is augmented to obtain first augmented data and second augmented data. The first augmented data is input into the first encoder to obtain the first positive sample feature, and the second augmented data is input into the second encoder to obtain the second positive sample feature.

[0019] The first positive sample features, the second positive sample features, and the negative sample features constructed based on the second positive sample features are input into the contrastive learning module to obtain the contrastive learning results.

[0020] The model parameters of the self-supervised training model are adjusted based on the results of the contrastive learning to obtain a feature extraction model, which is used to achieve the segmentation process of at least two cells.

[0021] According to another aspect of this disclosure, a cell segmentation method is provided, which may include:

[0022] Obtain a feature extraction model trained according to the model training method provided in any embodiment of this disclosure, and a first dataset used in the training process of the feature extraction model, wherein the first dataset includes data generated using spatial omics principles for each cell in at least two cells;

[0023] The first dataset is input into the feature extraction model to obtain the cell features of each cell, and the cell features of each cell are clustered to obtain the segmentation results of at least two cells.

[0024] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions thereon, which can be used to cause a processor to execute the model training method or cell segmentation method provided in any embodiment of this disclosure.

[0025] The technical solution of this disclosure, for the first encoder, second encoder, and contrastive learning module in the self-supervised training model to be trained, involves augmenting the first dataset, then inputting the obtained first augmented data into the first encoder to obtain first positive sample features, and inputting the obtained second augmented data into the second encoder to obtain second positive sample features. These two positive sample features are used to construct positive sample pairs. For negative sample features that can form negative sample pairs with the second positive sample features, the two positive sample features and the negative sample features are input into the contrastive learning module, so that the contrastive learning module performs contrastive learning through positive sample pairs and negative sample pairs to obtain contrastive learning results. Then, based on the contrastive learning results, the model parameters of the self-supervised training model are adjusted to obtain the trained feature extraction model. The above technical solution, through the feature extraction model trained by contrastive learning, can extract deep features from the first dataset. Compared with the original first dataset or the shallow features extracted from the first dataset, these deep features do not contain too much interference information. Therefore, when performing cell segmentation using these deep features, better segmentation accuracy can be obtained.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is an execution flowchart of the first processor in a model training system provided according to an embodiment of the present disclosure;

[0029] Figure 2 This is an execution flowchart of the first processor in another model training system provided according to an embodiment of the present disclosure;

[0030] Figure 3 This is an execution flowchart of the first processor in another model training system provided according to an embodiment of the present disclosure;

[0031] Figure 4 This is an execution flowchart of the first processor in another model training system provided according to an embodiment of the present disclosure;

[0032] Figure 5 This is an execution flowchart of a second processor in a cell segmentation system provided according to an embodiment of the present disclosure;

[0033] Figure 6a This is a schematic diagram illustrating an example of data preprocessing in a cell segmentation system provided according to an embodiment of the present disclosure;

[0034] Figure 6b This is a schematic diagram illustrating an example of data fusion, feature extraction, and encoding in a cell segmentation system provided according to embodiments of this disclosure;

[0035] Figure 6c This is a schematic diagram illustrating a contrastive learning example in a cell segmentation system provided according to an embodiment of the present disclosure;

[0036] Figure 7 This is a schematic diagram of the structure of an electronic device that implements the model training system or cell segmentation system in the embodiments of this disclosure. Detailed Implementation

[0037] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The same applies to "target," "original," etc., and will not be repeated here. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0039] Figure 1This is a flowchart illustrating the steps configured in a first processor within a model training system provided in this embodiment. This embodiment is applicable to situations where a feature extraction model for extracting deep features from a first dataset is trained through self-supervised learning, and is particularly applicable to situations where the aforementioned feature extraction model is trained through contrastive learning. This step can be performed by a model training device, which can be implemented in software and / or hardware, and can be integrated into the first processor.

[0040] See Figure 1 The model training system provided in this embodiment may include a first processor, which may be configured to perform the following steps:

[0041] S110. Obtain the first dataset and the self-supervised training model to be trained. The first dataset includes data generated using spatial omics principles for each cell in at least two cells. The self-supervised training model includes a first encoder, a second encoder, and a contrastive learning module.

[0042] The first dataset may include data generated using spatial omics principles for each cell in at least two cell groups. This data may include at least one of spatial data, protein data, and nucleic acid data. For example, protein data could be spatial proteomics data, characterized by spatial coordinates and protein abundance information. Similarly, nucleic acid data could be in situ sequencing (spatial genomics data), characterized by nucleic acid presence sites and expression levels. In practical applications, in addition to the aforementioned spatial proteomics and spatial genomics data, the data may also include at least one of spatial transcriptomics data, spatial mass spectrometry, spatial metabolomics data, spatial epigenomics, and spatial multi-omics data, without specific limitations.

[0043] For example, protein data can be represented by X∈R. N×D Let X be the first dataset, N be the number of cells in X (at least two), and D be the number of proteins in each cell. In practical applications, optionally, the first dataset may include other data besides protein data, such as the location data (x, y) of each cell, in which case it can be represented by X∈R. N×(D+2) Let D+2 represent the sum of the number of proteins and (x,y).

[0044] It should be noted that many of the following examples use protein data as illustrations, but this is only for illustrative purposes and is not a specific limitation on the first dataset.

[0045] A self-supervised training model can be understood as a deep learning model that requires training to achieve feature extraction. It includes a first encoder, a second encoder, and a contrastive learning module. The specific functions of these three components will be explained in the following steps. Furthermore, a self-supervised training model may also include a first ensemble mapping module and a second ensemble mapping module, and / or a first nonlinear encoding module and a second nonlinear encoding module, etc., without specific limitations here.

[0046] S120. Enhance the first dataset to obtain first enhanced data and second enhanced data. Input the first enhanced data into the first encoder to obtain the first positive sample feature, and input the second enhanced data into the second encoder to obtain the second positive sample feature.

[0047] Since the self-supervised training model needs to be trained based on contrastive learning, the first dataset is augmented to obtain first augmented data and second augmented data that can be used to construct positive sample pairs. Based on this, the first augmented data is input into the first encoder to extract features from it, obtaining the first positive sample features. Similarly, the second augmented data is input into the second encoder to extract features from it, obtaining the second positive sample features. The combination of the first and second positive sample features can be used as positive sample pairs in subsequent contrastive learning.

[0048] S130. Input the first positive sample features, the second positive sample features, and the negative sample features constructed based on the second positive sample features into the contrastive learning module to obtain the contrastive learning results.

[0049] Here, negative sample features can be understood as negative sample features constructed based on the second positive sample features, that is, negative sample features that can be combined with the second positive sample features to form negative sample pairs. On this basis, the first positive sample features, the second positive sample features, and the negative sample features are input into the contrastive learning module, so that the contrastive learning module can perform contrastive learning based on these three features to obtain the corresponding contrastive learning results.

[0050] S140. Adjust the model parameters of the self-supervised training model based on the contrastive learning results to obtain the feature extraction model, wherein the feature extraction model is used to achieve the segmentation process of at least two cells.

[0051] Specifically, the model parameters of the self-supervised training model are adjusted by comparing the learning results. After multiple rounds of comparative learning, a feature extraction model can be obtained, which includes a trained first encoder. Optionally, if the self-supervised training model also includes a first and a second ensemble mapping module, and / or a first and a second nonlinear encoding module, the feature extraction model may also include a trained first ensemble mapping module.

[0052] It should be noted that the feature extraction model obtained by the above-mentioned comparative learning training can extract deep features of the first dataset. Compared with the original first dataset or the shallow features extracted from the first dataset, the deep features do not contain too much interference information. This means that compared with the scheme of applying the first dataset or shallow features for cell segmentation, the present embodiment can better guarantee the segmentation accuracy by applying deep features for cell segmentation.

[0053] The technical solution of this disclosure, for the first encoder, second encoder, and contrastive learning module in the self-supervised training model to be trained, involves augmenting the first dataset, then inputting the obtained first augmented data into the first encoder to obtain first positive sample features, and inputting the obtained second augmented data into the second encoder to obtain second positive sample features. These two positive sample features are used to construct positive sample pairs. For negative sample features that can form negative sample pairs with the second positive sample features, the two positive sample features and the negative sample features are input into the contrastive learning module, so that the contrastive learning module performs contrastive learning through positive sample pairs and negative sample pairs to obtain contrastive learning results. Then, based on the contrastive learning results, the model parameters of the self-supervised training model are adjusted to obtain the trained feature extraction model. The above technical solution, through the feature extraction model trained by contrastive learning, can extract deep features from the first dataset. Compared with the original first dataset or the shallow features extracted from the first dataset, these deep features do not contain too much interference information. Therefore, when performing cell segmentation using these deep features, better segmentation accuracy can be obtained.

[0054] In one alternative technical solution, the second positive sample feature is obtained during the current round of training of the self-supervised training model based on the second augmented data, and the first processor is further configured to:

[0055] Obtain historical sample features obtained during the training of the self-supervised training model based on the second augmented data for historical rounds, wherein the historical rounds occurred before the current round;

[0056] Based on the characteristics of historical samples, negative sample characteristics are constructed.

[0057] To ensure effective model training, multiple rounds of training can be performed based on the first dataset, i.e., iterative training multiple times using the first dataset. Furthermore, if the second positive sample features are obtained during the current round of training of the self-supervised model using the second augmented data, historical sample features obtained during previous rounds of training using the second augmented data can be applied as negative sample features. In practice, the number of historical rounds can be one, two, or more, and correspondingly, the number of negative sample features can be one, two, or more. Given two or more negative sample features, a negative sample queue can be constructed for application. By constructing a negative sample queue and applying a sufficient number of negative sample features, model collapse can be avoided, and GPU resource utilization can be saved.

[0058] The above technical solution constructs negative sample pairs based on the sample features obtained after encoding the second augmented data in different training rounds. The invariance of the second augmented data can guarantee the construction effect of the negative sample pairs, thereby ensuring the model training effect.

[0059] Another alternative technical solution is that the self-supervised training model may include a third branch, a fourth branch, and a contrastive learning module. The third branch includes a first encoder and a first nonlinear encoding module, and the fourth branch includes a second encoder and a second nonlinear encoding module. The first processor is further configured to:

[0060] The first positive sample feature is input into the first nonlinear encoding module to obtain the first nonlinear feature, and the second positive sample feature is input into the second nonlinear encoding module to obtain the second nonlinear feature;

[0061] The first positive sample features, the second positive sample features, and the negative sample features constructed based on the second positive sample features are input into the contrastive learning module to obtain the contrastive learning results, including:

[0062] The first nonlinear feature, the second nonlinear feature, and the negative sample features constructed based on the second nonlinear feature are input into the contrastive learning module to obtain the contrastive learning results.

[0063] The self-supervised training model employs a two-branch structure, with both branches having identical module compositions, each including a nonlinear encoder module and an encoder. Taking the two branches as the third and fourth branches as an example, the third branch includes a first encoder and a first nonlinear encoder module, while the fourth branch includes a second encoder and a second nonlinear encoder module. It should be noted that the first and second nonlinear encoder modules are essentially the same—both are nonlinear encoder modules. Therefore, the different names used here are merely to distinguish the nonlinear encoder modules on the two branches and do not represent a specific limitation on their actual meaning.

[0064] The nonlinear encoding (Head Projection) module can be understood as a module specifically designed for contrastive learning. In practical applications, it can optionally include a first linear mapping unit, a rectified linear unit (RELU), and a second linear mapping unit connected in sequence. Positive sample features extracted by the encoder are assigned to the corresponding Head Projection modules, and the nonlinear features output by the Head Projection modules are then used for contrastive learning training, thereby ensuring the model training effect.

[0065] It should be noted that since the Head Projection module is designed for the comparison task and is very close to the comparison task, while the downstream task (clustering and segmentation task) is a different new task, the Head Projection module, which is close to the comparison task, does not need to be involved in the downstream task. It can be removed when extracting features. That is, the Head Projection module does not need to be included in the trained feature extraction model.

[0066] Figure 2 This is a flowchart illustrating the steps configured in a first processor within another model training system provided in this disclosure embodiment. This embodiment is an optimization based on the aforementioned technical solutions. In this embodiment, optionally, the data includes the location data of each cell; enhancing the first dataset to obtain first enhanced data and second enhanced data includes: mapping the first dataset onto a three-dimensional space based on the location data of each cell to obtain three-dimensional spatial data; enhancing the three-dimensional spatial data to obtain the first enhanced data and second enhanced data. Explanations of terms identical or corresponding to those in the above embodiments are not repeated here.

[0067] See Figure 2 The first processor in this embodiment can be configured to perform the following steps:

[0068] S210. Obtain the first dataset and the self-supervised training model to be trained. The first dataset includes location data generated for each cell in at least two cells using spatial omics principles. The self-supervised training model includes a first encoder, a second encoder, and a contrastive learning module.

[0069] S220. Based on the position data of each cell, map the first dataset onto a three-dimensional space to obtain three-dimensional spatial data.

[0070] To better represent the first dataset, in the modeling process, in addition to utilizing the protein data of each cell, the location data of each cell can also be used. Specifically, based on the location data of each cell, the first dataset is mapped onto a three-dimensional space to obtain three-dimensional spatial data. For example, the protein data of all cells can be filled into a three-dimensional spatial data according to the location data of each cell. In this way, when performing comparative learning through three-dimensional spatial data, both the semantics of spatial omics itself (i.e., gene expression information) and the real spatial information can be learned.

[0071] S230. Enhance the three-dimensional spatial data to obtain first enhanced data and second enhanced data. Input the first enhanced data into the first encoder to obtain first positive sample features, and input the second enhanced data into the second encoder to obtain second positive sample features.

[0072] S240. Input the first positive sample features, the second positive sample features, and the negative sample features constructed based on the second positive sample features into the contrastive learning module to obtain the contrastive learning results.

[0073] S250. Adjust the model parameters of the self-supervised training model based on the contrastive learning results to obtain the feature extraction model, wherein the feature extraction model is used to achieve the segmentation process of at least two cells.

[0074] In the technical solution of this disclosure embodiment, when modeling the first dataset, the deep features extracted by the feature extraction model obtained by combining real spatial information with protein information (i.e. protein data) are more conducive to ensuring the segmentation accuracy of cell segmentation.

[0075] An optional technical solution involves enhancing three-dimensional spatial data to obtain first enhanced data and second enhanced data, including:

[0076] The three-dimensional spatial data is cut into at least two block spatial data.

[0077] For each of the at least two slice space data, the slice space data is augmented to obtain first augmented data and second augmented data.

[0078] When the size of 3D spatial data is relatively large, the augmented data obtained from it cannot be directly input into a self-supervised training model. To address this issue, the 3D spatial data can be segmented into at least two blocks. Since each of these blocks is relatively small (or medium-sized), augmented data can be obtained by augmenting the blocks, allowing this medium-sized augmented data to be input into the self-supervised training model. This technical solution, by segmenting the 3D spatial data, can handle 3D spatial data of arbitrarily large sizes, thus ensuring that large-sized 3D spatial data is trainable.

[0079] Figure 3 This is a flowchart illustrating the steps configured in a first processor within another model training system provided in this disclosure embodiment. This embodiment is optimized based on the aforementioned switching space data. Optionally, in this embodiment, enhancing the chunk space data to obtain first enhanced data and second enhanced data may include: dividing the chunk space data to obtain at least two slice space data; and enhancing the at least two slice space data to obtain the first enhanced data and the second enhanced data. Explanations of terms identical or corresponding to those in the above embodiments will not be repeated here.

[0080] See Figure 3 The first processor in this embodiment can be configured to perform the following steps:

[0081] S310. Obtain the first dataset and the self-supervised training model to be trained. The first dataset includes location data generated for each cell in at least two cells using spatial omics principles. The self-supervised training model includes a first encoder, a second encoder, and a contrastive learning module.

[0082] S320. Based on the position data of each cell, map the first dataset onto a three-dimensional space to obtain three-dimensional spatial data, and cut the three-dimensional spatial data to obtain at least two block spatial data.

[0083] S330. For each of the at least two slice space data, divide the slice space data to obtain at least two slice space data.

[0084] Specifically, for each of the at least two slice spatial data, the slice spatial data can be divided into at least two slice spatial data that can represent local information. In this way, when subsequent enhancements are applied using the at least two slice spatial data, both the global information represented by the slice spatial data (or the global information of all slice spatial data under the slice spatial data) and the local information represented by the slice spatial data can be applied. The combination of global and local information helps improve the model training effect, ensuring that the deep features extracted by the feature extraction model trained in this way can better represent the first dataset. In practical applications, optionally, both the aforementioned global and local information can be understood as semantic information.

[0085] S340. Enhance at least two slice spatial data to obtain first enhanced data and second enhanced data, and input the first enhanced data into the first encoder to obtain first positive sample features, and input the second enhanced data into the second encoder to obtain second positive sample features.

[0086] Specifically, the at least two sliced ​​spatial data obtained from the division can be treated as a whole and enhanced to obtain the first enhanced data and the second enhanced data. In practical applications, enhancement can be performed in various ways. For example, considering the characteristics of the first dataset itself, common enhancement methods such as flipping, stretching, or color adjustment cannot be used. When the data includes protein data, a target dimension in the protein quantity dimension of the at least two sliced ​​spatial data is determined, such as a dimension randomly selected from the protein quantity dimension as the target dimension. Then, the data in the at least two sliced ​​spatial data under the target dimension is perturbed to obtain the first enhanced data and the second enhanced data. Of course, other methods can also be used to enhance the at least two sliced ​​spatial data, which are not limited here.

[0087] Based on this, optionally, considering that the above-mentioned augmented data is relatively sparse, in order to ensure that the interaction within the sparse augmented data can be learned, that is, the correlation between the spatial data of each slice can be learned, the first encoder and / or the second encoder can be implemented based on the Vision Transformer (ViT), that is, ViT is used as the encoder to replace the convolutional structure, so that the correlation between the spatial data of each slice can be learned by utilizing the self-attention mechanism.

[0088] S350. Input the first positive sample features, the second positive sample features, and the negative sample features constructed based on the second positive sample features into the contrastive learning module to obtain the contrastive learning results.

[0089] S360. Adjust the model parameters of the self-supervised training model based on the contrastive learning results to obtain the feature extraction model, wherein the feature extraction model is used to achieve the segmentation process of at least two cells.

[0090] The technical solution of this disclosure divides the block space data to obtain at least two slice space data. When the at least two slice space data are subsequently enhanced and applied, both the global information represented by the block space data and the local information represented by the slice space data can be applied. The combined application of global and local information helps the deep features extracted by the trained feature extraction model to better represent the first dataset.

[0091] An optional technical solution is that the self-supervised training model includes a first branch, a second branch, and a contrastive learning module. The first branch includes a first encoder and a first ensemble mapping module, and the second branch includes a second encoder and a second ensemble mapping module.

[0092] The first processor was also configured as follows:

[0093] Obtain the number of cells contained in each of at least two slice spatial data sets;

[0094] The quantity and the first enhancement data are input into the first integrated mapping module to obtain the first mapping data, and the quantity and the second enhancement data are input into the second integrated mapping module to obtain the second mapping data;

[0095] The first augmented data is input into the first encoder to obtain the first positive sample features, and the second augmented data is input into the second encoder to obtain the second positive sample features, including:

[0096] The first mapping data is input into the first encoder to obtain the first positive sample feature, and the second mapping data is input into the second encoder to obtain the second positive sample feature;

[0097] The feature extraction model also includes a first integrated mapping module that has been trained.

[0098] The self-supervised training model employs a two-branch structure, with both branches having identical module compositions, including an embedding module and an encoder. Taking the two branches as the first and second branches as an example, the first branch includes a first encoder and a first embedding module, while the second branch includes a second encoder and a second embedding module. It should be noted that the first and second embedding modules are essentially the same—both are embedding modules. Therefore, the different names used here are merely to distinguish the embedding modules on the two branches and do not represent a specific limitation on their actual meaning. The aforementioned embedding modules can also be called fusion modules.

[0099] When dividing the block space data, the number of cells contained in each of the at least two resulting block space data may differ; for example, it could be zero, one, or two. Here, the number of cells contained in each block space data can be obtained. At this point, at least two block space data and the number of cells contained in each block space data have been obtained, both of which are used in the model training process. Further, to improve the computational convenience of subsequent feature extraction, the aforementioned quantities and the first augmented data can be input into the first integration mapping module for integration and mapping to obtain the first mapped data. Similarly, the aforementioned quantities and the second augmented data can be input into the second integration mapping module for integration and mapping to obtain the second mapped data. Based on this, the first and second mapped data can be encoded separately for feature extraction. It should be noted that, as mentioned above, the Embedding module's role is to improve the computational convenience of subsequent feature extraction; therefore, the feature extraction model trained in this way can also include the trained first integration mapping module.

[0100] Based on this, optionally, the first integration mapping module can realize data integration and data mapping through the following steps: summing the first augmented data in the long and wide dimensions of space, and dividing the summation result by the quantity to obtain integrated data; obtaining predefined trainable parameters, and performing linear mapping on the integrated data based on the trainable parameters to obtain linearly mapped data; and obtaining the first mapping data based on the integrated data and the linearly mapped data.

[0101] Specifically, the first augmented data is summed along both the long and wide spatial dimensions, and the sum is divided by the number of cells in each cell to obtain the integrated data. In practical applications, optionally, to ensure a uniform data distribution, layer normalization can be performed on the integrated data, and the integrated data can be updated based on the obtained layer normalization results.

[0102] Further, predefined trainable parameters are obtained. Optionally, these trainable parameters may be related to the protein quantity dimension and a pre-set mapping dimension. Then, the integrated data is linearly mapped based on the trainable parameters to obtain linearly mapped data. For example, based on the trainable parameters, a vector of the mapping dimension is assigned to the protein quantity dimension of the integrated data to obtain linearly mapped data. Furthermore, a first mapping data is obtained based on the integrated data and the linearly mapped data. For example, the integrated data and the linearly mapped data are multiplied and averaged to obtain the first mapping data.

[0103] Figure 4 A flowchart illustrating the steps configured in another model training system provided in this disclosure embodiment. This embodiment optimizes the slice space data described above. Optionally, after obtaining the first augmented data and the second augmented data, the first processor is further configured to: determine the global semantic parameters of the slice space data; concatenate the global semantic parameters with the first augmented data to obtain first connection data global semantic parameters; input the first augmented data into the first encoder to obtain first positive sample features, including: inputting the first connection data into the first encoder to obtain first positive sample features; furthermore, optionally, the first processor is further configured to: concatenate the global semantic parameters with the second augmented data to obtain second connection data; input the second augmented data into the second encoder to obtain second positive sample features, including: inputting the second connection data into the second encoder to obtain second positive sample features. Explanations of terms identical or corresponding to those in the above embodiments are not repeated here.

[0104] See Figure 4 The first processor in this embodiment can be configured to perform the following steps:

[0105] S410. Obtain the first dataset and the self-supervised training model to be trained. The first dataset includes location data generated for each cell in at least two cells using spatial omics principles. The self-supervised training model includes a first encoder, a second encoder, and a contrastive learning module.

[0106] S420. Based on the position data of each cell, map the first dataset onto a three-dimensional space to obtain three-dimensional spatial data, and cut the three-dimensional spatial data to obtain at least two block spatial data.

[0107] S430. For each of the at least two slice space data, divide the slice space data to obtain at least two slice space data, and enhance the at least two slice space data to obtain first enhanced data and second enhanced data.

[0108] S440. Determine the global semantic parameters of the segmented spatial data, and connect the global semantic parameters with the first augmented data to obtain the first connected data, and connect the global semantic parameters with the second augmented data to obtain the second connected data.

[0109] The global semantic parameter can be understood as a parameter used to characterize the global semantics of the segmented spatial data. In practical applications, optionally, this global semantic parameter can be represented by a custom [CLS] label; alternatively, this global semantic parameter can be updated during model training. The global semantic parameter is concatenated with the first augmented data to obtain first connected data that simultaneously possesses global and local information; correspondingly, the global semantic parameter is concatenated with the second augmented data to obtain second connected data that simultaneously possesses global and local information.

[0110] S450. Input the first connection data into the first encoder to obtain the first positive sample feature, and input the second connection data into the second encoder to obtain the second positive sample feature.

[0111] Since the first connected data possesses both global and local information, it can be encoded by a first encoder to obtain the first positive sample feature, ensuring that the first positive sample feature also possesses both global and local information. The situation for the second positive sample feature is similar.

[0112] S460. Input the first positive sample features, the second positive sample features, and the negative sample features constructed based on the second positive sample features into the contrastive learning module to obtain the contrastive learning results.

[0113] Since the first positive sample feature, the second positive sample feature, and the negative sample feature all possess both global and local information, comparative learning based on these three types of sample features can be performed both globally (i.e., between block space data) and locally (i.e., between slice space data). In other words, it can consider both the correlation at the level of block space data in the neighborhood space and the correlation at the level of slice space data. The combination of global and local comparative learning helps to improve the model training effect.

[0114] S470. Adjust the model parameters of the self-supervised training model based on the contrastive learning results to obtain the feature extraction model, wherein the feature extraction model is used to achieve the segmentation process of at least two cells.

[0115] The technical solution of this disclosure embodiment connects global semantic parameters with augmented data that can represent local semantics to obtain connected data that simultaneously possesses global and local information. This means that the positive sample features obtained by encoding the connected data also simultaneously possess global and local information. This allows for comparative learning between global and local components in the comparative learning module. The combination of global and local comparative learning helps to improve the model training effect.

[0116] An optional technical solution is that the first positive sample feature includes a first positive global feature corresponding to the global semantic parameters and a first positive local feature corresponding to the first augmented data;

[0117] The second positive sample features include the second positive global features corresponding to the global semantic parameters and the second positive local features corresponding to the second augmented data;

[0118] Negative sample features include negative global features corresponding to the second positive global features and negative local features corresponding to the second positive local features;

[0119] The comparative learning module performs comparative learning through the following steps:

[0120] Global positive sample pairs are constructed based on the first positive global feature and the second positive global feature, and global negative sample pairs are constructed based on the first positive global feature and the negative global feature. Comparative learning is then performed based on the global positive sample pairs and the global negative sample pairs to obtain the global comparative learning loss.

[0121] Local positive sample pairs are constructed based on the first positive local features and the second positive local features, and local negative sample pairs are constructed based on the first positive local features and the negative local features. Comparative learning is then performed based on the local positive sample pairs and the local negative sample pairs to obtain the local comparative learning loss.

[0122] Based on the global contrastive learning loss and the local contrastive learning loss, the target contrastive learning loss is obtained, and the target contrastive learning loss is used as the contrastive learning result.

[0123] In other words, a global contrastive learning loss can be obtained by constructing global positive and negative sample pairs and performing comparative learning based on these pairs. Similarly, a local contrastive learning loss can be obtained by constructing local positive and negative sample pairs and performing comparative learning based on these pairs. Furthermore, the global and local contrastive learning losses are combined to obtain the final target contrastive learning loss. This target contrastive learning loss is then used as the result of the contrastive learning to adjust the model parameters, thus achieving an effective combination of global and local contrastive learning.

[0124] Figure 5This is a flowchart illustrating the steps configured in a second processor within a cell segmentation system provided in this embodiment. This embodiment is applicable to cases where cell segmentation of a first dataset is achieved through self-supervised learning and clustering. This step can be performed by a cell segmentation device, which can be implemented in software and / or hardware. This device can be integrated into a second processor, which may be the same as or different from the first processor mentioned above.

[0125] See Figure 5 The cell segmentation system provided in this embodiment may include a second processor, which may be configured to perform the following steps:

[0126] S510. Obtain the feature extraction model trained by the model training system provided in any embodiment of this disclosure, and the first dataset used in the training process of the feature extraction model, wherein the first dataset includes data generated using spatial omics principles for each cell in at least two cells.

[0127] S520. Input the first dataset into the feature extraction model to obtain the cell features of each cell, and cluster the cell features of each cell to obtain the segmentation results of at least two cells.

[0128] The technical solution of this disclosure embodiment obtains a feature extraction model based on self-supervised training, and extracts the cell features of each cell in the first dataset based on the feature extraction model. Since the training process of the feature extraction model makes the extracted cell features deep features, accurate segmentation of at least two cells can be achieved by clustering the cell features of each cell.

[0129] One optional technical solution is that the data includes the location data of each cell, and the training process applies the location data of each cell;

[0130] The first dataset is input into the feature extraction model to obtain the cell features of each cell. The cell features of each cell are then clustered to obtain segmentation results for at least two cells, including:

[0131] Based on the location data of each cell, the first dataset is mapped to three-dimensional space to obtain three-dimensional spatial data, and the three-dimensional spatial data is input into the feature extraction model to obtain the cell features of each cell;

[0132] For each cell feature in each cell feature, based on the location data corresponding to the cell feature and the location data of each cell, determine the target cell corresponding to the cell feature from at least two cells, and assign the cell feature to the target cell;

[0133] Cluster the cell features of each cell, and based on the clustering results and the allocation results of the cell features of each cell, obtain the segmentation results of at least two cells.

[0134] In the training process of the feature extraction model, when the location data of each cell is involved, for each cell feature among all cell features, the cell feature can be assigned to the corresponding cell (i.e., the target cell) according to the location data corresponding to the cell feature and the location data of each cell. The target cell can be understood as the cell with the cell feature. Furthermore, the cell features of each cell are clustered, and based on the obtained clustering results and the assignment results of the cell features of each cell (i.e. the cells to which each cell feature belongs), the segmentation results of at least two cells are obtained.

[0135] To better understand the various technical solutions described above, specific examples are provided below for illustrative purposes. For example,

[0136] 1) Data preprocessing, see Figure 6a :

[0137] Given protein data X∈R N×(D+2) Where N is the number of cells contained in X, and D is the number of proteins contained in each cell. Figure 6a Taking D=46 as an example, D+2 represents the sum of the number and location data (x,y) of proteins. X is then converted into three-dimensional spatial data X′∈R according to (x,y). H×W×D Here, H is the height after mapping, and W is the width after mapping; these two can be determined based on the maximum value among all (x, y). Then, X′ is cut into n non-overlapping patch data, denoted as P. i ∈R 100×100×D For i = 1, 2, ..., n, P can be... i It is processed as a sample data. P i This is further divided into 100 slices of spatial data (tokens), each with a length and width of 10, denoted as T∈R. 100×10×10×D And record the number of cells contained in each token, denoted as C∈R. 100 C i The number of elements is between 0 and 2, i.e., C i ∈(0,2], i=1,…,100. For T, 0.5% of the dimensions in dimension D (i.e., the protein quantity dimension) are randomly selected for random perturbation to generate the augmented sample corresponding to the token (i.e., the first augmented data Q). Input Second augmented data K Input ).

[0138] 2) Embedding module, see Figure 6b :

[0139] After data preprocessing, we obtain T and its corresponding C, which are input into the Embedding module. First, for T... i Summing is performed along the long and wide dimensions of the space, and then divided by the corresponding C. i Let i = 1, ..., 100. To ensure a uniform data distribution, Layer Normalization will be performed next:

[0140]

[0141] e x =LayerNorm(e x ),

[0142] Among them, e x ∈R 100×D Then, define a trainable parameter of (D, embed_size) as e. x Assign embed_size dimension vectors to the D dimensions and perform embedding operations (table lookup), that is, look up the corresponding vector in the trainable parameters for the index of dimension D:

[0143] e d =Embedding(e x ),

[0144] Among them, e d ∈R 100×D×embed_size Finally, e x With e d Multiply the results, take the average, and use this average as the output of the Embedding module after Layer Normalization:

[0145]

[0146] e = LayerNorm(e),

[0147] Where, e∈R 100×embed_size ( Figure 6b (Taking embed_size=128 as an example).

[0148] 3) ViT module (i.e., encoder), see Figure 6b :

[0149] A custom [CLS] label is assigned to the first position of the output 'e', ​​which represents the global semantic data of the current patch. Then, a corresponding positional code is assigned and added to 'e' as input to the encoder. The encoder here is modeled using ViT(V) to represent the local relationships between tokens within the patch.

[0150] e = Concat([CLS], e),

[0151] X e =V(e),

[0152] Among them, X e ∈R 101×embed_size .

[0153] 4) Head Projection module, see Figure 6b :

[0154] X is projected through the Head Projection (H) module. e For further nonlinear encoding, the Head Projection module can be structured as follows: first a linear mapping, then ReLU, and finally another linear mapping. X... e When input into the Head Projection module, the first position corresponds to the encoding of [CLS], and the remaining positions correspond to the encoding of the token:

[0155] Y CLS =H(X) e (:,0,:)),

[0156] Y tokens =H(X) e (:,1:,:)),

[0157] Among them, Y CLS ∈R d Y tokens ∈R 100×d d represents the dimension output by the Head Projection module. The feature vector representation (i.e., positive sample features) of each cell obtained so far is denoted as Y∈R. N×d :

[0158] Y = f(X′)

[0159] It should be noted that f adopts a two-branch structure, encapsulating the Embedding module, ViT module, and HeadProjection module, respectively denoted as f. Q (Similar to the first and third branches above) with f K(Similar to the second and fourth branches above), the module structures of the two branches are identical. During reverse updates, f Q Perform gradient updates, f K Then only momentum updates will be performed:

[0160] θ k =mθ k +(1-m)θ q m = 0.999

[0161] Where θ represents the model parameters of f.

[0162] Regarding the first augmented data Q described above Input Second augmented data K Input The two augmented data are processed through two separate branches to generate the first positive global feature q of the patch. CLS Second positive global feature k CLS And the first positive local feature q that generates tokens tokens Second positive local feature k tokens :

[0163] q CLS ,q tokens =f Q (Q Input ),

[0164] k CLS ,k tokens =f K (K Input ),

[0165] Where, q CLS, k CLS ∈R d ,q tokens, k tokens ∈R 100×d , q CLS and k CLS As a global positive sample pair, and q tokens and k tokens As local positive sample pairs.

[0166] 5) MoCo module (i.e., contrastive learning module), see [link / reference]. Figure 6c :

[0167] Construct two negative sample queues, denoted as k. queue1 and k queue2 , where k queue1 Used to store the current f K k under the branch CLS q serves as a negative global feature in the next training round. CLS With kqueue1 As a global negative sample pair; k queue2 Used to store the current f K k under the branch tokens q serves as a negative local feature in the next training round. tokens With k queue2 As global positive sample pairs, each queue's enqueue and dequeue operations follow a first-in, first-out (FIFO) principle, meaning that the earliest feature stored in the queue is removed and the current feature is added. Finally, two different levels of comparison tasks are performed: comparison learning based on [CLS] and comparison learning based on tokens.

[0168] This example uses the InfoNCE loss function, which mainly consists of two losses: one is a loss function based on contrastive learning using [CLS]. Another is a loss function that compares tokens. Ultimate target loss Pick and Weighted sum:

[0169]

[0170]

[0171]

[0172] Where τ1 and τ2 represent the values ​​used for adjustment. and Temperature coefficients; k1 and k2 are respectively k queue1 With k queue2 The size; set ε to 0.5.

[0173] 6) Clustering

[0174] After comparative learning, the learned cell features of each cell are clustered and segmented, which is a downstream task. First, the learned token feature vector is fused with the corresponding [CLS] feature vector to obtain the final feature vector of the token; then, the (x,y) corresponding to the token is mapped back to its original position to find the corresponding cell and assign it the final feature vector; finally, KMeans is used for clustering and segmentation.

[0175] To verify the effectiveness of the above example, an experiment was conducted based on CRC data and TNBC data. The experimental process and results are shown below:

[0176] Datasets and preprocessing methods:

[0177] CRC data: Data from 35 patients with advanced colorectal cancer (CRC) acquired by CODEX, including 140 tissue regions and 56 protein markers along with their corresponding location data. The total data contains 238,385 cells; after removing functional protein markers, 46 proteins remain. In the preprocessing stage, the data is divided according to tissue regions and then mapped to 3D space. The data is first divided into 100×100 blocks, then further divided into 10×10 tokens, and finally, data augmentation is performed.

[0178] TNBC data: Data from 41 triple-negative breast cancer patients (TNBC) obtained by MIBI-TOF. First, functional protein markers were removed, leaving 27 protein data and their corresponding location data. After mapping into three-dimensional spatial data, the data was first cut into 128×128 blocks, then divided into 8×8 tokens, and finally data augmentation was performed.

[0179] Parameter settings: The encoding part uses an 8-layer, 8-head Transformer Block, with the embedding layer dimension set to 128 and the head projection layer dimension set to 128. The length of the negative sample queue storing [CLS] is set to 64000, and the length of the negative sample queue storing tokens is set to 65536. τ1 and τ2 are set to 0.07 and 0.03, respectively. The batch size during training is 32, and the learning rate is set to 1e-6. The Adam optimizer is used for parameter updates, and after approximately 100 training epochs, the model with the smaller loss is selected.

[0180] Training process: Observe the decrease in loss and select the optimal model parameters for f and store them.

[0181] Prediction process: Load stored model parameters, select only f Q The module structure and model parameters under the branch are as follows: The Head Projection module is removed, and the output is only the result of the ViT module, namely the feature vector of [CLS] and the feature vector of each token. Next, the encoding vector of the filling position needs to be masked, recording the token's feature vector, its corresponding [CLS] feature vector, (x,y), and the corresponding tissue region encoding. Then, the recorded data is further processed. Based on the mapping relationship between (x,y) and the original cell position, the corresponding cell is found, and the token's feature vector and the corresponding [CLS] feature vector are assigned to it. Finally, the token's feature vector and the [CLS] feature vector are summed in a ratio to obtain the final feature vector. Then, KMeans clustering is performed on the final feature vector to obtain the final segmentation result.

[0182] Experimental Analysis: The above experiments summarize the classic follicle structure and verify that the enriched T cell-macrophage crossover can extract the bounded layout of the tertiary lymphoid structure without supervision, indicating the classic Crohn's-like response behavior.

[0183] This disclosure provides a model training method. This embodiment is applicable to situations where a feature extraction model for extracting deep features from a first dataset is trained through self-supervised learning, and is particularly applicable to situations where the aforementioned feature extraction model is trained through contrastive learning. This method can be executed by a model training device, which can be implemented in software and / or hardware, and can be integrated into the first processor described in any of the above embodiments.

[0184] The method of this disclosure specifically includes the following steps:

[0185] S610. Obtain the first dataset and the self-supervised training model to be trained. The first dataset includes data generated using spatial omics principles for each cell in at least two cells. The self-supervised training model includes a first encoder, a second encoder, and a contrastive learning module.

[0186] S620. Enhance the first dataset to obtain first enhanced data and second enhanced data, and input the first enhanced data into the first encoder to obtain the first positive sample feature, and input the second enhanced data into the second encoder to obtain the second positive sample feature.

[0187] S630. Input the first positive sample features, the second positive sample features, and the negative sample features constructed based on the second positive sample features into the contrastive learning module to obtain the contrastive learning results.

[0188] S640. Adjust the model parameters of the self-supervised training model based on the contrastive learning results to obtain the feature extraction model, wherein the feature extraction model is used to achieve the segmentation process of at least two cells.

[0189] The technical solution of this disclosure, for the first encoder, second encoder, and contrastive learning module in the self-supervised training model to be trained, involves augmenting the first dataset, then inputting the obtained first augmented data into the first encoder to obtain first positive sample features, and inputting the obtained second augmented data into the second encoder to obtain second positive sample features. These two positive sample features are used to construct positive sample pairs. For negative sample features that can form negative sample pairs with the second positive sample features, the two positive sample features and the negative sample features are input into the contrastive learning module, so that the contrastive learning module performs contrastive learning through positive sample pairs and negative sample pairs to obtain contrastive learning results. Then, based on the contrastive learning results, the model parameters of the self-supervised training model are adjusted to obtain the trained feature extraction model. The above technical solution, through the feature extraction model trained by contrastive learning, can extract deep features of a dataset. Compared with the original first dataset or the shallow features extracted from the first dataset, these deep features do not contain too much interference information. Therefore, when performing cell segmentation using these deep features, better segmentation accuracy can be obtained.

[0190] This disclosure provides a cell segmentation method. This embodiment is applicable to situations where cell segmentation of a first dataset is achieved through self-supervised learning and clustering. The method can be executed by a cell segmentation device, which can be implemented in software and / or hardware, and can be integrated into the second processor described in any of the above embodiments.

[0191] The method of this disclosure specifically includes the following steps:

[0192] S710. Obtain a feature extraction model trained according to the model training method provided in any embodiment of this disclosure, and a first dataset used in the training process of the feature extraction model, wherein the first dataset includes data generated using spatial omics principles for each cell in at least two cells.

[0193] S720. Input the first dataset into the feature extraction model to obtain the cell features of each cell, and cluster the cell features of each cell to obtain the segmentation results of at least two cells.

[0194] The technical solution of this disclosure embodiment obtains a feature extraction model based on self-supervised training, and extracts the cell features of each cell in the first dataset based on the feature extraction model. Since the training process of the feature extraction model makes the extracted cell features deep features, accurate segmentation of at least two cells can be achieved by clustering the cell features of each cell.

[0195] As used herein, the term "sequencing instrument" generally refers to the instrument used to determine the sequence of the genetic material of a sample. Sequencing instruments can function in a variety of ways and based on a variety of technologies, including sequencing using labeled or unlabeled nucleotides via primer extension, such as sequencing-while-ligating or pyrosequencing, for example using Sanger dideoxy sequencing, nanopore sequencing, or any of the "NexGen" sequencing methods in the art (e.g., using MGI sequencing platforms, ROCHE 454 sequencing platforms, ILLUMINA™ SOLEXA™ sequencing platforms, LIFE TECHNOLOGIES / APPLIED BIOSYSTEMS' SOLID™ sequencing platforms, PACIFIC BIOSCIENCES' SMRT™ sequencing platforms, POLLONATOR Polony sequencing platforms, COMPLETE GENOMICS sequencing platforms, INTELLIGENT BIOSYSTEMS' sequencing platforms, HELICOS sequencing platforms, or any other sequencing instruments and systems in the art).

[0196] Quantitative real-time PCR (QPCR) is a method that uses fluorescent chemicals to measure the total amount of product after each polymerase chain reaction (PCR) cycle in a DNA amplification reaction. It is a method for quantitative analysis of specific DNA sequences in a sample using internal or external controls.

[0197] Quantitative PCR (qPCR) is a technique built upon immunology, biochemistry, and microscopy. Based on the principle of antigen-antibody reactions, it involves labeling known antigens or antibodies with fluorescent groups, then using this fluorescent antibody (or antigen) as a probe to detect the corresponding antigen (or antibody) within cells or tissues. Fluorescence microscopy allows visualization of the cells or tissues where fluorescence is present, thus determining the nature and location of the antigen or antibody, and enabling quantitative techniques (such as flow cytometry) to measure its concentration. However, it employs fiber optic transmission technology for detection, which suffers from light attenuation, has a long transmission path, and a complex structure.

[0198] The aforementioned model training system or cell segmentation system can be implemented in an electronic device designed to represent various forms of digital computers, such as laptops, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, sequencers, gene sequencing systems, large-scale population genomics platforms, PCR instruments, spatiotemporal omics systems, laboratory automation systems, sample preparation equipment, dispensing equipment, library production equipment, pipetting equipment, magnetic bead detection equipment, nucleic acid purification equipment, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0199] For example, such as Figure 7 The illustrated electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0200] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0201] Processor 11 is a computing unit and can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 can perform the various methods and processes described above, such as model training methods or cell segmentation methods.

[0202] In some embodiments, the model training method or cell segmentation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the model training method or cell segmentation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the model training method or cell segmentation method by any other suitable means (e.g., by means of firmware).

[0203] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0204] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0205] In the context of this disclosure, a computer-readable storage medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be a machine-readable signal medium or a machine-readable storage medium. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0206] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0207] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0208] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0209] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0210] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A model training system, characterized in that, include: The first processor is configured as follows: Obtain a first dataset and a self-supervised training model to be trained, wherein the first dataset includes data generated using spatial omics principles for each cell in at least two cells, and the self-supervised training model includes a first encoder, a second encoder, and a contrastive learning module, wherein the data includes at least one of the location data, protein data, spatial data, and nucleic acid data of each cell; The first dataset is augmented to obtain first augmented data and second augmented data. The first augmented data is input into the first encoder to obtain first positive sample features, and the second augmented data is input into the second encoder to obtain second positive sample features. The first positive sample feature, the second positive sample feature, and the negative sample feature constructed based on the second positive sample feature are input into the contrast learning module to obtain the contrast learning result; The model parameters of the self-supervised training model are adjusted based on the contrastive learning results to obtain a feature extraction model, wherein the feature extraction model includes the trained first encoder and is used to extract cell features of each cell to achieve the segmentation process of the at least two cells.

2. The system according to claim 1, characterized in that, The second positive sample feature is obtained during the current round of training of the self-supervised training model based on the second augmented data; The first processor is also configured to: Obtain historical sample features obtained during the training of the self-supervised training model based on the second augmented data for historical rounds, wherein the historical rounds occurred before the current round; Based on the historical sample features, the negative sample features are constructed.

3. The system according to claim 1, characterized in that, The data generated using spatial omics principles includes the location data of each cell. The augmentation of the first dataset to obtain first augmented data and second augmented data includes: Based on the location data of each cell, the first dataset is mapped to a three-dimensional space to obtain three-dimensional spatial data; The three-dimensional spatial data is enhanced to obtain first enhanced data and second enhanced data.

4. The system according to claim 3, characterized in that, The enhancement of the three-dimensional spatial data to obtain first enhanced data and second enhanced data includes: The three-dimensional spatial data is cut to obtain at least two block spatial data. For each of the at least two slice space data, the slice space data is enhanced to obtain first enhanced data and second enhanced data.

5. The system according to claim 4, characterized in that, The enhancement of the segmented spatial data to obtain first enhanced data and second enhanced data includes: The block space data is divided to obtain at least two slice space data. The at least two slice spatial data are enhanced to obtain first enhanced data and second enhanced data.

6. The system according to claim 5, characterized in that, After obtaining the first enhanced data and the second enhanced data, the first processor is further configured to: Determine the global semantic parameters of the segmented spatial data; The global semantic parameters are concatenated with the first enhanced data to obtain the first concatenated data; The step of inputting the first enhanced data into the first encoder to obtain the first positive sample feature includes: inputting the first connection data into the first encoder to obtain the first positive sample feature.

7. The system according to claim 6, characterized in that: The first positive sample features include a first positive global feature corresponding to the global semantic parameters and a first positive local feature corresponding to the first augmented data; The second positive sample features include a second positive global feature corresponding to the global semantic parameters and a second positive local feature corresponding to the second augmented data; The negative sample features include negative global features corresponding to the second positive global features and negative local features corresponding to the second positive local features; The comparative learning module performs comparative learning through the following steps: A global positive sample pair is constructed based on the first positive global feature and the second positive global feature, and a global negative sample pair is constructed based on the first positive global feature and the negative global feature. A comparative learning is performed based on the global positive sample pair and the global negative sample pair to obtain a global comparative learning loss. Local positive sample pairs are formed based on the first positive local features and the second positive local features, and local negative sample pairs are formed based on the first positive local features and the negative local features. Comparative learning is then performed based on the local positive sample pairs and the local negative sample pairs to obtain a local comparative learning loss. Based on the global contrastive learning loss and the local contrastive learning loss, the target contrastive learning loss is obtained, and the target contrastive learning loss is used as the contrastive learning result.

8. The system according to claim 5, characterized in that, The self-supervised training model includes a first branch, a second branch, and the contrastive learning module. The first branch includes a first encoder and a first integration mapping module, and the second branch includes a second encoder and a second integration mapping module. The first processor is also configured to: Obtain the number of cells contained in each of the at least two slice spatial data; The quantity and the first enhanced data are input into the first integrated mapping module to obtain the first mapping data, and the quantity and the second enhanced data are input into the second integrated mapping module to obtain the second mapping data; The step of inputting the first enhanced data into the first encoder to obtain the first positive sample feature, and inputting the second enhanced data into the second encoder to obtain the second positive sample feature, includes: The first mapping data is input into the first encoder to obtain the first positive sample feature, and the second mapping data is input into the second encoder to obtain the second positive sample feature; The feature extraction model further includes the first integrated mapping module that has been trained.

9. The system according to claim 8, characterized in that, The first integration and mapping module achieves data integration and data mapping through the following steps: The first augmented data is summed in both the long and wide spatial dimensions, and the summation result is divided by the quantity to obtain the integrated data; Obtain predefined trainable parameters, and perform a linear mapping on the integrated data based on the trainable parameters to obtain linearly mapped data; Based on the integrated data and the linear mapping data, the first mapping data is obtained.

10. The system according to claim 5, characterized in that, The data also includes protein data for each cell, and the enhancement of the at least two slice spatial data to obtain first enhanced data and second enhanced data includes: Determine the target dimension in the protein quantity dimension of the at least two slice spatial data; The data in the target dimension of the at least two slice spatial data are perturbed to obtain the first enhanced data and the second enhanced data.

11. The system according to claim 1, characterized in that, The self-supervised training model includes a third branch, a fourth branch, and the contrastive learning module. The third branch includes the first encoder and a first nonlinear coding module, and the fourth branch includes the second encoder and a second nonlinear coding module. The first processor is further configured to: The first positive sample feature is input into the first nonlinear encoding module to obtain the first nonlinear feature, and the second positive sample feature is input into the second nonlinear encoding module to obtain the second nonlinear feature; The step of inputting the first positive sample feature, the second positive sample feature, and the negative sample feature constructed based on the second positive sample feature into the contrastive learning module to obtain the contrastive learning result includes: The first nonlinear feature, the second nonlinear feature, and the negative sample feature constructed based on the second nonlinear feature are input into the contrastive learning module to obtain the contrastive learning result.

12. A cell segmentation system, characterized in that, include: The second processor is configured as follows: Obtain a feature extraction model trained by the model training system according to any one of claims 1-11, and a first dataset used in the training process of the feature extraction model, wherein the first dataset includes data generated using spatial omics principles for each cell in at least two cells. The first dataset is input into the feature extraction model to obtain the cell features of each cell, and the cell features of each cell are clustered to obtain the segmentation results of at least two cells.

13. The system according to claim 12, characterized in that, The data includes the location data of each cell, and the training process applies the location data of each cell. The step of inputting the first dataset into the feature extraction model to obtain the cell features of each cell, and clustering the cell features of each cell to obtain the segmentation results of at least two cells, includes: Based on the location data of each cell, the first dataset is mapped to a three-dimensional space to obtain three-dimensional spatial data, and the three-dimensional spatial data is input into the feature extraction model to obtain the cell features of each cell; For each cell feature in the cell features of each cell, based on the location data corresponding to the cell feature and the location data of each cell, a target cell corresponding to the cell feature is determined from at least two cells, and the cell feature is assigned to the target cell; The cell features of each cell are clustered, and the segmentation results of the at least two cells are obtained based on the clustering results and the allocation results of the cell features of each cell.

14. A model training method, characterized in that, include: Obtain a first dataset and a self-supervised training model to be trained, wherein the first dataset includes data generated using spatial omics principles for each cell in at least two cells, and the self-supervised training model includes a first encoder, a second encoder, and a contrastive learning module, wherein the data includes at least one of the location data, protein data, spatial data, and nucleic acid data of each cell; The first dataset is augmented to obtain first augmented data and second augmented data. The first augmented data is input into the first encoder to obtain first positive sample features, and the second augmented data is input into the second encoder to obtain second positive sample features. The first positive sample feature, the second positive sample feature, and the negative sample feature constructed based on the second positive sample feature are input into the contrast learning module to obtain the contrast learning result; The model parameters of the self-supervised training model are adjusted based on the contrastive learning results to obtain a feature extraction model, wherein the feature extraction model includes the trained first encoder and is used to extract cell features of each cell to achieve the segmentation process of the at least two cells.

15. A cell segmentation method, characterized in that, include: Obtain a feature extraction model trained according to the model training method of claim 14, and a first dataset used in the training process of the feature extraction model, wherein the first dataset includes data generated using spatial omics principles for each cell in at least two cells; The first dataset is input into the feature extraction model to obtain the cell features of each cell, and the cell features of each cell are clustered to obtain the segmentation results of at least two cells.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the model training method of claim 14 or the cell segmentation method of claim 15.

Images