Hierarchical clustering acceleration method, system, device, and medium based on dynamic sampling

Abstract

The application discloses a hierarchical clustering acceleration method, system, device and medium based on dynamic sampling, the method comprises the following steps: obtaining a gene expression dataset to be clustered; constructing an approximate nearest neighbor graph index and a dynamic sampling model according to the gene expression dataset; initializing the clustering state of the gene expression dataset to obtain an initialized cluster set; performing N times of iteration processing based on the initialized cluster set, the approximate nearest neighbor graph index and the dynamic sampling model until the initialized cluster set in the Nth iteration meets a preset iteration condition, and outputting the initialized cluster set in the Nth iteration; and generating a clustering label of each cell sample in the gene expression dataset according to the initialized cluster set in the Nth iteration, wherein the clustering label is used for identifying a cell subpopulation to which each cell sample belongs, and the dynamic sampling and the nearest neighbor index retrieval can be used to reduce the calculation cost, suppress long chain deformity and realize efficient and accurate clustering of the gene expression data.

Description

Technical Field

[0001] This application relates to the interdisciplinary field of bioinformatics and machine learning, and in particular to hierarchical clustering acceleration methods, systems, devices and media based on dynamic sampling. Background Technology

[0002] With the rapid development of single-cell RNA sequencing technology, researchers can acquire massive amounts of cellular gene expression data at single-cell resolution. For this type of data, cluster analysis of different cells is typically required to identify cell types, cell subpopulations, and hierarchical relationships between different cell states. Hierarchical clustering algorithms are widely used for cellular heterogeneity analysis and cell subpopulation relationship analysis in single-cell data because they can construct tree-like cluster structures without pre-specifying the number of clusters.

[0003] In this application scenario, each cell can be considered as a data point to be clustered, and each data point is represented by the cell's gene expression characteristics or a dimension-reduced expression feature vector. Existing hierarchical clustering acceleration methods often narrow down the candidate merging range by approximating nearest neighbor indices. In scenarios with large differences in cell subpopulation density, low proportion of rare cell groups, or continuous transitions in cell states, deterministic greedy merging strategies are prone to selecting unreasonable merging targets, leading to the continuous accumulation of erroneous merging along the hierarchical structure, causing long chain distortion problems, and thus affecting the accuracy of the final cluster tree and cell subpopulation labels, making it difficult to guarantee clustering accuracy with low computational overhead. Summary of the Invention

[0004] The main objective of this disclosure is to propose a hierarchical clustering acceleration method, system, device, and storage medium based on dynamic sampling, which can solve the technical problem of low cell subpopulation labeling accuracy caused by long chain distortion and excessive computational overhead in hierarchical clustering of gene expression data.

[0005] A first aspect of this application provides a hierarchical clustering acceleration method based on dynamic sampling, the method comprising: Obtain the gene expression dataset to be clustered, wherein each data point in the gene expression dataset corresponds to a cell sample, and each data point is represented by the gene expression feature vector of the corresponding cell sample; Based on the gene expression dataset, construct an approximate nearest neighbor graph index and a dynamic sampling model; The gene expression dataset is initialized with cluster state to obtain an initialized cluster set; Based on the initial cluster set, the approximate nearest neighbor graph index, and the dynamic sampling model, perform N iterations until the initial cluster set in the Nth iteration meets the preset iteration conditions, and output the initial cluster set in the Nth iteration; Based on the initial cluster set in the Nth iteration, a clustering label is generated for each cell sample in the gene expression dataset. The clustering label is used to identify the cell subpopulation to which each cell sample belongs. The process of any one of the N iterations is as follows: The initial cluster set is input into the dynamic sampling model for sampling to obtain the initial source cluster set; Based on the approximate nearest neighbor graph index, determine the target cluster for each source cluster in the initial screening source cluster set; Based on each source cluster in the initial screening source cluster set and the target cluster of each source cluster, construct a target merge set; Based on the target merge set, update the dynamic sampling model and the initial cluster set.

[0006] The first aspect of this application provides a hierarchical clustering acceleration method based on dynamic sampling. This method involves: acquiring a gene expression dataset to be clustered; constructing an approximate nearest neighbor graph index and a dynamic sampling model based on the gene expression dataset; initializing the clustering state of the gene expression dataset to obtain an initial cluster set; performing N iterations based on the initial cluster set, the approximate nearest neighbor graph index, and the dynamic sampling model until the initial cluster set in the Nth iteration meets a preset iteration condition, and outputting the initial cluster set in the Nth iteration; and generating clustering labels for each cell sample in the gene expression dataset based on the initial cluster set in the Nth iteration. These clustering labels identify the cell subpopulation to which each cell sample belongs. Through dynamic sampling and nearest neighbor index retrieval, this method reduces computational overhead while suppressing long-chain distortions, achieving efficient and accurate clustering of gene expression data.

[0007] To achieve the above objectives, a second aspect of the present invention provides a hierarchical clustering acceleration system based on dynamic sampling, the system comprising: The acquisition module is used to acquire the gene expression dataset to be clustered, wherein each data point in the gene expression dataset corresponds to a cell sample, and each data point is represented by the gene expression feature vector of the corresponding cell sample; The module is used to construct an approximate nearest neighbor graph index and a dynamic sampling model based on the gene expression dataset; The clustering module is used to initialize the clustering state of the gene expression dataset to obtain an initialized cluster set; The iteration module is used to perform N iterations based on the initial cluster set, the approximate nearest neighbor graph index, and the dynamic sampling model until the initial cluster set in the Nth iteration meets the preset iteration conditions, and then output the initial cluster set in the Nth iteration. The generation module is used to generate clustering labels for each cell sample in the gene expression dataset based on the initial cluster set in the Nth iteration. The clustering labels are used to identify the cell subpopulation to which each cell sample belongs. The process of any one of the N iterations is as follows: The initial cluster set is input into the dynamic sampling model for sampling to obtain the initial source cluster set; Based on the approximate nearest neighbor graph index, determine the target cluster for each source cluster in the initial screening source cluster set; Based on each source cluster in the initial screening source cluster set and the target cluster of each source cluster, construct a target merge set; Based on the target merge set, update the dynamic sampling model and the initial cluster set.

[0008] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: at least one processor and a memory for communicatively connecting to the at least one processor; the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the above-described hierarchical clustering acceleration method based on dynamic sampling.

[0009] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described hierarchical clustering acceleration method based on dynamic sampling.

[0010] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0011] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating a hierarchical clustering acceleration method based on dynamic sampling provided in an embodiment of this application. Figure 2 This is a schematic diagram of the structure of a hierarchical clustering acceleration system based on dynamic sampling provided in an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0013] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0014] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0015] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0016] With the rapid development of single-cell RNA sequencing technology, researchers can acquire large amounts of cellular gene expression data at single-cell resolution. For this type of data, cluster analysis of different cells is typically required to identify cell types, cell subpopulations, and hierarchical relationships between different cell states. Hierarchical clustering algorithms are widely used for cellular heterogeneity analysis and cell subpopulation relationship analysis in single-cell data because they can construct tree-like cluster structures without pre-specifying the number of clusters. In this application scenario, each cell can be considered as a data point to be clustered, and each data point is represented by the cell's gene expression characteristics or a dimension-reduced expression feature vector.

[0017] However, with the continuous increase in the scale of single-cell sequencing, the number of data points to be clustered has increased significantly, and each data point usually has a high feature dimension. Traditional hierarchical clustering algorithms typically need to calculate or maintain the distance relationships between a large number of data points during the clustering process, and continuously update the merging relationships in multiple iterations, resulting in large computational and storage costs, making it difficult to meet the needs of rapid processing of large-scale single-cell data. Existing acceleration schemes usually narrow down the candidate merging range through approximate nearest neighbor graph indexing or local neighborhood search to reduce the global distance calculation volume; however, such schemes are prone to prematurely selecting unsuitable merging objects due to limited local search range or deterministic greedy merging strategies when there are large differences in cell subpopulation density, a small number of rare cell groups, or continuous transitions in cell states. This causes erroneous merging to accumulate in subsequent hierarchical structures, thus affecting the accuracy of the final cluster tree and cell subpopulation labels. Therefore, how to improve the rationality of candidate merging object selection while reducing the computational complexity of hierarchical clustering in the processing of large-scale high-dimensional single-cell data has become an urgent technical problem to be solved.

[0018] Based on this, embodiments of this application provide a hierarchical clustering acceleration method, system, electronic device, and medium based on dynamic sampling. The aim is to iteratively filter source clusters through a dynamic sampling model, quickly retrieve target clusters by combining an approximate nearest neighbor index, and use the merging distance as a reward signal to optimize subsequent sampling. This reduces the global distance calculation overhead while avoiding the accumulation of erroneous merging, effectively suppressing long chain distortions, and achieving high-precision and high-efficiency hierarchical clustering.

[0019] The hierarchical clustering acceleration method, system, electronic device, and medium based on dynamic sampling provided in this application are specifically described through the following embodiments. First, the hierarchical clustering acceleration method based on dynamic sampling in this application embodiment is described.

[0020] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0021] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0022] The hierarchical clustering acceleration method based on dynamic sampling provided in this application relates to the fields of data mining and machine learning technology. This method can be applied to terminals, servers, or software running on either a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the hierarchical clustering acceleration method based on dynamic sampling, but is not limited to the above forms.

[0023] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0024] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0025] Therefore, referring to Figure 1This application provides a hierarchical clustering acceleration method based on dynamic sampling. This method is applied to a central controller, which can be a server, an electronic device, or a mobile terminal, etc. There are no specific limitations here. The method includes the following steps S110 to S150.

[0026] Step S110: Obtain the gene expression dataset to be clustered. Each data point in the gene expression dataset corresponds to a cell sample, and each data point is represented by the gene expression feature vector of the corresponding cell sample.

[0027] The gene expression dataset to be clustered described above can be understood as a collection of gene expression data obtained through single-cell RNA sequencing technology that requires cluster analysis. Each data point in this dataset represents an independent cell sample, and each data point is represented by the gene expression feature vector of that cell sample. The gene expression feature vector is a vector formed by quantifying the expression level of each gene in the cell, with each dimension of the vector corresponding to the expression level of a gene.

[0028] For example, in one specific application scenario, the gene expression dataset to be clustered may include 50,000 cell samples, each cell sample corresponding to the expression levels of 20,000 genes, then each data point is represented by a 20,000-dimensional gene expression feature vector; in another application scenario, the gene expression dataset to be clustered may include 10,000 cell samples, each cell sample corresponding to a 50-dimensional feature vector after dimensionality reduction by principal component analysis, then each data point is represented by a 50-dimensional reduced feature vector.

[0029] Step S120: Based on the gene expression dataset, construct an approximate nearest neighbor graph index and a dynamic sampling model.

[0030] The aforementioned approximate nearest neighbor graph index can be understood as a data structure used to accelerate nearest neighbor retrieval. It organizes the nearest neighbor relationships between data points by constructing a graph model, thus enabling the rapid finding of nearest neighbors without traversing all data points during retrieval. The aforementioned dynamic sampling model can be understood as a model that adaptively adjusts the sampling strategy based on historical feedback information. In each iteration, it samples a portion of clusters from the initial cluster set as source clusters to participate in the merging operation.

[0031] For example, the near nearest neighbor graph index can adopt the hierarchical navigable small world graph index (HNSW), which achieves fast nearest neighbor retrieval by constructing a multi-layer graph structure; the dynamic sampling model can adopt the multi-armed slot machine model, which maintains the rocker arm and its parameter distribution corresponding to each cluster and adaptively adjusts the probability of each cluster being sampled based on the historical reward signal.

[0032] Step S130: Initialize the clustering state of the gene expression dataset to obtain the initialized cluster set.

[0033] The above cluster state initialization can be understood as the process of assigning initial cluster identifiers and cluster state information to each data point in the gene expression dataset before hierarchical clustering begins. The initialized cluster set refers to the set of clusters obtained after initialization, where each cluster contains at least one data point, and each cluster has attributes such as cluster identifier, cluster centroid vector, cluster size, and cluster state.

[0034] For example, in one specific implementation, each data point in the gene expression dataset can be initialized as an independent cluster, so 50,000 data points correspond to 50,000 initialized clusters, each cluster has a cluster size of 1, the cluster centroid vector is the gene expression feature vector of the data point itself, and the cluster state is active; in another implementation, k-means clustering can also be used to pre-cluster the data points, pre-clustering the 50,000 data points into 1,000 clusters, each cluster contains several data points, and the cluster centroid vector is the mean vector of the data points within the cluster.

[0035] Step S140: Perform N iterations based on the initial cluster set, the approximate nearest neighbor graph index, and the dynamic sampling model until the initial cluster set in the Nth iteration meets the preset iteration conditions, and output the initial cluster set in the Nth iteration. The above N iterations can be understood as a repeated clustering and merging process. Each iteration includes operations such as sampling, determining the target cluster, constructing the target merge set, updating the dynamic sampling model, and initializing the cluster set. The preset iteration condition refers to the condition used to determine whether to terminate the iteration process. When the initial cluster set in the Nth iteration meets this condition, the iteration process terminates and the current initial cluster set is output.

[0036] The aforementioned preset iteration conditions include that the number of active clusters in the initial cluster set is less than a preset threshold. The preset threshold can be understood as a pre-set threshold for the number of active clusters used to determine whether to terminate the iteration process. When the number of active clusters drops below this threshold, the iteration process terminates.

[0037] For example, a preset threshold can be set to 100. When the number of active clusters in the initial cluster set drops below 100, the iterative process terminates and enters the relay clustering stage. In another implementation, the preset threshold can be set to 50 or 200, and the specific value can be adjusted according to the data scale and computing resources. The setting of the preset threshold needs to balance the computational cost of the iterative stage and the relay clustering stage. If the preset threshold is too large, the computational cost of the relay clustering stage will be too high, and if the preset threshold is too small, the computational cost of the iterative stage will be too high.

[0038] The aforementioned dynamic sampling model includes the multi-armed slot machine model. The multi-armed slot machine model can be understood as a decision-making model used to weigh exploration against exploitation. This model maintains multiple arms, each corresponding to a cluster. The parameter distribution of each arm is updated through historical reward signals. In each round of sampling, a portion of the clusters corresponding to each arm are sampled as source clusters based on the parameter distribution of each arm.

[0039] For example, a multi-armed slot machine model can employ a Thompson sampling strategy, where each arm corresponds to a Beta distribution. The alpha and beta parameters of the Beta distribution are updated based on historical reward signals. During each round of sampling, a random number is sampled from the Beta distribution of each arm, and the clusters corresponding to the arms with the largest random numbers are selected as the source clusters. In another implementation, the multi-armed slot machine model can also employ an upper confidence bound algorithm or an epsilon greedy strategy.

[0040] In this embodiment, by setting the preset iteration condition to the number of active clusters being less than a preset threshold, the iteration process automatically terminates when the number of active clusters drops to a reasonable range, balancing the computational overhead of the iteration phase and the relay clustering phase. By setting the dynamic sampling model to a multi-armed slot machine model, the adaptive trade-off between exploration and utilization of the multi-armed slot machine model is utilized, enabling the sampling strategy to be dynamically adjusted according to historical reward signals. This avoids the limitations of deterministic greedy merging strategies and improves the rationality of merging object selection.

[0041] In any of the N iterations, the processing is as follows: Step S141: Input the initial cluster set into the dynamic sampling model for sampling to obtain the initial source cluster set.

[0042] The above-mentioned inputting the initial cluster set into the dynamic sampling model can be understood as using all active clusters in the current initial cluster set as input to the dynamic sampling model. The dynamic sampling model then samples a portion of the clusters as source clusters to participate in this round of merging operations based on the historical reward information and parameter distribution of each cluster. The initial source cluster set refers to the set of source clusters obtained after sampling by the dynamic sampling model.

[0043] For example, in one specific implementation, the initial cluster set contains 1,000 active clusters. The dynamic sampling model samples 300 clusters as source clusters based on the Beta distribution parameters of the rocker arms corresponding to each cluster. These 300 source clusters constitute the initial screening source cluster set. In another implementation, the dynamic sampling model can sort the expected reward values ​​of the rocker arms corresponding to each cluster and select the 200 clusters with the highest expected reward values ​​as source clusters.

[0044] Step S142: Determine the target cluster for each source cluster in the initial screening source cluster set based on the approximate nearest neighbor graph index.

[0045] The above-mentioned determination of the target cluster for each source cluster in the initial screening source cluster set can be understood as follows: for each source cluster in the initial screening source cluster set, its nearest neighbor clusters are retrieved using an approximate nearest neighbor graph index, and the most suitable cluster is selected from the nearest neighbor clusters as the merging object of that source cluster, i.e., the target cluster. The target cluster refers to the cluster that is merged with the source cluster.

[0046] For example, for source cluster A in the initial screening source cluster set, its nearest neighbor clusters are retrieved by the approximate nearest neighbor graph index to obtain nearest neighbor clusters B, C and D. Based on features such as cluster weight and cluster centroid vector, nearest neighbor cluster B is selected as the target cluster of source cluster A.

[0047] Step S143: Construct a target merge set based on each source cluster and its target cluster in the initial screening source cluster set.

[0048] The construction of the target merge set can be understood as forming merge pairs by combining each source cluster in the initial source cluster set with its corresponding target cluster, and then filtering and removing anomalies from these merge pairs to obtain the final set of merge pairs. Each merge pair in the target merge set contains one source cluster and one target cluster, and the actual merge operation will be performed in subsequent steps.

[0049] For example, the initial screening source cluster set contains 300 source clusters, each source cluster corresponds to a target cluster, forming 300 candidate merging pairs; after weight ratio constraint screening and anomaly removal, the remaining 250 merging pairs constitute the target merging set.

[0050] Step S144: Update the dynamic sampling model and initialize the cluster set based on the target merge set.

[0051] The above-mentioned updating of the dynamic sampling model and initialization of the cluster set can be understood as calculating the reward signal based on the merging results of each merging pair in the target merging set and feeding it back to the dynamic sampling model to adjust its parameter distribution. At the same time, the merged new cluster is added to the initialization cluster set and the merged source cluster and target cluster are marked as inactive, thereby completing the update of the initialization cluster set.

[0052] For example, the target merge set contains 250 merge pairs. After the merge operation is performed, 250 new clusters are obtained. The reward signal is calculated based on the merge distance of each merge pair and fed back to the dynamic sampling model to update the Beta distribution parameters of each rocker arm. At the same time, the 250 new clusters are added to the initial cluster set and the corresponding 500 original clusters are marked as inactive.

[0053] Step S150: Based on the initial cluster set in the Nth iteration, generate cluster labels for each cell sample in the gene expression dataset. The cluster labels are used to identify the cell subpopulation to which each cell sample belongs.

[0054] The clustering labels mentioned above can be understood as identification information used to identify the cell subpopulation to which each cell sample belongs. Generating clustering labels based on the initial cluster set in the Nth iteration refers to performing global hierarchical clustering and merging on the remaining active clusters after the iteration terminates, constructing a complete clustering tree, and assigning a corresponding subpopulation label to each cell sample according to the structure of the clustering tree.

[0055] For example, after the iteration terminates, 100 active clusters remain. Global hierarchical clustering is performed on these 100 active clusters to construct a clustering tree. Based on the cluster division results of the clustering tree at a certain cutting level, 50,000 cell samples are assigned to different cell subpopulations, such as T cell subpopulation, B cell subpopulation, NK cell subpopulation, etc.

[0056] In this embodiment, by acquiring the gene expression dataset to be clustered, an approximate nearest neighbor graph index and a dynamic sampling model are constructed. The gene expression dataset is then initialized with clustering states to obtain an initial cluster set. Based on the initial cluster set, the approximate nearest neighbor graph index, and the dynamic sampling model, N iterations are performed until a preset iteration condition is met. Clustering labels are generated based on the initial cluster set in the Nth iteration, thereby achieving rapid hierarchical clustering of large-scale high-dimensional single-cell gene expression data. Specifically, the approximate nearest neighbor graph index accelerates nearest neighbor retrieval, reducing the computational complexity of each iteration. The dynamic sampling model adaptively adjusts the sampling strategy based on historical reward signals, avoiding the problem of deterministic greedy merging strategies selecting unreasonable merging objects in scenarios with large differences in cell subpopulation density, low proportion of rare cell groups, or continuous transitions in cell states. This reduces the continuous accumulation of erroneous merging along the hierarchical structure, alleviates the long chain distortion problem, and improves the accuracy of the final cluster tree and cell subpopulation labels. Therefore, the method provided in this embodiment solves the technical problems of high computational complexity and unreasonable selection of candidate merging objects in the processing of large-scale high-dimensional single-cell data proposed in the background art, ensuring clustering accuracy with low computational overhead.

[0057] In one embodiment, step S120 may include the following steps S121 to S125.

[0058] Step S121: Extract the total number of cell samples and gene feature dimensions from the gene expression dataset.

[0059] The total number of cell samples mentioned above can be understood as the number of cell samples contained in the gene expression dataset. The gene feature dimension mentioned above can be understood as the dimension of the gene expression feature vector of each cell sample, that is, the number of gene expression values ​​contained in the vector.

[0060] For example, the gene expression dataset contains 50,000 cell samples, and the gene expression feature vector of each cell sample is reduced to 50 dimensions after principal component analysis. In this case, the total number of cell samples is 50,000 and the gene feature dimension is 50. In another embodiment, the gene expression dataset contains 10,000 cell samples, and the gene expression feature vector of each cell sample is 20,000 dimensions (without dimensionality reduction). In this case, the total number of cell samples is 10,000 and the gene feature dimension is 20,000.

[0061] Step S122: Preprocess the gene expression dataset to obtain the set of gene feature vectors corresponding to each cell sample in the gene expression dataset.

[0062] The preprocessing described above can be understood as cleaning, normalizing, and reducing the dimensionality of the original gene expression data to obtain a set of gene feature vectors suitable for constructing an approximate nearest neighbor graph index. The set of gene feature vectors refers to the collection of gene feature vectors from all cell samples after preprocessing.

[0063] For example, preprocessing may include the following operations: first, quality control of the original gene expression matrix is ​​performed to remove cell samples with excessively high mitochondrial gene expression; then, normalization methods such as SCRAN are used to normalize the expression levels to eliminate differences in cell sequencing depth; next, dimensionality reduction methods such as principal component analysis or variational autoencoders are used to reduce the high-dimensional expression vector to 50 dimensions; finally, the dimensionality-reduced feature vectors of all cell samples are used to construct a set of gene feature vectors.

[0064] Step S123: Based on the total number of cell samples, gene feature dimensions, and gene feature vector set, construct an initial hierarchical navigable small-world graph index model.

[0065] The aforementioned initial hierarchical navigable small-world graph index model can be understood as the initial framework for a hierarchical navigable small-world graph (HNSW) index. This framework determines the index's structural parameters based on the total number of cell samples and gene feature dimensions, including the number of layers, the maximum number of connections per layer, and the size of the candidate neighbor list. The hierarchical navigable small-world graph index model achieves fast nearest neighbor retrieval by constructing a multi-layered graph structure. The upper-layer graph has sparse nodes for quickly locating the retrieval region, while the lower-layer graph has dense nodes for precise retrieval.

[0066] For example, when the total number of cell samples is 50,000 and the gene feature dimension is 50, the initial hierarchical navigable small-world graph index model can be set to a 3-layer graph structure, the maximum number of connections M is set to 16, the candidate neighbor list size efConstruction is set to 200, and the node sampling probability of the upper graph is set to 0.1; in another embodiment, when the total number of cell samples is 10,000, it can be set to a 2-layer graph structure, and the maximum number of connections M is set to 12.

[0067] Step S124: Add all cell samples from the gene feature vector set to the initial hierarchical navigable small-world graph index model to obtain an approximate nearest neighbor graph index.

[0068] The above process of adding all cell samples to the initial hierarchical navigable small-world graph index model can be understood as inserting each cell sample in the gene feature vector set as a node into the initial hierarchical navigable small-world graph index model, and establishing a connection relationship between each node and its nearest neighbor nodes, thereby constructing a complete approximate nearest neighbor graph index.

[0069] For example, 50,000 cell samples are inserted one by one into the initial hierarchical navigable small-world graph index model. When inserting each cell sample, the search starts from the highest level and establishes connections with its nearest neighbor nodes, and so on down the layers until the bottom level. Finally, after all 50,000 cell samples are inserted, a complete approximate nearest neighbor graph index is obtained. During retrieval, given a query vector, the search starts from the entry node of the highest level and greedily searches for nearest neighbors, refining the search down the layers until a preset number of nearest neighbor cell samples are found at the bottom level.

[0070] Step S125: Construct a dynamic sampling model based on the total number of cell samples.

[0071] The above-mentioned dynamic sampling model based on the total number of cell samples can be understood as determining the number of rocker arms and initial parameters in the dynamic sampling model according to the total number of cell samples. Each cell sample or each initial cluster corresponds to one rocker arm, and the initial parameters of the rocker arm are set according to prior information.

[0072] For example, when the total number of cell samples is 50,000 and each cell sample is initialized as a cluster, the dynamic sampling model contains 50,000 rocker arms, and the Beta distribution parameters alpha and beta of each rocker arm are initialized to 1, indicating a uniform distribution; in another embodiment, when the 50,000 cell samples are clustered into 1,000 clusters using k-means pre-clustering, the dynamic sampling model contains 1,000 rocker arms, and the Beta distribution parameters alpha and beta of each rocker arm are initialized to 1.

[0073] In this embodiment, the gene expression dataset is preprocessed by extracting the total number of cell samples and the gene feature dimension to obtain a set of gene feature vectors. An initial hierarchical navigable small-world graph index model is constructed based on the total number of cell samples, the gene feature dimension, and the set of gene feature vectors. All cell samples are added to this initial hierarchical navigable small-world graph index model to obtain an approximate nearest neighbor graph index. A dynamic sampling model is then constructed based on the total number of cell samples, thus achieving automated construction of the approximate nearest neighbor graph index and the dynamic sampling model. Specifically, the hierarchical navigable small-world graph index model achieves logarithmic-level retrieval complexity through a multi-layered graph structure, significantly reducing the computational overhead of nearest neighbor retrieval. The dynamic sampling model adaptively determines the number of rocker arms based on the total number of cell samples, providing a foundation for dynamic sampling in subsequent iterative processing.

[0074] In one embodiment, step S150 may include the following steps S151 to S153.

[0075] Step S151: Extract the centroid vectors of all active clusters in the initial cluster set during the Nth iteration to form a relay feature matrix.

[0076] The aforementioned active clusters can be understood as those that remain active and have not been merged after the Nth iteration. The cluster centroid vector can be understood as the mean vector of the gene expression feature vectors of all data points within the cluster, used to characterize the overall features of the cluster. The relay feature matrix can be understood as a matrix formed by arranging the centroid vectors of all active clusters row-wise, with each row corresponding to the centroid vector of one active cluster.

[0077] For example, after the Nth iteration, there are 100 active clusters remaining, each with a 50-dimensional centroid vector. The 100 cluster centroid vectors are arranged in rows to form a 100-row, 50-column relay feature matrix. In another embodiment, after the Nth iteration, there are 50 active clusters remaining, each with a 30-dimensional centroid vector. The relay feature matrix is ​​then 50-row, 30-column.

[0078] Step S152: Perform global hierarchical clustering and merging on the force feature matrix to obtain the global hierarchical clustering and merging result.

[0079] The aforementioned global hierarchical clustering merging can be understood as performing traditional global hierarchical clustering algorithms, such as Ward's method, single-linked method, or fully linked method, on all active clusters in the force feature matrix, gradually merging all active clusters until a root cluster is formed, constructing a complete clustering tree. The result of global hierarchical clustering merging refers to the clustering tree structure obtained after global hierarchical clustering.

[0080] For example, the Ward method is used to perform global hierarchical clustering merging on 100 active clusters in the force feature matrix. In each iteration, the Ward distance between all cluster pairs is calculated, and the cluster pair with the smallest Ward distance is selected for merging. After repeating the merging operation 99 times, all clusters are merged into a root cluster to obtain a complete cluster tree. In another embodiment, the fully linked method can also be used to perform global hierarchical clustering merging.

[0081] Step S153: Based on the global hierarchical clustering and merging results, generate clustering labels for each cell sample in the gene expression dataset.

[0082] The above-mentioned generation of cluster labels based on the global hierarchical clustering and merging results can be understood as dividing the clustering tree into several subtrees at a certain cutting level according to the structure of the clustering tree. Each subtree corresponds to a cell subpopulation. Then, each cell sample is assigned to the corresponding cell subpopulation according to its active cluster, thereby assigning a cluster label to each cell sample.

[0083] For example, based on the structure of the clustering tree, the clustering tree is divided into 10 subtrees at the cutting level of 10. Each subtree corresponds to a cell subpopulation, such as T helper cell subpopulation, T killer cell subpopulation, B cell subpopulation, NK cell subpopulation, monocyte subpopulation, etc. Then, 50,000 cell samples are assigned to the corresponding subpopulation according to their active clusters, and each cell sample obtains a clustering label.

[0084] In this embodiment, a relay feature matrix is ​​formed by extracting the centroid vectors of all active clusters in the Nth iteration. Global hierarchical clustering merging is then performed on the relay feature matrix to obtain the global hierarchical clustering merging result. Based on the global hierarchical clustering merging result, cluster labels for each cell sample are generated, thus realizing the relay construction from the active clusters after the iteration terminates to the complete cluster tree. Specifically, the relay feature matrix uses the active clusters retained during the dynamic sampling iteration phase as the relay starting point, avoiding the high computational overhead of performing global hierarchical clustering from scratch. The global hierarchical clustering merging operation is performed when the number of active clusters is small, keeping the computational complexity controllable and ensuring the integrity and accuracy of the cluster tree.

[0085] In one embodiment, step S142 may include steps S1421 to S1425.

[0086] Step S1421: Based on the centroid vector of each source cluster in the initial screening source cluster set, retrieve a preset number of neighboring cell samples for each source cluster in the approximate nearest neighbor graph index.

[0087] The cluster centroid vector mentioned above can be understood as the mean vector of gene expression feature vectors of all data points within the source cluster, used to characterize the overall features of the source cluster. The preset quantity mentioned above can be understood as the pre-defined number of nearest neighbor cell samples returned for each retrieval. The nearest neighbor cell samples mentioned above can be understood as the several cell samples in the approximate nearest neighbor graph index that are closest to the cluster centroid vector of the source cluster.

[0088] For example, the preset number can be set to 50. For source cluster A in the initial screening source cluster set, its cluster centroid vector is used as the query vector, and 50 nearest neighbor cell samples are retrieved in the approximate nearest neighbor graph index. The Euclidean distance or cosine distance between these 50 nearest neighbor cell samples and the cluster centroid vector of source cluster A is the smallest. In another embodiment, the preset number can be set to 100 or 200.

[0089] Step S1422: Based on each neighboring cell sample, determine the initial neighboring cluster corresponding to each source cluster.

[0090] The initial nearest neighbor cluster mentioned above can be understood as a set of clusters adjacent to the source cluster, determined based on the clusters to which the retrieved nearest cell samples belong. Each nearest cell sample belongs to one cluster, and the initial nearest neighbor cluster is formed by removing duplicates from the clusters to which all nearest cell samples belong.

[0091] For example, for source cluster A, if 50 neighboring cell samples are retrieved, and these 50 neighboring cell samples belong to clusters B, C, D, E and F respectively, then the initial neighboring clusters include clusters B, C, D, E and F; in another embodiment, if all 50 neighboring cell samples belong to cluster B, then the initial neighboring clusters only include cluster B.

[0092] Step S1423: Remove the clusters that belong to the same cluster as the source cluster from each initial nearest neighbor cluster to obtain the candidate nearest neighbor cluster set of the corresponding source cluster.

[0093] The clusters that belong to the same cluster as the source cluster can be understood as those clusters in the initial nearest neighbor clusters that belong to the same cluster as the source cluster. Since a cluster cannot be merged with itself, these clusters need to be removed from the initial nearest neighbor clusters. The candidate nearest neighbor cluster set can be understood as the set of nearest neighbor clusters remaining after removing the clusters that belong to the same cluster as the source cluster.

[0094] For example, the initial nearest neighbor clusters of source cluster A include clusters B, C, D, E, and F, where cluster D is the same as source cluster A (i.e., cluster D is cluster A). Then, cluster D is removed from the initial nearest neighbor clusters, resulting in a candidate nearest neighbor cluster set including clusters B, C, E, and F. In another embodiment, if there is no cluster in the initial nearest neighbor clusters that belongs to the same cluster as the source cluster, then the candidate nearest neighbor cluster set is the same as the initial nearest neighbor cluster set.

[0095] Step S1424: Based on the cluster weight of each source cluster and the cluster weight of each nearest neighbor cluster in the candidate nearest neighbor cluster set of each source cluster, determine the candidate target cluster of the corresponding source cluster in the candidate nearest neighbor cluster set.

[0096] The cluster weights mentioned above can be understood as a measure of cluster size or the number of data points, used to characterize the size of the cluster. The candidate target clusters mentioned above can be understood as nearest neighbor clusters selected from the candidate nearest neighbor cluster set based on cluster weights, satisfying the weight ratio constraint. The weight ratio constraint means that the ratio of the cluster weights of the source cluster and the nearest neighbor clusters does not exceed a preset ratio threshold, in order to avoid merging clusters with excessively large size differences.

[0097] For example, the cluster weight of source cluster A is 100, and the cluster weights of candidate nearest neighbor clusters are 80 for cluster B, 150 for cluster C, 50 for cluster E, and 300 for cluster F. The preset ratio threshold is 3. Then, cluster B (weight ratio 100:80 = 1.25), cluster C (weight ratio 100:150 = 0.67), and cluster E (weight ratio 100:50 = 2) satisfy the weight ratio constraint, while cluster F (weight ratio 100:300 = 0.33, i.e., the target cluster is 3 times the size of the source cluster) does not satisfy the constraint. Therefore, the candidate target clusters include cluster B, cluster C, and cluster E. In another embodiment, the preset ratio threshold can be set to 5 or 10.

[0098] Step S1425: Determine the target cluster of each source cluster in the initial screening source cluster set based on the cluster centroid vector of each source cluster and the cluster centroid vector of the candidate target cluster of each source cluster.

[0099] The above method of determining the target cluster based on the cluster centroid vector can be understood as calculating the centroid vector distance between the source cluster and each candidate target cluster, and selecting the candidate target cluster with the smallest distance as the target cluster of the source cluster. The centroid vector distance can be measured using Euclidean distance, cosine distance, or Mahalanobis distance, etc.

[0100] For example, the candidate target clusters of source cluster A include clusters B, C, and E. The Euclidean distances between the centroid vector of source cluster A and the centroid vectors of clusters B, C, and E are calculated to be 0.3, 0.5, and 0.4, respectively. Among them, the distance of cluster B is the smallest, so cluster B is determined as the target cluster of source cluster A. In another embodiment, cosine distance can also be used as a metric.

[0101] In this embodiment, by retrieving neighboring cell samples from the approximate nearest neighbor graph index based on the centroid vector of the source cluster, an initial nearest neighbor cluster is determined based on the neighboring cell samples. Clusters belonging to the same cluster as the source cluster are removed to obtain a candidate nearest neighbor cluster set. The candidate target cluster is determined from the candidate nearest neighbor cluster set based on the cluster weight, and the target cluster is determined based on the cluster centroid vector, thus achieving accurate determination of the target cluster. Specifically, the retrieval operation of the approximate nearest neighbor graph index avoids the high computational overhead of global traversal; the weight ratio constraint avoids merging clusters with excessively large size differences, reducing unreasonable merging problems caused by size imbalance; and the selection of the cluster centroid vector distance ensures the similarity between the target cluster and the source cluster in the feature space, improving the rationality of the merging operation.

[0102] In one embodiment, step S143 may include the following steps S1431 to S1433.

[0103] Step S1431: Based on each source cluster in the initial screening source cluster set and the target cluster of the corresponding source cluster, construct the corresponding candidate merging pair to obtain the candidate merging pair set.

[0104] The aforementioned candidate merge pairs can be understood as a binary tuple consisting of a source cluster and its corresponding target cluster, indicating a merge intention between the source cluster and the target cluster. The aforementioned set of candidate merge pairs can be understood as the set of all candidate merge pairs consisting of all source clusters and their corresponding target clusters in the initial screening source cluster set.

[0105] For example, if there are 300 source clusters in the initial screening source cluster set, and each source cluster corresponds to a target cluster, then 300 candidate merging pairs are constructed, such as (source cluster A, target cluster B), (source cluster C, target cluster D), etc. These 300 candidate merging pairs constitute the candidate merging pair set.

[0106] Step S1432: Select candidate merging pairs that meet the preset screening conditions from the candidate merging pair set to obtain the candidate merging pair set.

[0107] The aforementioned preset screening criteria can be understood as conditions used to further screen candidate merge pairs, including weight ratio constraints, etc. The aforementioned set of candidate merge pairs can be understood as a subset of candidate merge pairs in the candidate merge pair set that meet the preset screening criteria.

[0108] For example, the preset screening conditions include a weight ratio constraint, that is, the ratio of the cluster weights of the source cluster and the target cluster is within a preset ratio threshold range. If there are 300 candidate merge pairs in the candidate merge pair set, and 250 of them satisfy the weight ratio constraint, then these 250 candidate merge pairs constitute the set of candidate merge pairs. In another embodiment, the preset screening conditions may also include a merge distance threshold constraint, that is, the distance between the cluster centroid vectors of the source cluster and the target cluster is less than a preset distance threshold.

[0109] Step S1433: Remove anomalies from the set to be merged to obtain the target merge set.

[0110] The aforementioned anomaly removal can be understood as the operation of removing merge pairs with anomalies from the set of candidate merge pairs. Anomalies include merge pairs with excessively large merge distances and merge pairs with conflicts. The aforementioned target merge set can be understood as the set of merge pairs retained after anomaly removal, and the merge pairs in this set will be subject to the actual merge operation in subsequent steps.

[0111] The process of removing anomalies from the set to be merged yields the target merge set, which includes: If at least two groups of candidate merge pairs contain the same active cluster, the candidate merge pairs that meet the preset elimination conditions in the at least two groups of candidate merge pairs are eliminated to obtain the target merge set.

[0112] The aforementioned "same active cluster" can be understood as an active cluster that appears in at least two sets of candidate merge pairs, meaning that the active cluster is selected as the target cluster by multiple source clusters or appears as both a source cluster and a target cluster in multiple merge pairs. The aforementioned preset elimination criteria can be understood as conditions used to determine which conflicting merge pairs to eliminate, such as retaining merge pairs with smaller merge distances and eliminating merge pairs with larger merge distances.

[0113] For example, in the set of candidate merge pairs, there are two merge pairs: (source cluster A, target cluster B) and (source cluster C, target cluster B). Target cluster B appears in both merge pairs, causing a conflict. Based on the preset elimination criteria, the merge distances of the two merge pairs are compared. Assuming the merge distance of (source cluster A, target cluster B) is 0.3 and the merge distance of (source cluster C, target cluster B) is 0.5, then (source cluster A, target cluster B) with the smaller merge distance is retained, and (source cluster C, target cluster B) is eliminated. Source cluster C needs to reselect a target cluster or abandon this round of merging.

[0114] In another embodiment, anomaly removal may also include merge distance anomaly removal, which involves calculating the statistical characteristics (such as mean and standard deviation) of the merge distances of all merge pairs in the candidate merge pair set, and identifying merge pairs whose merge distances are greater than the mean plus a certain multiple of the standard deviation as anomalies and removing them. For example, if the mean merge distance of all merge pairs in the candidate merge pair set is 0.2 and the standard deviation is 0.1, then merge pairs whose merge distances are greater than 0.2 plus 3 multiplied by 0.1 equals 0.5 are identified as anomalies and removed.

[0115] In this embodiment, a candidate merge pair set is obtained by constructing candidate merge pairs based on each source cluster and its corresponding target cluster in the initial screening source cluster set. Candidate merge pairs that meet preset screening conditions are then selected from this set to obtain a set of candidate merge pairs to be merged. Anomaly removal is then performed on the candidate merge pairs to obtain the target merge set, thus achieving precise construction of the target merge set. Specifically, the preset screening conditions, through weight ratio constraints, avoid merging cluster pairs with excessively large differences in size. The anomaly removal operation, through conflict resolution, ensures that each active cluster participates in at most one merge operation in each iteration, avoiding cluster state chaos caused by repeated merging of the same cluster. Simultaneously, anomaly removal through merge distance removes abnormal merge pairs with excessively high merge costs, ensuring the rationality and correctness of the merge operation.

[0116] In one embodiment, step S144 may include steps S1441 to S1444.

[0117] Step S1441: Merge the target clusters in the target merge set into the source clusters in the target merge set to obtain the final merged clusters.

[0118] The above process of merging the target cluster into the source cluster can be understood as incorporating all data points from the target cluster into the source cluster, updating the cluster centroid vector and cluster size of the source cluster, and marking the target cluster as inactive. The final merged cluster can be understood as the new cluster obtained after the merge operation, which contains all data points from both the source and target clusters.

[0119] For example, if there is a merge pair (source cluster A, target cluster B) in the target merge set, the cluster size of source cluster A is 100 and the cluster centroid vector is v_A, and the cluster size of target cluster B is 80 and the cluster centroid vector is v_B, then the final merged cluster size is 100 plus 80 equals 180, and the cluster centroid vector is (100 multiplied by v_A plus 80 multiplied by v_B) divided by 180. Target cluster B is marked as inactive.

[0120] Step S1442: Use the merging distance between the source cluster and the target cluster in the target merge set as a reward signal and feed it back to the dynamic sampling model to update the dynamic sampling model.

[0121] The aforementioned merging distance can be understood as the distance between the centroid vectors of the source and target clusters, used to characterize the cost of the merging operation. The aforementioned reward signal can be understood as a signal fed back to the dynamic sampling model, used to adjust the parameter distribution of each arm. The aforementioned updating of the dynamic sampling model can be understood as adjusting the Beta distribution parameters of each arm according to the reward signal, increasing the probability of arms generating high rewards being selected in subsequent iterations, and decreasing the probability of arms generating low rewards being selected in subsequent iterations.

[0122] For example, if the rocker arm corresponding to source cluster A produces a high reward (reward value greater than 1) after merging, then the alpha parameter of its Beta distribution is incremented by 1, increasing the probability of source cluster A being selected in subsequent iterations; if the rocker arm corresponding to source cluster C produces a low reward (reward value less than 1) after merging, then the beta parameter of its Beta distribution is incremented by 1, decreasing the probability of source cluster C being selected in subsequent iterations. The reward signal can be calculated as the ratio of the compactness of the new cluster after merging to the sum of the compactnesses of the two clusters before merging. Compactness can be measured by the average distance between cluster members and the centroid; the smaller the average distance, the higher the compactness.

[0123] Step S1443: Extract the clusters from the initial cluster set excluding those in the initial screening source cluster set to obtain the first cluster set.

[0124] The first cluster set mentioned above can be understood as the set of clusters in the initial cluster set other than the initial screening source cluster set. These clusters were not sampled as source clusters in this round of iteration and remain unchanged in their original state.

[0125] For example, if there are 1,000 active clusters in the initial cluster set and 300 source clusters in the initial screening source cluster set, then the first cluster set includes 700 clusters in the initial cluster set excluding these 300 source clusters. These 700 clusters are not sampled as source clusters in this iteration and remain unchanged.

[0126] Step S1444: Update the initial cluster set based on the first cluster set and the final merged cluster.

[0127] The above-mentioned update of the initial cluster set can be understood as merging the first cluster set and the final merged cluster set to form a new initial cluster set for the next round of iteration. The new initial cluster set includes the unsampled clusters (the first cluster set) and the merged new clusters (the final merged clusters), and the source and target clusters that are merged are marked as inactive.

[0128] For example, the first cluster set includes 700 clusters, and the final merged clusters include 250 new clusters (assuming that 50 of the 300 source clusters give up merging due to conflict resolution). The updated initial cluster set includes 700 clusters that did not participate in the merge and 250 new merged clusters, for a total of 950 active clusters. The source and target clusters that were merged are marked as inactive, and the next round of iteration is performed based on these 950 active clusters.

[0129] In this embodiment, the final merged cluster is obtained by merging target clusters in the target merging set into the source clusters. The merging distance is used as a reward signal to feed back to the dynamic sampling model to update the model. Clusters excluding those in the initial screening source cluster set are extracted from the initial cluster set to obtain the first cluster set. The initial cluster set is updated based on the first cluster set and the final merged cluster, thereby achieving synchronous updates between the dynamic sampling model and the initial cluster set. The reward signal feedback mechanism enables the dynamic sampling model to adaptively adjust the sampling strategy based on historical merging quality, improving the rationality of merging object selection in subsequent iterations. The extraction of the first cluster set ensures that unsampled clusters still have a chance to be sampled in the next iteration, avoiding the problem of some clusters being ignored for a long time.

[0130] To enable those skilled in the art to better understand the technical solution of this application, the following uses a specific single-cell RNA sequencing data clustering scenario as an example to fully explain the hierarchical clustering acceleration method based on dynamic sampling provided in this application.

[0131] Step 1: Data Acquisition and Preprocessing. A peripheral blood mononuclear cell (PBMC) dataset was acquired, containing 50,000 cell samples, each corresponding to the expression levels of 20,000 genes. The original gene expression matrix underwent quality control, removing cell samples with mitochondrial gene expression exceeding 10%, leaving 48,000 cell samples. SCARN normalization was used to normalize the expression levels. Principal component analysis was used to reduce the 20,000-dimensional expression vector to 50 dimensions, resulting in a set of 48,000 50-dimensional gene feature vectors.

[0132] Step 2: Construct an approximate nearest neighbor graph index and dynamic sampling model. Extract a total of 48,000 cell samples and a gene feature dimension of 50. Based on the total number of cell samples and the gene feature dimension, construct an initial hierarchical navigable small-world graph index model, setting the number of layers to 3, the maximum number of connections M to 16, and the candidate neighbor list size efConstruction to 200. Insert each of the 48,000 cell samples into the initial hierarchical navigable small-world graph index model to obtain the approximate nearest neighbor graph index. Based on the total number of cell samples (48,000), construct a multi-armed slot machine model, with each cell sample corresponding to one arm. The Beta distribution parameters alpha and beta of each arm are initialized to 1.

[0133] Step 3: Cluster state initialization. Each of the 48,000 cell samples is initialized as an independent cluster, resulting in 48,000 initialized clusters. Each cluster has a size of 1, a centroid vector that is the 50-dimensional gene feature vector of the cell sample, and an active cluster state. The preset iteration condition is that the number of active clusters is less than 100.

[0134] Step 4: Dynamic Sampling. Input 48,000 active clusters into the multi-armed slot machine model for sampling. Random numbers are sampled based on the Beta distribution parameters of each arm, and the clusters corresponding to the 300 arms with the largest random numbers are selected as source clusters to form the initial source cluster set.

[0135] Step 5: Determine the target cluster. For each source cluster in the initial screening source cluster set, use its cluster centroid vector as the query vector to retrieve 50 nearest neighbor cell samples in the approximate nearest neighbor graph index; determine the initial nearest neighbor cluster based on the clusters to which the 50 nearest neighbor cell samples belong; remove clusters in the initial nearest neighbor cluster that belong to the same cluster as the source cluster to obtain a candidate nearest neighbor cluster set; based on the cluster weight of the source cluster and the cluster weight of each nearest neighbor cluster in the candidate nearest neighbor cluster set, remove nearest neighbor clusters whose weight ratio is not within the preset ratio threshold of 3 to obtain the candidate target clusters; calculate the Euclidean distance between the cluster centroid vector of the source cluster and the cluster centroid vector of each candidate target cluster, and select the candidate target cluster with the smallest distance as the target cluster of the source cluster.

[0136] Step Six: Construct the Target Merge Set. Based on the 300 source clusters in the initial screening set and their corresponding target clusters, construct 300 candidate merge pairs to obtain a candidate merge pair set. From the candidate merge pair set, select candidate merge pairs that meet the weight ratio constraints, assuming 250 remain, to obtain the set of candidate merge pairs. Perform anomaly removal on the set of candidate merge pairs. First, perform conflict resolution. When multiple source clusters select the same target cluster, retain the merge pair with the smallest merge distance and remove other conflicting merge pairs, assuming 20 are removed, leaving 230. Then, perform anomaly removal based on merge distance. Calculate the mean merge distance of the 230 merge pairs (0.2, standard deviation 0.1). Remove merge pairs with a merge distance greater than 0.2 plus 3 multiplied by 0.1, which equals 0.5, assuming 10 are removed, finally leaving 220 merge pairs to form the target merge set.

[0137] Step 7: Update the dynamic sampling model and initialize the cluster set. The target clusters of the 220 merge pairs in the target merge set are merged into their corresponding source clusters to obtain 220 final merged clusters. The centroid vector of each final merged cluster is the vector obtained by weighting the centroid vectors of the source cluster and the target cluster according to the cluster size. The cluster size is the sum of the cluster sizes of the source cluster and the target cluster. The merge distance of the 220 merge pairs is used as a reward signal and fed back to the multi-armed slot machine model. For the scissor corresponding to the source cluster that generates a high reward (reward value greater than 1), the alpha parameter of its Beta distribution is incremented by 1. For the scissor corresponding to the source cluster that generates a low reward (reward value less than 1), the beta parameter of its Beta distribution is incremented by 1. 47,700 clusters are extracted from the 48,000 initial clusters, excluding 300 source clusters, to obtain the first cluster set. The initial cluster set is updated according to the first cluster set and the 220 final merged clusters. The updated initial cluster set includes 47,700 clusters that did not participate in the merge and 220 new merged clusters. The merged target clusters are marked as inactive.

[0138] Step 8: Iterative Clustering and Relay Clustering. Repeat steps 4 through 7 until the number of active clusters drops below 100. Then, extract the centroid vectors of the 100 active clusters to form a 100x50 relay feature matrix. Perform Ward's global hierarchical clustering and merging on the relay feature matrix to construct a complete clustering tree. Divide the clustering tree into 10 subtrees based on the cutting level of 10. Each subtree corresponds to a cell subpopulation. Assign the 48,000 cell samples to their respective active clusters, and assign each cell sample a clustering label, such as T helper cell subpopulation, T killer cell subpopulation, B cell subpopulation, NK cell subpopulation, monocyte subpopulation, etc.

[0139] As can be seen from the above specific application examples, the hierarchical clustering acceleration method based on dynamic sampling provided in this application can efficiently complete hierarchical clustering operations on a single-cell dataset of 50,000 cells. It accelerates nearest neighbor retrieval through an approximate nearest neighbor graph index, adaptively adjusts the sampling strategy through a multi-armed slot machine model, and ensures the rationality of the merging operation through weight ratio constraints and anomaly removal. Finally, it obtains accurate cell subpopulation labels with low computational overhead.

[0140] like Figure 2 As shown in some embodiments of this application, a hierarchical clustering acceleration system based on dynamic sampling is provided. The system includes an acquisition module 210, a construction module 220, a clustering module 230, an iteration module 240, and a generation module 250. Specifically: The acquisition module 210 is used to acquire the gene expression dataset to be clustered. Each data point in the gene expression dataset corresponds to a cell sample, and each data point is represented by the gene expression feature vector of the corresponding cell sample. Module 220 is used to build an approximate nearest neighbor graph index and a dynamic sampling model based on the gene expression dataset; Clustering module 230 is used to initialize the clustering state of the gene expression dataset to obtain an initialized cluster set; The iteration module 240 is used to perform N iterations based on the initial cluster set, the approximate nearest neighbor graph index, and the dynamic sampling model until the initial cluster set in the Nth iteration meets the preset iteration conditions, and then outputs the initial cluster set in the Nth iteration. The generation module 250 is used to generate clustering labels for each cell sample in the gene expression dataset based on the initial cluster set in the Nth iteration. The clustering labels are used to identify the cell subpopulation to which each cell sample belongs. In any of the N iterations, the processing is as follows: The initial cluster set is input into the dynamic sampling model for sampling to obtain the initial source cluster set; Based on the approximate nearest neighbor graph index, determine the target cluster for each source cluster in the initial screening source cluster set; Based on each source cluster and its target cluster in the initial source cluster set, construct a target merge set; Based on the target merge set, update the dynamic sampling model and initialize the cluster set.

[0141] It should be noted that the hierarchical clustering acceleration system based on dynamic sampling provided in this embodiment is based on the same inventive concept as the hierarchical clustering acceleration method based on dynamic sampling described above. Therefore, the relevant content of the hierarchical clustering acceleration method based on dynamic sampling described above is also applicable to the hierarchical clustering acceleration system based on dynamic sampling, and will not be repeated here.

[0142] The system acquires the gene expression dataset to be clustered; based on the gene expression dataset, it constructs an approximate nearest neighbor graph index and a dynamic sampling model; it initializes the clustering state of the gene expression dataset to obtain an initial cluster set; based on the initial cluster set, the approximate nearest neighbor graph index, and the dynamic sampling model, it performs N iterations until the initial cluster set in the Nth iteration meets the preset iteration conditions, and outputs the initial cluster set in the Nth iteration; based on the initial cluster set in the Nth iteration, it generates clustering labels for each cell sample in the gene expression dataset, which are used to identify the cell subpopulation to which each cell sample belongs. In this way, through dynamic sampling and nearest neighbor index retrieval, it can reduce computational overhead while suppressing long chain distortions, achieving efficient and accurate clustering of gene expression data.

[0143] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described hierarchical clustering acceleration method based on dynamic sampling.

[0144] like Figure 3 , Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes: At least one battery; At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the above-described hierarchical clustering acceleration method based on dynamic sampling as described in this disclosure.

[0145] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0146] The electronic devices according to embodiments of this application will now be described in detail.

[0147] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to implement a hierarchical clustering acceleration method based on dynamic sampling according to an embodiment of this disclosure.

[0148] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0149] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described hierarchical clustering acceleration method based on dynamic sampling.

[0150] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0151] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0152] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0155] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any related variations, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0156] It should be understood that in this application, "at least one (item)" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0158] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0160] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0161] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

[0162] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A hierarchical clustering acceleration method based on dynamic sampling, characterized in that, The method includes: Obtain the gene expression dataset to be clustered, wherein each data point in the gene expression dataset corresponds to a cell sample, and each data point is represented by the gene expression feature vector of the corresponding cell sample; Based on the gene expression dataset, construct an approximate nearest neighbor graph index and a dynamic sampling model; The gene expression dataset is initialized with cluster state to obtain an initialized cluster set; Based on the initial cluster set, the approximate nearest neighbor graph index, and the dynamic sampling model, perform N iterations until the initial cluster set in the Nth iteration meets the preset iteration conditions, and output the initial cluster set in the Nth iteration; Based on the initial cluster set in the Nth iteration, a clustering label is generated for each cell sample in the gene expression dataset. The clustering label is used to identify the cell subpopulation to which each cell sample belongs. The process of any one of the N iterations is as follows: The initial cluster set is input into the dynamic sampling model for sampling to obtain the initial source cluster set; Based on the approximate nearest neighbor graph index, determine the target cluster for each source cluster in the initial screening source cluster set; Based on each source cluster in the initial screening source cluster set and the target cluster of each source cluster, construct a target merge set; Based on the target merge set, update the dynamic sampling model and the initial cluster set; The step of constructing an approximate nearest neighbor graph index and a dynamic sampling model based on the gene expression dataset includes: Extract the total number of cell samples and gene feature dimensions from the gene expression dataset; The gene expression dataset is preprocessed to obtain a set of gene feature vectors corresponding to each cell sample in the gene expression dataset. Based on the total number of cell samples, the gene feature dimensions, and the set of gene feature vectors, an initial hierarchical navigable small-world graph index model is constructed. All cell samples in the gene feature vector set are added to the initial hierarchical navigable small-world graph index model to obtain the approximate nearest neighbor graph index; Based on the total number of cell samples, the dynamic sampling model is constructed; The step of updating the dynamic sampling model and the initial cluster set based on the target merge set includes: The target clusters in the target merge set are merged into the source clusters in the target merge set to obtain the final merged clusters; The merging distance between the source cluster and the target cluster in the target merging set is used as a reward signal and fed back to the dynamic sampling model to update the dynamic sampling model; The first cluster set is obtained by extracting clusters from the initial cluster set excluding those from the initial screening source cluster set; The initial cluster set is updated based on the first cluster set and the final merged cluster.

2. The hierarchical clustering acceleration method based on dynamic sampling according to claim 1, characterized in that, The step of generating cluster labels for each cell sample in the gene expression dataset based on the initial cluster set in the Nth iteration includes: Extract the centroid vectors of all active clusters in the initial cluster set during the Nth iteration to form a relay feature matrix; Perform global hierarchical clustering and merging on the relay feature matrix to obtain the global hierarchical clustering and merging result; Based on the global hierarchical clustering and merging results, clustering labels are generated for each cell sample in the gene expression dataset.

3. The hierarchical clustering acceleration method based on dynamic sampling according to claim 1, characterized in that, The preset iteration conditions include that the number of clusters in the initial cluster set that are in an active state is less than a preset threshold; the dynamic sampling model includes a multi-armed slot machine model.

4. The hierarchical clustering acceleration method based on dynamic sampling according to claim 1, characterized in that, The step of determining the target cluster for each source cluster in the initial screening source cluster set based on the approximate nearest neighbor graph index includes: Based on the centroid vector of each source cluster in the initial screening source cluster set, a preset number of neighboring cell samples are retrieved for each source cluster in the approximate nearest neighbor graph index; Based on each of the neighboring cell samples, determine the initial neighboring cluster corresponding to each of the source clusters; In each of the initial nearest neighbor clusters, the clusters that belong to the same cluster as the source cluster are removed to obtain the candidate nearest neighbor cluster set of the corresponding source cluster; Based on the cluster weight of each source cluster and the cluster weight of each nearest neighbor cluster in the candidate nearest neighbor cluster set of each source cluster, the candidate target cluster corresponding to the source cluster is determined in the candidate nearest neighbor cluster set. The target cluster of each source cluster in the initial screening source cluster set is determined based on the cluster centroid vector of each source cluster and the cluster centroid vector of the candidate target cluster of each source cluster.

5. The hierarchical clustering acceleration method based on dynamic sampling according to claim 1, characterized in that, The step of constructing a target merging set based on each source cluster in the initial screening source cluster set and the target cluster of each source cluster includes: Based on each source cluster in the initial screening source cluster set and the target cluster of the corresponding source cluster, a corresponding candidate merging pair is constructed to obtain a candidate merging pair set. From the set of candidate merge pairs, candidate merge pairs that meet the preset screening conditions are selected to obtain a set of candidate merge pairs; The set of candidate merge pairs is subjected to anomaly removal to obtain the target merge set; The step of removing anomalies from the candidate merge pair set to obtain the target merge set includes: If at least two groups of candidate merge pairs in the candidate merge pair set contain the same active cluster, the candidate merge pairs that meet the preset elimination conditions in the at least two groups of candidate merge pairs are eliminated to obtain the target merge set.

6. A hierarchical clustering acceleration system based on dynamic sampling, characterized in that, The system includes: The acquisition module is used to acquire the gene expression dataset to be clustered, wherein each data point in the gene expression dataset corresponds to a cell sample, and each data point is represented by the gene expression feature vector of the corresponding cell sample; The module is used to construct an approximate nearest neighbor graph index and a dynamic sampling model based on the gene expression dataset; The clustering module is used to initialize the clustering state of the gene expression dataset to obtain an initialized cluster set; The iteration module is used to perform N iterations based on the initial cluster set, the approximate nearest neighbor graph index, and the dynamic sampling model until the initial cluster set in the Nth iteration meets the preset iteration conditions, and then output the initial cluster set in the Nth iteration. The generation module is used to generate clustering labels for each cell sample in the gene expression dataset based on the initial cluster set in the Nth iteration. The clustering labels are used to identify the cell subpopulation to which each cell sample belongs. The process of any one of the N iterations is as follows: The initial cluster set is input into the dynamic sampling model for sampling to obtain the initial source cluster set; Based on the approximate nearest neighbor graph index, determine the target cluster for each source cluster in the initial screening source cluster set; Based on each source cluster in the initial screening source cluster set and the target cluster of each source cluster, construct a target merge set; Based on the target merge set, update the dynamic sampling model and the initial cluster set; The step of constructing an approximate nearest neighbor graph index and a dynamic sampling model based on the gene expression dataset includes: Extract the total number of cell samples and gene feature dimensions from the gene expression dataset; The gene expression dataset is preprocessed to obtain a set of gene feature vectors corresponding to each cell sample in the gene expression dataset. Based on the total number of cell samples, the gene feature dimensions, and the set of gene feature vectors, an initial hierarchical navigable small-world graph index model is constructed. All cell samples in the gene feature vector set are added to the initial hierarchical navigable small-world graph index model to obtain the approximate nearest neighbor graph index; Based on the total number of cell samples, the dynamic sampling model is constructed; The step of updating the dynamic sampling model and the initial cluster set based on the target merge set includes: The target clusters in the target merge set are merged into the source clusters in the target merge set to obtain the final merged clusters; The merging distance between the source cluster and the target cluster in the target merging set is used as a reward signal and fed back to the dynamic sampling model to update the dynamic sampling model; The first cluster set is obtained by extracting clusters from the initial cluster set excluding those from the initial screening source cluster set; The initial cluster set is updated based on the first cluster set and the final merged cluster.

7. An electronic device, characterized in that, It includes at least one processor and a memory for communicatively connecting to the at least one processor; the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the hierarchical clustering acceleration method based on dynamic sampling as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the hierarchical clustering acceleration method based on dynamic sampling as described in any one of claims 1 to 5.

Images