Single cell data analysis acceleration method and device based on cloud elasticity calculation and medium
By using a distributed execution method based on cloud elastic computing, the single-cell data analysis process is broken down into independent and dependent tasks, which solves the problems of high memory consumption and long time in single-cell data analysis and achieves efficient data processing and result integration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG MEIGE GENE TECH CO LTD
- Filing Date
- 2026-04-20
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies suffer from high memory consumption and long analysis time in single-cell data analysis, resulting in low research efficiency, especially when dealing with large-scale data, as they cannot be effectively executed in parallel.
By adopting a cloud-based elastic computing approach, the single-cell data analysis process is broken down into an independent first subtask and a result-dependent second subtask. Cloud servers are used for distributed execution, processing the first subtask in parallel, and then executing the second subtask sequentially, ensuring the integrity and efficiency of the analysis results.
It significantly accelerates single-cell data analysis while ensuring computational reliability, improves research efficiency, and avoids memory overflow and long waiting times.
Smart Images
Figure CN122067598A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of bioinformatics analysis technology, and in particular to a method, device and storage medium for accelerating single-cell data analysis based on cloud elastic computing. Background Technology
[0002] Single-cell transcriptome sequencing technology has become a cornerstone of life science research, enabling the revelation of cell population heterogeneity, the discovery of new cell types, and the elucidation of complex biological processes at single-cell resolution. However, with the widespread adoption of this technology and the increase in sequencing throughput, research is rapidly expanding from a scale of thousands of cells to ultra-large-scale studies involving hundreds of thousands or even millions of cells. This explosive growth in data volume poses extremely severe challenges to the computational infrastructure for downstream bioinformatics analysis.
[0003] Currently, single-cell data analysis workflows (including gene expression quantification, quality control filtering, dimensionality reduction clustering, and differential expression analysis) typically rely on users performing these analyses on their local high-performance workstations or servers. This traditional centralized analysis model suffers from the following significant, interrelated technical bottlenecks: (1) Extremely high memory consumption: Single-cell data analysis, especially in core steps such as dimensionality reduction clustering and cell-to-cell relationship construction, requires loading the entire expression matrix (cell × gene) into memory and performing large-scale matrix operations. Its memory consumption is proportional to the square of the number of cells. When processing datasets of millions of cells, the required memory easily exceeds hundreds of GB or even reaches the TB level. In addition, the high-concurrency memory load brought about by the parallelization of analysis tasks means that the configuration of local computing nodes in most research institutions cannot meet the requirements, directly causing analysis tasks to be interrupted due to insufficient memory, which seriously limits the ability of researchers to process large-scale data.
[0004] (2) Lengthy analysis time: Single-cell analysis is complex, involves numerous steps, and many algorithms have high computational complexity. With limited local computing resources, a complete data analysis process typically takes several hours to several days to complete. This long waiting time severely slows down the iteration speed of scientific discovery, reduces research efficiency, and is particularly detrimental to exploratory research that requires rapid screening of multiple parameters and conditions.
[0005] Therefore, those skilled in the art face a dilemma: serial execution is too time-consuming, impacting research efficiency; while attempting parallel execution is limited by local, fixed hardware resources, easily leading to memory overflows and preventing the analysis task from being completed. Furthermore, in the single-cell data analysis process, some subtasks are interdependent, making a fully parallel execution solution unusable. Current technology lacks an effective solution that can significantly accelerate large-scale single-cell data analysis while ensuring computational reliability. Summary of the Invention
[0006] This application provides a method, device, and medium for accelerating single-cell data analysis based on cloud elastic computing, in order to at least solve the above-mentioned technical problems existing in the prior art.
[0007] According to a first aspect of this application, a method for accelerating single-cell data analysis based on cloud elastic computing is provided, comprising the following steps: S1, upload the raw single-cell sequencing data to the object storage unit of the cloud server; S2, parse the single-cell data analysis process, and break down some steps of the single-cell data analysis process into a first sub-task and another part of the steps into a second sub-task; the second sub-task is configured with corresponding initiation conditions, and the initiation conditions include the analysis results of the first sub-task of some steps; S3, configure the corresponding number of cloud servers based on the number of the first and second subtasks that are split out; S4, the first and second subtasks are distributed to various cloud servers for distributed execution. The cloud servers first execute the first subtask in parallel, and then execute the second subtask after the conditions for initiating the second subtask are met. S5 transmits the analysis results of the first and second subtasks to the object storage unit for integration and summarization.
[0008] In some embodiments of the first aspect of this application, in step S2, gene expression quantification, cell filtering, cell clustering, and cell annotation of the single-cell data analysis process are broken down into first sub-tasks. The gene expression quantification is performed by splitting the sample, with each sample's expression matrix obtained independently, and then the matrices are combined into a single expression matrix. The cell filtering process is performed by splitting the sample, with each sample being processed independently to obtain a list of cell barcodes that need to be removed, which are then merged and removed. The cell clusters are divided according to the analysis parameters, and the analysis results are obtained by executing the analysis independently with different parameter settings. Cell annotation, based on software-automated annotation, is split according to different annotation databases.
[0009] In some embodiments of the first aspect of this application, in step S2, the differential expression analysis and functional enrichment analysis of the single-cell data analysis process are split into a second sub-task; The differential expression analysis was first performed by splitting the cells according to the analysis parameters, with each cell population obtained from different analysis parameters being analyzed independently. Secondly, in the bioinformatics analysis and plotting stage, the analysis was performed by splitting the cells according to cell subpopulations. The functional enrichment analysis was first divided into KEGG enrichment analysis and GO enrichment analysis based on function; then it was divided into cell subpopulations, with each cell subpopulation being analyzed independently to obtain its own enrichment analysis results.
[0010] In some embodiments of the first aspect of this application, the initiation condition for the second subtask of differential expression analysis and functional enrichment analysis includes the analysis results of the first subtask of cell clustering.
[0011] In some embodiments of the first aspect of this application, in step S4, after the first subtask or the second subtask is completed, the corresponding cloud server is released. Before the cloud server is released, the analysis results are transmitted to the object storage unit.
[0012] In some embodiments of the first aspect of this application, in step S4, the running status of the first subtask on each cloud server is detected in real time, and the execution of the second subtask is initiated only after all the first subtasks included in the initiation conditions have been completed.
[0013] In some embodiments of the first aspect of this application, step S4 further includes detecting the integrity of the execution results of the first subtask / second subtask; Check the status of the cloud server at fixed intervals. If the cloud server has been released, check if the result file generated by the last analysis command of the first subtask / second subtask exists. If the result file generated by the last analysis command exists, it means that the execution results of the first subtask / second subtask are complete.
[0014] In some embodiments of the first aspect of this application, when it is detected that the second subtask has met the initiation conditions, the second subtask is added to the list to be executed.
[0015] According to a second aspect of this application, a single-cell data analysis acceleration system based on cloud elastic computing is provided, comprising: The data upload unit uploads raw single-cell sequencing data to the object storage unit of the cloud server. The subtask splitting unit parses the single-cell data analysis process, splitting a portion of the steps in the single-cell data analysis process into a first subtask and another portion into a second subtask; the second subtask is configured with corresponding initiation conditions, the initiation conditions including the analysis results of the first subtask for a portion of the steps; The cloud resource scheduling unit configures a corresponding number of cloud servers based on the number of the first and second subtasks. The distributed execution unit distributes the split first subtask and second subtask to various cloud servers for distributed execution. The cloud servers first execute the first subtask in parallel, and then execute the second subtask after the conditions for initiating the second subtask are met. The results integration unit transmits the analysis results of the first and second subtasks to the object storage unit for integration and summarization.
[0016] In some embodiments of the second aspect of this application, the subtask splitting unit splits the gene expression quantification, cell filtering, cell clustering, and cell annotation of the single-cell data analysis process into first subtasks; The gene expression quantification is performed by splitting the sample, with each sample's expression matrix obtained independently, and then the matrices are combined into a single expression matrix. The cell filtering process is performed by splitting the sample, with each sample being processed independently to obtain a list of cell barcodes that need to be removed, which are then merged and removed. The cell clusters are divided according to the analysis parameters, and the analysis results are obtained by executing the analysis independently with different parameter settings. Cell annotation, based on software-automated annotation, is split according to different annotation databases.
[0017] In some embodiments of the second aspect of this application, the subtask splitting unit splits the differential expression analysis and functional enrichment analysis of the single-cell data analysis process into a second subtask; The differential expression analysis was first performed by splitting the cells according to the analysis parameters, with each cell population obtained from different analysis parameters being analyzed independently. Secondly, in the bioinformatics analysis and plotting stage, the analysis was performed by splitting the cells according to cell subpopulations. The functional enrichment analysis was first divided into KEGG enrichment analysis and GO enrichment analysis based on function; then it was divided into cell subpopulations, with each cell subpopulation being analyzed independently to obtain its own enrichment analysis results.
[0018] In some embodiments of the second aspect of this application, the initiation condition for the second subtask of differential expression analysis and functional enrichment analysis includes the analysis results of the first subtask of cell clustering.
[0019] In some embodiments of the second aspect of this application, after the distributed execution unit completes the first subtask or the second subtask, the corresponding cloud server is released. Before the cloud server is released, the analysis results are transmitted to the object storage unit.
[0020] In some embodiments of the second aspect of this application, the distributed execution unit monitors the running status of the first subtask on each cloud server in real time, and initiates the execution of the second subtask only after all the first subtasks included in the initiation conditions have been completed.
[0021] In some embodiments of the second aspect of this application, the distributed execution unit further detects the integrity of the execution results of the first subtask / second subtask; Check the status of the cloud server at fixed intervals. If the cloud server has been released, check if the result file generated by the last analysis command of the first subtask / second subtask exists. If the result file generated by the last analysis command exists, it means that the execution results of the first subtask / second subtask are complete.
[0022] In some embodiments of the second aspect of this application, when it is detected that the second subtask has met the conditions for initiation, the second subtask is added to the list to be executed.
[0023] According to a third aspect of this application, an electronic device is provided, comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.
[0024] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.
[0025] Compared with the prior art, this application has the following advantages: This application distributes the split first and second subtasks to various servers, and first executes the relatively independent first subtask in a distributed manner on each server, while setting initiation conditions for the second subtask. Since the initiation conditions include the analysis results of some steps of the first subtask, the second subtask will only be executed after all the first subtasks of a step have been completed. This divides the relatively independent steps in the single-cell data analysis process and the steps that depend on the analysis results of other steps into two categories and execute them sequentially. This ensures both the efficiency of distributed parallel execution and the sequential execution of steps with dependencies.
[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description
[0027] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
[0028] Figure 1 A schematic diagram of the overall method of Embodiment 1 of this application is shown.
[0029] Figure 2 A schematic diagram of the mgbash system structure according to Embodiment 1 of this application is shown.
[0030] Figure 3 A system schematic diagram of Embodiment 2 of this application is shown.
[0031] Figure 4 A schematic diagram of the composition structure of an electronic device according to Embodiment 3 of this application is shown. Detailed Implementation
[0032] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0033] Example 1: A method for accelerating single-cell data analysis based on cloud elastic computing This first embodiment provides a method for accelerating single-cell data analysis based on cloud elastic computing. Please refer to... Figure 1 This includes the following steps: S1 uploads raw single-cell sequencing data to the object storage unit of the cloud server.
[0034] Specifically, in this embodiment, Volcano Cloud, Alibaba Cloud, or Baidu Cloud are used as cloud servers. The raw single-cell sequencing data is uploaded from the local machine to a designated path on the cloud server. After successful upload, the raw single-cell sequencing data is stored in the object storage unit of the cloud server.
[0035] S2, parse the single-cell data analysis process, and split a portion of all steps in the single-cell data analysis process into a first sub-task and another portion into a second sub-task. The second sub-task is configured with corresponding initiation conditions, and the initiation conditions include the analysis results of the first sub-task of a portion of the steps.
[0036] The single-cell data analysis process mainly includes the following steps: The gene expression quantification is performed by splitting the sample, with each sample's expression matrix obtained independently, and then the results are combined into a single expression matrix.
[0037] The cell filtering process is performed by splitting the sample, with each sample being processed independently to obtain a list of cell barcodes that need to be removed, which are then merged and removed.
[0038] The cell population is segmented according to analytical parameters, with different parameter settings executing independently to obtain analysis results and cell subpopulations. These analytical parameters include: expected number of cells detected (expect_cells), gene count range, UMI range, mitochondrial percentage (percent_mt_max), doublet rate, batch correction, normalization method, principal component count (ndims), resolution, etc. Since single-cell analysis often requires parameter adjustments and re-analysis based on results, this design ensures that each set of parameters generates a separate result, with the final result selected by the analyst based on their judgment.
[0039] The differential expression analysis is first performed by splitting the cells according to the analysis parameters, and the cell population results obtained by different analysis parameters are performed independently for subsequent differential expression analysis; secondly, in the bioinformatics analysis and plotting stage, the analysis is performed by splitting the cells according to cell subpopulations.
[0040] The functional enrichment analysis was first divided into KEGG enrichment analysis and GO enrichment analysis based on function; then it was divided into cell subpopulations, with each cell subpopulation being analyzed independently to obtain its own enrichment analysis results.
[0041] Cell annotation, based on software-automated annotation (e.g., SingleR), is split according to different annotation databases.
[0042] Among them, gene expression quantification, cell filtering, cell clustering, and cell annotation are independent of each other and can be executed in parallel. Therefore, the gene expression quantification, cell filtering, cell clustering, and cell annotation in the single-cell data analysis process are split into the first sub-task.
[0043] Differential expression analysis and functional enrichment analysis depend on the analysis results of cell clusters before they can be further implemented. Therefore, the differential expression analysis and functional enrichment analysis in the single-cell data analysis workflow are broken down into a second sub-task. Furthermore, corresponding initiation conditions are configured for the second sub-task, including the analysis results of the first sub-task derived from the cell cluster analysis. In other words, the second sub-tasks derived from the differential expression analysis and functional enrichment analysis can only be executed sequentially after all the first sub-tasks of the cell cluster analysis have been completed.
[0044] S3, configure the corresponding number of cloud servers based on the number of the first and second subtasks.
[0045] Specifically, such as Figure 2 As shown, the mgbash tool is used to connect with cloud services (such as Huawei Cloud, Volcengine, etc.) based on the API interface and keys provided by the cloud vendor. According to the number of subtasks currently split, mgbash is invoked to automatically request, configure, and start the corresponding number of cloud server instances. Each instance has independent and sufficient memory and CPU resources to ensure that it can fully load the expression matrix and run efficiently.
[0046] The functions of the mgbash tool include: (1) config command: Configure user information. The user configuration information function is used to configure the user's username, password and default file path.
[0047] (2) ls command: View files, view files in the specified path on the system's object storage.
[0048] (3) upload command: Upload files, upload local files to the specified path on the system.
[0049] (4) download command: Download file, download the system-specified file to the local machine.
[0050] (5) rm command: deletes a file, deletes a specified file on the system.
[0051] (6) run / parallel command: Create a task, create a specified number of servers to run a specified script / command. You can specify the number of CPU cores, memory size, system disk size, data disk size, and task name of the server for the task.
[0052] (7) jobs command: View tasks and view information about currently running tasks, including server ID, task ID, task name, creation time, running time, number of CPU cores, memory size, system disk size, data disk size, system image ID, and the name of the running script.
[0053] (8) stop command: Stop the task. Stop the task that is currently being executed. You can stop the task that is currently being executed based on the server ID or the task ID.
[0054] Therefore, in step S3, based on the type and number of subtasks to be executed, the specified number and specifications of cloud servers are created by calling the `run / parallel` command of mgbash. The `run / parallel` command also specifies the task scripts that each cloud server needs to run.
[0055] S4. The first and second subtasks are distributed to various cloud servers for distributed execution. The cloud servers first execute the first subtask in parallel, and then execute the second subtask after the conditions for initiating the second subtask are met.
[0056] Each cloud server independently executes the subtasks assigned to it, and there is no need for communication between them, thus avoiding the complex coordination overhead in distributed computing.
[0057] Furthermore, because there are dependencies between the analysis results of the steps (e.g., differential expression analysis and functional enrichment analysis depend on the analysis results of cell subpopulations, meaning they can only be implemented after obtaining the cell subpopulations), the running status of the first subtask on each cloud server is monitored in real time. The second subtask is only initiated after all the first subtasks included in the initiation conditions have been completed. It is worth noting that the corresponding cloud server is released after the first or second subtask is completed. Before the cloud server is released, the analysis results are transferred to the object storage unit.
[0058] Specifically, it is implemented through the following process: The `mgbash download` command downloads and analyzes the input file to local storage on the cloud server.
[0059] Execute the bioinformatics analysis command corresponding to this subtask, such as cell clustering.
[0060] Execute `mgbash upload` to upload the analysis results to object storage.
[0061] Simultaneously, the integrity of the execution results of the first subtask / second subtask is also checked, using the following method: The `mgbash jobs` command is executed at fixed intervals to check the status of the cloud server. Upon receiving the command, the status of the currently created cloud service (released / not released) is returned. If the cloud server has been released, the `mgbash ls` command is used to check if the result file generated by the last analysis command of the first / second subtask exists. If the result file generated by the last analysis command exists, it indicates that the execution results of the first / second subtask are complete, and the corresponding subtask has ended normally.
[0062] By executing this step, the steps in the single-cell data analysis process can be divided into two categories: those that are relatively independent and those that depend on the analysis results of other steps. These steps are executed sequentially, which ensures both the efficiency of distributed parallel execution and the sequential execution of steps with dependencies.
[0063] When it is detected that the second subtask has met the conditions for initiation, the second subtask is added to the list to be executed, and the second subtask is executed in sequence.
[0064] S5 transfers the analysis results of the first and second subtasks to the object storage unit for integration and summarization; finally, it generates the final result consistent with the local centralized analysis.
[0065] Example 2: A Single-Cell Data Analysis Acceleration System Based on Cloud Elastic Computing This embodiment provides a single-cell data analysis acceleration system based on cloud elastic computing. Please refer to... Figure 3 It includes the following five units: The data upload unit uploads raw single-cell sequencing data to the object storage unit of the cloud server.
[0066] Specifically, in this embodiment, Volcano Cloud, Alibaba Cloud, or Baidu Cloud are used as cloud servers. The raw single-cell sequencing data is uploaded from the local machine to a designated path on the cloud server. After successful upload, the raw single-cell sequencing data is stored in the object storage unit of the cloud server.
[0067] The subtask splitting unit parses the single-cell data analysis process, splitting a portion of all steps in the single-cell data analysis process into a first subtask and another portion into a second subtask. The second subtask is configured with corresponding initiation conditions, which include the analysis results of the first subtask for a portion of the steps.
[0068] The single-cell data analysis process mainly includes the following steps: The gene expression quantification is performed by splitting the sample, with each sample's expression matrix obtained independently, and then the results are combined into a single expression matrix.
[0069] The cell filtering process is performed by splitting the sample, with each sample being processed independently to obtain a list of cell barcodes that need to be removed, which are then merged and removed.
[0070] The cell population is segmented according to analytical parameters, with different parameter settings executing independently to obtain analysis results and cell subpopulations. These analytical parameters include: expected number of cells detected (expect_cells), gene count range, UMI range, mitochondrial percentage (percent_mt_max), doublet rate, batch correction, normalization method, principal component count (ndims), resolution, etc. Since single-cell analysis often requires parameter adjustments and re-analysis based on results, this design ensures that each set of parameters generates a separate result, with the final result selected by the analyst based on their judgment.
[0071] The differential expression analysis is first performed by splitting the cells according to the analysis parameters, and the cell population results obtained by different analysis parameters are performed independently for subsequent differential expression analysis; secondly, in the bioinformatics analysis and plotting stage, the analysis is performed by splitting the cells according to cell subpopulations.
[0072] The functional enrichment analysis was first divided into KEGG enrichment analysis and GO enrichment analysis based on function; then it was divided into cell subpopulations, with each cell subpopulation being analyzed independently to obtain its own enrichment analysis results.
[0073] Cell annotation, based on software-automated annotation (e.g., SingleR), is split according to different annotation databases.
[0074] Among them, gene expression quantification, cell filtering, cell clustering, and cell annotation are independent of each other and can be executed in parallel. Therefore, the gene expression quantification, cell filtering, cell clustering, and cell annotation in the single-cell data analysis process are split into the first sub-task.
[0075] Differential expression analysis and functional enrichment analysis depend on the analysis results of cell clusters before they can be further implemented. Therefore, the differential expression analysis and functional enrichment analysis in the single-cell data analysis workflow are broken down into a second sub-task. Furthermore, corresponding initiation conditions are configured for the second sub-task, including the analysis results of the first sub-task derived from the cell cluster analysis. In other words, the second sub-tasks derived from the differential expression analysis and functional enrichment analysis can only be executed sequentially after all the first sub-tasks of the cell cluster analysis have been completed.
[0076] The cloud resource scheduling unit configures a corresponding number of cloud servers based on the number of the first and second subtasks.
[0077] Specifically, such as Figure 2 As shown, the mgbash tool is used to connect with cloud services (such as Huawei Cloud, Volcengine, etc.) based on the API interface and keys provided by the cloud vendor. According to the number of subtasks currently split, mgbash is invoked to automatically request, configure, and start the corresponding number of cloud server instances. Each instance has independent and sufficient memory and CPU resources to ensure that it can fully load the expression matrix and run efficiently.
[0078] The distributed execution unit distributes the split first and second subtasks to various cloud servers for distributed execution. The cloud servers first execute the first subtask in parallel, and then execute the second subtask after the conditions for initiating the second subtask are met.
[0079] Each cloud server independently executes the subtasks assigned to it, and there is no need for communication between them, thus avoiding the complex coordination overhead in distributed computing.
[0080] Furthermore, because there are dependencies between the analysis results of the steps (e.g., differential expression analysis and functional enrichment analysis depend on the analysis results of cell subpopulations, meaning they can only be implemented after obtaining the cell subpopulations), the running status of the first subtask on each cloud server is monitored in real time. The second subtask is only initiated after all the first subtasks included in the initiation conditions have been completed. It is worth noting that the corresponding cloud server is released after the first or second subtask is completed. Before the cloud server is released, the analysis results are transferred to the object storage unit.
[0081] Check the status of the cloud server at fixed intervals. Upon receiving a command, return the status of the currently created cloud service (released / not released). If the cloud server has been released, check if the result file generated by the last analysis command of the first / second subtask exists. If the result file generated by the last analysis command exists, it indicates that the execution result of the first / second subtask is complete, and the corresponding subtask has ended normally.
[0082] When it is detected that the second subtask has met the conditions for initiation, the second subtask is added to the list to be executed, and the second subtask is executed in sequence.
[0083] The results integration unit transfers the analysis results of the first and second subtasks to the object storage unit for integration and summarization; ultimately, it generates the final results consistent with the local centralized analysis.
[0084] The specific implementation principle of the single-cell data analysis acceleration system based on cloud elastic computing in Embodiment 2 is the same as that of the single-cell data analysis acceleration method based on cloud elastic computing described in Embodiment 1, and will not be repeated here.
[0085] Example 3: This embodiment also provides an electronic device and a readable storage medium.
[0086] Figure 4 A schematic block diagram of an example electronic device that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.
[0087] like Figure 4 As shown, the device includes a computing unit that can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM can also store various programs and data required for device operation. The computing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.
[0088] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0089] The computing unit can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing units include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit performs the various methods and processes described above, such as the cloud-based elastic computing-based single-cell data analysis acceleration method described in Embodiment 1. For example, in some embodiments, the cloud-based elastic computing-based single-cell data analysis acceleration method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the computing unit, one or more steps of the cloud-based elastic computing-based single-cell data analysis acceleration method described above can be performed. Alternatively, in other embodiments, the computing unit can be configured to perform the cloud-based elastic computing-based single-cell data analysis acceleration method by any other suitable means (e.g., by means of firmware).
[0090] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0091] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0092] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0094] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0095] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0096] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.
[0097] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.
[0098] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for accelerating single-cell data analysis based on cloud elastic computing, characterized in that, Includes the following steps: S1, upload raw single-cell sequencing data to the object storage unit of the cloud server; S2, parse the single-cell data analysis process, and break down some of the steps of the single-cell data analysis process into a first sub-task and another part of the steps into a second sub-task; The second subtask is configured with corresponding initiation conditions, which include the analysis results of the first subtask in some steps; S3, configure the corresponding number of cloud servers based on the number of the first and second subtasks. S4, the first and second subtasks are distributed to various cloud servers for distributed execution. The cloud servers first execute the first subtask in parallel, and then execute the second subtask after the conditions for initiating the second subtask are met. S5 transmits the analysis results of the first and second subtasks to the object storage unit for integration and summarization.
2. The method for accelerating single-cell data analysis based on cloud elastic computing according to claim 1, characterized in that, In step S2, the gene expression quantification, cell filtering, cell clustering, and cell annotation of the single-cell data analysis process are broken down into the first sub-task. The gene expression quantification is performed by splitting the sample, with each sample's expression matrix obtained independently, and then the matrices are combined into a single expression matrix. The cell filtering process is performed by splitting the sample, with each sample being processed independently to obtain a list of cell barcodes that need to be removed, which are then merged and removed. The cell clusters are divided according to the analysis parameters, and the analysis results are obtained by executing the analysis independently with different parameter settings. Cell annotation, based on software-automated annotation, is split according to different annotation databases.
3. The method for accelerating single-cell data analysis based on cloud elastic computing according to claim 2, characterized in that, In step S2, the differential expression analysis and functional enrichment analysis of the single-cell data analysis process are broken down into a second sub-task. The differential expression analysis was first performed by splitting the cells according to the analysis parameters, with each cell population obtained from different analysis parameters being analyzed independently. Secondly, in the bioinformatics analysis and plotting stage, the analysis was performed by splitting the cells according to cell subpopulations. The functional enrichment analysis was first divided into KEGG enrichment analysis and GO enrichment analysis based on function; then it was divided into cell subpopulations, with each cell subpopulation being analyzed independently to obtain its own enrichment analysis results.
4. The method for accelerating single-cell data analysis based on cloud elastic computing according to claim 3, characterized in that, The initiation conditions for the second subtask of differential expression analysis and functional enrichment analysis include the analysis results of the first subtask of cell clustering.
5. The method for accelerating single-cell data analysis based on cloud elastic computing according to claim 1, characterized in that, In step S4, after the first or second subtask is completed, the corresponding cloud server will be released. Before the cloud server is released, the analysis results are transmitted to the object storage unit.
6. The method for accelerating single-cell data analysis based on cloud elastic computing according to claim 5, characterized in that, In step S4, the running status of the first subtask on each cloud server is monitored in real time. The second subtask is initiated only after all the first subtasks included in the initiation conditions have been completed.
7. The method for accelerating single-cell data analysis based on cloud elastic computing according to claim 5, characterized in that, In step S4, the integrity of the execution results of the first subtask / second subtask is also checked. Check the status of the cloud server at fixed intervals. If the cloud server has been released, check if the result file generated by the last analysis command of the first subtask / second subtask exists. If the result file generated by the last analysis command exists, it means that the execution results of the first subtask / second subtask are complete.
8. The method for accelerating single-cell data analysis based on cloud elastic computing according to claim 7, characterized in that, When it is detected that the second subtask has met the conditions for initiation, the second subtask is added to the list of tasks to be executed.
9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.