A micro-service-based genetic molecule sequence task scheduling method

Abstract

The application discloses a kind of genetic molecule sequence task scheduling methods based on microservice, the method includes the following steps: S1, set the server platform for container operation, and genetic molecule sequence analysis algorithm is deployed to server platform;S2, based on the running state calculation server platform resource utilization of algorithm task in container;S3, to be analyzed genetic molecule sequence file is read into server platform to generate target task, and whether server platform meets the required resources of target task is judged, if meet, the target task is scheduled to server platform, if can not meet, return step S2;S4, the target task is analyzed to obtain corresponding analysis result.The genetic molecule sequence analysis with the characteristics of microservice of the application is organically combined with scheduling algorithm, and by early calculation mode, resources are selectively scheduled to server platform, which can effectively reduce the processing time of server platform, improve the processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer software technology, and in particular to a microservice-based genetic molecular sequence task scheduling method for cloud computing. Background Technology

[0002] Currently, most genetic molecular sequence analysis involves single-unit applications and interdisciplinary learning in bioinformatics. Single-unit applications often result in large datasets, requiring deployment as WAR files. This approach hinders algorithm updates and maintenance within the interdisciplinary field of bioinformatics. Updating an algorithm necessitates a complete overhaul, especially for deep learning-based models, which may inadvertently halt their computation, leading to training failures and requiring re-runs—a significant time drain. Furthermore, while interdisciplinary learning in bioinformatics boasts cutting-edge research, current technology lacks a solution to integrate highly accurate models into a unified analysis platform, thus failing to provide a solution that drives the discipline's development.

[0003] Furthermore, there is currently no microservice scheduling method for genetic sequence analysis. Traditional microservice architecture focuses on container deployment and operation, using threshold processing to control container load, but does not consider resource utilization. Unused algorithms may also consume resources due to their containerized deployment, which is not conducive to improving the computing speed of other algorithms. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a microservice-based genetic molecular sequence task scheduling method, the method comprising the following steps:

[0005] S1. Set up a server platform for the container to run and deploy the genetic molecular sequence analysis algorithm to the server platform;

[0006] S2. Calculate the resource utilization of the server platform based on the running status of algorithm tasks within the container;

[0007] S3. Read the genetic molecular sequence file to be analyzed from the disk into the server platform to generate the target task, and determine whether the remaining resources of the server platform meet the requirements of the target task. If they do, schedule the target task to the server platform. If they do not, return to step S2.

[0008] S4. Analyze the target task to obtain the corresponding analysis results.

[0009] Preferably, step S2 specifically includes:

[0010] S21. Collect the running status of the algorithm task in the container on the server platform once every first preset time interval, and persist it.

[0011] S22. Based on the data collected in step S21, the resource utilization rate of the server platform is predicted once every second preset time interval;

[0012] S23. Based on the prediction results of step S22, calculate the remaining resource space of the server platform, expressed by the formula:

[0013]

[0014] In the formula, U t This represents the remaining resource space of the server platform at time t. This represents the predicted resource utilization rate of the server platform at time t.

[0015] S24. Compare the remaining space of the current server platform resources U. t The relationship between U and the first preset threshold δ1, if U t If <δ1, then return to step S22. If U t If ≥δ1, proceed to step S25;

[0016] S25. Calculate the difference between the predicted server platform resource utilization rate and the actual server platform resource utilization rate at time t. The calculation formula is as follows:

[0017]

[0018] In the formula, n represents the algorithm task currently running on the server platform, and R... used This represents the resource utilization rate of the server platform at time t.

[0019] S26. Compare the current server platform resource utilization prediction results. The relationship between the magnitude of the second preset threshold δ2 and the value of the second preset threshold δ2 is as follows: Then wait for the current moment to end, if Then proceed to step S27;

[0020] S27, If N t If N < 0, then perform a shrinkage operation; if N < 0, then perform a shrinkage operation. t If the value is greater than 0, then an expansion operation will be performed.

[0021] Preferably, the first preset time is 5 minutes, the second preset time is 60 minutes, the first preset threshold is 0.85, and the second preset threshold is 0.5.

[0022] Preferably, in step S21, if the algorithm task in the container of the server platform is in a non-working state, the algorithm task in the non-working state will be hot-released or taken offline.

[0023] Preferably, in step S22, the formula for calculating the predicted resource utilization rate of the server platform at time t is:

[0024]

[0025] In the formula, Indicates the server platform at t m The predicted result of time-based resource utilization, where m represents the number of time-based moments.

[0026] Preferably, step S3 further includes: if the remaining resources of the server platform do not meet the resources required by the target task, then determine whether there is an algorithm task that is about to end in the server platform container. If there is, then schedule the target task to the server platform. If there is no such task, then return to step S2.

[0027] Preferably, step S4 specifically includes:

[0028] S41. Verify the rationality of the genetic molecular sequence file to be analyzed;

[0029] S42. Analyze the basic characteristics of genetic analysis, including multilocus, high variability, and genetic stability;

[0030] S43. Remove interfering information from genetic molecular sequence tasks;

[0031] S44. Obtain the corresponding analysis results and output them.

[0032] Preferably, step S44 specifically involves: summarizing and analyzing the obtained analysis results and generating an analysis report, which is then sent to the user in the form of a report or email.

[0033] Preferably, the server platform is Kubernetes and the container is Docker.

[0034] Compared with existing technologies, this invention organically combines genetic molecular sequence analysis with scheduling algorithms that have microservice characteristics, and selectively schedules resources to the server platform based on pre-calculation. By considering the utilization of server platform resources, it effectively reduces server processing time and improves processing efficiency. At the same time, by hot-deploying or taking offline algorithm tasks in non-working states within the container, it is only necessary to focus on the characteristics of the genetic molecular sequence analysis algorithm without affecting the global algorithm state, further improving processing and maintenance efficiency. Attached Figure Description

[0035] Figure 1 This is a flowchart of a microservice-based genetic molecular sequence task scheduling method according to the present invention. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0037] like Figure 1 As shown, this invention provides a microservice-based genetic molecular sequence task scheduling method, the method comprising the following steps:

[0038] S1. Set up a server platform for the container to run and deploy the genetic molecular sequence analysis algorithm to the server platform;

[0039] In this step, the server platform is Kubernetes (K8s, an abbreviation of "ubernete" which uses "8" to represent the eight characters in the name; it is an open-source application for managing containerized applications across multiple hosts in a cloud platform. Kubernetes aims to make deploying containerized applications simple and efficient, providing a mechanism for application deployment, planning, updating, and maintenance), and the container is Docker (an open-source application container engine). The server platform contains multiple containers occupying different resource spaces. Each container has a basic system kernel, and each scheduled algorithm task can run within a container that meets its resource requirements. Each container runs within a pod (Plainold data structure, a type of data structure defined in the C++ language standard), and then exposes the application (i.e., one of the algorithm tasks) to other applications for use.

[0040] S2. Calculate the resource utilization of the server platform based on the running status of algorithm tasks within the container;

[0041] This step specifically includes:

[0042] S21. Collect the running status of the algorithm task in the container on the server platform once every first preset time interval, and persist it.

[0043] S22. Based on the data collected in step S21, the resource utilization rate of the server platform is predicted every second preset time interval. The prediction formula for the resource utilization rate of the server platform at time t is as follows:

[0044]

[0045] In the formula, Indicates the server platform at tm The predicted result of resource utilization at each time point, where m represents the number of time points;

[0046] In this step, due to network speed fluctuations during the processing of algorithm task requests by the server platform, the number of algorithm task requests may experience temporary fluctuations. These temporary fluctuations are something the algorithm needs to avoid. Therefore, a sliding window algorithm is introduced to update the load count for the next time step of the prediction result. The algorithm sets the width to m as the width of the sliding window. When the algorithm needs t... m When using the predicted value at time as a decision-making reference, to avoid system jitter caused by frequent scaling up and down due to network speed fluctuations, t will be used. m The prediction result at time t is replaced with time t1 and t m The average of all predictions at any given time is used as a reference value for the algorithm's scaling decisions.

[0047] S23. Based on the prediction results of step S22, calculate the remaining resource space of the server platform, expressed by the formula:

[0048]

[0049] In the formula, U t This represents the remaining resource space of the server platform at time t. This represents the predicted resource utilization rate of the server platform at time t.

[0050] S24. Compare the remaining space of the current server platform resources U. t The relationship between U and the first preset threshold δ1, if U t If <δ1, then return to step S22. If U t If ≥δ1, proceed to step S25;

[0051] In this step, when the prediction result reaches the first preset threshold δ1, it indicates that the resource utilization of the current server platform is relatively high, and it is not advisable to receive more network requests for algorithm tasks. If there are still network requests for algorithm tasks that need to be processed at this time, the scheduling component needs to be requested to allocate resources to the container and perform expansion processing to ensure that new requests can be processed and alleviate the pressure of resource shortage.

[0052] S25. Calculate the difference between the predicted server platform resource utilization rate and the actual server platform resource utilization rate at time t. The calculation formula is as follows:

[0053]

[0054] In the formula, n represents the algorithm task currently running on the server platform, and R... usedThis represents the resource utilization rate of the server platform at time t;

[0055] In this step, the difference between the predicted resource utilization rate of the server platform at time t and the actual resource utilization rate of the server platform is calculated. Based on this difference, it is determined whether to expand or shrink the server platform to ensure that subsequent new algorithm task requests can be processed and alleviate the pressure on server platform resources.

[0056] S26. Compare the current server platform resource utilization prediction results. The relationship between the magnitude of the second preset threshold δ2 and the value of the second preset threshold δ2 is as follows: Then wait for the current moment to end, if Then proceed to step S27;

[0057] In this step, a second preset threshold δ2 is set to ensure that the server platform can perform scaling up and down operations when the service load is low, thereby reducing server processing time and improving processing efficiency.

[0058] S27, If N t If N < 0, then perform a shrinkage operation; if N < 0, then perform a shrinkage operation. t If the value is greater than 0, then an expansion operation will be performed.

[0059] In this embodiment, the first preset time is 5 minutes, the second preset time is 60 minutes, the first preset threshold is 0.85, and the second preset threshold is 0.5. In other embodiments, the first preset time, the second preset time, the first preset threshold, and the second preset threshold can all be set according to the actual situation.

[0060] In step S21, if the algorithm task in the container on the server platform is in a non-working state, the algorithm task in the non-working state will be hot-released or taken offline.

[0061] In this embodiment, the set services are registered with a service registry center to obtain services that maintain the corresponding algorithm tasks. Each algorithm task is maintained by a dedicated service. Each time an algorithm task is executed, the corresponding service is scheduled to the service registry center, facilitating service consumption in multi-service scenarios. A thread pool is started to track the status of all services registered with the service registry center. Based on the service status tracking results, the resource utilization of the server platform is calculated. The service status includes various states: if there is a "heartbeat" between multiple services, it is considered to be in a running state; if there is no "heartbeat," the task or service has stopped. The resource utilization of the server platform is then calculated based on whether the service status is running or stopped to deploy the algorithm tasks. After tracking the status of all services registered with the service registry center using the thread pool, non-working algorithm tasks in the container are either hot-released or taken offline. Hot-release refers to synchronously or rollingly updating the algorithm task while other services are running; offline refers to stopping the corresponding algorithm task through the algorithm task scheduling mechanism.

[0062] S3. Read the genetic molecular sequence file to be analyzed from the disk into the server platform to generate the target task, and determine whether the remaining resources of the server platform meet the requirements of the target task. If they do, schedule the target task to the server platform. If they do not, return to step S2.

[0063] If the remaining resources on the server platform are insufficient to meet the resource requirements of the target task, it is determined whether there is an algorithm task that is about to end in the server platform container. If there is, the target task is scheduled to the server platform; if not, the process returns to step S2.

[0064] S4. Analyze the target task to obtain the corresponding analysis results.

[0065] This step specifically includes:

[0066] S41. Verify the rationality of the genetic molecular sequence file to be analyzed;

[0067] S42. Analyze the basic characteristics of genetic analysis, including multilocus, high variability, and genetic stability;

[0068] S43. Remove interfering information from genetic molecular sequence tasks;

[0069] S44. Obtain and output the corresponding analysis results, specifically: summarize and analyze the obtained analysis results and generate an analysis report, which is then sent to the user in the form of a report or email.

[0070] In this step, by verifying, analyzing, and cleaning the genetic molecular sequence files to be analyzed, it is possible to effectively determine whether the files have been tampered with or damaged during the scheduling process, thereby ensuring the compliance of the files.

[0071] In this embodiment, by organically combining genetic molecular sequence analysis with microservice characteristics with scheduling algorithms, and selectively scheduling resources to the server platform based on pre-calculation, the processing time of the server is effectively reduced and the processing efficiency is improved by taking into account the utilization of server platform resources. At the same time, by hot-deploying or taking offline algorithm tasks in the container that are not in working state, only the characteristics of the genetic molecular sequence analysis algorithm need to be focused on without affecting the global algorithm state, which further improves the processing efficiency and maintenance efficiency.

[0072] It should be noted that if a deployment represents an instance file under a multi-cloud container cluster, it represents the resources and configuration required by an application. The task scheduling can be completed by executing this application file in a single-node environment or a cluster environment. After the required resources are scheduled, the application file is deployed to the server platform, thus completing one task scheduling.

[0073] The foregoing has provided a detailed description of a microservice-based genetic molecular sequence task scheduling method for cloud computing provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention, and the descriptions of the embodiments above are merely for the purpose of helping to understand the core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this invention.

Claims

1. A microservice-based genetic molecular sequence task scheduling method, characterized in that, The method includes the following steps: S1. Set up a server platform for the container to run and deploy the genetic molecular sequence analysis algorithm to the server platform; S2. Resource utilization of the computing server platform based on the running status of in-container algorithm tasks, specifically including: S21. Collect the running status of the algorithm task in the container on the server platform once every first preset time interval, and persist it. S22. Based on the data collected in step S21, the resource utilization rate of the server platform is predicted once every second preset time interval; S23. Based on the prediction results of step S22, calculate the remaining resource space of the server platform, expressed by the formula: In the formula, Indicates the server platform is The remaining space of resources at any given time. Indicates the server platform is Predicted results of time-based resource utilization; S24. Compare the remaining space of current server platform resources. With the first preset threshold Size relationship, if Then return to step S22, if Then proceed to step S25; S25, Calculation The difference between the predicted server platform resource utilization and the actual server platform resource utilization is calculated using the following formula: In the formula, This indicates the current running state of the algorithm task on the current server platform. Indicates the server platform is Time-based resource utilization rate; S26. Compare the current server platform resource utilization prediction results. With the second preset threshold The size relationship, if Then wait for the current moment to end, if Then proceed to step S27; S27, if If so, then perform a shrinkage operation. If so, then perform the expansion operation; S3. Read the genetic molecular sequence file to be analyzed from the disk into the server platform to generate the target task, and determine whether the remaining resources of the server platform meet the requirements of the target task. If they do, schedule the target task to the server platform. If they do not, return to step S2. S4. Analyze the target task to obtain the corresponding analysis results.

2. The microservice-based genetic molecular sequence task scheduling method as described in claim 1, characterized in that, The first preset time is 5 minutes, the second preset time is 60 minutes, the first preset threshold is 0.85, and the second preset threshold is 0.

5.

3. The microservice-based genetic molecular sequence task scheduling method as described in claim 2, characterized in that, In step S21, if the algorithm task in the container of the server platform is in a non-working state, the algorithm task in the non-working state will be hot-released or taken offline.

4. The microservice-based genetic molecular sequence task scheduling method as described in claim 3, characterized in that, In step S22, the server platform is The formula for calculating the predicted resource utilization rate at any given time is: In the formula, Indicates the server platform is Prediction results of time-based resource utilization. Indicates the number of moments.

5. The microservice-based genetic molecular sequence task scheduling method as described in claim 4, characterized in that, Step S3 further includes: if the remaining resources of the server platform do not meet the resources required by the target task, determine whether there is an algorithm task that is about to end in the server platform container. If there is, schedule the target task to the server platform. If there is not, return to step S2.

6. The microservice-based genetic molecular sequence task scheduling method as described in claim 5, characterized in that, Step S4 specifically includes: S41. Verify the rationality of the genetic molecular sequence file to be analyzed; S42. Analyze the basic characteristics of genetic analysis, including multilocus, high variability, and genetic stability; S43. Remove interfering information from genetic molecular sequence tasks; S44. Obtain the corresponding analysis results and output them.

7. The microservice-based genetic molecular sequence task scheduling method as described in claim 6, characterized in that, Step S44 specifically involves summarizing and analyzing the obtained analysis results and generating an analysis report, which is then sent to the user in the form of a report or email.

8. The microservice-based genetic molecular sequence task scheduling method as described in any one of claims 1-7, characterized in that, The server platform is Kubernetes, and the container is Docker.

Images