Computing power sharing method and device based on time-sharing management and control

Abstract

The invention relates to a computing power sharing method and device based on time-sharing management and control, and the method comprises the steps: separating user data and computing resources, dividing the user data and the computing resources into two different clusters for management, including a computing power cluster for combining all the computing resources and a storage cluster for combining all the hard disk resources, the computing power resources are virtualized; constructing a container mirror image and a virtual machine resource library; resource request information of a user is obtained from a user resource application form, then the resource request information of the user is analyzed according to a resource allocation and recovery rule, a computing power cluster is called in combination with the current resource condition to allocate computing power resources for the computing power cluster, and corresponding configuration data is obtained from a container mirror image and a virtual machine resource library to be allocated to the user for use; the method further comprises the steps of monitoring the use condition of the allocated computing power resources in real time, judging the obtained effective utilization rate, and determining whether the allocated computing power resources need to be recycled or not according to a judgment result, so that the effective utilization rate of the resources is improved.

Description

Technical Field

[0001] The present invention relates to the field of computing power allocation technology, and more specifically, to a computing power sharing method and device based on time-sharing control. Background Art

[0002] At present, with the rapid development of artificial intelligence and its wide application in the field of scientific research, more and more universities and scientific research institutions have or are deploying a large number of high-performance professional computing facilities. Compared with traditional CPU computing power, dedicated AI computing power equipment represented by GPU (hereinafter referred to as "computing power equipment") is usually expensive and has limited resources. Therefore, effective management and efficient use of these computing power equipment have become the key to improving scientific research efficiency and reducing operating costs. From the perspective of scientific research applications, there is an urgent need to achieve efficient management and dynamic sharing of these computing power devices in order to improve the utilization rate of computing power equipment and reduce the investment cost of computing power equipment. However, in actual universities / scientific research institutions, computing power equipment often has the following disadvantages when in use:

[0003] 1. Static allocation of computing power equipment. In most universities and research institutions, once computing power equipment is allocated to a specific project, it is difficult for other projects to use it even when the project is not active. However, this static allocation method makes it impossible to respond flexibly when resource demand changes, especially when resource utilization fluctuates greatly between peak and off-peak periods of scientific research, resulting in inefficient resource allocation and high costs.

[0004] 2. Computing power equipment is short of resources and over-resourced at critical stages. In a scientific research environment, the demand for computing power equipment usually increases sharply at critical stages (such as the data analysis period or before the paper submission deadline), especially when performing computationally intensive tasks such as data processing and model training. This periodic demand peak causes computing resources to be extremely tight in a short period of time, and is common when multiple scientific research projects reach a critical progress stage at the same time. This may not only lead to insufficient resources, seriously affecting the timely completion of the project and scientific research efficiency, but also cause a significant surplus of resources after the peak of demand. In order to cope with peak demand, scientific research teams often apply for a large number of computing power equipment, and these equipment are significantly inefficiently used after the critical stage, resulting in a surplus of resources.

[0005] 3. In particular, some teachers or teams did not actually use the computing resources after applying for them, or only used part of them, but other teachers did not know about it. This resulted in some teachers having no computing resources available, while some computing power was allocated to other teachers but was actually idle.

[0006] 4. When a device is shared by multiple people, there is often a problem that data is deleted by other teams, or the data cannot be saved continuously when a scientific research team uses it discontinuously. For example, Team A uses it in the morning and it is assigned to Team B in the afternoon. When Team A uses it again the next morning, yesterday's data may no longer exist or be deleted by Team B by mistake, which makes it impossible for Team A to use it continuously and intermittently. Summary of the invention

[0007] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art and provide a computing power sharing method and device based on time-sharing control, which is used to solve the problems of static resource allocation, inability to monitor allocated resources, and inability to continuously save data in existing resource usage.

[0008] The technical solution adopted by the present invention is a computing power sharing method based on time-sharing control, and the method comprises:

[0009] S1: User data and computing resources are separated and divided into two different clusters for management, including a computing cluster that combines all computing resources and a storage cluster that combines all hard disk resources. For the computing cluster, computing resources are virtualized using computing virtualization technology.

[0010] S2: Build container images and virtual machine resource libraries;

[0011] S3: Obtain the user's resource request information from the user's resource application form, and then parse the user's resource request information according to the resource allocation and recovery rules, and call the computing power cluster to allocate computing power resources for it based on the current resource situation, and obtain the corresponding configuration data from the container image and virtual machine resource library to allocate to the user;

[0012] In this application, user data and computing resources are stored separately so that user data can be continuously preserved, ensuring the continuity of user data when users use computing resources, and effectively avoiding the loss or deletion of user data after each reallocation of computing resources; and both the computing cluster and the storage cluster are built directly using existing computer equipment resources, without the need to add additional hardware, which is economical and practical; at the same time, the computing resources are virtualized and provided to users in the form of containers or virtual machines, so that the resource environment is isolated, and the scientific research environment of each user is independent and does not interfere with each other, and the user's computing operation process can be made transparent, which helps to monitor the use of computing power in real time and make real-time intelligent allocation and adjustment, thereby improving the effective utilization of resources.

[0013] Preferably, the method further includes real-time monitoring of the usage of the allocated computing resources, including:

[0014] S41: Collect usage information of allocated computing resources at fixed time intervals according to actual needs;

[0015] S42: Calculate the effective utilization rate of the allocated computing resources using a relevant algorithm according to the obtained usage information of the allocated computing resources;

[0016] S43: Determine the obtained effective utilization rate, and decide whether to recycle the allocated computing resources according to the determination result.

[0017] In this application, in order to further improve the effective utilization rate of computing power resources, the usage of computing power resources that have been allocated to users is also monitored in real time, so that the usage of the allocated computing power resources is open and transparent, avoiding the situation where the computing power resources applied for by users are not used or are not fully used. Adjustments can be made in time for these situations to achieve more effective sharing and use of computing power resources.

[0018] Further preferably, in step S42, the algorithm includes:

[0019] S421: For all computing power card resources applied for by a user in an application form, select a data with the highest utilization rate every n minutes to represent the utilization rate of the n minutes;

[0020] S422: For all utilization rates of all computing power card resources applied for by a user in an application form, when the utilization rate of a computing power card is greater than 0, it is recorded as 1, and then the number of computing power cards with utilization rates greater than 0 is finally calculated and recorded as u, and the number of all computing power cards applied for in the application form is recorded as a, and then the effective utilization rate η of the computing power cards applied for by a user in an application form within n minutes is calculated, η=u / a*100%;

[0021] S423: Count all effective utilization rates of the computing power cards in each time period of all time periods allocated in an application form of the user in one day, and take the maximum effective utilization rate in each time period as the effective utilization rate of the computing power cards in the application form in the time period;

[0022] S424: Collect statistics on all effective utilization rates calculated for all time periods allocated to an application form of a user within one day, and take the maximum effective utilization rate among all time periods as the effective utilization rate of all computing power cards applied for by the user for the application form on that day.

[0023] The effective utilization rate is calculated by dividing the user's computing power resource usage into several different time nodes, and the highest effective utilization rate calculated at each different time node is selected as the current effective utilization rate, thereby obtaining the most accurate effective utilization rate of the current user's computing power resource usage. The obtained effective utilization rate can intuitively observe the user's usage of the allocated computing power resources, so as to dynamically adjust the allocated resources in time, effectively avoiding the waste of computing power resources.

[0024] Preferably, the step S43 further includes setting a threshold of effective utilization rate.

[0025] When the final effective utilization rate η is 0, it means that the user has not used any computing power card resources applied for in the application form on that day, and the computing power card recycling action can be directly triggered according to the preset rules to automatically recycle the resources;

[0026] When the final effective utilization rate η is greater than 0 but less than the set threshold, it means that the user has only used part of the computing power card resources for the application on that day, and some computing power card resources can be recycled. The situation will be sent to the administrator and the user. If the user confirms that the computing power resources can be released, the computing power resources of the user will be automatically adjusted to release the idle part. If the user has no feedback, the administrator will assist in adjusting the idle computing power resources of the user, thereby realizing dynamic adjustment of resources during operation.

[0027] By using the preset effective utilization threshold and the calculated effective utilization rate, it can be determined whether the current user's use of the allocated computing resources meets the automatic or semi-automatic recovery conditions. If it meets the automatic recovery conditions, the user's computing resources whose utilization rate has always been 0 will be directly recovered and released; if it meets the semi-automatic recovery conditions, the user will be first asked whether the computing resources whose utilization rate has always been low can be recovered and released. If so, it will be released automatically. Otherwise, the administrator will access to assist in the adjustment, thereby achieving effective utilization of computing resources and avoiding resource waste.

[0028] Preferably, in step S1, the storage cluster uses distributed storage, and for each user, corresponding data storage space is allocated on the distributed storage according to his / her name; and multiple backup storage is used at the same time, and each user's data can be automatically saved in multiple copies to prevent loss.

[0029] By storing data in a distributed manner, corresponding storage space is allocated to each user in the distributed storage with the user's name. As a result, when each user uses it uninterruptedly, their data can be stored continuously and persistently as if it were stored on their own hard disk. This ensures that the data will not be released after the applied computing resources expire, and the next time you apply for computing resources again, you can continue to use the previous data directly by user name. This method of storing computing resources and data separately effectively improves the user experience.

[0030] Preferably, the method also includes time-sharing management and control of computing resources. When the computing resources are used in multiple time periods, and a user only uses them in a certain time period a day, the computing resources will be recovered and released and allocated to other users. The data in the computing resources will be stored in the data storage space of the storage cluster corresponding to the user's name. When the user is allocated computing resources in the next time period, the data stored in the previous time period will be automatically mounted to the newly allocated computing resources for the user to use again.

[0031] In actual use, the computing resources allocated to users are not necessarily used around the clock. Therefore, in order to improve the effective utilization of resources, computing resources are managed through time-sharing control, which can achieve cross-use of computing resources. In addition, with the help of user data stored in the storage cluster, it can also ensure that users can re-mount previously used data when computing resources are allocated in the next time period, thereby achieving effective utilization of resources while ensuring user experience.

[0032] Preferably, in S2, the configuration resources allocated by the container image and virtual machine resource library are automatically run only within the validity period of the computing resource allocation, and are promptly closed outside the validity period; and during the user's subsequent use, the required image resources can be uploaded for use according to actual needs.

[0033] Controlling the opening and closing of container images and virtual machines according to their validity period can effectively avoid resource waste. If there are no suitable image resources in the resource library, users can also upload images that meet their needs for use, thereby further improving the flexibility of computing resource use.

[0034] Preferably, the method also includes setting a timed task for the user's resource request information obtained from the user resource application form through a dedicated thread pool, and executing the opening and closing tasks according to the time nodes of the set computing power resource validity period.

[0035] By setting up a dedicated thread pool to set scheduled tasks, it is possible to effectively achieve automated, accurate and efficient management of user computing resource requests. Without the need for human intervention, resources can be automatically turned on and off according to user needs, optimizing resource scheduling, reducing resource waste, and improving system stability and performance.

[0036] On the other hand, the present application also provides a computing power sharing device based on time-sharing control, the device comprising:

[0037] A resource storage module, used to separate user data from computing resources, in which two different clusters are divided for management, including a computing power cluster that combines all computing resources and a storage cluster that combines all hard disk resources;

[0038] The container image and virtual machine resource library module is used to store the configuration resources required for the use of computing resources, including at least container images and virtual machine resources;

[0039] The resource allocation and recycling rule configuration module is used to receive the user's resource application form and obtain the user's resource request information from it, then parse the user's resource request information according to the resource allocation and recycling rules, and call the computing power cluster to allocate computing power resources for it in combination with the current resource situation, and then use the computing power card virtualization module to virtualize the computing power resources, obtain the corresponding configuration data from the container image and virtual machine resource library and allocate it to the user;

[0040] The computing power usage monitoring module is used to continuously monitor the computing power resources allocated to users to achieve more efficient sharing and use of resources;

[0041] The redundant resource notification module is used to send the monitoring results of the computing power usage monitoring module to the user or administrator, and to process the redundant computing resources in a timely manner.

[0042] In this device, different clusters are divided in the resource storage module to manage computing resources and user data, so that user data can be continuously saved, ensuring the continuity of user data when users use computing resources, and effectively avoiding the loss or deletion of user data after each reallocation of computing resources. In addition, both the computing cluster and the storage cluster are directly built using existing computer equipment resources, without the need to add additional hardware, which is economical and practical. At the same time, the computing resources are virtualized and provided to users in the form of containers or virtual machines, so that the resource environment is isolated, and the scientific research environment of each user is independent and does not interfere with each other. The user's computing power operation process can also be made transparent, which helps to monitor the use of computing power in real time and make real-time intelligent allocation and adjustment, improve the effective utilization of resources, and avoid the situation where the computing resources applied for by the user are not used or are not fully used. In response to these situations, the redundant resource notification module can be used to timely enable users and administrators to make corresponding adjustments, so as to achieve more effective sharing and use of computing resources.

[0043] Preferably, the computing power usage monitoring module mainly includes:

[0044] Utilization calculation component: used to calculate the allocated computing resources and obtain the effective utilization rate of the computing resources requested by the user;

[0045] Resource recovery judgment component: used to compare the obtained effective utilization value with the preset threshold, and judge whether the user's computing power resource usage meets the automatic recovery conditions based on the comparison result. If it meets the conditions, the computing power resource recovery action is directly triggered to automatically recycle the resources; if it does not meet the conditions, the situation is sent to the user or administrator through the redundant resource notification module for further processing.

[0046] In the computing power usage monitoring module, by setting relevant components, the effective utilization rate of the allocated computing power resources can be directly calculated, so that the usage of the resources allocated to the user can be intuitively judged, thereby avoiding resource waste.

[0047] Preferably, in the redundant resource notification module, if a resource usage situation that does not meet the automatic recovery conditions is received from the resource recovery judgment component, the situation is sent to the administrator and the user. If the user confirms that the computing power resources can be released, the computing power resources of the user are automatically adjusted to release the idle part. If the user does not provide feedback, the administrator will assist in adjusting the user's idle computing power resources, thereby realizing dynamic adjustment of resources during the operation process.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. Improved resource utilization: The time-sharing allocation of computing resources improves the utilization efficiency of computing equipment and avoids equipment idleness.

[0050] 2. Continuous preservation of user data to ensure the continuity of user business data: Based on the mechanism of separating computing power clusters and storage clusters, user usage process data is stored based on distributed storage, thereby ensuring the continuity of user business data during the dynamic adjustment and allocation of computing resources, and avoiding the loss or accidental deletion of user data after each reallocation.

[0051] 3. Transparent use of computing power cards and intelligent dynamic adjustment: The main feature of the traditional allocation method is the static allocation method. During the use process after allocation, the use of the computing power cards is unclear and opaque, which may lead to some user research teams applying for computing power resources but not actually using them or not reusing them, resulting in redundancy. Therefore, the present invention monitors the effective utilization rate of the computing power cards in real time during use, can promptly discover the situation of non-use or under-use, and through automatic or semi-automatic adjustment, realize more effective sharing and use of computing power resources.

[0052] 4. Resource and environment isolation: The scientific research environments of each user team are independent of each other and will not interfere with each other, ensuring the stability and security of the research work. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of the method flow provided in this embodiment.

[0054] Figure 2 This is a schematic diagram of the device structure provided in this embodiment. DETAILED DESCRIPTION

[0055] The drawings of the present invention are only for illustrative purposes and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0056] Example 1

[0057] like Figure 1 As shown, this embodiment provides a computing power sharing method based on time-sharing control, and the method includes:

[0058] Step S1: user data and computing resources are separated and divided into two different clusters for management, including a computing power cluster that combines all computing resources and a storage cluster that combines all hard disk resources. For the computing power cluster, computing power virtualization technology is used to virtualize the computing power resources; wherein the computing resources can be CPU, computing power card, memory resources, etc.

[0059] Preferably, in step S1, the storage cluster uses distributed storage, and for each user, corresponding data storage space is allocated on the distributed storage according to his / her name; and multiple backup storage is used at the same time, and each user's data can be automatically saved in multiple copies to prevent loss.

[0060] By storing data in a distributed manner, corresponding storage space is allocated to each user in the distributed storage with the user's name. As a result, when each user uses it uninterruptedly, their data can be stored continuously and persistently as if it were stored on their own hard disk. This ensures that the data will not be released after the applied computing resources expire, and the next time you apply for computing resources again, you can continue to use the previous data directly by user name. This method of storing computing resources and data separately effectively improves the user experience.

[0061] Step S2: Build a container image and virtual machine resource library; specifically, container or virtual machine images that support various machine learning environments such as Python, PyTorch, Jupyter Notebook, MATLAB, etc. can be built through container image and virtual machine resource management, and stored in the container image and virtual machine resource library.

[0062] Preferably, in step S2, the configuration resources allocated by the container image and virtual machine resource library are automatically run only within the validity period of the computing resource allocation, and are promptly closed outside the validity period; and during the user's subsequent use, the required image resources can be uploaded for use according to actual needs.

[0063] Controlling the opening and closing of container images and virtual machines according to their validity period can effectively avoid resource waste. If there are no suitable image resources in the resource library, users can also upload images that meet their needs for use, thereby further improving the flexibility of computing resource use.

[0064] Step S3: Obtain the user's resource request information from the user resource application form, and then parse the user's resource request information according to the resource allocation and recovery rules, and call the computing power cluster to allocate virtualized computing power resources to it in combination with the current resource situation, and obtain the corresponding configuration data from the container image and virtual machine resource library to allocate to the user;

[0065] In this embodiment, when computing power virtualization technology is used to virtualize computing power resources, the memory partitioning tool provided by the computing power card manufacturer is used to achieve fine-grained segmentation of the memory of the high-performance computing power card. Taking a computing power card equipped with 48GB of video memory as an example, it is evenly divided into 6 independent 8GB video memory units and allocated to different virtual machines or containers. Through this fine-grained resource segmentation, a single physical computing power card can simultaneously support the parallel computing needs of multiple virtual machines or containers.

[0066] Therefore, in this embodiment, by storing user data and computing resources separately, user data can be continuously preserved, ensuring the continuity of user data when users use computing resources, and effectively avoiding the loss or deletion of user data after each reallocation of computing resources; and both the computing power cluster and the storage cluster are directly built using existing computer equipment resources, without the need to add additional hardware, which is economical and practical; at the same time, the computing power resources are virtualized and provided to users in the form of containers or virtual machines, so that the resource environment is isolated, and the scientific research environment of each user is independent and does not interfere with each other. It can also make the user's computing power operation process transparent, which helps to monitor the use of computing power in real time and make real-time intelligent allocation and adjustment, thereby improving the effective utilization of resources.

[0067] Preferably, in this embodiment, a special thread pool is also set up to set scheduled tasks for the user's resource request information obtained from the user resource application form, and to execute the opening and closing tasks according to the time nodes of the set computing power resource validity period.

[0068] Specifically, when the scientific research team puts forward resource requirements, the administrator can configure or adjust the computing resources mounted on the container / virtual machine and set the validity period of the computing resources according to the actual situation of the scientific research team through resource allocation and recycling rules configuration, wherein the computing resources are mainly configured as: number of CPU cores, memory size, disk size and computing power card video memory size. When computing resources are allocated, the real-time hardware configuration data of the current device will be obtained to effectively prevent the equipment from crashing due to excessive allocation of hardware configuration; and when allocating resources, a certain period of time can be specified to divide the computing power resources of a certain configuration, for example, from 8:00 to 12:00 every day from the 1st of the next month to the 5th of the next month, a virtual machine or container with a video memory size of 24GB is divided, and the background can realize the task of opening and closing the container or virtual machine at the set time node (the start and end time node of the computing power resource validity period) through the scheduled task set in the thread pool. And when some containers or virtual machines cannot be closed on time, the next batch of containers / virtual machines cannot be opened, which effectively prevents the computing power equipment from crashing due to the simultaneous use of too many resources.

[0069] Therefore, in this embodiment, by setting a dedicated thread pool to set scheduled tasks, it is possible to effectively realize the automated, accurate and efficient management of user computing resource requests. Without the need for human intervention, resources can be automatically turned on and off according to user needs, thereby optimizing resource scheduling, reducing resource waste, and improving system stability and performance.

[0070] Preferably, in this embodiment, the method further includes real-time monitoring of the usage of the allocated computing resources when the user logs in to use the computing resources, including:

[0071] Step S41: Collect usage information of allocated computing resources at fixed time intervals according to actual needs;

[0072] Step S42: Calculate the effective utilization rate of the allocated computing resources using a relevant algorithm according to the usage information of the allocated computing resources;

[0073] Further preferably, in step S42, the algorithm includes the following steps, and in this embodiment, the fixed time interval is set to 1 minute and 5 minutes, specifically:

[0074] Step S421: For all computing power card resources applied for by a user in an application form, select a data with the highest utilization rate every 1 minute to represent the utilization rate of that minute, and select a point with the highest utilization rate every 5 minutes as the utilization rate of the computing power card within 5 minutes;

[0075] Step S422: For all utilization rates of all computing power card resources applied for in one application form by one user, when the utilization rate of a computing power card is greater than 0, it is recorded as 1, and then the number of computing power cards with utilization rates greater than 0 is finally calculated as u, and the number of all computing power cards applied for in the application form is recorded as a, and then the effective utilization rate η of the computing power cards applied for in one application form by one user within 5 minutes is calculated, η=u / a*100%;

[0076] Step S423: collect statistics on all effective utilization rates of the computing power cards in each time period of all time periods allocated to an application form of the user in one day, and take the maximum effective utilization rate in each time period as the effective utilization rate of the computing power card in the application form in the time period;

[0077] Step S424: statistics are collected on all effective utilization rates calculated for all time periods allocated to an application form of a user within one day, and the maximum effective utilization rate among all time periods is taken as the effective utilization rate of all computing power cards applied for by the user for the application form on that day.

[0078] Therefore, by dividing the user's computing power resource usage into several different time nodes to calculate its effective utilization rate, and selecting the highest calculated effective utilization rate at each different time node as the current effective utilization rate, the most accurate effective utilization rate of the current user's computing power resource usage can be obtained. The obtained effective utilization rate can be used to intuitively observe the user's usage of the allocated computing power resources, so as to dynamically adjust the allocated resources in time, effectively avoiding the waste of computing power resources.

[0079] Preferably, step S43: judge the effective utilization rate obtained, and decide whether to recycle the allocated computing resources according to the judgment result, which can effectively prevent some scientific research teams from occupying the resources for a long time without using them after obtaining them, and set up an automatic recycling mechanism. For example, if the resources applied for by a scientific research team are not used for 24 consecutive hours, the system will automatically perform the recycling operation, thereby avoiding the situation where resources are occupied by a team for a long time without use.

[0080] Specifically, in step S43, a threshold of the effective utilization rate is first set, and then the obtained effective utilization rate is compared with the set threshold. If the computing power is not fully utilized, the effective utilization rate can be adjusted accordingly.

[0081] When the final effective utilization rate η is 0, it means that the user has not used any computing power card resources applied for in the application form on that day, and the computing power card recycling action can be directly triggered according to the preset rules to automatically recycle the resources;

[0082] When the final effective utilization rate η is greater than 0 but less than the set threshold, it means that the user has only used part of the computing power card resources for the application on that day, and some computing power card resources can be recycled. The situation will be sent to the administrator and the user. If the user confirms that the computing power resources can be released, the computing power resources of the user will be automatically adjusted to release the idle part. If the user has no feedback, the administrator will assist in adjusting the idle computing power resources of the user, thereby realizing dynamic adjustment of resources during operation.

[0083] Among them, the situation where computing power is not fully utilized is shown in Table 1, and corresponding adjustments can be made.

[0084]

[0085]

[0086] Table 1

[0087] Therefore, based on the preset effective utilization rate threshold and the calculated effective utilization rate, it can be determined whether the current user's use of the allocated computing resources meets the automatic or semi-automatic recovery conditions. If it meets the automatic recovery conditions, the part of the user's computing resources whose utilization rate has always been 0 will be directly recovered and released; if it meets the semi-automatic recovery conditions, the user will be first asked whether the part of the computing resources whose utilization rate has always been low can be recovered and released. If so, it will be automatically released. Otherwise, the administrator will access to assist in the adjustment, thereby achieving effective utilization of computing resources and avoiding waste of resources.

[0088] Preferably, the method described in this embodiment also includes time-sharing management and control of computing resources. When the computing resources are used in multiple time periods, and the user only uses a certain time period a day, the computing resources will be recovered and released and allocated to other users. The data in the computing resources will be stored in the data storage space of the storage cluster corresponding to the user's name. When the user is allocated computing resources in the next time period, the data stored in the previous time period will be automatically mounted to the newly allocated computing resources for the user to use again.

[0089] In actual use, when the user's scientific research team completes scientific research tasks, the computing equipment will not be fully loaded all the time. Therefore, in order to improve the effective utilization of resources, computing resources are managed through time-sharing control. For example, when Team A uses it in time period T1, it uses the computing resources on server S1. When it uses it in time period T2, if it is dynamically allocated to the computing resources of server S2, it can use the same data, which does not affect the use and experience of Team A at all.

[0090] This enables cross-use of computing resources, and with the help of user data stored in the storage cluster, it can also ensure that users can re-mount previously used data when computing resources are allocated in the next time period, thereby achieving effective utilization of resources while ensuring that users have a good user experience.

[0091] Example 2

[0092] like Figure 2 As shown, this embodiment provides a computing power sharing device based on time-sharing control, and the device at least includes:

[0093] A container image and virtual machine resource management module is connected to the container image and virtual machine resource library and is used to manage the container image and virtual machine resource library;

[0094] The container image and virtual machine resource library module is used to store the configuration resources required for the use of computing resources, including at least container images and virtual machine resources;

[0095] The resource storage module is used to separate user data from computing resources. Two different clusters are divided in the resource storage module for management, including a computing cluster that combines all computing resources and a storage cluster that combines all hard disk resources. The computing resources include CPU, computing card, memory, etc.

[0096] The resource allocation and recycling rule configuration module is used to receive the user's resource application form and obtain the user's resource request information from it, then parse the user's resource request information according to the resource allocation and recycling rules, and call the computing power cluster to allocate computing power resources for it in combination with the current resource situation, and then use the computing power card virtualization module to virtualize the computing power resources, obtain the corresponding configuration data from the container image and virtual machine resource library and allocate it to the user;

[0097] The computing power usage monitoring module is used to continuously monitor the computing power resources allocated to users to achieve more efficient sharing and use of resources; wherein, the computing power usage monitoring module also includes:

[0098] Utilization calculation component: used to calculate the allocated computing resources and obtain the effective utilization rate of the computing resources requested by the user;

[0099] Resource recovery judgment component: used to compare the obtained effective utilization value with the preset threshold, and judge whether the user's computing power resource usage meets the automatic recovery conditions based on the comparison result. If it meets the conditions, the computing power resource recovery action is directly triggered to automatically recycle the resources; if it does not meet the conditions, the situation is sent to the user or administrator through the redundant resource notification module for further processing.

[0100] In the computing power usage monitoring module, by setting relevant components, the effective utilization rate of the allocated computing power resources can be directly calculated, so that the usage of the resources allocated to the user can be intuitively judged, thereby avoiding resource waste.

[0101] The redundant resource notification module is used to send the monitoring results of the computing power usage monitoring module to the user or administrator, and to handle the redundant computing resources in a timely manner.

[0102] Specifically, in the redundant resource notification module, if a resource usage situation that does not meet the automatic recovery conditions is received from the resource recovery judgment component, the situation is sent to the administrator and the user. If the user confirms that the computing resources can be released, the computing resources of the user are automatically adjusted to release the idle part. If the user does not provide feedback, the administrator will assist in adjusting the user's idle computing resources, thereby realizing dynamic adjustment of resources during the operation process.

[0103] Among them, when allocating resources, firstly, the computing power card virtualization module is used to virtualize the computing power resources stored in the resource storage module using the computing power virtualization technology, and then the resource allocation and recycling rule configuration module is used to parse the user application form, and the resource allocation information is transmitted to the resource allocation component for allocation processing according to the application form information obtained by the analysis, and at the same time, the container image and virtual machine information resource library are called to hang in the corresponding configuration environment. After the allocation, the user logs in through the resource login management module, and then the resource allocation component allocates resources to the user according to the user login and opens the storage space of the storage cluster corresponding to the user name to store its data;

[0104] In addition, during the user's use process, it also includes using the computing power usage monitoring module to continuously monitor the computing power resources allocated to the user, and using the utilization calculation component to calculate the effective utilization rate of the computing power resources applied for by the user, including: first, collecting the usage information of the allocated computing power resources at fixed time intervals according to actual needs, and then using the relevant algorithm to calculate its effective utilization rate based on the obtained usage information of the allocated computing power resources, and finally using the resource recovery judgment component to judge the obtained effective utilization rate, and decide whether the allocated computing power resources need to be recovered based on the judgment result.

[0105] Specifically:

[0106] For all the computing power card resources applied for by a user in an application form, select the data with the highest utilization rate every 1 minute to represent the utilization rate of that minute, and select the point with the highest utilization rate every 5 minutes as the utilization rate of the computing power card within 5 minutes;

[0107] For all utilization rates of all computing power card resources applied for by a user in an application form, when the utilization rate of a computing power card is greater than 0, it is recorded as 1, and then the number of computing power cards with utilization rates greater than 0 is recorded as u, and the number of all computing power cards applied for in the application form is recorded as a. Then calculate the effective utilization rate η of the computing power cards applied for by a user in an application form within 5 minutes, η=u / a*100%;

[0108] Collect statistics on all effective utilization rates of the hashing cards in each time period of all time periods allocated to an application form of the user in one day, and take the maximum effective utilization rate in each time period as the effective utilization rate of the hashing card in the application form in that time period;

[0109] Statistics are collected for all effective utilization rates calculated for all time periods allocated to an application form of a user within one day, and the maximum effective utilization rate among all time periods is taken as the effective utilization rate of all computing power cards applied for by the user for the application form on that day.

[0110] Therefore, by dividing the user's computing power resource usage into several different time nodes to calculate its effective utilization rate, and selecting the highest calculated effective utilization rate at each different time node as the current effective utilization rate, the most accurate effective utilization rate of the current user's computing power resource usage can be obtained. The obtained effective utilization rate can be used to intuitively observe the user's usage of the allocated computing power resources, so as to dynamically adjust the allocated resources in time, effectively avoiding the waste of computing power resources.

[0111] In the resource recovery judgment component, the resource allocation and recovery rules are used to determine whether the user's computing power resource usage meets the automatic recovery conditions, and the situation of insufficient computing power usage is sent to the user and administrator through the redundant resource notification module, and the situation of insufficient computing power usage is adjusted in time with the help of the resource release adjustment component. Among them, the situation of insufficient computing power usage mainly includes the following two situations:

[0112] ① If the effective utilization rate η of some computing power cards is always 0, it means that the user has not used any computing power card resources applied for in the application form on that day. The computing power card recycling action can be directly triggered according to the preset rules to automatically release the resources with an effective utilization rate η of 0;

[0113] ② There are situations where the utilization rate of multiple computing power cards is continuously low. For example, the final effective utilization rate η is greater than 0 but less than the set threshold, which indicates that the user has only used part of the computing power card resources for the application form on that day, and some computing power card resources can be recycled. In this case, the situation will be sent to the administrator and the user. If the user confirms that the computing power resources can be released, the computing power resources of the user will be automatically adjusted to release the idle part. If the user has no feedback, the administrator will assist in adjusting the idle computing power resources of the user. Tasks used on multiple cards can be merged into the same card to release the excess computing power card resources, thereby realizing dynamic adjustment of resources during operation.

[0114] Therefore, in this device, different clusters are divided in the resource storage module to manage computing resources and user data, so that user data can be continuously saved, ensuring the continuity of user data when users use computing resources, and effectively avoiding the loss or deletion of user data after each reallocation of computing resources. In addition, both the computing cluster and the storage cluster are directly built using existing computer equipment resources, without the need to add additional hardware, which is economical and practical. At the same time, the computing resources are virtualized and provided to users in the form of containers or virtual machines, so that the resource environment is isolated, and the scientific research environment of each user is independent and does not interfere with each other. It can also make the user's computing power operation process transparent, which is helpful to monitor the use of computing power in real time and make real-time intelligent allocation and adjustment, improve the effective utilization of resources, and avoid the situation where the computing resources applied by the user are not used or are not fully used. In response to these situations, with the help of the redundant resource notification module, users and administrators can make corresponding adjustments in time to achieve more effective sharing and use of computing resources.

[0115] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A computing power sharing method based on time-sharing control, the method comprising: S1: User data and computing resources are separated and divided into two different clusters for management, including a computing cluster that combines all computing resources and a storage cluster that combines all hard disk resources. For the computing cluster, computing resources are virtualized using computing virtualization technology. S2: Build container images and virtual machine resource libraries; S3: Obtain the user's resource request information from the user's resource application form, and then parse the user's resource request information according to the resource allocation and recovery rules, and call the computing power cluster to allocate computing power resources for it based on the current resource situation, and obtain the corresponding configuration data from the container image and virtual machine resource library to allocate to the user; Characterized in that the method further includes real-time monitoring of the usage of the allocated computing resources, including: S41: Collect usage information of allocated computing resources at fixed time intervals according to actual needs; S42: Calculate the effective utilization rate of the allocated computing resources using a relevant algorithm according to the obtained usage information of the allocated computing resources; S43: Determine the obtained effective utilization rate, and decide whether to recycle the allocated computing resources according to the determination result.

2. A computing power sharing method based on time-sharing control according to claim 1, characterized in that: In step S42, the algorithm includes: S421: For all computing power card resources applied for by a user in an application form, select a data with the highest utilization rate every n minutes to represent the utilization rate of the n minutes; S422: For all utilization rates of all computing power card resources applied for by a user in an application form, when the utilization rate of a computing power card is greater than 0, it is recorded as 1, and then the number of computing power cards with utilization rates greater than 0 is finally calculated and recorded as u, and the number of all computing power cards applied for in the application form is recorded as a, and then the effective utilization rate η of the computing power cards applied for by a user in an application form within n minutes is calculated, η=u / a*100%; S423: Count all effective utilization rates of the computing power cards in each time period of all time periods allocated in an application form of the user in one day, and take the maximum effective utilization rate in each time period as the effective utilization rate of the computing power cards in the application form in the time period; S424: Collect statistics on all effective utilization rates calculated for all time periods allocated to an application form of a user within one day, and take the maximum effective utilization rate among all time periods as the effective utilization rate of all computing power cards applied for by the user for the application form on that day.

3. A computing power sharing method based on time-sharing control according to claim 2, characterized in that: The step S43 also includes setting a threshold value of effective utilization rate. When the final effective utilization rate η is 0, it means that the user has not used any computing power card resources applied for in the application form on that day, and the computing power card recycling action can be directly triggered according to the preset rules to automatically recycle the resources; When the final effective utilization rate η is greater than 0 but less than the set threshold, it means that the user has only used part of the computing power card resources for the application on that day, and some computing power card resources can be recycled. The situation will be sent to the administrator and the user. If the user confirms that the computing power resources can be released, the computing power resources of the user will be automatically adjusted to release the idle part. If the user has no feedback, the administrator will assist in adjusting the idle computing power resources of the user, thereby realizing dynamic adjustment of resources during operation.

4. A computing power sharing method based on time-sharing control according to claim 3, characterized in that: In step S1, the storage cluster uses distributed storage, and allocates corresponding data storage space on the distributed storage according to the name of each user; and using multi-backup storage, each user's data can be automatically saved in multiple copies to prevent loss.

5. A computing power sharing method based on time-sharing control according to claim 4, characterized in that: The method also includes time-sharing management and control of computing resources. When the computing resources are used in multiple time periods, if a user only uses them in a certain time period in a day, the computing resources will be recovered and released and allocated to other users. The data in the computing resources will be stored in the data storage space of the storage cluster corresponding to the user's name. When the user is allocated computing resources in the next time period, the data stored in the previous time period will be automatically mounted to the newly allocated computing resources for the user to use again.

6. A computing power sharing method based on time-sharing control according to claim 5, characterized in that: In S2, the configuration resources allocated by the container image and virtual machine resource library are automatically run only within the validity period of the computing resource allocation, and are promptly closed outside the validity period; and in the subsequent use process of the user, the required image resources can be uploaded for use according to actual needs.

7. A computing power sharing method based on time-sharing control according to any one of claims 1 to 6, characterized in that: The method also includes setting a timed task for the user's resource request information obtained from the user resource application form through a dedicated thread pool, and executing the opening and closing tasks according to the time nodes of the set computing power resource validity period.

8. A computing power sharing device based on time-sharing control, characterized in that: The device comprises: A resource storage module, used to separate user data from computing resources, in which two different clusters are divided for management, including a computing power cluster that combines all computing resources and a storage cluster that combines all hard disk resources; The container image and virtual machine resource library module is used to store the configuration resources required for the use of computing resources, including at least container images and virtual machine resources; The resource allocation and recycling rule configuration module is used to receive the user's resource application form and obtain the user's resource request information from it, then parse the user's resource request information according to the resource allocation and recycling rules, and call the computing power cluster to allocate computing power resources for it in combination with the current resource situation. At the same time, according to the allocated computing power resources, the corresponding configuration data is obtained from the container image and virtual machine resource library and allocated to the user; The computing power usage monitoring module is used to continuously monitor the computing power resources allocated to users to achieve more efficient sharing and use of resources; The redundant resource notification module is used to send the monitoring results of the computing power usage monitoring module to the user or administrator, and to process the redundant computing resources in a timely manner.

9. A computing power sharing device based on time-sharing control according to claim 8, characterized in that: The computing power usage monitoring module mainly includes: Utilization calculation component: used to calculate the allocated computing resources and obtain the effective utilization rate of the computing resources requested by the user; Resource recovery judgment component: used to compare the obtained effective utilization value with the preset threshold, and judge whether the user's computing power resource usage meets the automatic recovery conditions based on the comparison result. If it meets the conditions, the computing power resource recovery action is directly triggered to automatically recycle the resources; if it does not meet the conditions, the situation is sent to the user or administrator through the redundant resource notification module for further processing.

10. A computing power sharing device based on time-sharing control according to claim 9, characterized in that: In the redundant resource notification module, if a resource usage situation that does not meet the automatic recovery conditions is received from the resource recovery judgment component, the situation will be sent to the administrator and the user. If the user confirms that the computing power resources can be released, the computing power resources of the user will be automatically adjusted to release the idle part. If the user does not provide feedback, the administrator will assist in adjusting the user's idle computing power resources, thereby realizing dynamic adjustment of resources during the operation process.

Images