Multi-level resource pool intelligent scheduling method and device for urban rail cloud platform

Abstract

The invention relates to an urban rail cloud platform-oriented multi-level resource pool intelligent scheduling method and equipment, which are applied to a rail transit service system, and the method comprises the following steps: automatically dividing resource pools of different levels by using an application system operation behavior model and an application health state analysis model according to a CPU super-division proportion, a memory super-division proportion and whether a KSM is forbidden or not; according to the importance and characteristics of each service module, the thermal migration capability of the virtual machine and the current time period, respectively scheduling the service modules to resource pools of different levels by using an application health state analysis model and an application peak running time analysis model; and receiving system operation condition feedback data, and optimizing the application system operation behavior model and the application health state analysis model according to the feedback data. Compared with the prior art, the method has the advantages that the memory resource demand is reduced and the effective utilization rate of the memory is improved while the allocated virtual machine can meet the actual operation demand of the system.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular, to a multi-level resource pool intelligent scheduling method and device for an urban rail cloud platform. Background Art

[0002] The cloudification of rail transit is one of the important directions for the development of urban rail transit systems. By building a unified cloud platform, realizing data interconnection and sharing, and promoting intelligent applications, etc., the operation efficiency can be improved, the operation cost can be reduced, and the intelligent development can be promoted. Urban rail cloud is a cloud computing platform designed for urban rail transit operation management. Currently, many cities have completed the construction of urban rail cloud, and non-safety systems have been running on the urban rail cloud platform. However, since rail transit-related systems are mainly designed for physical machines, after moving to the cloud, only the physical machines are replaced with virtual machines, and the required resources have not been reduced, resulting in that the overall resource volume of the cloud platform has not decreased significantly compared with the use of physical machines. At the same time, due to system design problems, the memory and CPU usage are unbalanced, the memory utilization rate is very high, the CPU utilization rate is very low, and the overall hardware resource utilization rate is very low.

[0003] Chinese Patent Application CN115686856A discloses a cloud computing resource prediction method. By analyzing the load data of different service types on the cloud platform, predicting its service type, and allocating load resources according to the prediction results, the accuracy of load situation prediction and the utilization rate of computing resources on the cloud platform are improved. At the same time, this invention applies control theory to the resource allocation system, making the cloud resource allocation a dynamically adjustable process, thus greatly improving the resource utilization rate of the load, reducing the latency of cloud service execution, and improving the security of the overall system.

[0004] The above patent application focuses on predicting and analyzing the existing load data of different service types on the cloud platform to optimize the resource allocation scheme, improve the resource utilization rate, and reduce the service execution delay. It is not highly adaptable to new services, especially rail transit-related systems, and does not well solve the problem of unbalanced memory and CPU usage caused by service systems different from ordinary services within the rail transit system, where the memory utilization rate is very high, the CPU utilization rate is very low, and the overall hardware resource utilization rate is very low. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a multi-level resource pool intelligent scheduling method and device for an urban rail cloud platform, which can ensure that the allocated virtual machines can meet the actual operation requirements of the system while reducing the memory resource requirements and improving the effective utilization rate of memory.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A multi-level resource pool intelligent scheduling method for an urban rail cloud platform, which is applied to a rail transit service system. The method includes:

[0008] Automatically divide resource pools of different levels according to the CPU overcommitment ratio, memory overcommitment ratio, and whether KSM is disabled, using the application system operation behavior model and the application health status analysis model; schedule business modules to resource pools of different levels respectively according to the importance, characteristics, virtual machine live migration ability, and current time period of each business module, using the application health status analysis model and the application peak operation time analysis model; receive system operation feedback data, and optimize the application system operation behavior model and the application health status analysis model according to the feedback data.

[0009] Further, the resource pools are divided into at least 3 levels according to the CPU overcommitment ratio, memory overcommitment ratio, and whether KSM is disabled, and each level of resource pool includes multiple virtual machines.

[0010] Furthermore, the hardware CPUs and memories of the multiple virtual machines are the same, and the actually allocable CPUs and memories of a single virtual machine in resource pools of different levels change with the resource pool level.

[0011] Further, the initial division process of the resource pools includes:

[0012] Collect the status data of business modules in the rail transit service system, label the data through the application system operation behavior model and annotate it using expert experience to obtain a rail transit multi-professional application portrait database;

[0013] Run the services on preset different resource pools, obtain the operation status data, and respectively score the operation status data using the application health status analysis model, sort and extract values from the operation status data using the resource pool matching algorithm, and annotate the operation status data using expert experience to obtain a resource pool grading database;

[0014] Match the operation status data with the rail transit multi-professional application portrait database to obtain application portrait labels, combine with the resource pool grading database to obtain the initial resource pools to which different business modules belong, and finally calculate the initial resource pool size, the initial resource pool grading division result, and the record of the resource pools to which the business modules belong in combination with the actual resource pool data.

[0015] Further, the business modules include algorithmic business modules, memory cache business modules, and offline analysis business modules.

[0016] Furthermore, the characteristics of the business modules include:

[0017] Algorithm-based business modules require high-performance CPUs; memory cache-based business modules require a large amount of memory; the CPU and memory requirements of offline analysis-based business modules peak and valley over time.

[0018] Furthermore, the training process of the application system operation behavior model is as follows:

[0019] Obtain the static attributes preset during system deployment and the CPU usage, memory occupancy, IO metrics, and call conditions of each business module in the resource pool during system operation, and perform data processing; establish an operation behavior analysis data set using the processed data; train the application system operation behavior model through mechanical algorithms based on the operation behavior analysis data set;

[0020] The training process of the application health status analysis model is as follows:

[0021] Obtain the health status data of the system at rest and during operation, and perform data processing; establish a health status analysis data set using the processed data; train the application health status analysis model through mechanical algorithms based on the health status analysis data set;

[0022] The training process of the application peak operation time analysis model is as follows:

[0023] Obtain the operation status data of each level of the application pool and the operation status data of each business module in the system, and perform data processing; establish an operation time analysis data set using the processed data; train the application peak operation time analysis model through mechanical algorithms based on the operation time analysis data set.

[0024] Furthermore, the processing includes: data cleaning, abnormal data processing, data formatting, data normalization, and text processing and extraction.

[0025] Furthermore, the process of scheduling business modules to different levels of resource pools includes:

[0026] Real-time collect the operation status data of business modules, score them through the application health status analysis model. If the score is less than the minimum threshold, schedule the business module to a higher-level resource pool; if the score is greater than the maximum threshold, after confirmation by the user, schedule the business module to a lower-level resource pool.

[0027] Furthermore, the process of scheduling business modules to different levels of resource pools includes:

[0028] Real-time collect the operation status data of business modules and the operation status data of resource pools. For the characteristics of each business module, use the application peak operation time analysis model to analyze the peak operation time periods of each business module;

[0029] Using the peak operation time periods of each business module and the operation status data of the resource pool, obtain the peak operation time period of the low-level resource pool and the operation gap period of the high-level resource pool, and obtain the overlapping time period between the two;

[0030] During the overlapping time period, schedule the business modules in the low-level resource pool to run in the high-level resource pool.

[0031] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the multi-level resource pool intelligent scheduling method for the urban rail cloud platform as described above are implemented.

[0032] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the steps of the multi-level resource pool intelligent scheduling method for the urban rail cloud platform as described above are implemented.

[0033] Compared with the prior art, the beneficial effects of the present invention include:

[0034] 1. By dividing resource pools of different levels, the present invention schedules different services to appropriate resource pools, making full use of the resources of each level of resource pool and ensuring the normal operation of each business module; each level of resource pool is automatically divided according to the CPU overcommitment ratio, memory overcommitment ratio, and whether to enable KSM, using the application system operation behavior model and the application health status analysis model, making the resource allocation of the resource pool more reasonable and reducing human subjectivity; according to the importance, characteristics, virtual machine live migration ability of the service, and the current time period, using the application health status analysis model and the application peak operation time analysis model, the business modules are respectively scheduled to resource pools of different levels, realizing the dynamic scheduling of services in resource pools of different levels without affecting the normal operation of the rail transit business system, reducing the overall hardware resource requirements of the system, and using the model makes the scheduling more timely and rapid, ensuring the maximization of resource utilization; optimizing the application system operation behavior model according to the feedback data, having self-adaptability;

[0035] 2. The present invention uses resource pools to re-divide the available resources of existing multiple virtual machines, can utilize the existing resources, has strong flexibility and scalability, and in addition, different resource pools are allocated according to different services, improving the effective utilization rate of memory while avoiding the decline of virtual machine performance;

[0036] 3. The present invention can perform dynamic scheduling on business modules, migrate business modules in the low-level resource pool to run during the overlapping time period of the high-level resource pool, realize the maximization of resource utilization, and save the overall hardware resource requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is the flowchart of the present invention;

[0038] Figure 2 This is the flowchart of the initial resource pool division of the present invention;

[0039] Figure 3 This is the flowchart of the dynamic scheduling of the resource pool;

[0040] Figure 4 This is the architecture diagram of the resource pool in an embodiment of the present invention;

[0041] Figure 5 This is the schematic diagram of the intelligent scheduling of the resource pool of the present invention;

[0042] Figure 6 This is the scheduling example diagram of the rail transit AFC and off-line data analysis system in an embodiment of the present invention;

[0043] Figure 7 This is the scheduling example diagram of the rail transit ATS and intelligent operation and maintenance system in an embodiment of the present invention. Detailed implementation manners

[0044] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0045] Embodiment 1

[0046] This embodiment aims to disclose a multi-level resource pool intelligent scheduling method for an urban rail cloud platform, which is applied to a rail transit service system. As Figure 1 shown, the specific method is as follows:

[0047] Step S1, according to the CPU overcommitment ratio, memory overcommitment ratio, and whether to disable KSM, use the application system operation behavior model and the application health status analysis model to automatically divide resource pools of different levels.

[0048] The resource pools are divided into at least three levels of resource pools according to the CPU overcommitment ratio, memory overcommitment ratio, and whether to disable KSM. Each level of resource pool includes multiple virtual machines, and the hardware CPUs and memories of each virtual machine are the same. The actually allocable CPUs and memories of a single virtual machine in resource pools of different levels change with the resource pool level.

[0049] The resource pool was initially divided according to manual experience and historical system data. However, the more resources are over-allocated, the worse the performance will be. Therefore, as the system runs, by means of artificial intelligence algorithms and manual feedback, it can be learned whether the running states of different business applications in different-level resource pools can meet the actual needs, and more accurate over-allocation ratios of the resource pool and whether to enable KSM can be given.

[0050] The initial division process of the resource pool is as Figure 2 shown, including:

[0051] Collect the status data of business modules in the rail transit business system, label the data through the application system running behavior model and annotate it using expert experience to obtain the rail transit multi-professional application portrait database;

[0052] Run the business on different preset resource pools, obtain the running status data, and respectively use the application health status analysis model to score the running status data, the resource pool matching algorithm to sort and extract values from the running status data, and expert experience to annotate the running status data to obtain the resource pool grading database;

[0053] Match the running status data with the rail transit multi-professional application portrait database to obtain application portrait labels, and combine with the resource pool grading database to obtain the initial resource pools to which different business modules belong. Finally, calculate the initial resource pool size, the initial grading division result of the resource pool, and the record of the resource pool to which the business module belongs by combining the data provided by the supplier.

[0054] The training process of the application system running behavior model is as follows:

[0055] Obtain the static attributes preset for system deployment and the CPU usage, memory occupancy, IO metrics, and call conditions of each business module in the resource pool during system operation, and perform data processing; use the processed data to establish a running behavior analysis data set; train the application system running behavior model through mechanical algorithms based on the running behavior analysis data set;

[0056] The training process of the application health status analysis model is as follows:

[0057] Obtain the health status data of the system at rest and during operation, and perform data processing; use the processed data to establish a health status analysis data set; train the application health status analysis model through mechanical algorithms based on the health status analysis data set;

[0058] The training process of the application peak running time analysis model is as follows:

[0059] Obtain the operation status data of application pools at all levels of the system and the operation status data of each business module, and perform data processing; establish an operation time analysis data set using the processed data; and train an application peak operation time analysis model through a mechanical algorithm based on the operation time analysis data set.

[0060] The above processing all includes: data cleaning, abnormal data processing, data formatting, data normalization, and text processing and extraction.

[0061] The system operation feedback data includes: static attributes preset for system deployment; CPU usage, memory occupancy, IO metrics, and call conditions of each business module in the resource pool during system operation; and the health status data of the system.

[0062] The process of scheduling business modules to resource pools at different levels is as Figure 3 shown, including two parts of scheduling:

[0063] 1. For the situation where the pre-divided resource pool may be inaccurate, collect the operation status data of business modules in real time, score it through an application health status analysis model. If the score is less than the minimum threshold, schedule the business module to a higher-level resource pool; if the score is greater than the maximum threshold, after confirmation by the user, schedule the business module to a lower-level resource pool.

[0064] 2. Collect the operation status data of business modules and the operation status data of resource pools in real time. Considering the different peak operation time periods of different applications, use the application peak operation time analysis model to analyze the peak operation time periods of each business module;

[0065] Use the peak operation time periods of each business module and the operation status data of the resource pool to obtain the peak operation time period of the lower-level resource pool and the operation gap period of the higher-level resource pool, and obtain the overlapping time period between the two;

[0066] During the overlapping time period, schedule the business modules in the lower-level resource pool to run in the higher-level resource pool to achieve the maximum utilization of resources and save the overall hardware resource requirements.

[0067] In this embodiment, according to the CPU overcommitment ratio, memory overcommitment ratio, and whether to disable KSM, the resource pools are divided into 3 levels as Figure 4 shown. The virtual machine hardware CPU in each level of resource pool is 52C, and the memory is 256G. The actual allocable CPU for a single virtual machine in different levels of resource pools is 104C, 156C, and 208C, and the allocable memory is 256G, 768G, and 1024G, which can save more than half of the hardware resources.

[0068] The rail transit service modules include algorithm-based service modules, memory cache-based service modules, and offline analysis-based service modules.

[0069] The characteristics of each service module include:

[0070] The algorithm-based service modules require high-performance CPUs; the memory cache-based service modules require a large amount of memory; the CPU and memory requirements of the offline analysis-based service modules show peak and valley situations over time.

[0071] Step S2: According to the importance, characteristics, virtual machine live migration ability, and the current time period of each service module, use the application health status analysis model and the application peak running time analysis model to schedule the service modules to different levels of resource pools respectively.

[0072] As Figure 5 shown, the service modules initially select the appropriate level of resource pool for scheduling according to manual experience and the importance, characteristics, and virtual machine live migration ability of each service module in different time periods in the historical system data. After a period of learning of the application system operation behavior model and long-term evaluation by humans, more appropriate intelligent scheduling will be performed on the service modules, and the service modules will be dynamically migrated from the initially determined resource pool to a more suitable resource pool to make full use of the resources of each level of resource pool.

[0073] Step S3: Receive the feedback data on the system operation situation, and optimize the application system operation behavior model and the application health status analysis model according to the feedback data.

[0074] Next, take some business application scenarios in the real system as examples:

[0075] As Figure 6 shown, the rail transit AFC (Automatic Fare Collection) and the offline data analysis system have typical peak and valley characteristics. This characteristic can be utilized to schedule the AFC operation virtual machine to the LEVEL2 resource pool during the daytime peak period and to the LEVEL3 resource pool during the nighttime valley period. At the same time, schedule the statistical task virtual machine of the offline data analysis system from the LEVEL3 to the LEVEL2 resource pool. On the premise of meeting the operation requirements of the AFC and the operation management system, the overall resource demand is reduced. At the same time, due to the highly similar operation environments of the offline data analysis systems, KSM can be fully utilized to achieve memory sharing.

[0076] As Figure 7As shown in the figure, the rail transit ATS (Automatic Train Supervision) system makes extensive use of memory caches and cannot accept memory over-allocation. The intelligent operation and maintenance system of rail transit can accept a certain degree of performance fluctuations. The ATS system can be scheduled to the LEVEL0 resource pool, and the intelligent operation and maintenance system can be scheduled to the LEVEL2 resource pool, thereby reducing the demand for the LEVEL0 resource pool.

[0077] Due to stability considerations and the inability to accurately evaluate resource requirements, each professional system of rail transit reserves resources to the greatest extent in advance, resulting in resource waste. By applying the system operation behavior model to actually monitor and learn based on feedback data, the actual resource requirements are accurately grasped, and the resources are dynamically adjusted to a more suitable resource pool, thereby reducing the overall resource requirements.

[0078] Embodiment 2

[0079] Based on Embodiment 1, this embodiment provides an electronic device, including: one or more processors and a memory. The memory stores one or more programs, and the one or more programs include instructions for executing the multi-level resource pool intelligent scheduling method for the urban rail transit cloud platform as described above.

[0080] At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the multi-level resource pool intelligent scheduling method for the urban rail transit cloud platform as described above. Of course, in addition to the software implementation method, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or a logic device.

[0081] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0082] A computer-readable medium includes permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0083] As described above, the above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A multi-level resource pool intelligent scheduling method for urban rail cloud platform, applied to rail transit business system, characterized in that: The method comprises: Based on the CPU over-allocation ratio, memory over-allocation ratio, and whether KSM is disabled, the application system operation behavior model and application health status analysis model are used to automatically divide resource pools into different levels. Based on the importance, characteristics, virtual machine hot migration capabilities, and current time period of each business module, the application health status analysis model and application peak operation time analysis model are used to schedule business modules to resource pools of different levels. Feedback data on system operation is received, and the application system operation behavior model and application health status analysis model are optimized based on the feedback data.

2. According to claim 1, a multi-level resource pool intelligent scheduling method for urban rail cloud platform is characterized in that: The resource pool is divided into no less than 3 levels according to the CPU over-allocation ratio, the memory over-allocation ratio and whether KSM is disabled, and each level of the resource pool includes multiple virtual machines.

3. According to claim 2, a multi-level resource pool intelligent scheduling method for urban rail cloud platform is characterized in that: The hardware CPU and memory of the multiple virtual machines are the same, and the CPU and memory that can be actually allocated to a single virtual machine in resource pools of different levels vary with the level of the resource pool.

4. According to the multi-level resource pool intelligent scheduling method for urban rail cloud platform according to claim 1, it is characterized in that: The initial division process of the resource pool includes: Collect the status data of business modules in the rail transit business system, label the data through the application system operation behavior model and annotate it using expert experience to obtain a multi-professional application portrait database for rail transit; Run the business on different preset resource pools to obtain operation status data, and use the application health status analysis model to score the operation status data, the resource pool matching algorithm to sort and value the operation status data, and the expert experience to annotate the operation status data to obtain a resource pool classification database; The operating status data is matched with the rail transit multi-professional application portrait database to obtain the application portrait label. Combined with the resource pool classification database, the initial resource pool belonging to different business modules is obtained. Finally, the initial resource pool size, the initial classification result of the resource pool, and the resource pool record to which the business module belongs are calculated based on the actual resource pool data.

5. According to the multi-level resource pool intelligent scheduling method for urban rail cloud platform according to claim 1, it is characterized in that: The business modules include an algorithmic business module, a memory cache business module and an offline analysis business module.

6. The multi-level resource pool intelligent scheduling method for urban rail cloud platform according to claim 5 is characterized in that: The characteristics of the business module include: Algorithmic business modules require high-performance CPUs; memory cache business modules require a large amount of memory; and the CPU and memory requirements of offline analytical business modules show peaks and troughs over time.

7. The multi-level resource pool intelligent scheduling method for urban rail cloud platform according to claim 1 is characterized in that: The training process of the application system operation behavior model is as follows: Obtain the static attributes preset by system deployment and the CPU usage, memory usage, IO indicators and call status of each business module in the resource pool when the system is running, and perform data processing; Using the processed data to establish an operation behavior analysis data set; based on the operation behavior analysis data set, training the application system operation behavior model through a mechanical algorithm; The training process of the application health status analysis model is as follows: Obtain the health status data of the system at static and runtime, and process the data; Using the processed data to establish a health status analysis data set; based on the health status analysis data set, applying a health status analysis model through mechanical algorithm training; The training process of the application peak running time analysis model is as follows: Obtain the operating status data of application pools at all levels of the system and the operating status data of each business module, and process the data; use the processed data to establish a running time analysis data set; based on the running time analysis data set, train the application peak running time analysis model through mechanical algorithms.

8. The multi-level resource pool intelligent scheduling method for urban rail cloud platform according to claim 7 is characterized in that: The processing includes: data cleaning, abnormal data processing, data formatting, data normalization and text processing extraction.

9. The multi-level resource pool intelligent scheduling method for urban rail cloud platform according to claim 1 is characterized in that: The process of scheduling business modules to resource pools of different levels includes: The business module operation status data is collected in real time and scored by applying the health status analysis model. If the score is less than the minimum threshold, the business module is scheduled to a higher-level resource pool. If the score is greater than the maximum threshold, the business module is scheduled to a lower-level resource pool after user confirmation.

10. The multi-level resource pool intelligent scheduling method for urban rail cloud platform according to claim 1 is characterized in that: The process of scheduling business modules to resource pools of different levels includes: Collect the business module operation status data and resource pool operation status data in real time, and analyze the peak operation time period of each business module by using the application peak operation time analysis model according to the characteristics of each business module; Using the peak operation time period of each business module and the operation status data of the resource pool, the peak operation time period of the low-level resource pool and the operation gap period of the high-level resource pool are obtained, and the overlapping time period between the two is obtained; During the overlapping time period, the business modules in the low-level resource pool are scheduled to run in the high-level resource pool.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the multi-level resource pool intelligent scheduling method for the urban rail cloud platform as described in any one of claims 1-10 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-level resource pool intelligent scheduling method for the urban rail cloud platform as described in any one of claims 1 to 10 are implemented.

Images