IaaS service performance optimization method, storage medium and equipment

Abstract

The invention discloses an IaaS service performance optimization method, a storage medium and equipment, and the IaaS service performance optimization method comprises the steps: caching static data in each task stage of life cycle management of a task; receiving a task operation instruction in a task execution process, and judging whether to-be-operated data corresponding to the task operation instruction is static data or not; and if the to-be-operated data is the static data, obtaining and operating the static data from a cache. According to the method, the caching mechanism is added, the storage center node of the IaaS does not need to query the database frequently, frequent data interaction with the database is not needed, and the static data is cached through the caching mechanism, so that the performance index of each task stage can be optimized, the IaaS service performance can be optimized, and the service response efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer application technology, and in particular to an IaaS service performance optimization method, storage medium and device. Background Art

[0002] Edge nodes are divided into two types: computing nodes and storage nodes. Computing nodes and storage nodes are independent of each other and work in pairs to provide complete services to the outside world.

[0003] In the prior art, when the storage center node interacts with the edge node to perform various tasks, it often needs to interact with the database, that is, frequently query the database, write to the database, etc., which leads to a decrease in the service performance of the storage center node. Summary of the invention

[0004] The purpose of this application is to provide an IaaS service performance optimization method, storage medium and device to solve the technical problems existing in the prior art. The various technical effects that can be produced by the optional technical solutions among the various technical solutions provided in this application are described in detail below.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] A first aspect of the present application provides an IaaS service performance optimization method, which is applied to a task execution system, wherein the task execution system includes a plurality of storage nodes, and the method includes:

[0007] In each task phase of task lifecycle management, static data is cached;

[0008] Receiving a task operation instruction during task execution, and determining whether the data to be operated corresponding to the task operation instruction is static data;

[0009] If the data to be operated is the static data, the static data is acquired from the cache and operated.

[0010] In some embodiments, the IaaS service performance optimization method further includes:

[0011] When arranging tasks in the task phase, creating specific subtask information to be performed;

[0012] If there is a situation where multiple tasks are executed concurrently, the information of the multiple specific subtasks is uniformly and asynchronously written into the database in batches, and the specific subtask information is updated to the cache.

[0013] In some embodiments, the IaaS service performance optimization method further includes:

[0014] When the task phase is a polling and pushing phase, querying the cache for a list of subtasks that have been scheduled to be executed;

[0015] The list of subtasks that have been scheduled to be executed is pushed to the edge node for execution.

[0016] In some embodiments, the IaaS service performance optimization method further includes:

[0017] When the task stage is reporting update, querying the storage subtask information reported by the edge service from the cache;

[0018] The storage subtask information is updated to the cache, and asynchronously updated to the database.

[0019] In some embodiments, the IaaS service performance optimization method further includes:

[0020] When the task phase is a polling update phase, querying the orchestration task information from the cache;

[0021] The subtasks corresponding to the orchestration task information are updated to the cache, and the subtasks are asynchronously updated in batches to the database.

[0022] In some embodiments, the IaaS service performance optimization method further includes:

[0023] When the task phase is a polling update phase or a polling update phase, the change state of the cache and the cache information corresponding to the subtask are monitored in real time.

[0024] In some embodiments, the IaaS service performance optimization method further includes:

[0025] When the task stage is read and loaded, the cache is periodically queried for storage task information.

[0026] In some embodiments, the IaaS service performance optimization method further includes:

[0027] If the data to be operated is dynamic data, the dynamic data is acquired from the database and operated.

[0028] A second aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the IaaS service performance optimization method described above is implemented.

[0029] The third aspect of the present application provides a processing device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors are used to execute the one or more computer programs stored in the memory, so that the one or more processors execute the IaaS service performance optimization method as described above.

[0030] Implementing one of the above technical solutions of the present application has the following advantages or beneficial effects: the IaaS service performance optimization method, storage medium and device of the present application caches static data in each task stage of the task life cycle management, receives task operation instructions during task execution, and determines whether the data to be operated corresponding to the task operation instructions is static data; if the data to be operated is the static data, the static data is obtained from the cache and operated. By adding a cache mechanism, the storage center node of IaaS does not need to frequently query the database and frequently interact with the database. By caching static data through the cache mechanism, the performance indicators of each task stage can be optimized, the IaaS service performance can be optimized, and the service response efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. It is obvious that the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0032] Figure 1 It is a schematic diagram of the deployment method of the edge node in the embodiment of the present application;

[0033] Figure 2 is a schematic diagram of the process of executing various tasks by the current task execution system of an embodiment of the present application;

[0034] Figure 3 It is a flowchart of the IaaS service performance optimization method according to an embodiment of the present application;

[0035] Figure 4 It is a structural block diagram of the service performance optimization device of the storage center node in an embodiment of the present application.

[0036] In the figure: 1. processing device; 10. memory; 11. processor. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present application clearer, the various exemplary embodiments to be described below will refer to the corresponding drawings, which constitute a part of the exemplary embodiments, wherein various exemplary embodiments that may be used to implement the present application are described. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present disclosure. It should be understood that they are only examples of processes, methods, and devices that are consistent with some aspects disclosed in the present application as detailed in the attached claims, and other embodiments may also be used, or the embodiments listed herein may be modified in structure and function without departing from the scope and essence of the present application.

[0038] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", etc. indicate the orientation or positional relationship based on the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the elements referred to must have a specific orientation, be constructed and operated in a specific orientation. The terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. The term "multiple" means two or more. The terms "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, an integral connection, a mechanical connection, an electrical connection, a communication connection, a direct connection, an indirect connection through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0039] In order to illustrate the technical solution described in the present application, a specific embodiment is provided below, and only the parts related to the embodiment of the present application are shown.

[0040] According to the logical division of the same computer room, computing nodes and storage nodes can be physically deployed in the same physical LAN, or they can be deployed in independent networks.

[0041] The storage node exposes standard access methods to the computing power node. One storage node can be used by multiple computing power nodes at the same time. One computing power node can only use one storage node, that is, the computing power node and storage node are in an N:1 ratio.

[0042] like Figure 1 As shown, Figure 1Indicates the deployment mode of the storage nodes of this application. Storage nodes include edge nodes mainly used for computing, central nodes, and other storage nodes mainly used for storage. Both edge nodes and central nodes have storage functions, but their storage capacity is limited. Therefore, edge nodes and central nodes can be collectively referred to as storage nodes.

[0043] exist Figure 1 In the figure, there is one central node and three edge nodes, namely edge node A, edge node B and edge node C. Each edge node has one computing node and two storage nodes. The computing nodes are computing node 1, computing node 2 and computing node 3, and the storage nodes are storage node 1 and storage node 2.

[0044] Therefore, in Figure 1 In the example, storage node 1 can be used by computing node 1 and computing node 2 at the same time. Computing node 1 uses storage node 1, computing node 2 uses storage node 1, and computing node 3 uses storage node 2.

[0045] Whether a storage node meets the deployment conditions needs to be measured from the following business perspectives:

[0046] 1. Input / Output BandWidth: The input / output bandwidth of a node determines the data transmission speed.

[0047] 2. Storage Capacity: Determined by the size of the game and the elimination strategy.

[0048] 3. IOPS (Input / Output Operations Per Second): The peak IOPS of the storage service NAS or CEPH.

[0049] The execution of tasks is based on the deployment of computing power. If all storage nodes and computing power nodes meet the deployment conditions, the deployment can realize the execution of tasks.

[0050] The tasks in the embodiments of the present application refer to the tasks of the IaaS (Infrastructure as a Service) storage module, and the execution of the tasks is achieved through the task execution system. IaaS refers to a service model that provides IT infrastructure as a service through the network and charges users based on their actual usage or occupation of resources. In this service model, ordinary users do not need to build a data center or other hardware facilities by themselves, but instead obtain computer infrastructure services from IaaS service providers through the Internet, including server, storage, and network services, by renting.

[0051] Among them, the task execution system includes IaaS gateway, IaaS storage center node, database, IaaS storage task, storage message queue, IaaS storage node, IaaS storage agent and IaaS task status node.

[0052] The current task execution system executes each task in the following process: Figure 2 shown.

[0053] exist Figure 2 In the IaaS gateway, you can create, unmount, mount, and deploy storage volumes on the IaaS storage center node. After receiving the task (creating, unmounting, mounting, or deploying a storage volume) sent by the IaaS gateway, the IaaS storage center node asynchronously sends the relevant data of each task to the database, and pushes the message of the asynchronous task to the storage message queue in real time as needed, and then returns the id of the asynchronous task to the IaaS gateway.

[0054] If the storage message queue has relevant messages for operations such as disk creation, mounting, and unmounting, the relevant messages for operations such as master disk production and distribution will be sent to the IaaS storage agent; the IaaS storage node and the IaaS storage agent will report the task execution results of operations such as master disk production and distribution to the storage message queue respectively, and the storage message queue will report the received task execution result message to the IaaS storage task.

[0055] The IaaS storage task reports the task execution results to the database for update processing.

[0056] During the polling task phase, the database will perform task orchestration. The storage task will asynchronously push the task orchestration information to the storage message queue. The storage message queue will report the task information to the storage task and update the task results to the database. The storage task will push the instance status update information to the storage message queue. The storage message queue will update the instance task occupancy status to the IaaS task status node.

[0057] The IaaS gateway can also query the IaaS storage center node for storage tasks, data information, etc.

[0058] Based on the above deployment method and task execution system, the IaaS service performance optimization method of this application is proposed and applied to the task execution system. The task execution system includes multiple storage nodes and is executed through a storage center node. The storage center node of this application refers to Figure 3 IaaS storage center node in. Figure 3 As shown, the IaaS service performance optimization method of the present application includes steps S1 to S4.

[0059] S1. In each task stage of the task lifecycle management, static data is cached.

[0060] The various task stages of task lifecycle management include task creation, reading, unloading, orchestration, execution, update, and reporting.

[0061] Among them, the task creation phase refers to the storage center node receiving the call from the upper-level IaaS gateway, creating a storage task to the database, and synchronously returning the task id to the IaaS gateway, and asynchronously pushing the task to the storage message queue on demand. During this period, it involves querying the computing service to obtain instance-related information, querying local storage resources and information in execution, and whether the task creation conditions are met. If so, new storage task information is created.

[0062] Static data refers to the non-dynamic data involved in a task, which is a collection of data that does not change or changes very slowly during the execution of a task. Static data is characterized by persistence, stability, and predictability. During the processing of a task, static data includes but is not limited to configuration parameters, parameter data, historical data, and metadata.

[0063] The storage center node helps the design and optimization of the task execution system by caching static data during each task stage of the task life cycle management, because it allows the task execution system to adopt caching, preloading and other strategies to improve efficiency when processing tasks, while reducing the frequency of queries to the database or other data sources, thereby reducing latency and resource consumption.

[0064] S2. Receive a task operation instruction during the task execution process, and determine whether the to-be-operated data corresponding to the task operation instruction is static data.

[0065] S3. If the data to be operated is the static data, obtain and operate the static data from the cache.

[0066] S4. If the data to be operated is dynamic data, the dynamic data is acquired from a database and operated.

[0067] Specifically, if the data to be operated is static data, the operation is performed from the cache, such as querying and reading; if the data to be operated is dynamic data, the operation is performed from the database, thereby reducing the query to the database.

[0068] In some embodiments, when the task stage is to create a task, the task operation instruction is a task creation operation instruction. Then, when the storage center node receives the task creation instruction from the upper-level gateway, when it involves querying the computing service to obtain instance-related information, local storage resources, and information in execution, it will determine whether the relevant data to be queried for these creation tasks is static data. If so, since the static data has been cached, the corresponding static data can be queried in the cache without directly querying the database, thereby reducing the query to the database.

[0069] In some embodiments, when the task stage is read loading, the cache is periodically queried for storage task information.

[0070] Specifically, the reading and loading task is executed by the IaaS storage task. Usually, the IaaS storage task listens to the task queue, or periodically polls and queries the database, and loads the read task information into the orchestration and polling stage. In order to reduce the query frequency of the database during the task reading and loading stage, a cache mechanism is added, that is, the storage task information can be periodically queried from the cache. This can reduce the query frequency of the database.

[0071] In some embodiments, the method may further include:

[0072] When arranging tasks in the task phase, creating specific subtask information to be performed;

[0073] If there is a situation where multiple tasks are executed concurrently, the information of the multiple specific subtasks is uniformly and asynchronously written into the database in batches, and the specific subtask information is updated to the cache.

[0074] Specifically, the task orchestration stage is also the execution of IaaS storage tasks. Usually, IaaS storage tasks create specific subtask information to be executed, and then update the orchestration information to the database, preparing to execute the orchestration information during the polling stage. The main processes involved in the task orchestration stage are: querying computing services, obtaining instance-related information, and creating orchestration storage subtask information.

[0075] In order to reduce the number of queries to computing services during the task scheduling phase, the static data related to the task scheduling phase is cached, and specific subtask information to be executed is created; if there are multiple tasks being executed concurrently, the multiple specific subtask information is asynchronously batch-written to the database, and the specific subtask information is updated to the cache, so that the specific subtask information can be quickly obtained during the subsequent task execution phase.

[0076] In some embodiments, the method may further include:

[0077] When the task phase is a polling and pushing phase, querying the cache for a list of subtasks that have been scheduled to be executed;

[0078] The list of subtasks that have been scheduled to be executed is pushed to the edge node for execution.

[0079] Specifically, when the task stage is the polling and pushing stage, under normal circumstances, the IaaS storage task periodically reads the list of subtasks that have been scheduled to be executed from the database, and polls and checks and pushes them to the edge node for execution. However, this requires querying the database. In order to reduce the query to the database, when the task stage is the polling and pushing stage, the embodiment of the present application, since the subtasks that have been scheduled to be executed are updated to the cache during the task scheduling stage, it is possible to directly query the list of subtasks that have been scheduled to be executed from the cache, and then push the list of subtasks that have been scheduled to be executed to the edge node for execution, thereby reducing the query to the database.

[0080] In some embodiments, when the task stage is the polling push stage, the cache change state and the cache information corresponding to the subtask can be monitored in real time. The continuously generated data is collected and calculated in real time to obtain the calculation results as quickly as possible. For example, an open source distributed real-time computing system Storm can be used to simply and reliably process a large amount of data streams, support horizontal expansion, have high fault tolerance, and ensure that each message will be processed, and the processing speed is very fast. For example, in a small cluster, each node can process millions of messages per second, which can enable the aforementioned change state or cache information to be processed in real time.

[0081] In some embodiments, the edge node executes the task, which means that the IaaS storage agent and the IaaS storage node receive the task, start to execute the task, and report the task progress (task result) to the message queue according to the stage of task execution, and report it to the storage task until the task is completed.

[0082] In some embodiments, the method may further include:

[0083] When the task stage is reporting update, querying the storage subtask information reported by the edge service from the cache;

[0084] The storage subtask information is updated to the cache, and asynchronously updated to the database.

[0085] Specifically, when the task stage is reporting and updating, the storage task usually receives the reporting information of the task edge service and updates it to the database. When receiving the reporting information, it is necessary to query the database for the storage subtask information, and then update the orchestration storage subtask information to the database. However, this will require querying the database. In order to reduce the query to the database, when the task stage is reporting and updating, in an embodiment of the present application, since the storage subtask information has been cached, the storage subtask information reported by the edge service can be queried from the cache; the storage subtask information is updated to the cache, and asynchronously updated to the database. Queries to the database can be reduced. Thereby improving the service performance of the storage center node.

[0086] In some embodiments, the method may further include:

[0087] When the task phase is a polling update phase, querying the orchestration task information from the cache;

[0088] The subtasks corresponding to the orchestration task information are updated to the cache, and the subtasks are asynchronously updated in batches to the database.

[0089] Specifically, when the task stage is the polling update stage, the IaaS storage task usually reads the list of orchestrated subtasks, and the polling check is pushed on demand. After all subtasks are completed, the main task status is updated. During this period, it involves querying the database to store the main task and subtask information, and updating the stored main task and subtask information to the database. This requires frequent database queries. In order to reduce the query to the database, when the task stage is the polling update stage in the embodiment of the present application, since the orchestration task information has been cached, it is possible to modify the query from the database to read from the cache, that is, query the orchestration task information from the cache; the subtask corresponding to the orchestration task information is updated to the cache first, and the subtask is updated to the database asynchronously in batches. Thereby reducing the query to the database.

[0090] In some embodiments, the IaaS service performance optimization method further includes:

[0091] When the task phase is a polling update phase, the change status of the cache and the cache information corresponding to the subtask are monitored in real time.

[0092] Corresponding to the polling and pushing stage in the task stage, the polling and updating stage can also monitor the changing status of the cache and the cache information corresponding to the subtask in real time. Moreover, the polling and updating stage has similar technical effects of real-time monitoring in the polling and pushing stage, which will not be repeated here.

[0093] After testing, the IaaS service performance optimization method of the embodiment of the present application obtained the performance indicators shown in Table 1 below.

[0094]

[0095]

[0096] The IaaS service performance optimization method of the embodiment of the present application caches static data in each task stage of the task life cycle management, receives task operation instructions during task execution, and determines whether the data to be operated corresponding to the task operation instructions is static data; if the data to be operated is the static data, the static data is obtained from the cache and operated. By adding a cache mechanism, the IaaS storage center node does not need to frequently query the database and frequently interact with the database. By caching static data through the cache mechanism, the performance indicators of each task stage can be optimized, the IaaS service performance can be optimized, and the service response efficiency can be improved.

[0097] Those skilled in the art can understand that all or part of the features / steps of the above-mentioned method embodiments can be implemented by methods, data processing systems or computer programs, and these features can be implemented without hardware, entirely by software, or by a combination of hardware and software. The aforementioned computer program can be stored in one or more computer-readable storage media, and the storage medium stores the computer program. When the computer program is executed (such as by a processor), the steps of the above-mentioned IaaS service performance optimization method embodiment are executed.

[0098] The aforementioned storage media that can store program codes include: static hard disks, solid-state hard disks, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks and / or combinations of the above devices, that is, they can be implemented by any type of volatile or non-volatile storage device or a combination thereof.

[0099] The present application also provides an embodiment of a processing device 1, such as Figure 4 As shown, it includes one or more processors 11 and a memory 10; wherein the memory 10 is used to store one or more computer programs, and the one or more processors 11 are used to execute one or more computer programs stored in the memory 10, so that the processor 11 executes the features / steps of the above-mentioned IaaS service performance optimization method embodiment.

[0100] This application identifies multiple tasks submitted by users, obtains the task type, and schedules each task to be executed on at least one Windows instance according to the task type and resource requirements. In this case, the computing resources of multiple Windows instances can be fully utilized, resource utilization efficiency can be improved, and costs can be reduced; at the same time, appropriate resources are automatically allocated according to different task types, which can improve task processing efficiency.

[0101] The above is only the preferred embodiment of the present application. Those skilled in the art will appreciate that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present application. In addition, under the guidance of the present application, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application belong to the protection scope of the present application.

Claims

1. A method for optimizing IaaS service performance, characterized in that: Applied to a task execution system, the task execution system includes a plurality of storage nodes, and the method includes: In each task phase of task lifecycle management, static data is cached; Receiving a task operation instruction during task execution, and determining whether the data to be operated corresponding to the task operation instruction is static data; If the data to be operated is the static data, the static data is acquired from the cache and operated.

2. The IaaS service performance optimization method according to claim 1, characterized in that: The method further comprises: When arranging tasks in the task phase, creating specific subtask information to be performed; If there is a situation where multiple tasks are executed concurrently, the information of the multiple specific subtasks is uniformly and asynchronously written into the database in batches, and the specific subtask information is updated to the cache.

3. The IaaS service performance optimization method according to claim 2, characterized in that: The method further comprises: When the task phase is a polling and pushing phase, querying the cache for a list of subtasks that have been scheduled to be executed; The list of subtasks that have been scheduled to be executed is pushed to the edge node for execution.

4. The IaaS service performance optimization method according to claim 2, characterized in that: The method further comprises: When the task stage is reporting update, querying the storage subtask information reported by the edge service from the cache; The storage subtask information is updated to the cache, and asynchronously updated to the database.

5. The IaaS service performance optimization method according to claim 2, characterized in that: The method further comprises: When the task phase is a polling update phase, querying the orchestration task information from the cache; The subtasks corresponding to the orchestration task information are updated to the cache, and the subtasks are asynchronously updated in batches to the database.

6. The IaaS service performance optimization method according to claim 5, characterized in that: The IaaS service performance optimization method further includes: When the task phase is a polling update phase or a polling update phase, the change state of the cache and the cache information corresponding to the subtask are monitored in real time.

7. The IaaS service performance optimization method according to claim 1, characterized in that: The method further comprises: When the task stage is read and loaded, the cache is periodically queried for storage task information.

8. The IaaS service performance optimization method according to claim 2, characterized in that: The method further comprises: If the data to be operated is dynamic data, the dynamic data is acquired from the database and operated.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed, the IaaS service performance optimization method according to any one of claims 1 to 8 is implemented.

10. A service performance optimization device for a storage center node, characterized in that: include: one or more processors; The memory is used to store one or more computer programs, and the one or more processors are used to execute the one or more computer programs stored in the memory, so that the one or more processors execute the IaaS service performance optimization method as described in any one of claims 1-8.

Images