Task processing method and system

Abstract

The invention relates to a task processing method and system. The method comprises the following steps that: a main service adjusts a rear-end task processing upper limit value of each controller according to front-end performance statistical information and rear-end performance statistical information of a controller which the main service belongs to, wherein the front-end performance statistical information and the rear-end performance statistical information are reported by each proxy service; and according to the adjusted rear-end task processing upper limit value and the adjusted front-end task processing upper limit value of each controller, distributing tokens to each proxy service, the tokens being used for the controller corresponding to each proxy service to process the corresponding front-end task and rear-end task. The main service is deployed on any controller in the storage device, and proxy services are deployed on the controllers of the storage device. By adopting the method, the availability and the stability of the storage equipment can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer storage technology, and in particular to a task processing method and system. Background Art

[0002] The storage device is composed of multiple controllers and multiple storage media. Each controller can process input / output service quality requests from the host and can also process input / output service quality requests of backend tasks of the storage device.

[0003] In the related art, the backend tasks are processed through the interaction between each controller and the host. Usually, the host determines the executable backend tasks according to the backend tasks to be executed sent by each controller, and instructs each controller to process these executable backend tasks.

[0004] However, the task processing method in the related art has high requirements on the stability of the link between the host and each controller, which easily affects the availability of the storage device. Summary of the invention

[0005] Based on this, it is necessary to provide a task processing method and system to address the above technical issues, which can improve the availability and stability of storage devices.

[0006] In a first aspect, the present application provides a task processing method, which is applied to a main service, where the main service is deployed on any controller in a storage device, and each controller of the storage device is deployed with an agent service; the method comprises:

[0007] According to the front-end performance statistics and back-end performance statistics of the controllers reported by each proxy service, the upper limit of the back-end task processing of each controller is adjusted;

[0008] Tokens are distributed to each proxy service according to the adjusted upper limit of backend task processing and upper limit of frontend task processing of each controller. The tokens are used by the controller corresponding to each proxy service to process the corresponding frontend task and backend task.

[0009] In the task processing method provided in the embodiment of the present application, the main service adjusts the upper limit of back-end task processing of each controller according to the front-end performance statistics and back-end performance statistics of the controller to which each proxy service belongs reported, and distributes tokens to each proxy service according to the adjusted upper limit of back-end task processing and upper limit of front-end task processing of each controller, and the tokens are used for the controller corresponding to each proxy service to process the corresponding front-end task and back-end task. In this method, the main service is deployed on any controller in the storage device, and each controller of the storage device is deployed with a proxy service. The main service obtains the front-end performance statistics and back-end performance statistics reported by each proxy service, which is equivalent to obtaining the processing status of each controller on the front-end task and the back-end task, so as to adjust the upper limit of back-end task processing of each controller so that the front-end task and the back-end task reach a balanced state. On this basis, the main service distributes tokens to each proxy service according to the adjusted upper limit of back-end task processing and front-end task processing of each controller, so that the controller corresponding to each proxy service consumes a corresponding number of tokens to process front-end tasks and back-end tasks. This is equivalent to the main service on the controller and each proxy service adaptively adjusting the front-end and back-end tasks to ensure that the front-end and back-end tasks can reach a balanced state, so that the storage device is maintained in a stable and efficient state. In this process, it is no longer constrained by the host, which ensures the stability of task processing and thereby improves the availability and stability of the storage device.

[0010] In one embodiment, the front-end performance statistics include front-end pressure information and front-end performance information in the current distribution cycle; the back-end performance statistics include back-end pressure information and back-end performance information in the current distribution cycle; and the back-end task processing upper limit of each controller is adjusted according to the front-end performance statistics and back-end performance statistics of the controller to which each proxy service belongs reported, including:

[0011] When each front-end pressure information and each back-end pressure information are not zero, the adjustment state of the back-end task of each controller in the previous distribution cycle is obtained;

[0012] According to each adjustment status, each front-end performance information and each back-end performance information, the upper limit value of each back-end task processing is adjusted.

[0013] In the task processing method provided by the embodiment of the present application, the front-end performance statistical information includes the front-end pressure information and the front-end performance information in the current distribution cycle, and the back-end performance statistical information includes the back-end pressure information and the back-end performance information in the current distribution cycle; when each front-end pressure information and each back-end pressure information are not zero, the adjustment state of the back-end task of each controller in the previous distribution cycle is obtained, and the upper limit value of each back-end task processing is adjusted according to each adjustment state, each front-end performance information and each back-end performance information. In this method, when each controller has both front-end tasks to be processed and back-end tasks to be processed, the adjustment state of the back-end task of each controller in the previous distribution cycle is obtained to obtain the adjustment situation of the previous distribution cycle, and based on the adjustment situation of the previous distribution cycle, combined with the front-end performance information and the back-end performance information of each controller, the upper limit value of the back-end task processing of each controller is adjusted, so that the adjusted upper limit value of the back-end task processing is more in line with the performance of each controller, and the adjustment is made based on the processing situation of the previous distribution cycle, to ensure the rationality of the back-end task adjustment and improve the adjustment accuracy of the upper limit value of the back-end task processing.

[0014] In one embodiment, adjusting the upper limit of each backend task processing according to each adjustment state, each frontend performance information and each backend performance information includes:

[0015] For any controller, when the adjustment state of the backend task of the controller in the previous distribution cycle is not adjusted, the upper limit value of the backend task processing of the controller is adjusted according to the frontend performance information and the backend performance information of the controller;

[0016] When the adjustment state of the backend task of the controller in the previous distribution cycle is adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the frontend performance information of the controller.

[0017] In the task processing method provided by the embodiment of the present application, for any controller, when the adjustment state of the backend task of the controller in the previous distribution cycle is not adjusted, the upper limit value of the backend task processing of the controller is adjusted according to the front-end performance information and the back-end performance information of the controller; when the adjustment state of the backend task of the controller in the previous distribution cycle is adjusted, the upper limit value of the backend task processing of the controller is adjusted according to the front-end performance information of the controller. In this method, when the adjustment state of the backend task of the controller in the previous distribution cycle is different, the adjustment method of the current distribution cycle is also different; if the adjustment state of the backend task of the controller in the previous distribution cycle is not adjusted, then the adjustment is based on the front-end performance information and the back-end performance information of the controller; if the adjustment state of the backend task of the controller in the previous distribution cycle is adjusted, then the adjustment is based on the front-end performance information of the controller, so that the adjustment method of the controller corresponds to the adjustment state of the previous distribution cycle, improves the accuracy of the adjustment, and makes the front-end and back-end tasks of each controller in a balanced state.

[0018] In one embodiment, adjusting the upper limit of backend task processing of the controller according to the frontend performance information and the backend performance information of the controller includes:

[0019] Determine comprehensive performance information of the controller according to the front-end performance information and the back-end performance information;

[0020] According to the ratio of the front-end performance information to the comprehensive performance information, the upper limit value of the back-end task processing of the controller is adjusted.

[0021] In the task processing method provided in the embodiment of the present application, the comprehensive performance information of the controller is determined based on the front-end performance information and the back-end performance information, and the upper limit value of the back-end task processing of the controller is adjusted based on the ratio of the front-end performance information to the comprehensive performance information. In this method, the comprehensive performance information of the controller is first determined, and then the proportion of the front-end performance to the entire system is determined based on the comprehensive performance information, so as to accurately adjust the upper limit value of the back-end task processing of the controller according to the proportion of the front-end performance.

[0022] In one embodiment, the upper limit of backend task processing of the controller is adjusted according to the ratio of the frontend performance information to the comprehensive performance information, including:

[0023] If the ratio of the front-end performance information to the comprehensive performance information is greater than a preset performance threshold, the controller's back-end task processing upper limit is adjusted upward according to the preset capability adjustment step size;

[0024] If the ratio of the front-end performance information to the comprehensive performance information is less than or equal to the performance threshold, the upper limit of the back-end task processing of the controller is lowered according to the capability adjustment step size.

[0025] In the task processing method provided by the embodiment of the present application, if the ratio of the front-end performance information to the comprehensive performance information is greater than the preset performance threshold, the upper limit value of the back-end task processing of the controller is adjusted upward according to the preset capability adjustment step; if the ratio of the front-end performance information to the comprehensive performance information is less than or equal to the performance threshold, the upper limit value of the back-end task processing of the controller is adjusted downward according to the capability adjustment step. In this method, the upper limit value of the back-end task processing of the controller is adjusted by the relationship between the ratio of the front-end performance information to the comprehensive performance information and the preset performance threshold; when the ratio is greater than the performance threshold, the upper limit value of the back-end task processing is adjusted upward, otherwise, the upper limit value of the back-end task processing is adjusted downward, so that the front-end and back-end tasks are balanced, and the capability adjustment step is used for adjustment at the same time, so that the adjustment is within a controllable range, and the accuracy of the adjustment is improved.

[0026] In one embodiment, adjusting the upper limit of backend task processing of the controller according to the frontend performance information of the controller includes:

[0027] When the adjustment state of the backend task of the controller in the previous distribution cycle is upward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the decrease status information of the front-end performance information compared with the previous distribution cycle;

[0028] When the adjustment state of the backend task of the controller in the previous distribution cycle is upward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the change status information of the front-end performance information compared with the previous distribution cycle.

[0029] In the task processing method provided by the embodiment of the present application, when the adjustment state of the backend task of the controller in the previous distribution cycle is an upward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the decline status information of the front-end performance information compared with the previous distribution cycle; when the adjustment state of the backend task of the controller in the previous distribution cycle is a downward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the change status information of the front-end performance information compared with the previous distribution cycle. In this method, the situation where the adjustment state of the backend task of the controller in the previous distribution cycle is an adjustment is divided more finely to refine the adjustment of the upper limit value of the backend task processing of the controller. In different adjustment situations, different methods are used to adjust the upper limit value of the backend task processing of the controller, so that the adjustment is more detailed, the adjustment accuracy is improved, and a solid foundation is provided for the balance of the front-end and back-end tasks.

[0030] In one embodiment, adjusting the upper limit of backend task processing of the controller according to the decrease of the frontend performance information compared with the previous distribution cycle includes:

[0031] If the degradation status information indicates that the front-end performance information has not decreased compared with the front-end performance information of the previous distribution cycle or the degradation range is within a preset range, the upper limit value of the back-end task processing of the controller is increased according to the preset capacity adjustment step size;

[0032] If the degradation status information indicates that the degradation range of the front-end performance information compared to the front-end performance information of the previous distribution cycle is not within the preset range, the upper limit value of the back-end task processing of the controller is lowered according to the historical capacity adjustment data.

[0033] In the task processing method provided by the embodiment of the present application, if the decline status information is that the front-end performance information has not declined compared with the front-end performance information of the previous distribution cycle or the decline range is within the preset range, then the upper limit value of the back-end task processing of the controller is adjusted upward according to the preset capacity adjustment step; if the decline status information is that the decline range of the front-end performance information compared with the front-end performance information of the previous distribution cycle is not within the preset range, then the upper limit value of the back-end task processing of the controller is adjusted downward according to the historical capacity adjustment data. In this method, when the range of the front-end performance decline is in different ranges, the corresponding back-end task processing upper limit value is adjusted in different ways. The back-end task processing upper limit value of the controller is adjusted according to different decline situations, so that the back-end task processing upper limit value of the current distribution cycle can be controlled based on the changes in the front-end performance, without affecting the processing of the front-end tasks, thereby improving the rationality of the adjustment.

[0034] In one embodiment, adjusting the total capacity of the backend task of the controller according to the change status information of the frontend performance information compared with the previous distribution cycle includes:

[0035] If the change status information indicates that the front-end performance information has increased compared to the front-end performance information of the previous distribution cycle, the upper limit value of the back-end task processing of the controller is lowered according to the preset capacity adjustment step size;

[0036] If the change status information indicates that the front-end performance information is lower than the front-end performance information of the previous distribution cycle, the upper limit value of the back-end task processing of the controller is increased according to the historical capacity adjustment data.

[0037] In the task processing method provided by the embodiment of the present application, if the change status information is that the front-end performance information has increased compared to the front-end performance information of the previous distribution cycle, then the upper limit value of the back-end task processing of the controller is lowered according to the preset capacity adjustment step; if the change status information is that the front-end performance information has decreased compared to the front-end performance information of the previous distribution cycle, then the upper limit value of the back-end task processing of the controller is increased according to the historical capacity adjustment data. In this method, when the adjustment state of the back-end task of the controller in the previous distribution cycle is downward, the upper limit value of the back-end task processing of the controller is adjusted accordingly according to the different changes in the front-end performance, so as to achieve a balance between the front-end and back-end tasks and improve the working efficiency and stability of the storage device.

[0038] In one embodiment, the method further comprises:

[0039] Sending an information acquisition request to each proxy service; the information acquisition request is used to instruct each proxy service to collect the front-end performance statistics information and back-end performance statistics information of the corresponding controller in the current distribution cycle;

[0040] Receive front-end performance statistics and back-end performance statistics reported by each proxy service according to the information acquisition request.

[0041] In the task processing method provided in the embodiment of the present application, an information acquisition request is sent to each proxy service, and the front-end performance statistics and back-end performance statistics reported by each proxy service according to the information acquisition request are received; wherein the information acquisition request is used to instruct each proxy service to collect the front-end performance statistics and back-end performance statistics of the corresponding controller in the current distribution cycle. In this method, through the interaction between the main service and each proxy service, the main service can timely obtain the front-end performance information and back-end performance information reported by each proxy service, so that the main service can make timely adjustments to the front-end and back-end tasks of the current distribution cycle, so that the front-end and back-end tasks reach a balanced state, thereby improving the availability of the storage device.

[0042] In a second aspect, an embodiment of the present application further provides a task processing method, which is applied to a proxy service, wherein a proxy service is deployed on each controller in a storage device; a main service is deployed on any controller in the storage device; the method comprises:

[0043] Report the front-end performance statistics and back-end performance statistics of the controller to which it belongs to to the main service;

[0044] According to the tokens distributed by the main service based on the front-end performance statistics and the back-end performance statistics, the front-end tasks and the back-end tasks are processed.

[0045] In the task processing method provided in the embodiment of the present application, the proxy service reports the front-end performance statistics and back-end performance statistics of the controller to which it belongs to the main service, and then implements the processing of the front-end tasks and back-end tasks according to the tokens distributed by the main service based on the front-end performance statistics and back-end performance statistics. In this method, the main service is deployed on any controller in the storage device, and the proxy service is deployed on each controller in the storage device. The main service obtains the front-end performance statistics and back-end performance statistics reported by each proxy service, which is equivalent to obtaining the processing status of each controller on the front-end tasks and back-end tasks, so as to adjust the upper limit of the back-end task processing of each controller so that the front-end tasks and back-end tasks reach a balanced state. On this basis, the main service distributes tokens to each proxy service according to the adjusted upper limit of back-end task processing and front-end task processing of each controller, so that the controller corresponding to each proxy service consumes a corresponding number of tokens to process front-end tasks and back-end tasks. This is equivalent to the main service on the controller and each proxy service adaptively adjusting the front-end and back-end tasks to ensure that the front-end and back-end tasks can reach a balanced state, so that the storage device is maintained in a stable and efficient state. In this process, it is no longer constrained by the host, which ensures the stability of task processing and thereby improves the availability and stability of the storage device.

[0046] In one embodiment, the method further comprises:

[0047] In the case where the token distributed by the main service is not received, a token for the current distribution cycle is generated based on the historical token of the previous distribution cycle of the current distribution cycle; the token is used by the controller corresponding to the proxy service to process the front-end tasks and the back-end tasks.

[0048] In the task processing method provided by the embodiment of the present application, in the case where the token distributed by the main service is not received, a token for the current distribution cycle is generated based on the historical token of the previous distribution cycle of the current distribution cycle, and the token is used by the controller corresponding to the proxy service to process the front-end task and the back-end task. In this method, if there is a fault between the main service and the proxy service link, the proxy service generates a token for the current distribution cycle based on the token of the previous distribution cycle, so that the proxy service can still schedule the back-end task, ensuring the stability of task processing and further improving the availability and stability of the storage device.

[0049] In a third aspect, an embodiment of the present application further provides a task processing system, the task processing system comprising a storage device; the storage device comprising a plurality of controllers; a main service is deployed on any one of the plurality of controllers, and a proxy service is deployed on each controller;

[0050] Each proxy service is used to report the front-end performance statistics and back-end performance statistics of its controller to the main service;

[0051] The main service is used to adjust the upper limit of backend task processing of each controller according to the frontend performance statistics and backend performance statistics reported by each proxy service; distribute tokens to each proxy service according to the adjusted upper limit of backend task processing and upper limit of frontend task processing of each controller, and the tokens are used for the controller corresponding to each proxy service to process the corresponding frontend tasks and backend tasks.

[0052] In a fourth aspect, an embodiment of the present application further provides a task processing device, including:

[0053] The upper limit adjustment module is used to adjust the upper limit value of the backend task processing of each controller according to the front-end performance statistics and back-end performance statistics of the controller reported by each proxy service;

[0054] The token distribution module is used to distribute tokens to each proxy service according to the backend task processing upper limit and frontend task processing upper limit adjusted by each controller. The token is used by the controller corresponding to each proxy service to process the corresponding frontend task and backend task.

[0055] In a fifth aspect, the embodiment of the present application further provides a task processing device, including:

[0056] The information reporting module is used to report the front-end performance statistics and back-end performance statistics of the controller to which it belongs to to the main service;

[0057] The token processing module is used to process the front-end tasks and the back-end tasks according to the tokens distributed by the main service based on the front-end performance statistics and the back-end performance statistics.

[0058] In a sixth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method in any one of the embodiments of the first aspect or the second aspect are implemented.

[0059] In the seventh aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in any one of the embodiments of the first aspect or the second aspect above.

[0060] In an eighth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method in any one of the embodiments of the first aspect above.

[0061] The task processing method and system provided in the embodiment of the present application, the main service adjusts the upper limit of back-end task processing of each controller according to the front-end performance statistics and back-end performance statistics of the controller to which each proxy service belongs reported, and distributes tokens to each proxy service according to the adjusted upper limit of back-end task processing and upper limit of front-end task processing of each controller, and the tokens are used for the controller corresponding to each proxy service to process the corresponding front-end task and back-end task. In this method, the main service is deployed on any controller in the storage device, and the proxy service is deployed on each controller of the storage device. The main service obtains the front-end performance statistics and back-end performance statistics reported by each proxy service, which is equivalent to obtaining the processing status of each controller on the front-end task and the back-end task, so as to adjust the upper limit of back-end task processing of each controller so that the front-end task and the back-end task reach a balanced state. On this basis, the main service distributes tokens to each proxy service according to the adjusted upper limit of back-end task processing and front-end task processing of each controller, so that the controller corresponding to each proxy service consumes a corresponding number of tokens to process front-end tasks and back-end tasks. This is equivalent to the main service on the controller and each proxy service adaptively adjusting the front-end and back-end tasks to ensure that the front-end and back-end tasks can reach a balanced state, so that the storage device is maintained in a stable and efficient state. In this process, it is no longer constrained by the host, which ensures the stability of task processing and thereby improves the availability and stability of the storage device. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] FIG1 is an application environment diagram of a task processing method in one embodiment;

[0064] FIG2 is a schematic diagram of a task processing method in one embodiment;

[0065] FIG3 is a schematic diagram of a process for adjusting the upper limit of backend task processing in one embodiment;

[0066] FIG4 is a schematic diagram of a process for adjusting the upper limit of backend task processing in another embodiment;

[0067] FIG5 is a schematic diagram of a process for adjusting the upper limit of backend task processing in another embodiment;

[0068] FIG6 is a schematic diagram of a process for adjusting the upper limit of backend task processing in another embodiment;

[0069] FIG7 is a schematic diagram of a process for adjusting the upper limit of backend task processing in another embodiment;

[0070] FIG8 is a schematic diagram of a process for adjusting the upper limit of backend task processing in another embodiment;

[0071] FIG9 is a schematic diagram of a process for adjusting the upper limit of backend task processing in another embodiment;

[0072] FIG10 is a schematic diagram of a process for obtaining front-end performance statistics and back-end performance statistics in one embodiment;

[0073] FIG11 is a schematic flow chart of a task processing method in another embodiment;

[0074] FIG12 is a schematic diagram of the structure of a task processing device in one embodiment;

[0075] FIG13 is a schematic diagram of the structure of a task processing device in another embodiment;

[0076] FIG. 14 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0078] The technical background of the embodiments of the present application is first described below.

[0079] Storage devices can store data from the host, and the host can also read data from the storage device. Storage devices are composed of controllers and storage media. The controller can process input / output (IO) quality of service (QoS) requests from the host device, and can also process QoS requests from the storage device's backend tasks, such as garbage collection (GC). However, when the storage device manages background tasks, the storage device's IO may be affected. The storage device's IO may come from the host IO, which we call front-end IO, or it may come from I / O generated by background tasks managed by the controller, which we call back-end IO. Front-end and back-end IO may conflict, thereby affecting I / O QoS requests and, in turn, the device's availability.

[0080] In order to resolve the conflict between front-end and back-end IO, the main technology is to receive a message from the storage device, which indicates which background tasks are to be executed on the storage device. The host determines which background tasks can be executed based on the current IO service status on the storage device, and sends a message to the storage device, which executes these background tasks. When the host detects that the current IO service status does not meet the scheduling strategy, the host will send a message again to notify the storage device to stop the background tasks being executed. This method has high requirements for the stability of the link between the host and the main controller. If the link is broken or delayed, the scheduling of the back-end tasks will be affected, which in turn affects the availability of the device.

[0081] Based on this, the embodiment of the present application provides a task processing method, the main service is deployed on any controller in the storage device, and the proxy service is deployed on each controller of the storage device. The main service obtains the front-end performance statistics and back-end performance statistics reported by each proxy service, which is equivalent to obtaining the processing status of each controller on the front-end task and the back-end task, so as to adjust the upper limit of the back-end task processing of each controller, so that the front-end task and the back-end task reach a balanced state. On this basis, the main service distributes tokens to each proxy service according to the adjusted upper limit of the back-end task processing and the upper limit of the front-end task processing of each controller, so that the controller corresponding to each proxy service consumes a corresponding number of tokens to process the front-end task and the back-end task, which is equivalent to the main service on the controller and each proxy service adaptively adjusting the front-end and back-end tasks to ensure that the front-end and back-end tasks can reach a balanced state, so that the storage device is maintained in a stable and efficient state, and is no longer constrained by the host in this process, which ensures the stability of task processing, thereby improving the availability and stability of the storage device.

[0082] It should be noted that the beneficial effects brought about by the embodiments of the present application or the technical problems solved are not limited to this one, but may also include other implicit or related problems. For details, please refer to the description of the following embodiments.

[0083] The application environment of the task processing method provided in the embodiment of the present application is described below, which can be applied to the storage device shown in Figure 1. Among them, the storage device includes n controllers, each controller has multiple logical volumes (Logical Unit Number, LUN) and multiple back-end tasks, and each logical volume can be accessed by multiple hosts (the interaction between the host and the logical volume is not shown in the figure). The main service is deployed on one of the controllers in the storage device, which is used for information collection, policy adjustment, token allocation, etc., and an agent service is deployed on each controller in the storage device. At the same time, the main service communicates with the agent service of each controller, and each controller can access all storage media of the storage device.

[0084] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0085] In an exemplary embodiment, as shown in FIG2 , a task processing method is provided, which is described by taking the method applied to the main service in FIG1 as an example, and includes the following steps 201 to 202. Among them:

[0086] S201, adjusting the upper limit of backend task processing of each controller according to the frontend performance statistics information and backend performance statistics information of the controller to which each proxy service belongs reported.

[0087] In the embodiment of the present application, the storage device includes multiple controllers, the main service is deployed on any one of the controllers in the storage device, and the proxy service is deployed on each controller of the storage device.

[0088] Taking the distributed storage device as an example, in actual application scenarios, the distributed storage device provides services for multiple hosts, and the hosts and controllers interact to process backend tasks. Considering that the stability of the link between each controller and the host is easily affected, if the link is disconnected or delayed, the scheduling of backend tasks will be affected, thereby affecting the availability and stability of the storage device.

[0089] Based on this, in an embodiment of the present application, a main service is deployed on one of the controllers in the storage device, and a proxy service is deployed on each controller. Through the interaction between the main service and the proxy services, specifically, each proxy service reports the front-end performance statistics and back-end performance statistics of its controller to the main service, so as to support the main service to timely know the performance of each proxy service in the entire storage device in processing the front-end tasks and back-end tasks, and based on this, adaptively adjust the front-end tasks and back-end tasks received by the entire storage device to ensure that the front-end and back-end tasks can reach a balanced state, so that the storage device is maintained in a stable and efficient state.

[0090] The front-end performance statistics reported by each proxy service include the front-end pressure information and front-end performance information of the controller in the current distribution cycle, wherein the front-end pressure information includes the number of front-end tasks to be processed by the controller, and the front-end performance information includes the speed and efficiency of the controller in processing the front-end tasks, such as the number of input / output operations per second, throughput and bandwidth, etc. The back-end performance statistics reported by each proxy service include the back-end pressure information and back-end performance information of the controller in the current distribution cycle, wherein the back-end pressure information includes the number of back-end tasks to be processed by the controller, and the back-end performance information includes the speed and efficiency of the controller in processing the back-end tasks, such as the number of input / output operations per second, throughput and bandwidth, etc.

[0091] Normally, the access task processing requirements in the storage device are described in terms of the service quality cycle, such as processing 500 IO requests per second, processing 500M bandwidth access tasks per second, etc. Based on this, the front-end performance statistics and back-end performance statistics reported by each proxy service are often obtained once in each service quality cycle. In order to achieve more fine-grained regulation in the embodiment of the present application, the service quality cycle is divided into n distribution cycles. In the embodiment of the present application, the main service obtains the front-end performance statistics and back-end performance statistics reported by each proxy service once in each distribution cycle, and adjusts the front-end tasks and back-end tasks.

[0092] In one scenario, the main service sends an information acquisition request to each proxy service at the beginning of the current distribution cycle to instruct each proxy service to report the front-end performance statistics information and the back-end performance statistics information to the main service. In another scenario, the main service sends an information acquisition request to each proxy service at the end of the previous distribution cycle to instruct each proxy service to report the front-end performance statistics information and the back-end performance statistics information to the main service.

[0093] In an embodiment of the present application, after receiving the front-end performance information and back-end performance information of the controllers to which each proxy service belongs as reported, the main service adjusts the upper limit of back-end task processing of each controller. For example, the upper limit of back-end task processing of each controller can be adjusted according to a preset adjustment strategy in combination with each front-end performance information and each back-end performance information. Among them, the adjustment strategy specifically refers to adjusting the upper limit of back-end task processing when the impact of front-end performance is controllable. By adjusting the upper limit of back-end task processing through such an adjustment strategy, the front-end tasks and the back-end tasks can reach a balanced state, thereby making the storage device stable and efficient, and improving the availability and stability of the storage device. Among them, the upper limit of back-end task processing refers to the upper limit of the controller's ability to process back-end tasks, such as the number of input / output operations per second and bandwidth.

[0094] S202, distributing tokens to each proxy service according to the adjusted upper limit of backend task processing and upper limit of frontend task processing of each controller, the tokens are used for the controller corresponding to each proxy service to process the corresponding frontend task and backend task.

[0095] In the embodiment of the present application, the token is generated by the main service according to the upper limit value of backend task processing and the upper limit value of frontend task processing adjusted by each controller.

[0096] Exemplarily, for any controller, the main service can convert the adjusted backend task processing upper limit value into a token based on the adjusted backend task processing upper limit value and the service quality policy or algorithm of the controller, and convert the front-end task processing upper limit value of the controller into a token based on a preset service quality policy or algorithm.

[0097] The main service adjusts the upper limit of backend task processing based on the frontend performance statistics and backend performance statistics reported by each proxy service, so that the frontend tasks and backend tasks of each controller reach a balanced state. Then, based on the adjusted upper limit of backend task processing and upper limit of frontend task processing of each controller, the main service determines the tokens required by each proxy service and distributes the corresponding tokens to each proxy service, so that the controller corresponding to each proxy service processes the corresponding frontend tasks and backend tasks. Each processing of a frontend task or a backend task consumes some tokens. When all tokens are consumed, the frontend and backend tasks reach a balance, and the availability and stability of the storage device are guaranteed.

[0098] In the task processing method provided in the embodiment of the present application, the main service adjusts the upper limit of back-end task processing of each controller according to the front-end performance statistics and back-end performance statistics of the controller to which each proxy service belongs reported, and distributes tokens to each proxy service according to the adjusted upper limit of back-end task processing and upper limit of front-end task processing of each controller, and the tokens are used for the controller corresponding to each proxy service to process the corresponding front-end task and back-end task. In this method, the main service is deployed on any controller in the storage device, and each controller of the storage device is deployed with a proxy service. The main service obtains the front-end performance statistics and back-end performance statistics reported by each proxy service, which is equivalent to obtaining the processing status of each controller on the front-end task and the back-end task, so as to adjust the upper limit of back-end task processing of each controller so that the front-end task and the back-end task reach a balanced state. On this basis, the main service distributes tokens to each proxy service according to the adjusted upper limit of back-end task processing and front-end task processing of each controller, so that the controller corresponding to each proxy service consumes a corresponding number of tokens to process front-end tasks and back-end tasks. This is equivalent to the main service on the controller and each proxy service adaptively adjusting the front-end and back-end tasks to ensure that the front-end and back-end tasks can reach a balanced state, so that the storage device is maintained in a stable and efficient state. In this process, it is no longer constrained by the host, which ensures the stability of task processing and thereby improves the availability and stability of the storage device.

[0099] Based on the above embodiment, an embodiment is provided to illustrate the process of adjusting the upper limit of backend task processing of each controller according to the frontend performance statistics information and backend performance statistics information of the controller reported by each proxy service.

[0100] In an exemplary embodiment, as shown in FIG3 , according to the front-end performance statistics and back-end performance statistics of the controllers to which each proxy service belongs reported, the back-end task processing upper limit of each controller is adjusted, including:

[0101] S301, when each front-end pressure information and each back-end pressure information are not zero, obtain the adjustment status of the back-end task of each controller in the previous distribution cycle.

[0102] The front-end performance statistics information includes the front-end pressure information and the front-end performance information in the current distribution cycle; and the back-end performance statistics information includes the back-end pressure information and the back-end performance information in the current distribution cycle.

[0103] In the embodiment of the present application, if there is only a front-end task or only a back-end task, that is, the front-end pressure information is zero, or the back-end pressure information is zero, there is no need to adjust the upper limit value of the back-end task processing.

[0104] When there are both front-end tasks and back-end tasks, that is, when both the front-end pressure information and the back-end pressure information are not zero, it is necessary to adjust the upper limit of the back-end task processing. At this time, it is necessary to determine how to adjust the upper limit of the back-end task processing of each controller based on the adjustment status of the back-end task of each controller in the previous distribution cycle. Among them, the adjustment status of the back-end task in the previous distribution cycle includes no adjustment, upward adjustment, and downward adjustment. If the adjustment status of the previous distribution cycle is different, the adjustment direction of the corresponding current distribution cycle is different.

[0105] The main service can obtain the adjustment status of the backend tasks of each controller in the previous distribution cycle from the database. The main service can adjust the upper limit of the backend task processing of each controller in each distribution cycle, and the adjusted data are recorded in the database. When needed, the main service can obtain the adjustment status of each controller in the previous distribution cycle from the database.

[0106] S302: Adjust the upper limit of each backend task processing according to each adjustment status, each frontend performance information and each backend performance information.

[0107] After obtaining the adjustment status of the backend tasks of each controller in the previous distribution cycle, the main service adjusts the upper limit of the backend task processing of each controller in combination with the frontend performance information and the backend performance information of each controller.

[0108] For example, if the adjustment status of the backend tasks of each controller in the previous distribution cycle is different, then the corresponding adjustment strategy for the current distribution cycle is different. The main service can obtain the matching adjustment strategy based on the adjustment status of the backend tasks of each controller in the previous distribution cycle, and adjust the upper limit value of the backend task processing of each controller based on the adjustment strategy and the front-end performance information and back-end performance information of each controller.

[0109] In the task processing method provided by the embodiment of the present application, the front-end performance statistical information includes the front-end pressure information and the front-end performance information in the current distribution cycle, and the back-end performance statistical information includes the back-end pressure information and the back-end performance information in the current distribution cycle; when each front-end pressure information and each back-end pressure information are not zero, the adjustment state of the back-end task of each controller in the previous distribution cycle is obtained, and the upper limit value of each back-end task processing is adjusted according to each adjustment state, each front-end performance information and each back-end performance information. In this method, when each controller has both front-end tasks to be processed and back-end tasks to be processed, the adjustment state of the back-end task of each controller in the previous distribution cycle is obtained to obtain the adjustment situation of the previous distribution cycle, and based on the adjustment situation of the previous distribution cycle, combined with the front-end performance information and the back-end performance information of each controller, the upper limit value of the back-end task processing of each controller is adjusted, so that the adjusted upper limit value of the back-end task processing is more in line with the performance of each controller, and the adjustment is made based on the processing situation of the previous distribution cycle, to ensure the rationality of the back-end task adjustment and improve the adjustment accuracy of the upper limit value of the back-end task processing.

[0110] Based on the above embodiment, an embodiment is provided to illustrate the process of adjusting the upper limit value of each backend task processing according to each adjustment status, each frontend performance information and each backend performance information.

[0111] In an exemplary embodiment, as shown in FIG4 , adjusting the upper limit of each backend task processing according to each adjustment state, each frontend performance information and each backend performance information includes:

[0112] S401, for any controller, when the adjustment state of the backend task of the controller in the previous distribution cycle is not adjusted, the upper limit value of the backend task processing of the controller is adjusted according to the frontend performance information and the backend performance information of the controller.

[0113] In the embodiment of the present application, taking any controller as an example, the method of adjusting the upper limit value of backend task processing is explained.

[0114] If the adjustment state of the backend task of the controller in the last distribution cycle is not adjusted, the backend task processing upper limit value of the controller is adjusted according to the frontend performance information and the backend performance information of the controller.

[0115] For example, when the adjustment state of the backend task of the controller in the previous distribution cycle is no adjustment, the main service can obtain an adjustment strategy that matches the no adjustment of the previous distribution cycle, and adjust the upper limit value of the backend task processing of the controller according to the adjustment strategy and the front-end performance information and back-end performance information of the controller, so that the front-end tasks and back-end tasks of the controller can reach a balanced state.

[0116] S402: When the adjustment state of the backend task of the controller in the previous distribution cycle is adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the frontend performance information of the controller.

[0117] If the adjustment state of the backend task of the controller in the previous distribution cycle is adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the front-end performance information of the controller.

[0118] For example, when the adjustment state of the backend task of the controller in the previous distribution cycle is adjustment, the main service can obtain an adjustment strategy that matches the adjustment of the previous distribution cycle, and adjust the upper limit value of the backend task processing of the controller according to the adjustment strategy and in combination with the front-end performance information of the controller, so that the front-end tasks and back-end tasks of the controller can reach a balanced state.

[0119] In the task processing method provided by the embodiment of the present application, for any controller, when the adjustment state of the backend task of the controller in the previous distribution cycle is not adjusted, the upper limit value of the backend task processing of the controller is adjusted according to the front-end performance information and the back-end performance information of the controller; when the adjustment state of the backend task of the controller in the previous distribution cycle is adjusted, the upper limit value of the backend task processing of the controller is adjusted according to the front-end performance information of the controller. In this method, when the adjustment state of the backend task of the controller in the previous distribution cycle is different, the adjustment method of the current distribution cycle is also different; if the adjustment state of the backend task of the controller in the previous distribution cycle is not adjusted, then the adjustment is based on the front-end performance information and the back-end performance information of the controller; if the adjustment state of the backend task of the controller in the previous distribution cycle is adjusted, then the adjustment is based on the front-end performance information of the controller, so that the adjustment method of the controller corresponds to the adjustment state of the previous distribution cycle, improves the accuracy of the adjustment, and makes the front-end and back-end tasks of each controller in a balanced state.

[0120] Based on the above embodiment, an embodiment is provided to illustrate the process of adjusting the upper limit value of the backend task processing of the controller according to the frontend performance information and the backend performance information of the controller.

[0121] In an exemplary embodiment, as shown in FIG5 , adjusting the upper limit of backend task processing of the controller according to the frontend performance information and the backend performance information of the controller includes:

[0122] S501, determining comprehensive performance information of the controller according to the front-end performance information and the back-end performance information.

[0123] In the embodiment of the present application, when adjusting the upper limit value of the back-end task processing of the controller, the comprehensive performance information of the controller is first determined. The main service can calculate the sum of the front-end performance information and the back-end performance information of the controller, and determine the sum of the front-end performance information and the back-end performance information of the controller as the comprehensive performance information of the controller.

[0124] S502, adjusting the upper limit of backend task processing of the controller according to the ratio of the frontend performance information to the comprehensive performance information.

[0125] After obtaining the comprehensive performance information of the controller, the main service adjusts the upper limit of the backend task processing of the controller based on the ratio of the frontend performance information to the comprehensive performance information.

[0126] Exemplarily, the main service can determine how to adjust the upper limit of the backend task processing of the controller according to the conditions satisfied by the ratio of the front-end performance information to the comprehensive performance information. For example, if the ratio of the front-end performance information to the comprehensive performance information is within a preset range, the upper limit of the back-end task processing of the controller is adjusted upward. Conversely, if the ratio of the front-end performance information to the comprehensive performance information is not within the preset range, the upper limit of the back-end task processing of the controller is adjusted downward. Among them, the preset range is determined according to actual needs, which may exist in the adjustment strategy or be obtained separately, and is not limited in the embodiments of the present application.

[0127] In the task processing method provided in the embodiment of the present application, the comprehensive performance information of the controller is determined based on the front-end performance information and the back-end performance information, and the upper limit value of the back-end task processing of the controller is adjusted based on the ratio of the front-end performance information to the comprehensive performance information. In this method, the comprehensive performance information of the controller is first determined, and then the proportion of the front-end performance to the entire system is determined based on the comprehensive performance information, so as to accurately adjust the upper limit value of the back-end task processing of the controller according to the proportion of the front-end performance.

[0128] Based on the above embodiment, an embodiment is provided to illustrate the process of adjusting the upper limit value of the back-end task processing of the controller according to the ratio of the front-end performance information to the comprehensive performance information.

[0129] In an exemplary embodiment, as shown in FIG6 , adjusting the upper limit of the backend task processing of the controller according to the ratio of the frontend performance information to the comprehensive performance information includes:

[0130] S601, if the ratio of the front-end performance information to the comprehensive performance information is greater than a preset performance threshold, the upper limit of the back-end task processing of the controller is increased according to a preset capability adjustment step.

[0131] In the embodiment of the present application, the performance threshold is determined according to actual needs. For example, if the user expects the controller's processing performance for front-end tasks to be higher than the processing performance for back-end tasks, the performance threshold can be set to above 50%.

[0132] The ratio of the front-end performance information to the comprehensive performance information is compared with a preset performance threshold. If the ratio of the front-end performance information to the comprehensive performance information is greater than the preset performance threshold, the upper limit of the back-end task processing of the controller is adjusted upward according to the preset capacity adjustment step. The capacity adjustment step is pre-configured. For example, if the processing upper limit is the number of input / output operations per second, the capacity adjustment step may be 50 input / output operations per second.

[0133] It should be noted that if the front-end and back-end tasks have not reached a balanced state after adjusting the step size once based on the capability, the step size can be adjusted again based on the capability until the front-end and back-end tasks reach a balanced state.

[0134] S602: If the ratio of the front-end performance information to the comprehensive performance information is less than or equal to the performance threshold, the upper limit of the back-end task processing of the controller is lowered according to the capability adjustment step.

[0135] If the ratio of the front-end performance information to the comprehensive performance information is less than or equal to the performance threshold, the upper limit of the back-end task processing of the controller is lowered according to the capability adjustment step. Similarly, if the front-end and back-end tasks have not reached a balanced state after one adjustment based on the capability adjustment step, the capability adjustment step can be continued to be adjusted until the front-end and back-end tasks reach a balanced state.

[0136] In the task processing method provided by the embodiment of the present application, if the ratio of the front-end performance information to the comprehensive performance information is greater than the preset performance threshold, the upper limit value of the back-end task processing of the controller is adjusted upward according to the preset capability adjustment step; if the ratio of the front-end performance information to the comprehensive performance information is less than or equal to the performance threshold, the upper limit value of the back-end task processing of the controller is adjusted downward according to the capability adjustment step. In this method, the upper limit value of the back-end task processing of the controller is adjusted by the relationship between the ratio of the front-end performance information to the comprehensive performance information and the preset performance threshold; when the ratio is greater than the performance threshold, the upper limit value of the back-end task processing is adjusted upward, otherwise, the upper limit value of the back-end task processing is adjusted downward, so that the front-end and back-end tasks are balanced, and the capability adjustment step is used for adjustment at the same time, so that the adjustment is within a controllable range, and the accuracy of the adjustment is improved.

[0137] Based on the above embodiment, an embodiment is provided to illustrate the process of adjusting the upper limit value of the back-end task processing of the controller according to the front-end performance information of the controller.

[0138] In an exemplary embodiment, as shown in FIG7 , adjusting the upper limit of backend task processing of the controller according to the frontend performance information of the controller includes:

[0139] S701, when the adjustment state of the backend task of the controller in the previous distribution cycle is upward, the upper limit value of the backend task processing of the controller is adjusted according to the decrease status information of the front-end performance information compared with the previous distribution cycle.

[0140] In the embodiment of the present application, the adjustment of the upper limit value of backend task processing is divided into more detailed categories, and the upper limit value of the backend task processing of the controller is adjusted according to whether the adjustment status of the backend task of the controller in the previous distribution cycle is increased or decreased.

[0141] If the adjustment state of the backend task of the controller in the last distribution cycle is upward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the decrease status information of the frontend performance information compared with the last distribution cycle.

[0142] The decline status information of the front-end performance information compared with the previous distribution cycle refers to the scope of decline of the front-end performance information compared with the front-end performance information of the previous distribution cycle, including whether the decline is within a controllable range or not within a controllable range.

[0143] For example, if the descent status information is within the controllable range, the upper limit value of the backend task processing of the controller is adjusted upward; if the descent status information is not within the controllable range, the upper limit value of the backend task processing of the controller is adjusted downward.

[0144] S702, when the adjustment state of the backend task of the controller in the previous distribution cycle is downward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the change status information of the front-end performance information compared with the previous distribution cycle.

[0145] If the adjustment state of the backend task of the controller in the previous distribution cycle is downward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the change status information of the front-end performance information compared with the previous distribution cycle.

[0146] The change status information of the front-end performance information compared with the previous distribution cycle refers to the change status of the front-end performance information compared with the front-end performance information of the previous distribution cycle, including increase or decrease.

[0147] For example, if the change status information is an increase, the upper limit value of the backend task processing of the controller is lowered; if the decrease status information is a decrease, the upper limit value of the backend task processing of the controller is increased.

[0148] In the task processing method provided by the embodiment of the present application, when the adjustment state of the backend task of the controller in the previous distribution cycle is an upward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the decline status information of the front-end performance information compared with the previous distribution cycle; when the adjustment state of the backend task of the controller in the previous distribution cycle is a downward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the change status information of the front-end performance information compared with the previous distribution cycle. In this method, the situation where the adjustment state of the backend task of the controller in the previous distribution cycle is an adjustment is divided more finely to refine the adjustment of the upper limit value of the backend task processing of the controller. In different adjustment situations, different methods are used to adjust the upper limit value of the backend task processing of the controller, so that the adjustment is more detailed, the adjustment accuracy is improved, and a solid foundation is provided for the balance of the front-end and back-end tasks.

[0149] Based on the above embodiment, an embodiment is provided to illustrate the process of adjusting the upper limit value of the backend task processing of the controller according to the decline status information of the frontend performance information compared with the previous distribution cycle.

[0150] In an exemplary embodiment, as shown in FIG8 , adjusting the upper limit of backend task processing of the controller according to the decrease status information of the frontend performance information compared with the previous distribution cycle includes:

[0151] S801, if the degradation status information is that the front-end performance information has not decreased compared with the front-end performance information of the previous distribution cycle or the degradation range is within a preset range, the upper limit value of the back-end task processing of the controller is increased according to the preset capacity adjustment step size.

[0152] In the embodiment of the present application, when the decline status information is that the front-end performance information has not declined compared to the front-end performance information of the previous distribution cycle or the decline range is within a preset range, the main service adjusts the upper limit of the back-end task processing of the controller according to the preset capacity adjustment step. Among them, the capacity adjustment step is consistent with the above and will not be repeated. Similarly, one or more adjustments can be made to achieve a balanced state of front-end and back-end task processing.

[0153] S802: If the degradation status information indicates that the degradation range of the front-end performance information compared with the front-end performance information of the previous distribution cycle is not within the preset range, the upper limit value of the back-end task processing of the controller is lowered according to the historical capacity adjustment data.

[0154] In an embodiment of the present application, when the degradation status information is that the degradation range of the front-end performance information compared to the front-end performance information of the previous distribution cycle is not within the preset range, the main service adjusts the upper limit of the controller's back-end task processing according to the historical capacity adjustment data.

[0155] The historical capacity adjustment data refers to the adjustment value corresponding to the last adjustment of the backend task. The main service can return the last adjustment value, that is, lower the upper limit of the controller's backend task processing.

[0156] In the task processing method provided by the embodiment of the present application, if the decline status information is that the front-end performance information has not declined compared with the front-end performance information of the previous distribution cycle or the decline range is within the preset range, then the upper limit value of the back-end task processing of the controller is adjusted upward according to the preset capacity adjustment step; if the decline status information is that the decline range of the front-end performance information compared with the front-end performance information of the previous distribution cycle is not within the preset range, then the upper limit value of the back-end task processing of the controller is adjusted downward according to the historical capacity adjustment data. In this method, when the range of the front-end performance decline is in different ranges, the corresponding back-end task processing upper limit value is adjusted in different ways. The back-end task processing upper limit value of the controller is adjusted according to different decline situations, so that the back-end task processing upper limit value of the current distribution cycle can be controlled based on the changes in the front-end performance, without affecting the processing of the front-end tasks, thereby improving the rationality of the adjustment.

[0157] Based on the above embodiment, an embodiment is provided to illustrate the process of adjusting the total capacity of the backend tasks of the controller according to the change status information of the frontend performance information compared with the previous distribution cycle.

[0158] In an exemplary embodiment, as shown in FIG9 , adjusting the total capacity of the backend tasks of the controller according to the change status information of the frontend performance information compared with the previous distribution cycle includes:

[0159] S901: If the change status information indicates that the front-end performance information has increased compared to the front-end performance information of the previous distribution cycle, the upper limit value of the back-end task processing of the controller is lowered according to a preset capacity adjustment step.

[0160] When the change status information indicates that the front-end performance information has increased compared to the front-end performance information of the previous distribution cycle, the main service adjusts the step size according to the preset capacity and lowers the upper limit of the back-end task processing of the controller.

[0161] In the embodiment of the present application, if the front-end performance increases compared to the previous distribution cycle and reaches a certain proportion, it means that the adjustment of the back-end task is beneficial, and the upper limit of the back-end task processing of the controller continues to be lowered.

[0162] S902: If the change status information indicates that the front-end performance information is lower than the front-end performance information of the previous distribution cycle, the upper limit value of the back-end task processing of the controller is increased according to the historical capacity adjustment data.

[0163] When the change status information indicates that the front-end performance information is lower than the front-end performance information of the previous distribution cycle, the main service adjusts the data according to the historical capacity and increases the upper limit of the back-end task processing of the controller.

[0164] The historical capacity adjustment data refers to the adjustment value corresponding to the last adjustment of the backend task. The main service can return the last adjustment value to increase the upper limit of the controller's backend task processing.

[0165] In the task processing method provided by the embodiment of the present application, if the change status information is that the front-end performance information has increased compared to the front-end performance information of the previous distribution cycle, then the upper limit value of the back-end task processing of the controller is lowered according to the preset capacity adjustment step; if the change status information is that the front-end performance information has decreased compared to the front-end performance information of the previous distribution cycle, then the upper limit value of the back-end task processing of the controller is increased according to the historical capacity adjustment data. In this method, when the adjustment state of the back-end task of the controller in the previous distribution cycle is downward, the upper limit value of the back-end task processing of the controller is adjusted accordingly according to the different changes in the front-end performance, so as to achieve a balance between the front-end and back-end tasks and improve the working efficiency and stability of the storage device.

[0166] Based on the above embodiment, an embodiment is provided to illustrate the process of obtaining the above front-end performance information and back-end performance information.

[0167] In an exemplary embodiment, as shown in FIG10 , the method further includes:

[0168] S1001, sending an information acquisition request to each proxy service; the information acquisition request is used to instruct each proxy service to collect the front-end performance statistics information and back-end performance statistics information of the corresponding controller in the current distribution cycle.

[0169] The main service sends an information acquisition request to each proxy service to instruct each proxy service to collect the front-end performance statistics information and back-end performance statistics information of its controller in the current distribution cycle and report them to the main service.

[0170] S1002, receiving front-end performance statistics information and back-end performance statistics information reported by each proxy service according to the information acquisition request.

[0171] Each proxy server collects the front-end performance statistics and back-end performance statistics of its controller and reports them to the main service.

[0172] In the task processing method provided in the embodiment of the present application, an information acquisition request is sent to each proxy service, and the front-end performance statistics and back-end performance statistics reported by each proxy service according to the information acquisition request are received; wherein the information acquisition request is used to instruct each proxy service to collect the front-end performance statistics and back-end performance statistics of the corresponding controller in the current distribution cycle. In this method, through the interaction between the main service and each proxy service, the main service can timely obtain the front-end performance information and back-end performance information reported by each proxy service, so that the main service can make timely adjustments to the front-end and back-end tasks of the current distribution cycle, so that the front-end and back-end tasks reach a balanced state, thereby improving the availability of the storage device.

[0173] The above is a description of the relevant embodiments with the main service as the execution subject. Based on this, the embodiment of the present application also provides an embodiment corresponding to the above process with the proxy service as the execution subject. Below, the relevant embodiments of the task processing method with the proxy service as the execution subject are described.

[0174] Since all the implementation principles and detailed processes in the aforementioned embodiment with the main service as the execution subject, as well as the technical effects that can be achieved are the same as those in the aforementioned embodiment with the proxy service as the execution subject, for the sake of brevity and clarity, the above embodiments will not be described one by one, and the implementation process and implementation effects of each embodiment can be found in the description of the aforementioned embodiment.

[0175] In an exemplary embodiment, as shown in FIG. 11 , a task processing method is provided, which is applied to a proxy service, wherein a proxy service is deployed on each controller in a storage device; a main service is deployed on any controller in the storage device; the method comprises:

[0176] S1101, reporting the front-end performance statistics information and the back-end performance statistics information of the controller to which it belongs to to the main service.

[0177] In an embodiment of the present application, the front-end performance statistical information reported by the proxy service to the main service includes the front-end pressure information and front-end performance information of the controller to which it belongs during the current distribution cycle, and the reported back-end performance statistical information includes the back-end pressure information and back-end performance information of the controller to which it belongs during the current distribution cycle.

[0178] S1102, processing of front-end tasks and back-end tasks is implemented according to tokens distributed by the main service based on front-end performance statistics information and back-end performance statistics information.

[0179] The token is generated by the main service based on the upper limit of backend task processing and the upper limit of frontend task processing adjusted by each controller.

[0180] The proxy service reports the front-end performance statistics and back-end performance statistics to the main service. The main service adjusts the upper limit of the controller's back-end task processing based on the front-end performance statistics and the back-end performance statistics, and generates a token according to the adjusted upper limit of the back-end task processing, and generates a token according to the upper limit of the front-end task processing, and sends the generated token to the proxy service. After receiving the token distributed by the main service, the proxy service processes the front-end tasks and back-end tasks based on the distributed tokens. The proxy service puts the token into the token bucket. Each time the processor corresponding to the proxy service processes a front-end task or a back-end task, it consumes a token from the token corresponding to the front-end task, or consumes a token from the token corresponding to the back-end task.

[0181] In the task processing method provided in the embodiment of the present application, the proxy service reports the front-end performance statistics and back-end performance statistics of the controller to which it belongs to the main service, and then implements the processing of the front-end tasks and back-end tasks according to the tokens distributed by the main service based on the front-end performance statistics and back-end performance statistics. In this method, the main service is deployed on any controller in the storage device, and the proxy service is deployed on each controller in the storage device. The main service obtains the front-end performance statistics and back-end performance statistics reported by each proxy service, which is equivalent to obtaining the processing status of each controller on the front-end tasks and back-end tasks, so as to adjust the upper limit of the back-end task processing of each controller so that the front-end tasks and back-end tasks reach a balanced state. On this basis, the main service distributes tokens to each proxy service according to the adjusted upper limit of back-end task processing and front-end task processing of each controller, so that the controller corresponding to each proxy service consumes a corresponding number of tokens to process front-end tasks and back-end tasks. This is equivalent to the main service on the controller and each proxy service adaptively adjusting the front-end and back-end tasks to ensure that the front-end and back-end tasks can reach a balanced state, so that the storage device is maintained in a stable and efficient state. In this process, it is no longer constrained by the host, which ensures the stability of task processing and thereby improves the availability and stability of the storage device.

[0182] Based on the above embodiment, another embodiment is provided to illustrate the token distribution process.

[0183] In an exemplary embodiment, the method further comprises:

[0184] In the case of not receiving the token distributed by the main service, a token for the current distribution cycle is generated based on the historical token of the previous distribution cycle of the current distribution cycle; the token is used by the controller corresponding to the proxy service to process front-end tasks and back-end tasks.

[0185] In an embodiment of the present application, if a link between the main service and a proxy service fails and the proxy service does not receive the token distributed by the main service, the proxy service generates a new token as the token for the current distribution cycle based on the token received last time, and processes the front-end tasks and back-end tasks to ensure that the system performance is not greatly fluctuated.

[0186] In the task processing method provided by the embodiment of the present application, in the case where the token distributed by the main service is not received, a token for the current distribution cycle is generated based on the historical token of the previous distribution cycle of the current distribution cycle, and the token is used by the controller corresponding to the proxy service to process the front-end task and the back-end task. In this method, if there is a fault between the main service and the proxy service link, the proxy service generates a token for the current distribution cycle based on the token of the previous distribution cycle, so that the proxy service can still schedule the back-end task, ensuring the stability of task processing and further improving the availability and stability of the storage device.

[0187] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0188] Based on the same inventive concept, the embodiment of the present application also provides a task processing system. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations of one or more task processing systems provided below can refer to the limitations of the task processing method above, and will not be repeated here.

[0189] In an exemplary embodiment, a task processing system includes a storage device; the storage device includes a plurality of controllers; a main service is deployed on any one of the plurality of controllers, and an agent service is deployed on each of the controllers;

[0190] Each proxy service is used to report the front-end performance statistics and back-end performance statistics of its controller to the main service;

[0191] The main service is used to adjust the upper limit of backend task processing of each controller according to the frontend performance statistics and backend performance statistics reported by each proxy service; distribute tokens to each proxy service according to the adjusted upper limit of backend task processing and upper limit of frontend task processing of each controller, and the tokens are used for the controller corresponding to each proxy service to process the corresponding frontend tasks and backend tasks.

[0192] Based on the same inventive concept, the embodiment of the present application also provides a task processing device for implementing the task processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more task processing device embodiments provided below can refer to the limitations on the task processing method above, and will not be repeated here.

[0193] In an exemplary embodiment, as shown in FIG12 , a task processing device 1 is provided, comprising: an upper limit adjustment module 10 and a token distribution module 20, wherein:

[0194] The upper limit adjustment module 10 is used to adjust the upper limit value of the backend task processing of each controller according to the front-end performance statistics information and the back-end performance statistics information of the controller to which each proxy service belongs reported;

[0195] The token distribution module 20 is used to distribute tokens to each proxy service according to the adjusted upper limit of backend task processing and upper limit of frontend task processing of each controller. The token is used for the controller corresponding to each proxy service to process the corresponding frontend task and backend task.

[0196] In one embodiment, the upper limit adjustment module 10 is further used for:

[0197] When the front-end pressure information and the back-end pressure information are not zero, the adjustment status of the back-end tasks of each controller in the previous distribution cycle is obtained; according to each adjustment status, each front-end performance information and each back-end performance information, the upper limit value of each back-end task processing is adjusted.

[0198] In one embodiment, the upper limit adjustment module 10 is further used for:

[0199] For any controller, when the adjustment state of the controller's backend task in the previous distribution cycle is not adjusted, the upper limit value of the backend task processing of the controller is adjusted according to the front-end performance information and back-end performance information of the controller; when the adjustment state of the controller's backend task in the previous distribution cycle is adjusted, the upper limit value of the backend task processing of the controller is adjusted according to the front-end performance information of the controller.

[0200] In one embodiment, the upper limit adjustment module 10 is further used for:

[0201] The comprehensive performance information of the controller is determined according to the front-end performance information and the back-end performance information; and the upper limit value of the back-end task processing of the controller is adjusted according to the ratio of the front-end performance information to the comprehensive performance information.

[0202] In one embodiment, the upper limit adjustment module 10 is further used for:

[0203] If the ratio of the front-end performance information to the comprehensive performance information is greater than the preset performance threshold, the upper limit value of the controller's back-end task processing is increased according to the preset capability adjustment step size; if the ratio of the front-end performance information to the comprehensive performance information is less than or equal to the performance threshold, the upper limit value of the controller's back-end task processing is decreased according to the capability adjustment step size.

[0204] In one embodiment, the upper limit adjustment module 10 is further used for:

[0205] When the adjustment status of the controller's backend task in the previous distribution cycle is an upward adjustment, the upper limit value of the controller's backend task processing is adjusted according to the decline status information of the front-end performance information compared with the previous distribution cycle; when the adjustment status of the controller's backend task in the previous distribution cycle is an upward adjustment, the upper limit value of the controller's backend task processing is adjusted according to the change status information of the front-end performance information compared with the previous distribution cycle.

[0206] In one embodiment, the upper limit adjustment module 10 is further used for:

[0207] If the degradation status information is that the front-end performance information has not decreased compared with the front-end performance information of the previous distribution cycle or the degradation range is within the preset range, then the upper limit value of the controller's back-end task processing is increased according to the preset capacity adjustment step; if the degradation status information is that the front-end performance information has not decreased compared with the front-end performance information of the previous distribution cycle, then the upper limit value of the controller's back-end task processing is decreased according to the historical capacity adjustment data.

[0208] In one embodiment, the upper limit adjustment module 10 is further used for:

[0209] If the change status information shows that the front-end performance information has increased compared to the front-end performance information of the previous distribution cycle, the upper limit value of the controller's back-end task processing will be lowered according to the preset capacity adjustment step; if the change status information shows that the front-end performance information has decreased compared to the front-end performance information of the previous distribution cycle, the upper limit value of the controller's back-end task processing will be increased according to the historical capacity adjustment data.

[0210] In one embodiment, the task processing device 1 further includes:

[0211] A request sending module is used to send an information acquisition request to each proxy service; the information acquisition request is used to instruct each proxy service to collect the front-end performance statistics information and back-end performance statistics information of the corresponding controller in the current distribution cycle;

[0212] The information receiving module is used to receive the front-end performance statistics information and the back-end performance statistics information reported by each proxy service according to the information acquisition request.

[0213] In an exemplary embodiment, as shown in FIG13 , a task processing device 2 is provided, comprising: an information reporting module 30 and a token processing module 40, wherein:

[0214] The information reporting module 30 is used to report the front-end performance statistics information and the back-end performance statistics information of the controller to which it belongs to to the main service;

[0215] The token processing module 40 is used to process the front-end tasks and the back-end tasks according to the tokens distributed by the main service based on the front-end performance statistics information and the back-end performance statistics information.

[0216] In one embodiment, the task processing device 2 further includes:

[0217] The token generation module is used to generate a token for the current distribution cycle based on the historical token of the previous distribution cycle of the current distribution cycle when the token distributed by the main service is not received; the token is used by the controller corresponding to the proxy service to process the front-end tasks and the back-end tasks.

[0218] Each module in the above task processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0219] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG14. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store task processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a task processing method is implemented.

[0220] Those skilled in the art will understand that the structure shown in FIG. 14 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0221] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0222] The implementation principles and technical effects of each step implemented by the processor in the embodiment of the present application are similar to the principles of the above-mentioned task processing method and will not be repeated here.

[0223] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0224] The implementation principles and technical effects of the various steps implemented when the computer program in the embodiment of the present application is executed by the processor are similar to the principles of the above-mentioned task processing method and will not be repeated here.

[0225] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0226] The implementation principles and technical effects of the various steps implemented when the computer program in the embodiment of the present application is executed by the processor are similar to the principles of the above-mentioned task processing method and will not be repeated here.

[0227] It should be noted that the personnel information (including but not limited to personnel equipment information, personnel personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the personnel or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0228] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0229] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0230] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A task processing method, characterized in that: Applied to a main service, the main service is deployed on any controller in a storage device, and a proxy service is deployed on each controller of the storage device; the method comprises: According to the front-end performance statistics and back-end performance statistics of the controllers reported by the proxy services, the upper limit of back-end task processing of each controller is adjusted; A token is distributed to each proxy service according to the adjusted upper limit of backend task processing and upper limit of frontend task processing by each controller, and the token is used for the controller corresponding to each proxy service to process the corresponding frontend task and backend task.

2. The method according to claim 1, characterized in that The front-end performance statistical information includes the front-end pressure information and the front-end performance information in the current distribution cycle; the back-end performance statistical information includes the back-end pressure information and the back-end performance information in the current distribution cycle; the back-end task processing upper limit value of each controller is adjusted according to the front-end performance statistical information and the back-end performance statistical information of the controller reported by each proxy service, including: When each of the front-end pressure information and each of the back-end pressure information is not zero, obtaining an adjustment state of the back-end task of each of the controllers in the previous distribution cycle; According to each of the adjustment states, each of the front-end performance information and each of the back-end performance information, the upper limit value of each of the back-end task processing is adjusted.

3. The method according to claim 2, characterized in that The adjusting the upper limit of each backend task processing according to each adjustment state, each frontend performance information and each backend performance information includes: For any controller, when the adjustment state of the backend task of the controller in the previous distribution cycle is not adjusted, adjusting the upper limit value of the backend task processing of the controller according to the frontend performance information and the backend performance information of the controller; When the adjustment state of the backend task of the controller in the last distribution cycle is adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the frontend performance information of the controller.

4. The method according to claim 3, characterized in that The adjusting the upper limit value of backend task processing of the controller according to the frontend performance information and the backend performance information of the controller includes: Determining comprehensive performance information of the controller according to the front-end performance information and the back-end performance information; The upper limit value of backend task processing of the controller is adjusted according to the ratio of the frontend performance information to the comprehensive performance information.

5. The method according to claim 4, characterized in that The adjusting the upper limit value of backend task processing of the controller according to the ratio of the frontend performance information to the comprehensive performance information includes: If the ratio of the front-end performance information to the comprehensive performance information is greater than a preset performance threshold, the upper limit of the back-end task processing of the controller is adjusted upward according to a preset capability adjustment step; If the ratio of the front-end performance information to the comprehensive performance information is less than or equal to the performance threshold, the upper limit value of the back-end task processing of the controller is lowered according to the capability adjustment step.

6. The method according to any one of claims 3 to 5, characterized in that: The adjusting the upper limit value of backend task processing of the controller according to the frontend performance information of the controller includes: When the adjustment state of the backend task of the controller in the previous distribution cycle is an upward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the decrease status information of the front-end performance information compared with the previous distribution cycle; When the adjustment state of the backend task of the controller in the previous distribution cycle is downward adjustment, the upper limit value of the backend task processing of the controller is adjusted according to the change status information of the front-end performance information compared with the previous distribution cycle.

7. The method according to claim 6, characterized in that The adjusting the upper limit value of backend task processing of the controller according to the decrease status information of the frontend performance information compared with the previous distribution cycle includes: If the degradation status information indicates that the front-end performance information has not decreased compared with the front-end performance information of the previous distribution cycle or the degradation range is within a preset range, the upper limit value of the back-end task processing of the controller is increased according to a preset capacity adjustment step size; If the degradation status information indicates that the degradation range of the front-end performance information compared to the front-end performance information of the previous distribution cycle is not within the preset range, the upper limit value of the back-end task processing of the controller is lowered according to the historical capacity adjustment data.

8. The method according to claim 6, characterized in that The adjusting the total capacity of the backend tasks of the controller according to the change status information of the frontend performance information compared with the previous distribution cycle includes: If the change status information indicates that the front-end performance information has increased compared to the front-end performance information of the previous distribution cycle, the upper limit value of the back-end task processing of the controller is adjusted downward according to a preset capacity adjustment step; If the change status information indicates that the front-end performance information is lower than the front-end performance information of the previous distribution cycle, the upper limit value of the back-end task processing of the controller is increased according to the historical capacity adjustment data.

9. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Sending an information acquisition request to each of the proxy services; the information acquisition request is used to instruct each of the proxy services to collect front-end performance statistics information and back-end performance statistics information of the corresponding controller in the current distribution cycle; Receive the front-end performance statistics information and the back-end performance statistics information reported by each of the proxy services according to the information acquisition request.

10. A task processing method, characterized in that: Applied to the proxy service, the proxy service is deployed on each controller in the storage device; a main service is deployed on any controller in the storage device; the method comprises: Reporting the front-end performance statistics and back-end performance statistics of the controller to which it belongs to to the main service; According to the tokens distributed by the main service based on the front-end performance statistical information and the back-end performance statistical information, the front-end tasks and the back-end tasks are processed.

11. The method according to claim 10, characterized in that The method further comprises: In the case where the token distributed by the main service is not received, a token for the current distribution cycle is generated based on the historical token of the previous distribution cycle of the current distribution cycle; the token is used by the controller corresponding to the proxy service to process the front-end task and the back-end task.

12. A task processing system, characterized in that: The task processing system includes a storage device; the storage device includes a plurality of controllers; a main service is deployed on any one of the plurality of controllers, and a proxy service is deployed on each of the controllers; Each of the proxy services is used to report the front-end performance statistics information and the back-end performance statistics information of the controller to which it belongs to to the main service; The main service is used to adjust the upper limit of backend task processing of each controller according to the front-end performance statistics and back-end performance statistics reported by each proxy service; and distribute tokens to each proxy service according to the adjusted upper limit of back-end task processing and upper limit of front-end task processing of each controller, and the tokens are used for the controller corresponding to each proxy service to process the corresponding front-end tasks and back-end tasks.

Images