Business processing method and device, equipment and storage medium

Abstract

The invention relates to a business processing method and device, equipment and a storage medium. The method comprises the steps that in response to a service processing instruction, average time delay information of user input and output IO on a target disk under the last service processing round and service association information of background services associated with the target disk under the current service processing round are acquired, the background service comprises at least one of an invalid data recovery service, a data restoration service, a data migration service and a data replication service, and determining a target IO request quantity of the background service under the current service processing round according to the average time delay information and the service association information of the background service. By adopting the method, the operation reliability of the storage system can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a business processing method, device, equipment and storage medium. Background Art

[0002] There are various background services in the object storage system, such as repair, garbage collection, hierarchical storage, asynchronous remote replication, etc. During the operation of the above background services, they will occupy disk resources, thus affecting the processing of user services. Therefore, in the existing business processing method, in order to ensure the normal operation of user services, the number of concurrent background services will be limited to limit the preemption of disk resources by background services.

[0003] However, using the existing method to simply limit the number of concurrent background businesses will cause problems such as the inability to quickly restore system data reliability, thereby reducing the reliability of object storage system operation. Summary of the invention

[0004] Based on this, it is necessary to provide a business processing method, device, equipment and storage medium that can improve the reliability of storage system operation in response to the above technical problems.

[0005] In a first aspect, the present application provides a business processing method, including:

[0006] In response to the business processing instruction, the average latency information of user input and output IO on the target disk in the previous business processing round and the business association information of the background business associated with the target disk in the current business processing round are obtained; wherein the background business includes at least one of the invalid data recovery business, the data repair business, the data migration business and the data replication business;

[0007] According to the average latency information and the business association information of the background business, the target IO request volume of the background business in the current business processing round is determined.

[0008] In an embodiment of the present application, by determining the target IO request volume that can be sent by the background business based on the average latency information of the user IO request and the business association information of the background business, it is possible to ensure the normal operation of the background business on the basis of reducing the impact of the background business on the user business, thereby improving the reliability of the storage system operation.

[0009] In one of the embodiments, when the background service includes an invalid data recovery service, the service association information includes the invalid data ratio in the logical disk set VSET and the remaining available capacity of the storage pool to which the VSET belongs; and according to the average latency information and the service association information of the background service, determining the target IO request amount of the background service in the current service processing round includes:

[0010] According to the proportion of invalid data and the remaining available capacity, the initial IO request amount of the background business in the current business processing round is determined; according to the average latency information, the initial IO request amount is adjusted to obtain the target IO request amount of the background business in the current business processing round.

[0011] In an embodiment of the present application, by determining the target IO request amount of the invalid data recovery service based on the invalid data ratio, the remaining available capacity and the average delay information, the target IO request amount corresponding to the invalid data recovery service in the current business processing round can be matched with the execution status of the user IO request, thereby ensuring the rationality of the target IO request amount.

[0012] In one of the embodiments, when the background service includes a data repair service, the service association information includes the current data redundancy of the VSET; determining the target IO request amount of the background service in the current service processing round according to the average delay information and the service association information of the background service includes:

[0013] Determine the number of invalid data copies based on the current data redundancy and the standard data redundancy corresponding to the data repair business; determine the target IO request volume of the background business in the current business processing round based on the number of invalid data copies and the average latency information.

[0014] In an embodiment of the present application, by determining the target IO request amount of the data repair service based on the number of invalidable data copies and the average latency information, the target IO request amount corresponding to the data repair service in the current business processing round can be matched with the execution status of the user IO request, thereby ensuring the rationality of the target IO request amount.

[0015] In one of the embodiments, when the background service includes a data migration service, the service association information includes a hierarchical storage priority and an available capacity of a high-performance layer in a VSET; and determining a target IO request amount of the background service in a current service processing round according to the average latency information and the service association information of the background service, including:

[0016] According to the available capacity and average latency information, the initial IO request amount of the background business in the current business processing round is determined; according to the hierarchical storage priority, the initial IO request amount is adjusted to obtain the target IO request amount of the background business in the current business processing round.

[0017] In an embodiment of the present application, by determining the target IO request amount of the data migration service based on available capacity, average latency information and hierarchical storage priority, the target IO request amount corresponding to the data migration service in the current service processing round can be matched with the execution status of the user IO request, thereby ensuring the rationality of the target IO request amount.

[0018] In one of the embodiments, when the background service includes a data replication service, the service association information includes a priority of the remote replication task.

[0019] In the embodiment of the present application, the priority of the remote replication task associated with the data replication service is introduced to ensure the rationality of determining the target IO request amount.

[0020] In one of the embodiments, when the background service includes at least two of an invalid data recovery service, a data repair service, a data migration service, and a data replication service, determining a target IO request amount of the background service in a current service processing round according to average latency information and service association information of the background service includes:

[0021] According to the average latency information, the total IO request volume of each background business is determined; according to the total IO request volume and the business association information of each background business, the target IO request volume of each background business in the current business processing round is determined.

[0022] In an embodiment of the present application, by determining the target IO request amount of each background business in the current business processing round based on the average latency information and the business association information of each background business, it is possible to ensure that the target IO request amount of each background business matches the execution status of the user's IO request and ensure the rationality of the allocation of the target IO request amount of each background business.

[0023] In one embodiment, the method further comprises:

[0024] Send the target IO request quantity current business tokens to the business management end corresponding to the background business; wherein each current business token is used for the business management end to initiate a business IO request; within the current business processing round, the business IO request initiated by the business management end based on the current business token is processed.

[0025] In an embodiment of the present application, by sending the target IO request quantity current business tokens to the business management end corresponding to the background business, it can be ensured that only the target IO request quantity business IO requests are processed within the current business processing round, thereby ensuring the reliability of user IO request processing.

[0026] In a second aspect, the present application also provides a service processing device, including:

[0027] An information acquisition module, which is used to respond to the business processing instruction and acquire the average latency information of user input and output IO on the target disk in the previous business processing round, and the business association information of the background business associated with the target disk in the current business processing round; wherein the background business includes at least one of the invalid data recovery business, the data repair business, the data migration business and the data replication business;

[0028] The determination module is used to determine the target IO request amount of the background business in the current business processing round according to the average delay information and the business association information of the background business.

[0029] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0030] In response to the business processing instruction, the average latency information of user input and output IO on the target disk in the previous business processing round and the business association information of the background business associated with the target disk in the current business processing round are obtained; wherein the background business includes at least one of the invalid data recovery business, the data repair business, the data migration business and the data replication business;

[0031] According to the average latency information and the business association information of the background business, the target IO request volume of the background business in the current business processing round is determined.

[0032] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0033] In response to the business processing instruction, the average latency information of user input and output IO on the target disk in the previous business processing round and the business association information of the background business associated with the target disk in the current business processing round are obtained; wherein the background business includes at least one of the invalid data recovery business, the data repair business, the data migration business and the data replication business;

[0034] According to the average latency information and the business association information of the background business, the target IO request volume of the background business in the current business processing round is determined.

[0035] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0036] In response to the business processing instruction, the average latency information of user input and output IO on the target disk in the previous business processing round and the business association information of the background business associated with the target disk in the current business processing round are obtained; wherein the background business includes at least one of the invalid data recovery business, the data repair business, the data migration business and the data replication business;

[0037] According to the average latency information and the business association information of the background business, the target IO request volume of the background business in the current business processing round is determined.

[0038] The above-mentioned business processing method, device, equipment and storage medium obtain the average delay information of user IO requests on the target disk in the previous business processing round, and the business association information of at least one background business of invalid data recovery business, data repair business, data migration business and data replication business associated with the target disk in the current business processing round by responding to the business processing instruction, and determine the target IO request amount of the background business in the current business processing round according to the average delay information and the business association information of the background business. Compared with the related art, which limits the preemption of disk resources by the background business by directly limiting the number of concurrent background businesses, the above-mentioned method determines the target IO request amount of IO requests that can be sent by the background business according to the average delay information of the user IO request and the business association information of the background business, which can ensure the normal operation of the background business on the basis of reducing the impact of the background business on the user business, thereby improving the reliability of the storage system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1A A flowchart of a business processing method in an embodiment;

[0041] Figure 1B A schematic diagram of business processing in one embodiment;

[0042] Figure 2 A schematic diagram of a process for determining a target IO request amount in one embodiment;

[0043] Figure 3 A schematic diagram of a process for determining a target IO request amount in another embodiment;

[0044] Figure 4 A schematic diagram of a process for determining a target IO request amount in another embodiment;

[0045] Figure 5A A schematic diagram of a process for determining a target IO request amount in yet another embodiment;

[0046] Figure 5B A schematic diagram of IO service processing in one embodiment;

[0047] Figure 6 A schematic diagram of a flow chart of backend business processing in an embodiment;

[0048] Figure 7 is a flowchart of a business processing method in another embodiment;

[0049] Figure 8 is a structural block diagram of a service processing device in an embodiment;

[0050] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] There are various background services in the object storage system, such as repair, garbage collection, hierarchical storage, asynchronous remote replication, etc. During the operation of the above background services, they will occupy disk resources, thus affecting the processing of user services. Therefore, in the existing business processing method, in order to ensure the normal operation of user services, the number of concurrent background services will be limited to limit the preemption of disk resources by background services.

[0053] However, using the existing method to simply limit the number of concurrent background businesses will cause problems such as the inability to quickly restore system data reliability, thereby reducing the reliability of object storage system operation.

[0054] Based on this, in an exemplary embodiment, Figure 1A As shown, a service processing method is provided, which is described by taking the method applied to a service processing device in a storage system as an example, and specifically includes the following steps:

[0055] S101, in response to a service processing instruction, obtaining average latency information of user input and output IO on a target disk in a previous service processing round, and service association information of a background service associated with the target disk in a current service processing round.

[0056] Among them, the business processing instruction is used to instruct the background business processing. Furthermore, the business processing instruction can be automatically generated based on a preset time interval, or it can be issued by the operation and maintenance end associated with the storage system. There is no specific restriction on this in the present application.

[0057] It can be understood that in order to ensure the reliability of background business processing, at least two adjacent business processing rounds can be determined based on the time interval of business processing, where the current business processing round is the processing round at the current moment; the previous business processing round is the previous business processing round adjacent to the current business processing round.

[0058] The target disk is the disk corresponding to the business processing instruction, that is, the disk with business processing requirements; the average latency information is the average latency for processing user IO requests. Furthermore, the user IO request is the IO request initiated by executing the user business; the background business is the business executed by the system backend on the target disk. In this application, the background business includes at least one of the invalid data recovery business, data repair business, data migration business and data replication business; the business association information is the relevant information under each type of background business.

[0059] For example, refer to Figure 1B As shown in the business processing diagram, in this application, the background business processing is processed at the disk granularity, wherein Disk_1 is Disk 1; OST_1 is the user IO request associated with Disk 1; RCVR_1 is the background IO request under the data repair business associated with Disk 1; GC_1 is the background IO request under the invalid data recovery business associated with Disk 1; REP_1 is the background IO request under the data replication business associated with Disk 1; OTIER_1 is the background IO request under the data migration business associated with Disk 1.

[0060] Optionally, after receiving a business processing instruction, the target disk can be located through the disk identification information contained in the business processing instruction; subsequently, the average latency information of the user IO request can be determined through the processing of the user IO request on the target disk in the previous business processing round, and the business association information in the business management terminal corresponding to the background business associated with the target disk in the current business processing round can be obtained.

[0061] Exemplarily, when determining the average latency information of user IO requests, for each disk, multiple cyclically used slots can be allocated on the disk for user service IO; then, the average latency information of user IO requests is determined based on the information and latency of user service IO on each slot.

[0062] For example, 10 cyclically used slots can be allocated for user service IO. Starting from slot 0, one slot is replaced every 3 seconds, and the information and latency of user service IO on each slot are counted. Then, the 10 slots are accumulated and calculated to obtain the average latency information of user IO requests in the past 10 cycles.

[0063] S102, determining a target IO request amount of the background service in the current service processing round according to the average delay information and the service association information of the background service.

[0064] The target IO request volume is the IO request volume that can be processed by the background business in the current business processing round.

[0065] Optionally, the disk IO resources required for user IO requests in the current business processing round can be predicted based on the average latency information of user IO requests on the target disk in the previous business processing round; then, the disk IO resources that can be allocated to the background business are determined based on the maximum disk IO resource amount of the target disk and the disk IO resources required for user IO requests in the current business processing round.

[0066] Furthermore, the target IO request amount of the background business in the current business processing round can be determined according to the business association information of the background business and the disk IO resources that can be allocated.

[0067] For example, if there is only one type of background business, the amount of disk IO resources required for each background IO request under the background processing order can be determined directly based on the business association information of the background business; then, based on the disk IO resources that can be allocated to the background business, the number of background IO requests that can be processed under the background processing order is determined, and the number of background IO requests is used as the target IO request amount. The background processing order can be generated based on the priority of the background IO request, or it can be generated based on the creation time of the background IO request, and this is not specifically limited in this application.

[0068] If there are multiple types of background businesses, it is necessary to allocate the disk IO resources that can be allocated to the background businesses to each type of background business in advance according to the business association information of each background business; then, for each background business, the number of background IO requests that can be processed under the background processing order corresponding to this type of background business can be determined based on the disk IO resources that can be allocated to this type of background business, and the number of background IO requests can be used as the target IO request amount corresponding to this type of background business.

[0069] Optionally, after determining the target IO request volume corresponding to the background business, the corresponding target IO request volume can be sent to the business management end corresponding to the background business to obtain the background IO requests to be processed in the current business processing round; at the same time, the processing results of the background IO requests processed in the previous business processing round are fed back to the business management end of the background business.

[0070] It is understandable that while the target disk is processing the background IO request, it is also processing the user IO request sent to the target disk to ensure that the processing of the user IO request is not affected by the processing of the background IO request.

[0071] In the above business processing method, by responding to the business processing instruction, the average delay information of the user IO request on the target disk in the previous business processing round and the business association information of at least one background business of the invalid data recovery business, data repair business, data migration business and data replication business associated with the target disk in the current business processing round are obtained, and the target IO request amount of the background business in the current business processing round is determined according to the average delay information and the business association information of the background business. Compared with the related art, which directly limits the number of concurrent background businesses to limit the preemption of disk resources by the background business, the above method is adopted to determine the target IO request amount of IO requests that can be sent by the background business according to the average delay information of the user IO request and the business association information of the background business, which can ensure the normal operation of the background business on the basis of reducing the impact of the background business on the user business, thereby improving the reliability of the storage system operation.

[0072] In order to ensure the rationality of the target IO request amount, based on the above embodiment, in this embodiment, when the background business includes an invalid data recovery business, the business association information includes the invalid data ratio in the logical disk set VSET and the remaining available capacity of the storage pool to which the VSET belongs; further, an optional method for determining the target IO request amount is provided, such as Figure 2 As shown, the specific steps include:

[0073] S201, determining the initial IO request amount of the background business in the current business processing round according to the invalid data ratio and the remaining available capacity.

[0074] In this application, the invalid data recovery service (GC background service) is executed in units of VSET, and the valid and discrete data in the source VSET is moved to the continuous address space of the target VSET. Among them, VSET is a set of logical disks aggregated by data redundancy, and each logical disk can only belong to one VSET. For example, if the storage system is in 3-copy mode, the VSET contains 3 logical disks; if the storage system is in redundant EC: 4+2 ratio mode, the VSET contains 6 logical disks.

[0075] The invalid data ratio in VSET is used to characterize the percentage of invalid data in VSET, which can also be called VSET fragmentation rate. Furthermore, the higher the invalid data ratio in VSET, the more invalid data in VSET is, and the more space should be recycled, and the more space will be released after post-processing. The remaining available capacity of the storage pool to which VSET belongs is used to characterize the space capacity that can be used in the storage pool to which VSET belongs.

[0076] It is understandable that since the invalid data ratio and the remaining available capacity can reflect to a certain extent the urgency of executing the invalid data recovery business in the current business processing round, the initial IO request amount can be determined based on the invalid data ratio and the remaining available capacity. Among them, the initial IO request amount is the IO request amount initially required by the background business.

[0077] Optionally, the invalid data percentage and the remaining available capacity may be input into a trained first determination model, and the first determination model outputs an initial IO request amount according to the invalid data percentage, the remaining available capacity and model parameters.

[0078] Alternatively, the invalid data ratio and the remaining available capacity may be used as indexes to query the corresponding relationship between the invalid data ratio and the remaining available capacity and the IO request amount to obtain the initial IO request amount.

[0079] S202: Adjust the initial IO request amount according to the average delay information to obtain the target IO request amount of the background business in the current business processing round.

[0080] Optionally, in order to ensure that the processing of user IO requests is not affected by background business, the amount of disk IO resources to be occupied by the user IO requests can be determined based on the average latency information of the user IO requests, and then the remaining IO resources of the target disk can be determined; then, based on the remaining IO resources, the initial IO request amount is adjusted to obtain the target IO request amount.

[0081] For example, if the remaining IO resources can execute the background IO requests under the initial IO request amount, the initial IO request amount can be directly used as the target IO request amount; if the remaining IO resources cannot execute all background IO requests under the initial IO request amount, the initial IO request amount can be adjusted according to the remaining IO resources and the disk IO resources required for each background IO request under the background processing order to obtain the target IO request amount.

[0082] It is understandable that, when the remaining IO resources are unable to execute all background IO requests under the initial IO request amount, if the invalid data ratio is higher than the preset ratio threshold and the remaining available capacity is lower than the preset capacity threshold, it proves that the urgency of executing the invalid data recovery service is high. At this time, it is necessary to limit the execution of user IO requests and directly use the initial IO request amount as the target IO request amount under the invalid data recovery service. Among them, the ratio threshold and capacity threshold can be set based on manual experience or determined by a large number of experiments, which is not limited in this application.

[0083] In an embodiment of the present application, by determining the target IO request amount of the invalid data recovery service based on the invalid data ratio, the remaining available capacity and the average delay information, the target IO request amount corresponding to the invalid data recovery service in the current business processing round can be matched with the execution status of the user IO request, thereby ensuring the rationality of the target IO request amount.

[0084] In order to ensure the rationality of the target IO request amount, based on the above embodiment, in this embodiment, when the background service includes a data repair service, the service association information includes the current data redundancy of the VSET; further, another optional method for determining the target IO request amount is provided, such as Figure 3 As shown, the specific steps include:

[0085] S301, determining the number of invalidated data copies according to the current data redundancy and the standard data redundancy corresponding to the data repair service.

[0086] In this embodiment, the data repair service (RCVR background service) is executed in units of VSET, and invalid data can be repaired based on valid data in VSET. Data redundancy is the number of repeated data storage in the storage system; current data redundancy is the data redundancy in the current business processing round; standard data redundancy is the minimum data redundancy required without affecting the reading of user services; the number of invalid data copies is the number of invalid valid data copies in the current business processing round.

[0087] Optionally, the number of invalid data copies of the data repair service can be calculated based on the standard data redundancy corresponding to the data repair service in the storage system and the current data redundancy. Exemplarily, the difference between the standard data redundancy corresponding to the data repair service and the current data redundancy can be used as the number of invalid data copies.

[0088] For example, when the storage system is in 3-copy mode, including data A1-A3, the standard data redundancy is 1. If the data in copy A1 fails, the data in copy A2 or copy A3 can be used to repair the data in copy A1. When 1 copy of data fails in the 3-copy mode, the current data redundancy is 2, and the number of copies of data that can fail is 1, that is, if another copy of data fails, it will not affect the reading of user services.

[0089] In the case of the redundant EC: 8+4 configuration mode, the storage system includes data D1-D8, P1-P4, and the standard data redundancy is 8. If data P1 fails, 8 copies of data can be selected from D1-D8, P2-P4, and data P1 can be obtained by calculation. When 1 copy of data fails in the redundant EC: 8+4 configuration mode, the current data redundancy is 11, and the number of copies of data that can fail is 3, that is, if 3 more copies of data fail, the reading of user services will not be affected.

[0090] S302, determining the target IO request amount of the background business in the current business processing round according to the number of invalidable data copies and the average latency information.

[0091] It is understandable that since the number of invalidable data copies can reflect to a certain extent the urgency of executing data repair services in the current business processing round, the target IO request volume can be determined based on the number of invalidable data copies and the average latency information.

[0092] Optionally, the number of invalidable data copies can be used as an index to query the corresponding relationship between each number of invalidable data copies and each initial IO request to obtain the initial IO request amount under the current business processing round; then, in order to ensure that the processing of user IO requests is not affected by background business, the initial IO request amount can be adjusted according to the average latency information of user IO requests to obtain the target IO request amount under the data repair business.

[0093] It is understandable that, when the remaining IO resources in the target disk cannot execute all background IO requests under the initial IO request amount, if the number of invalid data copies is less than the preset copy number threshold, it proves that the urgency of executing the data repair business is high. At this time, it is necessary to limit the execution of user IO requests, and directly use the initial IO request amount as the target IO request amount under the data repair business. Among them, the copy number threshold can be set based on manual experience or determined by a large number of experiments, which is not limited in this application.

[0094] In an embodiment of the present application, by determining the target IO request amount of the data repair service based on the number of invalidable data copies and the average latency information, the target IO request amount corresponding to the data repair service in the current business processing round can be matched with the execution status of the user IO request, thereby ensuring the rationality of the target IO request amount.

[0095] In order to ensure the rationality of the target IO request amount, based on the above embodiment, in this embodiment, when the background business includes the data migration business, the business association information includes the hierarchical storage priority and the available capacity of the high-performance layer in the VSET; further, another optional method for determining the target IO request amount is provided, such as Figure 4As shown, the specific steps include:

[0096] S401, determining the initial IO request amount of the background service in the current service processing round according to the available capacity and the average delay information.

[0097] In this application, the data migration service (OTIER background service) is executed in units of VSET. According to the task configuration rules, data is migrated from high-performance storage media to low-performance storage media, thereby reducing the storage cost of data. Among them, the storage cost of high-performance storage media is high, and the capacity of high-performance storage media is often limited. The available capacity of the high-performance layer in VSET is the remaining available capacity of the high-performance storage medium in VSET.

[0098] It is understandable that since the available capacity of the high-performance layer in the VSET can reflect to a certain extent the urgency of executing the data migration service in the current service processing round, the initial IO request amount can be determined based on the available capacity and average latency information.

[0099] Optionally, the available capacity and average latency information can be input into a trained second determination model, and the second determination model outputs an initial IO request amount based on the available capacity, average latency information and model parameters. At this time, the initial IO request amount is the maximum IO quantity that the data migration service can execute without affecting the user's IO request.

[0100] S402, adjusting the initial IO request amount according to the hierarchical storage priority to obtain a target IO request amount of the background business in the current business processing round.

[0101] The hierarchical storage priority is the priority of each background IO request determined according to the hierarchical storage situation of the background IO request.

[0102] Optionally, in order to ensure the reliability of data migration business execution, the number of high-priority IOs with a priority higher than the preset first priority threshold can be determined according to the hierarchical storage priority of each background IO request. If the number of high-priority IOs is greater than the initial IO request amount, the initial IO request amount can be adjusted according to the difference between the number of high-priority IOs and the initial IO request amount, that is, by limiting the user IO request, the normal execution of the high-priority data migration task can be ensured. Among them, the first priority threshold can be set based on manual experience or determined by a large number of experiments, which is not limited in this application.

[0103] If the number of high-priority IOs is less than or equal to the initial IO request amount, the initial IO request amount is directly used as the target IO request amount of the data migration service in the current service processing round.

[0104] It is understandable that, during the execution of the data migration service, the execution order of the background IO requests may also be determined according to the hierarchical storage priority of the background IO requests, that is, the background IO requests with high priority are executed first.

[0105] In an embodiment of the present application, by determining the target IO request amount of the data migration service based on available capacity, average latency information and hierarchical storage priority, the target IO request amount corresponding to the data migration service in the current service processing round can be matched with the execution status of the user IO request, thereby ensuring the rationality of the target IO request amount.

[0106] In order to ensure the rationality of the target IO request amount, based on the above embodiments, in this embodiment, when the background business includes a data replication business, the business association information includes the priority of the remote replication task; further, another optional method for determining the target IO request amount is provided, specifically, according to the priority and average delay information of the remote replication task, the target IO request amount of the background business in the current business processing round is obtained.

[0107] In this application, the data replication service (REP background service) is executed in units of objects. The master cluster reads data according to the storage location information (layout) stored in the metadata information of the data object to be migrated, and newly allocates a layout in the slave cluster to write the read data to the slave cluster. The priority of the remote replication task is the priority of each background IO request determined according to the configuration of the background IO request.

[0108] Optionally, the initial IO request amount of the data replication service in the current business processing round can be determined based on the average latency information of the user IO requests; then, the initial IO request amount is adjusted based on the priority of the remote replication task of each background IO request to obtain the target IO request amount of the data replication service in the current business processing round.

[0109] Exemplarily, in order to ensure the reliability of data replication service execution, the number of high-priority IOs with a priority higher than the preset second priority threshold can be determined according to the priority of the remote replication task of each background IO request. If the number of high-priority IOs is greater than the initial IO request amount, the initial IO request amount can be adjusted according to the difference between the number of high-priority IOs and the initial IO request amount, that is, by limiting the user IO request, the normal execution of the high-priority data replication task can be ensured. Among them, the second priority threshold can be set based on manual experience or determined by a large number of experiments, which is not limited in this application.

[0110] If the number of high-priority IOs is less than or equal to the initial IO request amount, the initial IO request amount is directly used as the target IO request amount of the data replication service in the current service processing round.

[0111] It is understandable that, during the execution of the data replication service, the execution order of the background IO requests may also be determined according to the priority of the remote replication task of each background IO request, that is, the background IO requests with high priority are executed first.

[0112] In an embodiment of the present application, by determining the target IO request amount of the data replication service based on the average latency information and the priority of the remote replication task, the target IO request amount corresponding to the data replication service in the current service processing round can be matched with the execution status of the user IO request, thereby ensuring the rationality of the target IO request amount.

[0113] In order to ensure the rationality of the target IO request amount, based on the above embodiment, in this embodiment, when the background business includes at least two of the invalid data recovery business, the data repair business, the data migration business and the data replication business, another optional method for determining the target IO request amount is provided, such as Figure 5A As shown, the specific steps include:

[0114] S501, determining the total IO request amount of each background service according to the average latency information.

[0115] The total IO request volume is the total number of IO requests that can be executed by various background businesses.

[0116] Optionally, the disk IO resources required for user IO requests in the current business processing round can be determined based on the average latency information of user IO requests; then, the disk IO resources that can be allocated to the background business are determined based on the maximum disk IO resource amount of the target disk and the disk IO resources required for user IO requests in the current business processing round, and the total IO request amount of each background business is determined based on the disk IO resources that can be allocated to the background business.

[0117] S502 : Determine a target IO request volume of each background service in a current service processing round according to the total IO request volume and service association information of each background service.

[0118] Optionally, the total IO request volume can be allocated to each background business according to its business association information to obtain the target IO request volume of each background business in the current business processing round, so as to achieve the purpose of dynamically adjusting the target IO request volume of each background business.

[0119] Exemplarily, the total IO request volume and the business association information of each background business can be simultaneously input into a trained third determination model, and the third determination model outputs the target IO request volume of each background business based on the total IO request volume, the business association information of each background business and the model parameters.

[0120] For example, refer to Figure 5B As shown in the schematic diagram of IO business processing, when the background business includes invalid data recovery business (GC), data repair business (RCVR), data migration business (OTIER) and data replication business (REP), the target IO request amount corresponding to each background business can be determined according to the average latency information of user IO and the business association information of GC, RCVR, OTIER and REP.

[0121] In addition, for background businesses with priority information (data migration business / data replication business), the target IO request amount allocated to each background business can be dynamically adjusted according to the priority of each background IO request in the background business. For example, if there are a large number of high-priority background IO requests in the data migration business, a larger target IO request amount can be allocated to the data migration business by reducing the target IO request amount of other background businesses.

[0122] In an embodiment of the present application, by determining the target IO request amount of each background business in the current business processing round based on the average latency information and the business association information of each background business, it is possible to ensure that the target IO request amount of each background business matches the execution status of the user's IO request and ensure the rationality of the allocation of the target IO request amount of each background business.

[0123] In order to ensure the reliability of user IO request processing, based on the above embodiment, in this embodiment, an optional method of background business processing is provided, such as Figure 6 As shown, the specific steps include:

[0124] S601, sending a target IO request quantity and a current business token to a business management terminal corresponding to a background business.

[0125] Among them, the current business token is the business token sent to the background business in the current business processing round; each current business token is used for the business management end to initiate a business IO request, that is, the business management end of the background business can use a current business token to initiate a background IO request.

[0126] Optionally, after determining the target IO request amount of the background business, a current business token of the target IO request amount may be generated, and the generated current business token of the target IO request amount may be issued to a business management terminal corresponding to the background business.

[0127] Alternatively, the target IO request amount can be used as an index to query the corresponding relationship between each IO request amount and the token gear to obtain the target token gear corresponding to the target IO request amount; then, the target token gear is sent to the business management end corresponding to the background business, so that the business management end can generate the target IO request amount and the current business token according to the target token gear.

[0128] S602: In the current business processing round, the business IO request initiated by the business management terminal based on the current business token is processed.

[0129] Optionally, within the current business processing round, the business management end corresponding to the background business can use the current business token to initiate a business IO request to the target disk; accordingly, after the current business token in the business IO request is checked, the business IO request corresponding to the current business token can be executed on the target disk.

[0130] For example, a business IO request can be initiated to the target disk in the form of "current business token + background IO request"; accordingly, for any business IO request initiated to the target disk, it can be determined whether the current business token is contained in the fixed position in the business IO request. If it is contained, the business IO request is executed on the target disk; if not, the business IO request is not executed.

[0131] In an embodiment of the present application, by sending the target IO request quantity current business tokens to the business management end corresponding to the background business, it can be ensured that only the target IO request quantity business IO requests are processed within the current business processing round, thereby ensuring the reliability of user IO request processing.

[0132] Figure 7 FIG. 1 is a flow chart of a service processing method in another embodiment. Based on the above embodiment, this embodiment provides an optional example of a service processing method. Figure 7 The specific implementation process is as follows:

[0133] S701, in response to a service processing instruction, obtaining average latency information of user IO on a target disk in a previous service processing round.

[0134] S702, obtaining the invalid data ratio in the VSET of the invalid data recovery service associated with the target disk in the current service processing round and the remaining available capacity of the storage pool to which the VSET belongs.

[0135] S703, obtaining the current data redundancy of the VSET of the data repair service associated with the target disk in the current service processing round.

[0136] S704, obtaining the hierarchical storage priority of the data migration service associated with the target disk in the current service processing round and the available capacity of the high-performance layer in the VSET.

[0137] S705: Obtain the priority of the remote replication task of the data replication service associated with the target disk in the current service processing round.

[0138] S706: Determine the total IO request amount of each background service according to the average latency information.

[0139] S707, determines the target IO request volume corresponding to each background business in the current business processing round based on the total IO request volume, the proportion of invalid data in the VSET of the invalid data recovery business and the remaining available capacity of the storage pool to which the VSET belongs, the current data redundancy of the VSET of the data repair business, the hierarchical storage priority of the data migration business and the available capacity of the high-performance layer in the VSET, and the priority of the remote replication task of the data replication business.

[0140] Optionally, if the background business only includes invalid data recovery business, the initial IO request amount of the background business in the current business processing round is determined based on the invalid data ratio and the remaining available capacity; the initial IO request amount is adjusted based on the average latency information to obtain the target IO request amount of the background business in the current business processing round.

[0141] If the background business only includes data repair business, the number of invalid data copies is determined based on the current data redundancy and the standard data redundancy corresponding to the data repair business; based on the number of invalid data copies and the average latency information, the target IO request volume of the background business in the current business processing round is determined.

[0142] If the background business only includes data migration business, the initial IO request amount of the background business in the current business processing round is determined based on the available capacity and average latency information; the initial IO request amount is adjusted according to the hierarchical storage priority to obtain the target IO request amount of the background business in the current business processing round.

[0143] If the background service only includes the data replication service, the target IO request amount of the background service in the current service processing round is obtained according to the priority and average delay information of the remote replication task.

[0144] S708, sending the target IO request quantity and current business tokens corresponding to each background business to the business management terminal corresponding to each background business.

[0145] Each current business token is used by the business management end to initiate a business IO request.

[0146] S709: In the current business processing round, the business IO request initiated by the business management terminal corresponding to each background business based on the current business token is processed.

[0147] The specific process of the above S701-S709 can be found in the description of the above method embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0148] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0149] Based on the same inventive concept, the embodiment of the present application also provides a business processing device for implementing the business processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more business processing device embodiments provided below can refer to the limitations on the business processing method above, and will not be repeated here.

[0150] In an exemplary embodiment, Figure 8 As shown, a service processing device 1 is provided, comprising: an information acquisition module 10 and a determination module 20, wherein:

[0151] The information acquisition module 10 is used to respond to the business processing instruction and obtain the average latency information of the user input and output IO on the target disk in the previous business processing round, and the business association information of the background business associated with the target disk in the current business processing round; wherein the background business includes at least one of the invalid data recovery business, the data repair business, the data migration business and the data replication business;

[0152] The determination module 20 is used to determine the target IO request amount of the background business in the current business processing round according to the average delay information and the business association information of the background business.

[0153] In an exemplary embodiment, when the background service includes an invalid data recovery service, the service association information includes the invalid data ratio in the logical disk set VSET and the remaining available capacity of the storage pool to which the VSET belongs; further, the determination module 20 is used to:

[0154] According to the proportion of invalid data and the remaining available capacity, the initial IO request amount of the background business in the current business processing round is determined; according to the average latency information, the initial IO request amount is adjusted to obtain the target IO request amount of the background business in the current business processing round.

[0155] In an exemplary embodiment, when the background service includes a data repair service, the service association information includes the current data redundancy of the VSET; further, the determination module 20 is further used to:

[0156] Determine the number of invalid data copies based on the current data redundancy and the standard data redundancy corresponding to the data repair business; determine the target IO request volume of the background business in the current business processing round based on the number of invalid data copies and the average latency information.

[0157] In an exemplary embodiment, when the background service includes a data migration service, the service association information includes a hierarchical storage priority and an available capacity of a high-performance layer in the VSET; further, the determination module 20 is further used to:

[0158] According to the available capacity and average latency information, the initial IO request amount of the background business in the current business processing round is determined; according to the hierarchical storage priority, the initial IO request amount is adjusted to obtain the target IO request amount of the background business in the current business processing round.

[0159] In an exemplary embodiment, when the background service includes a data replication service, the service association information includes a priority of the remote replication task.

[0160] In an exemplary embodiment, when the background service includes at least two of an invalid data recovery service, a data repair service, a data migration service, and a data replication service, the determination module 20 is further configured to:

[0161] According to the average latency information, the total IO request volume of each background business is determined; according to the total IO request volume and the business association information of each background business, the target IO request volume of each background business in the current business processing round is determined.

[0162] In an exemplary embodiment, the service processing device 1 further includes a service processing module, wherein the service processing module is specifically used to:

[0163] Send the target IO request quantity current business tokens to the business management end corresponding to the background business; wherein each current business token is used for the business management end to initiate a business IO request; within the current business processing round, the business IO request initiated by the business management end based on the current business token is processed.

[0164] Each module in the above-mentioned business processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0165] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store user IO data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a business processing method is implemented.

[0166] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0167] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0168] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0169] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0170] It should be noted that the data involved in this application (including but not limited to user IO data, etc.) are all data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0171] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0172] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0173] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A business processing method, characterized in that: The method comprises: In response to the business processing instruction, the average latency information of user input and output IO on the target disk in the previous business processing round and the business association information of the background business associated with the target disk in the current business processing round are obtained; wherein the background business includes at least one of an invalid data recovery business, a data repair business, a data migration business and a data replication business; According to the average delay information and the service association information of the background service, a target IO request amount of the background service in the current service processing round is determined.

2. The method according to claim 1, characterized in that In the case where the background service includes an invalid data recovery service, the service association information includes the invalid data ratio in the logical disk set VSET and the remaining available capacity of the storage pool to which the VSET belongs; The determining, according to the average delay information and the service association information of the background service, a target IO request amount of the background service in the current service processing round includes: Determine, according to the invalid data ratio and the remaining available capacity, an initial IO request amount of the background business in the current business processing round; The initial IO request amount is adjusted according to the average delay information to obtain a target IO request amount for the background service in the current service processing round.

3. The method according to claim 1, characterized in that: In the case where the background service includes a data repair service, the service association information includes the current data redundancy of the VSET; The determining, according to the average delay information and the service association information of the background service, a target IO request amount of the background service in the current service processing round includes: Determining the number of invalidated data copies according to the current data redundancy and the standard data redundancy corresponding to the data repair service; According to the number of invalidable data copies and the average latency information, a target IO request amount of the background business in the current business processing round is determined.

4. The method according to claim 1, characterized in that In the case where the background service includes a data migration service, the service association information includes a hierarchical storage priority and available capacity of a high-performance layer in the VSET; The determining, according to the average delay information and the service association information of the background service, a target IO request amount of the background service in the current service processing round includes: Determine, according to the available capacity and the average latency information, an initial IO request amount of the background service in the current service processing round; The initial IO request amount is adjusted according to the hierarchical storage priority to obtain a target IO request amount for the background business in the current business processing round.

5. The method according to claim 1, characterized in that In the case where the background service includes a data replication service, the service association information includes a priority of a remote replication task.

6. The method according to any one of claims 1 to 5, characterized in that: In a case where the background service includes at least two of an invalid data recovery service, a data repair service, a data migration service, and a data replication service, determining, according to the average latency information and the service association information of the background service, a target IO request amount of the background service in the current service processing round includes: Determine the total IO request amount of each background service according to the average latency information; According to the total IO request amount and the business association information of each background business, a target IO request amount of each background business in the current business processing round is determined.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Sending the target IO request quantity current business tokens to the business management end corresponding to the background business; wherein each current business token is used for the business management end to initiate a business IO request; In the current business processing round, the business IO request initiated by the business management end based on the current business token is processed.

8. A service processing device, characterized in that: The device comprises: An information acquisition module, configured to respond to a business processing instruction and acquire the average latency information of user input and output IO on the target disk in the previous business processing round, and business association information of the background business associated with the target disk in the current business processing round; wherein the background business includes at least one of an invalid data recovery business, a data repair business, a data migration business, and a data replication business; A determination module is used to determine the target IO request amount of the background business in the current business processing round according to the average delay information and the business association information of the background business.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images