Data processing method, device, equipment and storage medium of distributed storage system

Abstract

This application provides a data processing method, apparatus, electronic device, and storage medium for a distributed storage system. The method includes: predicting a first transfer control parameter that the distributed storage system can use in response to a transfer event in a second time period based on transfer control parameters corresponding to a storage location transfer event of distributed business data within the distributed storage system in a first time period; obtaining a second transfer control parameter that the distributed storage system can provide in the second time period if the transfer event does not occur; and controlling the transfer of the distributed business data in the second time period based on the first transfer control parameter and the second transfer control parameter.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a data processing method, apparatus, device and storage medium for a distributed storage system. Background Technology

[0002] In related technologies, in distributed business data storage solutions, data for the same business can be stored on different nodes of the storage system. In the event of a node failure, the business data stored on that node needs to be redistributed to other nodes in the storage system. This redistribution is equivalent to reading data originally stored on the failed node and writing it to other nodes. In the industry, read and write rates are used as I / O capability parameters of a storage system. Node failures result in a large volume of read and write operations, causing abnormal I / O capability parameters. This anomaly can trigger a cascading failure, not only preventing the redistributed business from responding to external requests but also affecting the accessibility of other business data stored in the storage system. Summary of the Invention

[0003] This application provides a data processing method, apparatus, device, and storage medium for a distributed storage system, to at least solve the above-mentioned technical problems existing in the prior art.

[0004] According to a first aspect of this application, a data processing method for a distributed storage system is provided, the method comprising:

[0005] Based on the transfer control parameters corresponding to the storage location transfer event of distributed business data in the distributed storage system in the first time period, the first transfer control parameters that the distributed storage system can adopt in the second time period in response to the transfer event are predicted.

[0006] Obtain the second transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event;

[0007] Based on the first transfer control parameter and the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled.

[0008] In one possible implementation, controlling the transfer of the distributed service data within the second time period based on the first transfer control parameter and the second transfer control parameter includes:

[0009] When the first transfer control parameter is less than or equal to the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled based on the difference between the second transfer control parameter and the first transfer control parameter.

[0010] In one possible implementation, controlling the transfer of the distributed service data within the second time period based on the first transfer control parameter and the second transfer control parameter includes:

[0011] When the first transfer control parameter is greater than the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled based on the difference between the first transfer control parameter and the second transfer control parameter and a reference transfer control parameter.

[0012] Wherein, the second transfer control parameter is the peak value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event; the reference transfer control parameter is the valley value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event.

[0013] In one embodiment, the distributed storage system includes multiple storage modules; the distributed business data is stored in at least one of the multiple storage modules within the distributed storage system; the method further includes:

[0014] Obtain the target attributes of each storage module;

[0015] Based on the target attributes of each storage module, the storage modules are grouped.

[0016] Based on the resource balancing parameters of each group, and / or based on the reference values ​​of each storage module in each group, determine whether the transfer event has occurred in the distributed storage system;

[0017] The reference value is at least one of the read and write capability parameters of the storage module.

[0018] In one possible implementation, a first storage module and a second storage model are determined from all storage modules of the distributed storage system where a storage location transfer event for distributed data occurs.

[0019] The control of the transfer of the distributed business data within the second time period based on the first transfer control parameter and the second transfer control parameter includes:

[0020] Based on the first transfer control parameter and the second transfer control parameter, the transfer of the distributed business data between the first storage module and the second storage module during the second time period is controlled.

[0021] In one possible implementation, obtaining the second transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event includes:

[0022] Based on historical data, the transfer control parameters provided by the distributed storage system within a historical time period that matches the second time are obtained in the case that the transfer event has not occurred in history.

[0023] The transfer control parameters provided by the distributed storage system within a historical time period that matches the second time are used as the second transfer control parameters.

[0024] According to a second aspect of this application, a data processing apparatus for a distributed storage system is provided, comprising:

[0025] The prediction unit is used to predict, based on the transfer control parameters corresponding to the storage location transfer event of distributed business data in the distributed storage system within the first time period, the first transfer control parameters that the distributed storage system can adopt in response to the transfer event within the second time period.

[0026] The acquisition unit is configured to acquire the second transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event;

[0027] The control unit is used to control the transfer of the distributed business data during the second time period based on the first transfer control parameters and the second transfer control parameters.

[0028] In one possible implementation, the control unit is used for

[0029] When the first transfer control parameter is less than or equal to the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled based on the difference between the second transfer control parameter and the first transfer control parameter.

[0030] and / or,

[0031] When the first transfer control parameter is greater than the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled based on the difference between the first transfer control parameter and the second transfer control parameter and a reference transfer control parameter.

[0032] Wherein, the second transfer control parameter is the peak value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event; the reference transfer control parameter is the valley value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event.

[0033] According to a third aspect of this application, an electronic device is provided, comprising:

[0034] At least one processor; and

[0035] A memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0037] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.

[0038] This application discloses a data processing method, apparatus, electronic device, and storage medium for a distributed storage system. The method includes: predicting, based on transfer control parameters corresponding to a storage location transfer event of distributed business data within the distributed storage system in a first time period, a first transfer control parameter that the distributed storage system can employ in a second time period in response to the transfer event; obtaining a second transfer control parameter that the distributed storage system can provide in the second time period if the transfer event does not occur; and controlling the transfer of the distributed business data in the second time period based on the first transfer control parameter and the second transfer control parameter.

[0039] By utilizing these two influencing factors (the first and second transfer control parameters), the transfer of business data in the second time period can be effectively and accurately controlled. Attached Figure Description

[0040] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which:

[0041] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0042] Figure 1 This application illustrates a schematic diagram of the implementation flow of a data processing method for a distributed storage system according to an embodiment of the present application. Figure 1 ;

[0043] Figure 2 This application illustrates a schematic diagram of the implementation flow of a data processing method for a distributed storage system according to an embodiment of the present application. Figure 2 ;

[0044] Figure 3 This application illustrates a schematic diagram of the implementation flow of a data processing method for a distributed storage system according to an embodiment of the present application. Figure 3 ;

[0045] Figure 4 A schematic diagram of disk grouping in an embodiment of this application is shown;

[0046] Figure 5 A schematic diagram of the fluctuation values ​​in an embodiment of this application is shown;

[0047] Figure 6 A schematic diagram of an application scenario from an embodiment of this application is shown;

[0048] Figure 7 A schematic diagram of application scenario two in the embodiments of this application is shown;

[0049] Figure 8 A schematic diagram of the composition structure of the data processing device of the distributed storage system according to an embodiment of this application is shown;

[0050] Figure 9 A schematic diagram of the composition structure of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0051] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0054] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0056] It should be understood that in the various embodiments of this application, the sequence number of each implementation process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0057] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0058] (1) Distributed business data, or simply business data, is business data in distributed platforms such as Elasticsearch (a search server based on Lucene) and Hadoop (a distributed system infrastructure).

[0059] (2) A distributed storage system, or simply a storage system, is a system that stores distributed business data (hereinafter referred to as business data unless otherwise specified). Within a distributed platform, the same distributed storage system can store data from different businesses. When executing a business function, the business system within the distributed platform accesses the business data stored in the storage system to perform the business function.

[0060] (3) Transfer control parameters

[0061] It's understandable that the redistribution of data originally stored on a failed node in the storage system essentially involves reading data from the failed node and writing it to other nodes to maintain normal business functionality. This process can be viewed as a data transfer event within the storage system. Data originally stored on the failed node is moved to other nodes for storage. In this storage location transfer scheme, data is read from the failed node at a certain rate (read rate) and written to other nodes at a certain rate (write rate).

[0062] In the technical solution of this application, during the transfer of business data storage location, it is necessary to control the read rate and / or write rate to avoid potential avalanche problems under conditions of heavy read and write operations. The parameters that need to be controlled during the transfer process, such as the read rate and / or write rate, can be used as the transfer rate in this application, i.e., the transfer control parameter.

[0063] In the industry, the process of reading data from a faulty node at a certain rate (read rate) and writing the read data to other nodes at a certain rate (write rate) can be considered a data recovery process. Data recovery aims to restore normal access to business data and enable normal business functions.

[0064] (4) The Gini coefficient is a parameter used to measure the resource balance of a storage system. In application scenarios where a storage system stores business data for various business processes within a business system, accessing this data requires a certain read rate to retrieve it. When business data is updated, a certain write rate is required to write the updated data back to the storage system. The read and write rates in these cases can be considered as resources of the storage system. The resource balance of a storage system can be reflected by the Gini coefficient, as detailed later.

[0065] In this application's technical solution, the processing logic of the data processing method for the distributed storage system can be deployed on a distributed platform, or on any suitable terminal or server. The terminal can be a desktop computer, all-in-one computer, tablet computer, mobile phone, or smart wearable device. The server includes, but is not limited to, ordinary servers and cloud servers.

[0066] This application provides a data processing method for a distributed storage system, such as... Figure 1 As shown, the method includes:

[0067] S101: Based on the transfer control parameters corresponding to the storage location transfer event of distributed business data in the distributed storage system within the first time period, predict the first transfer control parameters that the distributed storage system can adopt in the second time period in response to the transfer event.

[0068] In this step, both "first time" and "second time" are time information. "First time" and "second time" are time information with a certain duration, or they can be specific moments. The duration can be a reasonable value in units of seconds, minutes, hours, and days, such as minutes or hours. "First time" and "second time" can be moments such as 8:00 and 13:00.

[0069] On the timeline, the first time is earlier than the second time, and the second time is later than the first time. Based on the transfer control parameters corresponding to the storage location transfer events of distributed business data within the distributed storage system during the earlier time, the transfer control parameters for the later time are predicted.

[0070] In this step, a storage location transfer event occurs either when business data is read from one node of the storage system and stored on another node within the first time period, or when a storage location transfer event occurs earlier than the first time period.

[0071] This invention monitors whether a storage location transfer event has occurred in the storage system. Once a storage location transfer event is detected, the read and / or write rates used during the transfer process can be monitored to determine the transfer control parameters used in the first instance. In the industry, monitoring whether a storage location transfer event has occurred is often used to monitor whether data recovery operations are taking place in the storage system. In the event of a storage location transfer event, a large number of read and write operations will occur in the storage system. This application's technical solution aims to avoid system avalanche caused by improper handling of a large number of read and write operations.

[0072] In some application scenarios, the first time can be a time that has already occurred and a storage location transfer event has taken place at that time; for ease of description, this is considered a historical time. The second time is a time that has not yet occurred and can be considered a predicted time or a future time. Because the storage location transfer process takes a certain amount of time, it is unknown whether the transfer will be completed by the time the transfer begins at the moment it occurs, so prediction is necessary to avoid a cascading failure effect in the future. Specifically, the transfer is considered complete once the task of writing the business data stored on the failed node to other nodes is completed.

[0073] The transfer control parameters corresponding to the storage location transfer event of distributed business data within the distributed storage system during a certain historical period can be obtained automatically by monitoring the distributed storage system during that historical period.

[0074] The transfer control parameters corresponding to a storage location transfer event within the storage system can be understood as: the read rate and / or write rate adopted by the storage system during the transfer event. In simpler terms, the read rate and / or write rate adopted by the storage system during the transfer event are, from the storage system's perspective, the read rate and / or write rate required to cope with the transfer event. Therefore, this step involves: based on automatically obtained transfer control parameters adopted by the storage system to cope with transfer events within a certain historical period, predicting the transfer control parameters that the storage system may adopt for transfer events at a future time; that is, predicting the transfer control parameters that the storage system may adopt to cope with transfer events at a future time.

[0075] S102: Obtain the second transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event;

[0076] In this step, the storage system is considered to be in a normal state if no storage location transfer event occurs, and no faulty nodes are generated. Therefore, this step involves obtaining the transfer control parameters that the storage system can provide at a future time when it is in a normal state. S101 and S102 do not have a strict order and can be performed simultaneously.

[0077] Here, both the first transfer control parameter and the second transfer control parameter pertain to the read and / or write rates of the storage system. The first transfer control parameter is the predicted read and / or write rate that the storage system can adopt or use at a future time during the transfer event. The second transfer control parameter is the predicted transfer control parameter that the storage system can provide to the data access party at that future time.

[0078] In layman's terms, a storage system, as storage hardware, has its read and write performance predetermined at the time of manufacture, which can be reflected by read and write rates. The second transfer control parameter refers to the read and write rates that the storage system can provide to external users, such as those accessing business data, at a future timeframe. It can be understood that to achieve normal access to business data, users typically use read and write rates less than or equal to those provided by the storage system. The first transfer control parameter, however, refers to the read and write rates that the storage system itself consumes to handle transfer events.

[0079] This is understandable, because the storage system stores a large amount of business data, and users will randomly access the business data stored in the storage system. Therefore, the read and / or write speeds that the storage system can provide at a certain point in time depend on the number of users accessing the system and the complexity of the access at that point in time.

[0080] Generally, at any given time, the more accessors there are, the lower the read and / or write rates the storage system can provide, since each accessor occupies a certain amount of storage resources. Conversely, the fewer accessors there are, the more unused storage resources there are, and the higher the read and / or write rates the storage system can provide. In other words, in this application, the read and / or write rates that the storage system can provide vary depending on actual usage. At different times, due to different actual usage conditions, the transfer control parameters provided by the storage system may differ.

[0081] In practical implementation, reasonable prediction algorithms can be used, such as time series-based machine learning, deep learning, or reinforcement learning models, to predict the first transition control parameters.

[0082] S103: Based on the first transfer control parameter and the second transfer control parameter, control the transfer of the distributed business data during the second time period.

[0083] In this step, based on the predicted transfer control parameters that can be used at a certain future time and the transfer control parameters provided by the storage system to external systems during that future time, the transfer of business data that will be transferred during that future time is controlled. For example, the transfer control parameters of the business data that will be transferred during that future time are controlled.

[0084] In steps S101 to S103, based on the transfer control parameters corresponding to storage location transfer events of business data within the storage system within a certain time period, the transfer control parameters that the storage system can adopt in response to transfer events within another time period are predicted; the transfer control parameters that the storage system can provide externally within that other time period in the absence of a transfer event are obtained; and based on these two transfer control parameters, the transfer of business data is controlled. Taking into account the impact of the transfer control parameters that the storage system can adopt in response to transfer events within a certain time period (the second time period), and the transfer control parameters that the storage system can provide externally within this time period (the second time period) on the transfer process, these two influencing factors (the first and second transfer control parameters) are used to effectively and accurately control the transfer of business data within that time period (the second time period).

[0085] In layman's terms, using the aforementioned solution, when there are a large number of read and write operations in the storage system, two transfer control parameters are used to reasonably, effectively and accurately control the read and / or write rates, so as to avoid system avalanche problems caused by improper reading and writing operations and ensure that the business can be accessed normally.

[0086] It is understood that this scheme supports the prediction of the first transfer control parameter and the acquisition of the second transfer control parameter at certain intervals, and uses these two parameters to control the transfer. That is, according to the time axis, the scheme S101 to S103 can be executed at regular intervals to control the entire transfer process, thus reducing the probability of system avalanche. In two adjacent cycles, the first time in the later cycle is the same as the second time in the previous cycle. The first time in the later cycle and the second time in the previous cycle are separated by a certain time interval.

[0087] In a scheme where the first time in a subsequent cycle is the same as the second time in a previous cycle, the transfer control parameter corresponding to the storage location transfer event of business data in the storage system during the first time in the subsequent cycle can be: the actual transfer control parameter adopted by the storage system in the second time of the previous cycle to cope with the transfer event, which can be obtained through actual monitoring.

[0088] As an optional solution, the scheme for controlling the transfer of the distributed business data during the second time period based on the first transfer control parameter and the second transfer control parameter includes the following two cases:

[0089] The first case, such as Figure 2 As shown,

[0090] S103 becomes S103a: When the first transfer control parameter is less than or equal to the second transfer control parameter, the transfer of the distributed service data during the second time period is controlled based on the difference between the second transfer control parameter and the first transfer control parameter.

[0091] For example, the difference between the second transfer control parameter and the first transfer control parameter is used as the transfer control parameter value adopted in the second time period. During the second time period, the transfer rate of business data is controlled according to this transfer control parameter value.

[0092] The aforementioned solution not only provides reasonable, effective, and accurate control over read and / or write speeds, but is also simple, easy to implement in engineering, and highly practical.

[0093] In this application, since the server-side of a business platform can be considered as an access party to the business data stored in the storage system, it needs to use a certain read / write rate when accessing the storage system. The storage system, as the consumer, needs to provide the data to the access party using a certain read / write rate in response to the access from the business platform. Based on this, the technical solution of this application can control the transfer rate at both the business level and the storage level.

[0094] The scheme shown in S103a is suitable for business-level transfer rate control.

[0095] The second scenario, such as Figure 3 As shown,

[0096] S103 becomes S103b: When the first transfer control parameter is greater than the second transfer control parameter, the transfer of the distributed business data in the second time period is controlled based on the difference between the first transfer control parameter and the second transfer control parameter and the reference transfer control parameter;

[0097] Wherein, the second transfer control parameter is the peak value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event; the reference transfer control parameter is the valley value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event.

[0098] In S103b, the second time period is a time interval during which the transfer control parameters are numerically fluctuating. The transfer control parameters that the storage system can provide externally during this time period, i.e., the second transfer control parameters, can be approximated by the peak values ​​of the fluctuating values. The trough values ​​of the fluctuating values ​​can be used to represent the reference transfer control parameters. The difference between the first and second transfer control parameters is added to the reference transfer control parameter to obtain a sum. This sum is used as the transfer control parameter value adopted in the second time period. During the second time period, the transfer rate of business data is controlled according to this sum.

[0099] The aforementioned scheme takes into account the case that the transfer control parameters fluctuate in value during the second time period. By utilizing the peak and trough values ​​in the fluctuation values ​​and the first transfer control parameter, accurate and effective control of the transfer rate of business data during the second time period is achieved.

[0100] The scheme shown in S103b is suitable for storage-level transfer rate control.

[0101] It can be understood that transfer rate control controls the read and write rates, which can be simply understood as limiting the read and write rates. Thus, the solution shown in S103a is suitable for business-level rate limiting, while the solution shown in S103b is suitable for storage-level rate limiting.

[0102] In one optional embodiment, the distributed storage system includes multiple storage modules. A storage module is any device capable of storing business data, such as a disk, hard drive, flash memory, USB flash drive, etc. The distributed business data is stored in at least one of the multiple storage modules within the distributed storage system. For example, business data for the same service can be stored in one or more storage devices within the storage system. If the storage location of the distributed business data changes, such as an event where it is moved from one storage device to another, a storage location transfer event is considered to have occurred. If the storage location of the business data in the storage system does not change, a storage location transfer event is considered not to have occurred.

[0103] In distributed systems, when a node, such as a storage module, fails, the distributed platform generates a storage location transfer event to ensure normal business access. This transfer mechanism preserves the business data stored on the failed node. Besides node failures, storage location transfer events also occur when the number of nodes in the storage system changes (e.g., increases or decreases), as data needs to be redistributed among the nodes.

[0104] The following scheme is for determining whether a storage location transfer event has occurred in the storage system. The scheme includes:

[0105] Obtain the target attributes of each storage module; group the storage modules based on their target attributes; determine whether the transfer event has occurred in the distributed storage system based on the resource balancing parameters of each group and / or the reference value of each storage module in each group; wherein the reference value is at least one of the read capability and write capability parameters of the storage module.

[0106] In the aforementioned scheme, the target attribute of the storage module can be the read / write capability value inherent in the storage module itself. The resource balancing parameter can be the Gini coefficient. The read capability parameter of the storage module can be the read rate, and the write capability parameter of the storage module can be the write rate.

[0107] like Figure 4As shown, nodes 1 through n in the storage system store business data for a specific service, such as application 1. Each node (1 through n) contains two disks. All disks in all nodes are grouped according to their individual read / write capabilities. Disks with identical or similar read / write capabilities (within a tolerable range) are grouped together. This allows all storage modules to be divided into one or more groups. Figure 4 As shown, all disks are divided into two groups: disk group 1 and disk group 2. The read and write capabilities of storage modules within the same group are consistent or similar.

[0108] The read and write rates of each disk within each group are monitored. At a given moment, one of the read or write rates of each disk within each group is read. Based on the read data, the Gini coefficient of each group at that time is calculated. Based on the Gini coefficient of each group at that time, it is determined whether a storage system migration event has occurred.

[0109] The read Gini coefficient of the disk group is calculated based on the read speed of each disk. The write Gini coefficient of the disk group is calculated based on the write speed of each disk. Based on Table 1, the value of the read and write Gini coefficients is used to determine whether a migration event has occurred.

[0110] As shown in Table 1, if both the read and / or write Gini coefficients are 0, it indicates that the read and write speeds of the disks within the disk group are absolutely average, and no data recovery action occurs, i.e., no transfer event occurs. If both the read and / or write Gini coefficients are 0.2 (0.3), it indicates that the read and write speeds of the disks within the disk group are relatively average (relatively reasonable), and no data recovery action (represented by N) occurs, i.e., no transfer event occurs.

[0111] If both the read and / or write Gini coefficients are 0.4 or 0.6, it indicates a large difference in read and write speeds between the disks within the disk group, resulting in data recovery actions (represented by Y), i.e., a data transfer event. The large difference in read and write speeds is due to the transfer of business data between disk storage locations.

[0112] Table 1

[0113] Read / Write Gini Coefficient Rating Is there any data recovery process? 0 / 0 absolute average N 0.2 / 0.2 Comparison of average N 0.3 / 0.3 Relatively reasonable N 0.4 / 0.4 Big gap Y 0.6 / 0.6 The gap is huge Y

[0114] In this scheme, in addition to using the Gini coefficient to determine whether a transfer event has occurred, industry-standard anomaly detection methods such as the 3sigma model can also be used to calculate the read rate and / or write rate of each group. If the calculated read rate reaches a threshold set for the read rate, and / or the calculated write rate reaches a threshold set for the write rate, then a data recovery action is considered to have occurred, i.e., a transfer event has occurred. Otherwise, it is considered that no transfer event has occurred.

[0115] In a storage system, periodic monitoring can be performed to check for migration events. If a migration event is detected immediately, the actual migration control parameters used by the storage system in response to the event are monitored, such as the migration control parameters used from the time the event is detected until its end. This allows for the prediction of migration control parameters within the predicted timeframe.

[0116] As an optional solution, the method further includes:

[0117] From all storage modules of the distributed storage system, determine the first storage module and the second storage model where the distributed data storage location transfer event occurred;

[0118] Accordingly, controlling the transfer of the distributed business data within the second time period based on the first transfer control parameter and the second transfer control parameter includes:

[0119] Based on the first transfer control parameter and the second transfer control parameter, the transfer of the distributed business data between the first storage module and the second storage module during the second time period is controlled.

[0120] In the aforementioned alternative schemes, if a migration event is determined using the Gini coefficient and / or 3sigma model, the disk group where the migration event occurred can be identified from all disk groups in the storage system using the Gini coefficient and / or 3sigma model, as shown in the aforementioned scheme. Within the disk group where the migration event occurred, the disk with the highest write speed is identified and designated as the disk in that disk group where the migration event occurred. For ease of description, the disk where the migration event occurred is considered an abnormal disk.

[0121] The aforementioned solution not only identifies the migration event but also, based on the identification of the disk group where the migration event occurred, identifies the disk where the migration event took place. This enables finer-grained identification of the storage module where the migration event occurred, thereby allowing for accurate rate limiting at the storage level.

[0122] Typically, in data recovery scenarios, i.e., when a data transfer event occurs, data is read from multiple disks and recovered (written) to a single disk. The disk with the highest write speed is designated as the second storage module (write storage module) in the distributed data transfer event. The read storage module, which reads data and writes it to this write storage module, is designated as the first storage module (read storage module) in the distributed data transfer event.

[0123] In layman's terms, the business data being transferred was stored in the first storage module before the transfer. Later, due to node failure or changes in the number of nodes, it was read from the first storage module and written to the second storage module.

[0124] Once the disk where the migration event occurred is identified, a storage-level rate limiting scheme can be used to limit the storage system's rate, as shown in S103b above. Rate limiting can be applied to the disk group where the migration event occurred, or to the individual disks where the migration event occurred.

[0125] Based on this, the technical solution of this application can achieve both business-level rate limiting and storage-level rate limiting. In the storage-level rate limiting scheme, rate limiting can be applied to disk groups experiencing migration events, and even more fine-grained rate limiting can be applied to individual disks experiencing migration events. This achieves effective and accurate control of read and write rates, preventing the occurrence of cascading failures.

[0126] As an optional solution, obtaining the second transfer control parameter that the distributed storage system can provide during the second time period in the absence of the transfer event includes: obtaining the transfer control parameter provided by the distributed storage system during a historical time period matching the second time period in the absence of the transfer event in the past, based on historical data; and using the transfer control parameter provided by the distributed storage system during the historical time period matching the second time period as the second transfer control parameter.

[0127] In the aforementioned optional scheme, the historical time matched with the second time can be a historical time corresponding to the second time. For example, if the second time is 17:00-20:00 on June 24th, then the historical time matched with the second time is any one or more days prior to June 24th, from 17:00 to 20:00. Historical data refers to the read and / or write rates that the storage system could provide to the accessor in the absence of any migration events.

[0128] In specific implementation, under the condition that no transfer event has occurred in historical time, the transfer control parameters that the storage system can provide externally within the historical time period matching the second time can be obtained by monitoring the actual transfer control parameters that the storage system can provide externally within the historical time period matching the second time. A preset statistical algorithm is used to statistically analyze the actual transfer control parameters that the storage system can provide externally within the historical time period matching the second time, and the statistical results are used as the transfer control parameters that the storage system can provide externally within the second time period under the condition that no transfer event has occurred.

[0129] For example, assuming that no migration events occurred in the storage system for several days prior to June 24th, the actual migration control parameters that the storage system could provide externally between 17:00 and 20:00 on each day of those days are monitored. Based on a preset statistical method, the actual migration control parameters that the storage system could provide externally between 17:00 and 20:00 on each day of those days prior to June 24th are statistically analyzed to obtain a statistical result.

[0130] It is understandable that the statistical result can be a single value or a fluctuating value. In one optional scheme, if it is a single value, it can be applied to S103a and subtracted from the first transfer control parameter to execute the scheme in S103a and achieve rate limiting at the service level. If it is a fluctuating value, the peak and trough values ​​of the fluctuation value also need to be known to execute the scheme in S103b and achieve rate limiting at the storage level.

[0131] Among them, the preset statistical algorithms include statistical methods based on the 95 peak value, statistical methods based on standard deviation and variance, etc.

[0132] Therefore, the second transfer control parameter is obtained based on the transfer control parameters actually provided by the storage system when no transfer event occurs. This ensures the accuracy of the second transfer control parameter, thereby enabling the storage system to effectively and accurately control the read and write rates during transfer events.

[0133] The following is combined Figures 5-7 The following diagram further illustrates the technical solution of this application.

[0134] This application's technical solution can be applied to two business scenarios. The first scenario is where the existence of data recovery actions cannot be directly determined from the business side. The second scenario is where the existence of data recovery actions can be determined from the business side. In the first scenario, if data recovery actions are occurring, rate limiting at the storage side is necessary to prevent cascading failures. This scenario involves rate limiting at the storage level, as shown in the attached diagram. Figure 6 As shown in the diagram. In the second business scenario, since it's possible to determine whether data recovery is underway from the business side, rate limiting can be directly applied to the business side to prevent cascading failures. This scenario involves rate limiting at the business level, as shown in the attached diagram. Figure 7 As shown.

[0135] against Figure 6The scenario illustrated involves rate limiting at the storage level. In this application, the Gini coefficient for each group can be calculated in real-time or periodically based on the grouping of nodes in the storage system. The calculation result of the Gini coefficient determines whether data recovery action should be initiated at a specific time or period. Alternatively, a 3sigma model can be used to determine whether data recovery action should be initiated at a specific time or period. For the specific calculation process, please refer to the aforementioned explanations; repeated details will not be elaborated upon.

[0136] If a calculation is performed at 15:00 on June 24, 2022, to determine whether a data recovery action occurred, it is found that a data recovery action did occur at that time. To avoid a potential cascading failure due to subsequent data recovery actions, the time after 15:00 on June 24, 2022, can be considered as the predicted time. The technical solution of this application can be executed periodically or in real time to achieve reasonable control of the read and write rates within the predicted time, thereby avoiding the cascading failure problem.

[0137] For example, consider predicting read and write rates for a future period (which can be flexibly set as needed) every few minutes (pre-configurable). At 15:20 on June 24, 2022, predict the read and write rates for the period from 15:30 to 16:00 on June 24, 2022 (the second time period or prediction time). If a data recovery operation is known to occur, such as at 15:00, the monitoring system begins monitoring the actual read and write rates used by the storage system during the data recovery operation. This allows the system to determine the actual read and write rates of the storage system during the period from 15:00 to 15:20 (the first time period). The actual read and write rates of the storage system from 15:00 to 15:20 are then input into the prediction model. Based on the actual read and write rates used by the storage system within the already arrived time, the system predicts the read and write rates that the storage system might use within the prediction time period, such as from 15:30 to 16:00.

[0138] The prediction model is a time-series-based machine learning, deep learning, or reinforcement learning model, which can predict the read and write rates in the future based on the read and write rates in a certain historical period.

[0139] Historically, the storage system's actual read and / or write rates during the 15:30-16:00 period (when no data recovery occurred) are used as historical data. For example, if no data recovery occurred between June 1st and June 23rd, 2022, the actual read and write rates during these days can be used as historical data. This data can be obtained by reading the monitoring system's read and write rate data. Using statistical algorithms such as the standard deviation algorithm, the peak and trough values ​​of the read and write rates that the storage system can provide externally during the 15:30-16:00 period are calculated from the historical data.

[0140] by Figure 5 Taking write speed as an example, based on historical data of the storage system, it is predicted that the peak write speed the storage system can provide between 15:30 and 16:00 on June 24, 2022, will be 4.2GB (gigabytes), and the valley will be 3.8GB. The predicted peak write speed of 4.2GB is subtracted from the predicted write speed of 4.4GB that the storage system might use between 15:30 and 16:00, and the absolute value is taken. Alternatively, the predicted peak write speed of 4.2GB is subtracted from the predicted write speed of 4.4GB that the storage system might use between 15:30 and 16:00, and then added to the predicted valley of 3.8GB, resulting in a storage-level rate limit of 4GB required for the storage system during the 15:30-16:00 period. To avoid a cascading failure during the 15:30-16:00 time period, the write rate of the storage system during this period needs to be limited to 4GB.

[0141] In addition to limiting the write rate during the 15:30-16:00 time period, the read rate also needs to be limited. The process for limiting the read rate is similar to that for limiting the write rate, and will not be elaborated upon.

[0142] It should be noted that, because Figure 6 The scenario shown is the first business scenario, so a storage-level rate limiting scheme can be used. The above rate limiting is applied to the storage system as a whole. Furthermore, when using a storage-level rate limiting scheme, in addition to limiting the rate of the entire storage system, rate limiting can also be applied to the disks undergoing data recovery, i.e., precisely to the disk level. An abnormal disk is identified from the storage system using a 3-sigma model, and the write rate of the abnormal disk is limited to 4GB during the time period from 15:30 to 16:00. The process of identifying abnormal disks is detailed in the aforementioned instructions and will not be repeated here.

[0143] The above solution is based on the first business scenario. In the second business scenario, since the existence of data recovery action can be directly determined from the business side based on the business status, there is no need to calculate whether data recovery action exists. If the business is running normally, it is assumed that no data recovery action exists. If the business cannot run normally, it is assumed that data recovery action exists. When the business side determines that data recovery action exists, the monitoring system begins to monitor the actual read and write rates used by the storage system under the data recovery action. From this, the actual read and write rates of the storage system during the time period of 15:00-15:20 (the first time) can be obtained. The actual read and write rates of the storage system from 15:00-15:20 are input into the prediction model to predict the read and write rates that the storage system may use within the prediction time, such as 15:30-16:00, for example, both 4.2GB.

[0144] Based on historical monitoring data, statistical algorithms such as the 95% peak rate algorithm are used to calculate the read and write rates that the storage system can provide externally during the 15:30-16:00 time period. For example, if the available read and write rates are both 5GB, the difference between the available 5GB and the predicted read and write rates of 4.2GB during the 15:30-16:00 time period is calculated to be 0.8GB. When the 15:30-16:00 time period arrives, rate limiting is implemented on the business side to control the data recovery traffic used by the business side to 0.8GB, thus limiting the data recovery traffic on the business side to 0.8GB and preventing a cascading failure.

[0145] The technical solutions in this application are suitable for rate limiting at the business level or at the storage level. Reasonable rate limiting at either the business or storage level can prevent cascading failures in data recovery scenarios. This ensures that if data recovery is required, other services will not be affected.

[0146] exist Figure 6 and Figure 7 In the scenario shown, the distributed platform includes a cluster system, a business system, a monitoring system, etc. The processing logic of this embodiment can be deployed in the monitoring system or other suitable systems within the distributed platform. Within the distributed platform, the storage system and business system can be cluster systems, providing extensive storage and cluster business functions. Distributed business clusters include, but are not limited to, big data environments, distributed applications, and traditional database clusters.

[0147] In a storage system, the information acquisition module in the monitoring system can be used to sample or collect stored data. The prediction and acquisition processes in this embodiment can be implemented based on the actual read / write rate and historical data of the storage system collected by the information acquisition module within the first time period.

[0148] For example, based on historical data collected by the information acquisition module, the read and write rates that the storage system can provide within the predicted time frame under the condition that no data recovery action has taken can be statistically analyzed or calculated. Based on the actual read and write rates of the storage system in the first time frame under the condition that data recovery action has taken, collected by the information acquisition module, the read and write rates used by the storage system within the predicted time frame can be predicted.

[0149] The alarm module in the monitoring system is used to generate an alarm when it detects a data recovery action, i.e., a storage location transfer event, so as to prompt relevant personnel to deal with it in a timely manner and avoid the storage system from collapsing.

[0150] Using the aforementioned scheme, when there are a large number of read and write operations in the storage system, the read and write rates that the storage system can adopt within a certain predicted time or time period, as well as the read and write rates that the storage system can provide to external systems within a time period that matches the predicted time or time period when there are no large number of read and write operations, can be reasonably, effectively and accurately controlled. This avoids system avalanche problems caused by improper handling of a large number of read and write operations and ensures that services can be accessed normally.

[0151] Figure 6 The scenario shown is storage-level rate limiting, which limits the speed directly on the disk side to prevent a cascading failure of the disk under heavy read and write operations. This rate limiting solution is relatively straightforward. Figure 7 The scenario shown is a business-level rate limiting. When a business, such as an application, provides a rate limiting interface, it is equivalent to rate limiting on the application side. This solution can achieve timely rate limiting.

[0152] This application provides a data processing device for a distributed storage system, such as... Figure 8 As shown, the device includes:

[0153] Prediction unit 801 is used to predict, based on the transfer control parameters corresponding to the storage location transfer event of distributed business data in the distributed storage system within the first time period, a first transfer control parameter that the distributed storage system can adopt in response to the transfer event within the second time period.

[0154] The acquisition unit 802 is used to acquire the second transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event;

[0155] The control unit 803 is used to control the transfer of the distributed business data during the second time period based on the first transfer control parameters and the second transfer control parameters.

[0156] In one alternative embodiment, the control unit 803 is configured to:

[0157] When the first transfer control parameter is less than or equal to the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled based on the difference between the second transfer control parameter and the first transfer control parameter.

[0158] And / or, if the first transfer control parameter is greater than the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled based on the difference between the first transfer control parameter and the second transfer control parameter and a reference transfer control parameter;

[0159] Wherein, the second transfer control parameter is the peak value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event; the reference transfer control parameter is the valley value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event.

[0160] In one alternative, acquisition unit 802 is used to acquire the target attributes of each storage module;

[0161] Based on the target attributes of each storage module, the storage modules are grouped.

[0162] Based on the resource balancing parameters of each group, and / or based on the reference values ​​of each storage module in each group, determine whether the transfer event has occurred in the distributed storage system;

[0163] The reference value is at least one of the read and write capability parameters of the storage module.

[0164] In one alternative, the acquisition unit 802 is used to determine, from all storage modules of the distributed storage system, the first storage module and the second storage model where the distributed data storage location transfer event occurs;

[0165] Accordingly, the control unit 803 is used to control the transfer of the distributed business data between the first storage module and the second storage module during the second time period based on the first transfer control parameters and the second transfer control parameters.

[0166] In one alternative, the acquisition unit 802 is used to obtain, based on historical data, the transfer control parameters provided by the distributed storage system within a historical time period matching the second time when the transfer event has not occurred in the past; and to use the transfer control parameters provided by the distributed storage system within the historical time period matching the second time as the second transfer control parameters.

[0167] It should be noted that the data processing device of the distributed storage system in this application embodiment solves the problem in a similar way to the data processing method of the aforementioned distributed storage system. Therefore, the implementation process and implementation principle of the device can be referred to the implementation process and implementation principle description of the aforementioned method, and the repeated parts will not be repeated.

[0168] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0169] Figure 9 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0170] like Figure 9 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0171] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0172] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as data processing methods for a distributed storage system. For example, in some embodiments, the data processing methods for a distributed storage system can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data processing methods for a distributed storage system described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform data processing methods for a distributed storage system by any other suitable means (e.g., by means of firmware).

[0173] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0175] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A data processing method for a distributed storage system, the method comprising: Based on the transfer control parameters corresponding to the storage location transfer event of distributed business data in the distributed storage system in the first time period, the first transfer control parameters that the distributed storage system can adopt in the second time period in response to the transfer event are predicted. Obtain the second transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event; When the first transfer control parameter is greater than the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled based on the difference between the first transfer control parameter and the second transfer control parameter and a reference transfer control parameter. Wherein, the second transfer control parameter is the peak value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event; the reference transfer control parameter is the valley value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event.

2. The method according to claim 1, wherein, The control of the transfer of the distributed business data within the second time period based on the first transfer control parameter and the second transfer control parameter includes: When the first transfer control parameter is less than or equal to the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled based on the difference between the second transfer control parameter and the first transfer control parameter.

3. The method according to claim 1, wherein, The distributed storage system includes multiple storage modules; the distributed business data is stored in at least one of the multiple storage modules within the distributed storage system; the method further includes: Obtain the target attributes of each storage module; Based on the target attributes of each storage module, the storage modules are grouped. Based on the resource balancing parameters of each group, and / or based on the reference values ​​of each storage module in each group, determine whether the transfer event has occurred in the distributed storage system; The reference value is at least one of the read and write capability parameters of the storage module.

4. The method according to claim 1 or 3, further comprising: From all storage modules of the distributed storage system, determine the first storage module and the second storage model where the distributed data storage location transfer event occurred; The control of the transfer of the distributed business data within the second time period based on the first transfer control parameter and the second transfer control parameter includes: Based on the first transfer control parameter and the second transfer control parameter, the transfer of the distributed business data between the first storage module and the second storage module during the second time period is controlled.

5. The method according to any one of claims 1 to 3, wherein, The step of obtaining the second transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event includes: Based on historical data, the transfer control parameters provided by the distributed storage system within a historical time period that matches the second time are obtained in the case that the transfer event has not occurred in history. The transfer control parameters provided by the distributed storage system within a historical time period that matches the second time are used as the second transfer control parameters.

6. A data processing apparatus for a distributed storage system, comprising: The prediction unit is used to predict, based on the transfer control parameters corresponding to the storage location transfer event of distributed business data in the distributed storage system within the first time period, the first transfer control parameters that the distributed storage system can adopt in response to the transfer event within the second time period. The acquisition unit is configured to acquire the second transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event; The control unit is configured to control the transfer of the distributed business data during the second time period based on the difference between the first transfer control parameter and the second transfer control parameter and a reference transfer control parameter when the first transfer control parameter is greater than the second transfer control parameter. Wherein, the second transfer control parameter is the peak value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event; the reference transfer control parameter is the valley value of the transfer control parameters that the distributed storage system can provide during the second time period in the absence of the transfer event.

7. The apparatus according to claim 6, wherein the control unit is configured to... When the first transfer control parameter is less than or equal to the second transfer control parameter, the transfer of the distributed business data during the second time period is controlled based on the difference between the second transfer control parameter and the first transfer control parameter.

8. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

Images