Data management method and storage system

Abstract

The data management method of the embodiments of the present specification is applied to a storage system, the system comprising a business server and a storage server, in the case that the storage server receives a data download request submitted by a user client for target data, the storage server sends data shard information corresponding to the data download request to the business server; the business server queries initial speed limiting information corresponding to the user client, and calculates target speed limiting information corresponding to the data download request based on the initial speed limiting information and the data shard information; the storage server adjusts download parameters corresponding to the data download request according to the target speed limiting information, and transmits the target data to the user client according to the adjusted download parameters. Different download requests of different users can be set with corresponding speed limiting information, achieving the purpose of flow limitation for a single user, meeting the scene of multiple data stream downloads of the project party to the user, and also achieving the demand of flow control.

Description

Technical Field

[0001] This specification relates to the field of Internet technology, and particularly to data management methods. One or more embodiments of this specification also relate to a storage system, a data management device, a computing device, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of internet technology, more and more internet users are uploading multimedia resources to cloud storage servers, which can then be downloaded via the internet, achieving resource sharing or cloud storage. However, the large number of online users poses a significant challenge to network bandwidth. Existing technologies typically limit download speeds for different users to reduce network bandwidth costs, but setting the same speed limit for all users may result in some users being unable to access the resources, causing inconvenience. Therefore, how to implement download speed limits for different users is a problem that urgently needs to be solved. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a data management method. One or more embodiments of this specification also relate to a storage system, a data management device, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data management method is provided, the method being applied to a storage system, wherein the system includes a business server and a storage server, wherein...

[0005] When the storage server receives a data download request for target data submitted by a user client, it sends the data fragmentation information corresponding to the data download request to the business server through the storage server.

[0006] The service server queries the initial speed limit information corresponding to the user client, and calculates the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragmentation information.

[0007] The storage server adjusts the download parameters corresponding to the data download request based on the target rate limit information, and transmits the target data to the user client according to the adjusted download parameters.

[0008] According to a second aspect of the embodiments of this specification, a storage system is provided, the system comprising a service server and a storage server, wherein...

[0009] The storage server is used to send data fragment information corresponding to the data download request to the business server in response to a data download request submitted by a user client for target data.

[0010] The service server is used to query the initial speed limit information corresponding to the user client, calculate the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragmentation information, and send the target speed limit information to the storage server.

[0011] The storage server is used to adjust the download parameters corresponding to the data download request according to the target speed limit information, and transmit the target data to the user client according to the adjusted download parameters.

[0012] According to a third aspect of the embodiments of this specification, a data management method is provided, applied to a storage server, comprising:

[0013] In response to a data download request submitted by a user client for target data, the data fragmentation information corresponding to the data download request is sent to the business server.

[0014] Receive the target rate limit information returned by the service server in response to the data fragmentation information;

[0015] Adjust the download parameters corresponding to the data download request based on the target speed limit information;

[0016] The target data is transmitted to the user client according to the adjusted download parameters.

[0017] According to a fourth aspect of the embodiments of this specification, a data management method is provided, applied to a business server, comprising:

[0018] Receive data fragment information from the storage server regarding the associated data download request, and query the initial rate limit information corresponding to the user client based on the data fragment information;

[0019] Based on the initial speed limit information and the data fragmentation information, the target speed limit information corresponding to the data download request is calculated;

[0020] Send the target rate limit information to the storage server.

[0021] According to a fifth aspect of the embodiments of this specification, a data management apparatus is provided, the apparatus being applied to a storage system, wherein the system includes a business server and a storage server, wherein...

[0022] The sending module is configured to send data fragmentation information corresponding to the data download request to the business server through the storage server when the storage server receives a data download request submitted by a user client for target data.

[0023] The calculation module is configured to query the initial speed limit information corresponding to the user client through the business server, and calculate the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragmentation information;

[0024] The adjustment module is configured to have the storage server adjust the download parameters corresponding to the data download request according to the target speed limit information, and transmit the target data to the user client according to the adjusted download parameters.

[0025] According to a sixth aspect of the embodiments of this specification, a data management apparatus is provided, applied to a storage server, comprising:

[0026] The sending module is configured to send data fragmentation information corresponding to the data download request to the business server in response to a data download request submitted by a user client for target data.

[0027] The receiving module is configured to receive target rate limiting information returned by the service server in response to the data fragmentation information;

[0028] The adjustment module is configured to adjust the download parameters corresponding to the data download request based on the target speed limit information;

[0029] The transmission module is configured to transmit the target data to the user client according to the adjusted download parameters.

[0030] According to a seventh aspect of the embodiments of this specification, a data management apparatus is provided, applied to a business server, comprising:

[0031] The query module is configured to receive data fragment information of the associated data download request sent by the storage server, and query the initial rate limit information corresponding to the user client based on the data fragment information;

[0032] The calculation module is configured to calculate the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragmentation information;

[0033] The sending module is configured to send the target rate limit information to the storage server.

[0034] According to an eighth aspect of the embodiments of this specification, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor executes the computer instructions to implement the steps of the data management method.

[0035] According to a ninth aspect of an embodiment of this specification, a computer-readable storage medium is provided that stores computer instructions which, when executed by a processor, implement the steps of the data management method.

[0036] According to a tenth aspect of an embodiment of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the data management method described above.

[0037] The data download management method provided in this specification is applied to a storage system, wherein the system includes a business server and a storage server. When the storage server receives a data download request from a user client for target data, it sends data fragment information corresponding to the data download request to the business server. The business server queries the initial rate limit information corresponding to the user client, and calculates the target rate limit information corresponding to the data download request based on the initial rate limit information and the data fragment information. The storage server adjusts the download parameters corresponding to the data download request according to the target rate limit information, and transmits the target data to the user client according to the adjusted download parameters.

[0038] One embodiment of this specification implements a system whereby a storage server sends data fragment information corresponding to a data download request submitted by a user client to a business server. The business server calculates the target rate limit information for this data download request based on the initial rate limit information and data fragment information corresponding to the user client, and sends it to the storage server. This allows the storage server to adjust the download parameters according to the target rate limit information and transmit the target data to the user client according to the download parameters. Thus, corresponding rate limit information can be set for different download requests from different users, achieving the purpose of traffic limiting for a single user. This meets the needs of project owners for traffic control in scenarios where users have multiple data streams downloading. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating a data management method provided in one embodiment of this specification;

[0040] Figure 2 This is a schematic diagram of the system architecture of a storage system provided in one embodiment of this specification;

[0041] Figure 3 This is a system processing flowchart of a storage system provided in one embodiment of this specification;

[0042] Figure 4 This is a flowchart illustrating a data management method applied to a storage server, as provided in one embodiment of this specification.

[0043] Figure 5 This is a flowchart illustrating a data management method applied to a business server, as provided in one embodiment of this specification.

[0044] Figure 6 This is a flowchart illustrating the processing procedure of a data management method provided in one embodiment of this specification;

[0045] Figure 7 This is a schematic diagram of the structure of a data management device provided in one embodiment of this specification;

[0046] Figure 8 This is a schematic diagram of a data management device applied to a storage server according to one embodiment of this specification;

[0047] Figure 9 This is a schematic diagram of the structure of a data management device applied to a business server according to one embodiment of this specification;

[0048] Figure 10 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0049] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0050] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to any or all possible combinations including one or more of the associated listed items.

[0051] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0052] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0053] PDS: Photo & Drive Service is an open data management platform for enterprises, teams, and individuals, providing one-stop data storage and analysis capabilities. For example, users can upload shared materials to the cloud through the photo drive service, which can then be downloaded by other users in the team or other clients of that user, achieving data sharing.

[0054] Cloud-based data management platforms may implement traffic controls for user downloads to adjust download speeds on user clients. In practical applications, this facilitates the management and handling of users involved in black-market activities and saves costs. A common rate-limiting strategy is to control download speeds based on user level. However, this method cannot handle scenarios where a single user is downloading multiple data streams because it is a single-stream rate-limiting method. Single-stream rate-limiting applies the rate to the download address of the text being downloaded. When a user makes multiple requests to that address, although each request has the same rate limit, the user can increase the download speed through concurrent requests. Therefore, it is necessary to consider how to set corresponding rate limits for each user's each download request.

[0055] Based on this, this specification provides a data management method applied to a storage system, which includes a business server and a storage server. By calculating the rate limit value corresponding to the download request based on the data fragmentation information in the data download request submitted by the user client, the method achieves the effect of calculating the rate limit value for each download request, thus achieving user request-level traffic control. This specification also relates to a storage system, a data management device, a computing device, a computer-readable storage medium, and a computer program, which will be described in detail in the following embodiments.

[0056] Figure 1This diagram illustrates the architecture of a storage system according to an embodiment of this specification. The storage system includes a service server and a storage server. The service server provides related business services to users, such as cloud storage services. The storage server stores user-uploaded data, such as shared files uploaded by users in the cloud storage service. When a user wants to download a previously uploaded data file, the user client first obtains the data download address from the service client, i.e., determines which storage server the data file is stored on. Then, the user client sends a data download request for the data file to the storage server, so that the storage server can transfer the data file to the user client. When the storage server receives a data download request, it generates a corresponding download callback request. This callback request can carry the data fragment information for this download request. After sending the download callback request with the data fragment information to the business server, if the business server determines that the user is subject to rate limiting, it needs to calculate the target rate limit information based on the data fragment information. The calculated target rate limit information for the data download request is then returned to the storage server, allowing the storage server to adjust the download parameters for the data download request. The data file that the user wants to download is then transmitted to the user client according to the adjusted download parameters. This enables the storage server to set corresponding rate limits for each user request and calculate the target rate limit information according to a pre-set rate limiting algorithm. This ensures that normal service is provided to users while also meeting the business's requirements for rate limiting.

[0057] Figure 2 A flowchart illustrating a data management method according to an embodiment of this specification is shown. The method is applied to a storage system, wherein the system includes a business server and a storage server, and wherein the method includes steps 202 to 206:

[0058] Step 202: When the storage server receives a data download request submitted by the user client for the target data, the storage server sends the data fragment information corresponding to the data download request to the business server.

[0059] When a user client wants to download a data file, it sends a download request to the storage server that stores that data file. The download request may include information such as the identifier of the data file to be downloaded and its size. The storage server can be understood as the physical storage server for the data, i.e., the OSS side. After receiving the download request, the storage server sends the corresponding data fragment information to the business server. The business server can be understood as the server side that provides business services, i.e., the PDS side. The data fragment information can be understood as the size of the data file that the user client wants to download in this download request. After sending the data fragment information to the business server, if the business server determines that the current user client needs to be rate-limited, it will calculate the rate limit value for this download request based on the data fragment information to implement rate control for this user's download.

[0060] In practical applications, before sending a data download request to the storage server, the user client sends a download link retrieval request to the business server. This request inquires which storage server the data file to be downloaded is stored on, allowing the business server to subsequently determine the storage server based on the download link and send the data download request to that server, thus enabling the user to actually obtain the data file they want to download. Therefore, the business server can also respond to the user client's download link retrieval request for target data, determine the storage server corresponding to the target data, and send the storage server identifier of that storage server to the user client. The user client can then use the storage server identifier to determine the target storage server and send the data download request to that target storage server.

[0061] In a specific embodiment of this specification, a user wants to download data resource file A. The user client submits a download link request to the business server. The business server determines the storage server corresponding to data resource file A and returns the storage server identifier to the user client. After receiving the storage server identifier, the user client can determine the storage server storing data resource file A and send a data download request to that storage server to obtain data resource file A. Upon receiving the data download request for data resource file A from the user client, the storage server sends the data fragment information corresponding to the data download request to the business server. This information is used by the business server to subsequently calculate the rate limit information corresponding to this download request.

[0062] Furthermore, to prevent the storage server from failing to send data fragment information to the application server according to the data download request, the storage server can send a download rate limiting request carrying the data fragment information to the application server. Specifically, sending the data fragment information corresponding to the data download request from the storage server to the application server includes: sending a download rate limiting request carrying the data fragment information to the application server from the storage server. The application server calculates the target rate limiting information corresponding to the data download request based on the initial rate limiting information and the data fragment information, including: parsing the download rate limiting request to obtain the data fragment information, and performing the calculation of the target rate limiting information corresponding to the data download request based on the initial rate limiting information and the data fragment information, and sending the target rate limiting information to the storage server.

[0063] In this context, the download rate limiting request can be understood as a download callback request. This is a request that the storage server sends to the application server after receiving a data download request from a user client. The download rate limiting request provides a callback address and some basic custom information. Only after receiving the data download request can the storage server initiate the download rate limiting request (i.e., the download callback request) through the callback address in the download rate limiting request.

[0064] In practical applications, download rate limiting requests may include not only data fragmentation information but also business server identification information and data identification information. The download rate limiting request contains the business identification information of which business server the storage server needs to send the download rate limiting request to. This business identification information can be a server identifier or URL address, etc., serving as the business server's identifier. Data identification information may include the user corresponding to the data download request and related identification information of the target data, such as user ID, data ID, and file ID. By sending the download rate limiting request to the business server, the business server can determine the user requiring rate limiting and the data to be downloaded based on other parameters in the download rate limiting request, facilitating subsequent judgment on whether the current data download request needs to be rate-limited.

[0065] In a specific embodiment of this specification, after the storage server sends a download rate limiting request carrying data fragment information to the business server, the business server parses the received download rate limiting request and obtains the data fragment information based on the parsing result. After determining the data fragment information corresponding to this data download request, the target rate limiting information corresponding to this data download request can be calculated based on the initial rate limiting information and data fragment information corresponding to the queried user client, and the calculated target rate limiting information is returned to the storage server.

[0066] Based on this, by sending a download rate limiting request carrying data fragment information to the business server, the business server can calculate the target rate limiting information corresponding to this data download request based on the data fragment information in the download rate limiting request, thereby achieving the purpose of setting corresponding rate limiting values ​​for different data download requests.

[0067] Furthermore, in order to enable the business server to accurately calculate the target speed limit information based on the download speed limit request, other parameter information about the user client can be added to the download speed limit request. Specifically, the storage server sends a download speed limit request carrying data fragment information to the business server, including: obtaining the user client's business configuration information through the storage server; determining the business identification information corresponding to the user client based on the business configuration information; parsing the data download request to obtain data identification information and data fragment information; generating a download speed limit request based on the business identification information, the data identification information, and the data fragment information; and sending the download speed limit request to the business server corresponding to the business identification information.

[0068] The business configuration information can be understood as the user configuration information configured by the business operator for the user client. This information may include the user's tenant domain name, username under the tenant domain, etc. The business identification information, such as the domainid (domain name) and userid (username), can be used to determine the business identifier information corresponding to the user client. The storage server parses the data download request to obtain data identification information and data fragmentation information. The data identification information, including the driveid (entity name) and fileid (file name), identifies the data to be downloaded. The data fragmentation information represents the size range of the data to be downloaded. Based on the business identification information, data identification information, and data fragmentation information, a download rate limiting request can be generated and subsequently sent to the business server corresponding to the business identification information.

[0069] In practical applications, the request sent by the storage server to the business server includes callback information. The callback information in the request body includes the aforementioned business identifier information, data identifier information, and data sharding information. Specifically, the field value is the `callbackUrl` field corresponding to the business identifier information. In implementation, multiple URLs can be configured, allowing the storage server to iterate through these URLs until it sends the download rate limiting request to the business server. The `callbackBody` field can include the data identifier information and the data sharding information. This field supports system variables and custom variables. System variables specified in the request will be replaced with their actual values ​​before the storage server initiates the download rate limiting request. Custom variables include fields corresponding to the data identifier information; for example, the field corresponding to the user ID is `x:userid`, and the field corresponding to the file ID is `x:fileid`. When using custom variables, the corresponding request header, such as `x-oss-callback-var`, also needs to be specified. In practice, the callback information can also include other parameters, such as the callback occurrence information callbackStage, with optional values ​​before-execute and after-commit, indicating whether the callback occurs before or after the data download request is executed. Currently, the default is to set it to after-commit if the data download request is executed.

[0070] In a specific embodiment of this specification, the storage server obtains the user client's service configuration information, i.e., domain configuration information, determines the service identification information corresponding to the user client based on the service configuration information, fills in the callbackUrl field based on the service identification information, parses the data download request to obtain data identification information and data fragment information, fills in the callbackBody field based on the data identification information and data fragment information, composes a download rate limiting request based on the callbackUrl and callbackBody, determines the corresponding service server based on the service identification information, and sends the download rate limiting request to the service server.

[0071] Therefore, by adding business identification information to the download rate limiting request, the situation where callbacks cannot be made when the storage server and the business server fail to connect can be avoided. Adding data identification information and data fragment information to the download rate limiting request allows the business server to query the initial rate limiting information of the corresponding user client based on the data identification information, and to calculate the corresponding target rate limiting value by combining the data fragment information.

[0072] Furthermore, since the callback interface provided by the business server allows any third party to initiate requests, in order to prevent malicious users from sending abnormal rate-limiting requests through this callback interface, affecting other users' normal use or modifying their own rate-limiting values, and causing losses to the business, a signature security mechanism can be added to the download rate-limiting request to verify the source of the request. Specifically, the method also includes: generating an initial signature string through the storage server based on the data identification information and the data fragmentation information, encrypting and encoding the initial signature string to obtain signature information, and writing the signature information and the initial signature string into the download rate-limiting request.

[0073] The initial signature string can be understood as a string composed of data identification information and data fragmentation information used by the storage server. The signature information can be obtained by encrypting and encoding the initial signature string. In practice, the storage server can combine the request URL and request body of the download rate-limiting request into an initial signature string, encrypt it with a private key, and then encode it using base64 to obtain the signature information.

[0074] In practical applications, business servers may receive download rate limiting requests from third parties. To verify these requests, signature information can be added to them, and a signature verification security mechanism can be used to filter out download rate limiting requests sent by non-storage servers.

[0075] In a specific embodiment of this specification, the storage server generates an initial signature string based on data identification information and data fragmentation information. The initial signature string includes information such as the request URL and request body of the download rate limiting request. The initial signature string is encrypted with a private key and encoded using base64 to generate signature information. Subsequently, the signature information and the initial signature string are written into the download rate limiting request so that the subsequent business server can perform security verification based on the signature information and the initial signature string.

[0076] Based on this, by writing signature information and the corresponding initial signature string into the download rate limiting request, subsequent business servers can verify the download rate limiting request based on the signature information and the initial signature string, filtering out download rate limiting requests sent by non-storage servers and ensuring the normal operation of the storage system.

[0077] Step 204: Query the initial speed limit information corresponding to the user client through the business server, and calculate the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragmentation information.

[0078] The initial rate limit information can be understood as the rate limit value and related information initially set by the user's client in this business application. This initial rate limit information may include the initial rate limit value, minimum download speed per chunk, and maximum number of chunks. The initial rate limit information is generally the rate limit value configured by the business operator for the user. When a user is rate-limited by the business operator, all download requests submitted by the user will be subject to the configured rate limit value, meaning that all data download requests submitted by the user will have the same download rate limit value. This leads to the user downloading data files in chunks, increasing the download speed through concurrency. However, by having the business provider's business server calculate the target rate limit information for each data download request based on the data chunk information corresponding to each data download request, a corresponding target rate limit value can be set for each data download request. This prevents users from increasing download speed through concurrent downloads, thus truly achieving the effect of rate-limited downloads for users. The calculation of the target rate limit information for this data download request requires the calculation of the target rate limit information based on the initial rate limit information set by the business operator for the user's client and the data chunk information corresponding to this data download request.

[0079] In practical applications, the business server provides an API interface for user quotas. Business operators can use this API to implement rate limiting for users. The API interface includes a new field, `rate_limit_config`, which sets the download rate limit. This field can include the initial rate limit (`QuotaSpeed`), the minimum download speed per segment (`MinSpeed`), and the maximum number of segments (`MaxPerts`). The initial rate limit restricts the user's maximum download speed, the minimum download speed per segment restricts the user's minimum download speed, and the maximum number of segments restricts the user's maximum concurrent downloads. After the business operator provides the limits for each aspect, the business server writes the corresponding data values ​​into the field, thus implementing the initial rate limiting for users.

[0080] When a user downloads a data file concurrently, multiple download requests are sent for the same file. However, each download request has different fragment information, meaning the size of the downloaded data file varies. For example, if the data file is 150MB, the user can send download request A with 50MB fragment information to download 50MB of the data file, and download request B with 100MB fragment information to download the remaining 100MB. The downloaded portions from these two requests are then combined to form the desired data file. Therefore, after obtaining the initial rate limit information, the application server needs to calculate the target rate limit for each download request based on the initial rate limit and the fragment information, thus setting different rate limits for different download requests.

[0081] In a specific embodiment of this specification, the service server first queries the initial rate limit information corresponding to the user client. The initial rate limit information includes the initial rate limit value, the minimum speed of the data fragment, the maximum number of data fragments, and other information. Based on the initial rate limit information and the data fragment information sent by the storage server, the service server calculates the target rate limit information corresponding to this data download request and sends the target rate limit information to the storage server.

[0082] Furthermore, to prevent malicious users from sending download speed limit requests to the business server, which could lead to the business server incorrectly limiting the download speed of users based on the download speed limit requests, a corresponding signature verification mechanism can be set up in the business server. Specifically, the business server parses the download speed limit request to obtain the data fragment information, and performs calculation of the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragment information. This includes: parsing the download speed limit request to obtain the initial signature string and the signature information; decrypting and encoding the signature information to obtain the target signature string; verifying the download speed limit request based on the target signature string and the initial signature string; if the verification is successful, querying the initial speed limit information corresponding to the user client based on the data identification information carried in the download speed limit request, and continuing to perform calculation of the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragment information.

[0083] The target signature string can be understood as the string obtained by the business server decrypting and encoding the signature information. The target signature string is used to compare with the initial signature string. If the comparison is consistent, it means that the download rate limit request was sent by the storage server, and the subsequent steps of calculating the target rate limit information can continue according to the download rate limit request.

[0084] In practical applications, the public key address can be read from the request header to obtain the public key. The public key can then be used to decrypt and encode the signature information to obtain the target signature string. The target signature string can then be compared with the initial signature string for signature verification. If the verification is successful, it indicates that the download rate limit request was sent by the storage server, and the download rate limit request can be responded to to continue the steps of calculating the target rate limit value.

[0085] In a specific embodiment of this specification, the service server receives and parses a download rate limiting request, obtains an initial signature string and corresponding signature information based on the parsing result, decrypts and encodes the signature information to obtain a target signature string, compares the target signature string with the initial signature string, and if they match, it means that the download rate limiting request has been verified. At this point, subsequent steps such as querying the initial rate limiting information corresponding to the user client can continue to be executed.

[0086] Therefore, by setting up a signature verification function on the business server, download speed limiting requests sent by non-storage servers can be filtered out, preventing third parties from sending download speed limiting requests to the business server, damaging the business server, and affecting the normal use of other users.

[0087] Furthermore, when the business server calculates the target speed limit information based on the download speed limit request, to avoid inaccurate target speed limit information calculated by the business server, the calculation can be based on the data fragment information carried in the download speed limit request. Specifically, the method further includes: if the business server detects that the data fragment information is empty, it uses the basic speed limit information as the target speed limit information corresponding to the data download request and continues to send the target speed limit information to the storage server; or, if the business server detects that the data fragment information is not empty, it calculates the download rate information and download time information based on the data fragment information, generates the target speed limit information based on the download rate information and the download time information, and continues to send the target speed limit information to the storage server.

[0088] In this context, an empty data chunk information can be interpreted as the user client's data download request not specifying the corresponding data chunk size or setting the corresponding range parameter. When the corresponding range parameter is not set or the download rate limiting request does not include the range parameter, the business server directly sets the target rate limiting information based on the basic rate limiting information in the initial rate limiting information. The basic rate limiting information can be understood as the minimum rate limiting value set in the initial rate limiting information. If the range parameter for this data download request is not set, the target rate limiting information is directly set to the minimum rate limiting value. If the request includes the range parameter, the target rate limiting information can be calculated based on a preset rate limiting algorithm. The target rate limiting information includes download speed information and download time information. The download speed information can be understood as the download speed at which the user client downloads the target data, and the download time information can be understood as the duration for which the user client downloads the target data based on the download speed information.

[0089] In practical applications, the preset rate-limiting algorithm relies on the data fragment information carried in the request. If the data fragment information is empty, it means that the data download request did not specify data fragment information or the data fragment information is empty. In this case, there is no need to calculate the target rate limit information using the preset rate-limiting algorithm; the minimum rate limit value is directly used as the target rate limit value. However, when data fragment information is specified, such as range=50, it means that the data fragment size is 50 bytes, and the user client needs to download 50 bytes of data. In this case, the preset rate-limiting algorithm can be used for download adaptation, i.e., to calculate the target rate limit information.

[0090] In a specific embodiment of this specification, when the service server detects that the data fragment information is empty, it uses the minimum rate limit value as the target rate limit value corresponding to the data download request and sends the target rate limit value to the storage server.

[0091] In another specific embodiment of this specification, when the service server detects that the data fragment information is not empty, it calculates the download rate information and download time information based on the data fragment information, generates target speed limit information based on the download rate information and download time information, and sends the target speed limit information to the storage server.

[0092] Based on this, the business server checks the data fragment information. If the data fragment information is found to be empty, the minimum rate limit can be directly used as the target rate limit for the current user client download request, improving the calculation efficiency of the target rate limit information. If the data fragment information is not found to be empty, the download speed and download time information can also be calculated using a preset rate limiting algorithm, achieving the goal of setting a corresponding rate limit for each data download request.

[0093] Furthermore, to prevent the business server from calculating incorrect target rate limit information based on the data fragment information, a preset rate limit algorithm can be used to calculate the target rate limit information. Specifically, when the business server detects that the data fragment information is not empty, it calculates the download rate information and download time information based on the data fragment information. This includes: when the business server detects that the data fragment information is not empty, it determines the token bucket corresponding to the data fragment information and the token capacity information of the token bucket; it calculates the download rate information corresponding to the data download request based on the initial rate limit information and the token capacity information; and it calculates the download time information corresponding to the data download request based on the download rate information and the data fragment information.

[0094] The preset rate limiting algorithm reuses the token bucket mechanism, changing the method from calculating the remaining download speed to calculating the remaining download rate. The token bucket mechanism is one of the most commonly used algorithms in network traffic shaping and rate limiting. Typically, the token bucket algorithm controls the amount of data sent to the network and allows for bursts of data transmission. Each data download request corresponds to a token bucket, and traffic is processed using tokens in the bucket. A token bucket with sufficient token capacity can handle multiple requests simultaneously. Therefore, the token bucket corresponding to the data fragment information can be understood as the token bucket that processes data download requests. The content stored in the token bucket may include the data fragment information of the current request, the currently allocated download rate information, and the download time information. By obtaining the token bucket corresponding to the data fragment information and the token capacity information in the current bucket, the download rate information corresponding to the data download request can be calculated. After obtaining the download rate information, the download time information spent downloading the target data according to the download rate information can be calculated.

[0095] In practical applications, download speed information can be characterized as the rate at which the user client downloads the target data, while download time information can be characterized as the time it takes for the user client to download the target data according to the download speed information. Specifically, the remaining total speed can be calculated by subtracting the sum of speeds already distributed within the token bucket from the initial speed limit. The download speed information equals the remaining total speed multiplied by a calculation factor, and the download time information equals the data fragment information divided by the download speed information. For example, if the initial speed limit is 10MB / s and the sum of speeds already distributed within the token bucket is 0, the calculated remaining total speed is 10MB / s. When the calculation factor is 0.6, the calculated download speed is 6MB / s, and when the data fragment range is 60MB, the calculated download time is 10 seconds. In practice, the download time information has an upper limit; if the upper limit is exceeded, the download speed information is set to the minimum speed.

[0096] In summary, by reusing the token bucket mechanism, the business server calculates the rate limit value based on the size of the data fragments that the user client needs to download, and provides corresponding target rate limit information for different data download requests.

[0097] Furthermore, since the storage server is located in a storage node cluster, which includes multiple storage servers, to prevent the user client from being unable to correctly download data from the storage client containing the target data, the server address information of the storage server containing the target data needs to be returned to the user client as the download address. Specifically, the method further includes: when the business server receives the address retrieval request submitted by the user client for the target data, the business server determines the storage server in the storage node cluster and the corresponding server address information of the storage server, and returns the server address information as the data download address to the user client, wherein the server address information is used by the user client to download the target data from the storage server.

[0098] In this context, the target data can be understood as the data previously uploaded by the user to the cloud server for storage via the client. The address retrieval request can be understood as a request to the business server to obtain the storage server address before downloading the data, used to download and retrieve the target data based on the storage server address. The data download address is the download address used by the user's client to download the target data. After determining the server address information, this server address information can be used as the data download address for this download. The storage node cluster can be a cloud storage node cluster or a local storage node cluster. The storage node cluster includes multiple storage servers, which can be used to store the data uploaded by the user.

[0099] In practical applications, after a user uploads data, the business server will call a relatively idle storage server to receive the uploaded data according to the load balancing strategy. Therefore, when downloading data later, the user client needs to first query the business server for the address information of the storage server that stores the target data, and then download the data according to the address information.

[0100] In a specific embodiment of this specification, the business server receives an address acquisition request sent by the user client. The address acquisition request is for the server address required to obtain the target data for download. The business server determines the storage server storing the target data in the storage node cluster and returns the server address information corresponding to the storage server as the data download address to the user client.

[0101] Based on this, by returning the server address information as the data download address to the user client, the user client can accurately download the target data from the storage server based on the data download address.

[0102] Furthermore, to avoid excessive load on storage servers in the storage node cluster, which could cause them to malfunction, a load balancing strategy can be implemented by the business server to balance the load on each storage server. Specifically, the method further includes: when the business server receives a data storage request submitted by the user client for the target data, the business server determines the storage server in the storage node cluster according to a preset load balancing strategy, and the corresponding server address information of the storage server, and returns the server address information to the user client, wherein the server address information is used by the user client to upload the target data to the storage server.

[0103] A data storage request can be understood as a request from a user client to upload target data for storage. In practice, the user client can upload data to a cloud drive (cloud storage) or to a file-sharing system (local shared storage). When the business server receives a data storage request from a user client, it should allocate a relatively idle storage server to store the target data. The business server will determine the storage server and its address information within the storage node cluster according to a preset load balancing strategy and return the server address information to the user client, enabling the user client to upload the target data to the storage server based on the server address information.

[0104] In a specific embodiment of this specification, the business server receives a data storage request sent by the user client. The data storage request is a request to upload target data to the cloud for storage. The business server selects a storage server in the storage node cluster according to a preset load balancing strategy. The storage server is the server with the lowest load pressure. The storage server performs the data storage task. The business server will then return the server address information corresponding to the storage server to the user client so that the user client can upload the target data to the storage server according to the server address information.

[0105] Based on this, a load balancing strategy is implemented through the business server to call idle storage servers to perform this data storage task, ensuring that each storage server in the storage node cluster will not crash due to excessive load pressure, thus improving system stability.

[0106] Step 206: The storage server adjusts the download parameters corresponding to the data download request according to the target speed limit information, and transmits the target data to the user client according to the adjusted download parameters.

[0107] Download parameters can be understood as the parameters of the download port provided by the storage server for the data download request. By adjusting the download parameters, the bandwidth between the storage server and the user client can be changed. For example, adjusting the download rate parameters can control the download speed of the user client. After the storage server adjusts the download parameters according to the target speed limit information, it can transmit the target data to the user client according to the adjusted download parameters.

[0108] In practical applications, a storage server may provide multiple user clients for downloading, each with its own download interface. Data transfer between the two clients is achieved through these interfaces. When it is necessary to limit the download speed of a data download request sent by a particular user client, the download rate parameters of the download interface corresponding to that request can be adjusted. Subsequently, the data file can be transferred to the user client at the target speed limit according to the adjusted download rate parameters.

[0109] In a specific embodiment of this specification, after receiving the target rate limit information sent by the business server, the storage server can determine the download parameters of the download interface corresponding to the data download request, adjust the download parameters based on the target rate limit information, and transmit the target data to the user client according to the adjusted download parameters, thereby fulfilling the requirement of setting a rate limit for the data download request sent by the user client.

[0110] Furthermore, after the business server calculates the target rate limit information, it sends the target rate limit information to the storage server. To prevent the storage server from failing to limit the data download request based on the target rate limit information, the storage server can adjust the download parameters based on the target rate limit information. Specifically, the storage server adjusts the download parameters corresponding to the data download request according to the target rate limit information, including: obtaining the download rate information by parsing the target rate limit information, determining the download parameters corresponding to the data download request, adjusting the download rate limit value in the download parameters based on the download rate information, and continuing to transmit the target data to the user client according to the adjusted download parameters.

[0111] The download parameters can be understood as the interface parameters on the storage server corresponding to this data download request. By adjusting the download parameters, the download speed can be limited for the user client.

[0112] In practical applications, when the business server returns the target rate limit value, it can carry other field parameters, specifically x-oss-callback-responseheader. This field can include the response information of the download rate limit request. The corresponding field of the response information is Allowance, which corresponds to either allow or deny. If the response information is empty, the storage server will consider the download rate limit request to have failed. The storage server will continue to send the download rate limit request or reply with failure to the user client. The specific execution strategy can be selected according to the actual situation, and this manual does not impose specific restrictions here.

[0113] In practice, download parameters can be response fields corresponding to new download speed limit requests, including `TrafficLimitAfter` (the limit data size). After setting the limit data size, speed limiting will begin after downloading this amount of data. It also includes the `TrafficLimit` value, where download rate information is written for download speed limiting. The storage server specifies the maximum and minimum speed limit values; if these values ​​are exceeded, the storage server will refuse to transmit data to the client.

[0114] In a specific embodiment of this specification, the storage server parses the target rate limit information to obtain the download rate information, determines the download parameters corresponding to the data download request, adjusts the download rate limit value TrafficLimit of the download parameters based on the download rate information, and transmits the target data to the user client according to the adjusted download parameters.

[0115] Based on this, the storage server adjusts the download parameters according to the target rate limit value returned by the business server, enabling the storage server to transmit the target data to the user client according to the adjusted download parameters, thereby achieving download rate limiting for the user client.

[0116] Furthermore, in order to accurately limit the download speed of the user client and prevent the user client from downloading data at the limited speed indefinitely, the method can adjust the download speed information of the user client according to the download time information. Specifically, the method further includes: obtaining the download time information by parsing the target speed limit information through the storage server, obtaining the data transmission time corresponding to the target data, and if the data transmission time is detected to be greater than the download time information, adjusting the download speed limit value in the download parameters based on the basic speed limit information, and continuing to transmit the target data according to the adjusted download parameters.

[0117] The download time information refers to the time it takes for the user client to download according to the target speed limit information. The data transmission time can be understood as the time that the current user client has been downloading according to the download speed information. If the data transmission time is greater than the download time information, it means that the calculated download time has been exceeded. At this time, the download speed can be adjusted to the minimum speed, and the target data can continue to be transmitted according to the adjusted download parameters.

[0118] In practical applications, due to factors such as changes in the user's network environment and network speed, the user's client may be unable to download the target data within the calculated download time and at the calculated download speed. If the target data is not downloaded within the download time, the download speed will be adjusted to the minimum speed to further limit the user's client speed.

[0119] In a specific embodiment of this specification, the download time information in the target speed limit information is obtained by parsing the target speed limit information, and the data transmission time corresponding to the target data is obtained. The data transmission time is the time that the user client has spent downloading the target data. If the data transmission time is greater than the download time information, the download speed limit value is adjusted to the minimum speed limit value, and the remaining target data continues to be downloaded according to the adjusted download parameters.

[0120] Based on this, by comparing download time information and data transmission time, the download speed information of the user client can be accurately controlled, avoiding incorrect download speed settings for the user client.

[0121] Furthermore, to prevent download speed limits from being imposed on normal users, speed limit detection can be performed on users. Specifically, the method further includes: receiving traffic statistics information from the user client through the business server; if the traffic statistics information exceeds a preset traffic threshold, determining that the user client is a speed-limited user client, and continuing to query the initial speed limit information corresponding to the user client.

[0122] The traffic statistics can be understood as the total download volume of a user's client within a specified window. For example, if a user's client downloads within one hour, and the traffic statistics exceed a preset traffic threshold, download speed throttling will be triggered for that user. The preset traffic threshold is a threshold set by the business unit; exceeding this threshold indicates that the user has triggered the speed throttling condition. For instance, if the traffic statistics are 100MB and the preset traffic threshold is 50MB, then download speed throttling will be applied to that user.

[0123] In practical applications, the business operator can collect traffic statistics and send them to the business server. The business server can then determine whether the current user needs to be speed-limited based on the traffic statistics and set a corresponding download token for them. The download token can contain parameters such as the user identifier and initial speed limit information.

[0124] In a specific embodiment of this specification, if the traffic statistics information received from the user client within a certain time interval is 20GB and the preset traffic threshold is 10GB, it indicates that the traffic statistics information is greater than the preset traffic threshold. At this time, the user needs to perform speed limiting. The user client corresponding to the user is determined to be the speed-limited user client. The download token corresponding to the user client is queried to obtain the corresponding initial speed limit information, and the subsequent steps of calculating the target speed limit value are continued.

[0125] This specification provides a data management method applied to a storage system, wherein the system includes a business server and a storage server. When the storage server receives a data download request from a user client for target data, it sends data fragment information corresponding to the data download request to the business server. The business server queries the initial rate limit information corresponding to the user client, and calculates the target rate limit information corresponding to the data download request based on the initial rate limit information and the data fragment information. The storage server adjusts the download parameters corresponding to the data download request according to the target rate limit information, and transmits the target data to the user client according to the adjusted download parameters. The storage server sends the data fragment information corresponding to the data download request submitted by the user client to the business server. The business server calculates the target rate limit information for this data download request based on the initial rate limit information and data fragment information of the user client, and sends it to the storage server. This allows the storage server to adjust the download parameters according to the target rate limit information and transmit the target data to the user client according to the download parameters. In this way, corresponding rate limit information can be set for different download requests from different users, achieving the purpose of traffic limiting for a single user. This meets the needs of project owners for traffic control in scenarios where users have multiple data streams downloading.

[0126] Figure 3 A system processing flowchart of a storage system according to an embodiment of this specification is shown. The system includes a storage server 302 and a service server 304, wherein...

[0127] The storage server 302 is used to send data fragment information corresponding to the data download request to the business server in response to a data download request submitted by a user client for target data.

[0128] When a user client wants to download a data file, it sends a download request to the storage server that stores that data file. The download request may include information such as the identifier of the data file to be downloaded and its size. The storage server can be understood as the physical storage server for the data, i.e., the OSS side. After receiving the download request, the storage server sends the corresponding data fragment information to the business server. The business server can be understood as the server side that provides business services, i.e., the PDS side. The data fragment information can be understood as the size of the data file that the user client wants to download in this download request. After sending the data fragment information to the business server, if the business server determines that the current user client needs to be rate-limited, it will calculate the rate limit value for this download request based on the data fragment information to implement rate control for this user's download.

[0129] In practical applications, before sending a data download request to the storage server, the user client sends a download link retrieval request to the business server. This request inquires which storage server the data file to be downloaded is stored on, allowing the business server to subsequently determine the storage server based on the download link and send the data download request to that server, thus enabling the user to actually obtain the data file they want to download. Therefore, the business server can also respond to the user client's download link retrieval request for target data, determine the storage server corresponding to the target data, and send the storage server identifier of that storage server to the user client. The user client can then use the storage server identifier to determine the target storage server and send the data download request to that target storage server.

[0130] In a specific embodiment of this specification, a user wants to download data resource file A. The user client submits a download link request to the business server. The business server determines the storage server corresponding to data resource file A and returns the storage server identifier to the user client. After receiving the storage server identifier, the user client can determine the storage server storing data resource file A and send a data download request to that storage server to obtain data resource file A. Upon receiving the data download request for data resource file A from the user client, the storage server sends the data fragment information corresponding to the data download request to the business server. This information is used by the business server to subsequently calculate the rate limit information corresponding to this download request.

[0131] The service server 304 is used to query the initial rate limit information corresponding to the user client, calculate the target rate limit information corresponding to the data download request based on the initial rate limit information and the data fragmentation information, and send the target rate limit information to the storage server.

[0132] The initial rate limit information can be understood as the rate limit value and related information initially set by the user's client in this business application. This initial rate limit information may include the initial rate limit value, minimum download speed per chunk, and maximum number of chunks. The initial rate limit information is generally the rate limit value configured by the business operator for the user. When a user is rate-limited by the business operator, all download requests submitted by the user will be subject to the configured rate limit value, meaning that all data download requests submitted by the user will have the same download rate limit value. This leads to the user downloading data files in chunks, increasing the download speed through concurrency. However, by having the business provider's business server calculate the target rate limit information for each data download request based on the data chunk information corresponding to each data download request, a corresponding target rate limit value can be set for each data download request. This prevents users from increasing download speed through concurrent downloads, thus truly achieving the effect of rate-limited downloads for users. The calculation of the target rate limit information for this data download request requires the calculation of the target rate limit information based on the initial rate limit information set by the business operator for the user's client and the data chunk information corresponding to this data download request.

[0133] In practical applications, the business server provides an API interface for user quotas. Business operators can use this API to implement rate limiting for users. The API interface includes a new field, `rate_limit_config`, which sets the download rate limit. This field can include the initial rate limit (`QuotaSpeed`), the minimum download speed per segment (`MinSpeed`), and the maximum number of segments (`MaxPerts`). The initial rate limit restricts the user's maximum download speed, the minimum download speed per segment restricts the user's minimum download speed, and the maximum number of segments restricts the user's maximum concurrent downloads. After the business operator provides the limits for each aspect, the business server writes the corresponding data values ​​into the field, thus implementing the initial rate limiting for users.

[0134] When a user downloads a data file concurrently, multiple download requests are sent for the same file. However, each download request has different fragment information, meaning the size of the downloaded data file varies. For example, if the data file is 150MB, the user can send download request A with 50MB fragment information to download 50MB of the data file, and download request B with 100MB fragment information to download the remaining 100MB. The downloaded portions from these two requests are then combined to form the desired data file. Therefore, after obtaining the initial rate limit information, the application server needs to calculate the target rate limit for each download request based on the initial rate limit and the fragment information, thus setting different rate limits for different download requests.

[0135] In a specific embodiment of this specification, the service server first queries the initial rate limit information corresponding to the user client. The initial rate limit information includes the initial rate limit value, the minimum speed of the data fragment, the maximum number of data fragments, and other information. Based on the initial rate limit information and the data fragment information sent by the storage server, the service server calculates the target rate limit information corresponding to this data download request and sends the target rate limit information to the storage server.

[0136] The storage server 302 is used to adjust the download parameters corresponding to the data download request according to the target speed limit information, and transmit the target data to the user client according to the adjusted download parameters.

[0137] Download parameters can be understood as the parameters of the download port provided by the storage server for the data download request. By adjusting the download parameters, the bandwidth between the storage server and the user client can be changed. For example, adjusting the download rate parameters can control the download speed of the user client. After the storage server adjusts the download parameters according to the target speed limit information, it can transmit the target data to the user client according to the adjusted download parameters.

[0138] In practical applications, a storage server may provide multiple user clients for downloading, each with its own download interface. Data transfer between the two clients is achieved through these interfaces. When it is necessary to limit the download speed of a data download request sent by a particular user client, the download rate parameters of the download interface corresponding to that request can be adjusted. Subsequently, the data file can be transferred to the user client at the target speed limit according to the adjusted download rate parameters.

[0139] In a specific embodiment of this specification, after receiving the target rate limit information sent by the business server, the storage server can determine the download parameters of the download interface corresponding to the data download request, adjust the download parameters based on the target rate limit information, and transmit the target data to the user client according to the adjusted download parameters, thereby fulfilling the requirement of setting a rate limit for the data download request sent by the user client.

[0140] Optionally, the storage server is configured to send a download rate limiting request carrying data fragment information to the service server; the service server is configured to parse the download rate limiting request to obtain the data fragment information, and perform calculation of the target rate limiting information corresponding to the data download request based on the initial rate limiting information and the data fragment information, and send the target rate limiting information to the storage server.

[0141] Optionally, the storage server is configured to obtain the service configuration information of the user client, determine the service identification information corresponding to the user client based on the service configuration information, parse the data download request to obtain data identification information and data fragment information, generate a download rate limiting request based on the service identification information, the data identification information and the data fragment information, and send the download rate limiting request to the service server corresponding to the service identification information.

[0142] Optionally, the storage server is further configured to generate an initial signature string based on the data identification information and the data fragmentation information, encrypt and encode the initial signature string to obtain signature information, and write the signature information and the initial signature string into the download speed limit request.

[0143] Optionally, the service server is configured to parse the download rate limiting request, obtain the initial signature string and the signature information, decrypt and encode the signature information to obtain the target signature string, verify the download rate limiting request based on the target signature string and the initial signature string, and if the verification is successful, query the initial rate limiting information corresponding to the user client according to the data identification information carried in the download rate limiting request, and continue to calculate the target rate limiting information corresponding to the data download request based on the initial rate limiting information and the data fragmentation information.

[0144] Optionally, the service server is configured to, when detecting that the data fragment information is empty, use the basic rate limiting information as the target rate limiting information corresponding to the data download request, and continue to send the target rate limiting information to the storage server; or, the service server is configured to, when detecting that the data fragment information is not empty, calculate the download rate information and download time information based on the data fragment information, generate the target rate limiting information based on the download rate information and the download time information, and continue to send the target rate limiting information to the storage server.

[0145] Optionally, the service server is configured to, when detecting that the data fragment information is not empty, determine the token bucket corresponding to the data fragment information and the token capacity information of the token bucket, calculate the download rate information corresponding to the data download request based on the initial rate limit information and the token capacity information, and calculate the download time information corresponding to the data download request based on the download rate information and the data fragment information.

[0146] Optionally, the storage server is configured to parse the target speed limit information to obtain the download speed information, determine the download parameters corresponding to the data download request, adjust the download speed limit value in the download parameters based on the download speed information, and continue to transmit the target data to the user client according to the adjusted download parameters.

[0147] Optionally, the storage server is further configured to parse the target speed limit information to obtain the download time information, acquire the data transmission time corresponding to the target data, and, if the data transmission time is detected to be greater than the download time information, adjust the download speed limit value in the download parameters based on the basic speed limit information, and continue to transmit the target data according to the adjusted download parameters.

[0148] Optionally, the service server is further configured to receive traffic statistics information from the user client, and if the traffic statistics information exceeds a preset traffic threshold, determine that the user client is a rate-limited user client, and continue to query the initial rate-limit information corresponding to the user client.

[0149] This specification provides a storage system comprising a service server and a storage server. The storage server is configured to, in response to a user client submitting a data download request for target data, send data fragment information corresponding to the data download request to the service server. The service server is configured to, query the initial rate limit information corresponding to the user client, calculate the target rate limit information corresponding to the data download request based on the initial rate limit information and the data fragment information, and send the target rate limit information to the storage server. The storage server is configured to, according to the target rate limit information, adjust the download parameters corresponding to the data download request, and transmit the target data to the user client according to the adjusted download parameters. The storage server sends the data fragment information corresponding to the data download request submitted by the user client to the business server. The business server calculates the target rate limit information for this data download request based on the initial rate limit information and data fragment information of the user client, and sends it to the storage server. This allows the storage server to adjust the download parameters according to the target rate limit information and transmit the target data to the user client according to the download parameters. In this way, corresponding rate limit information can be set for different download requests from different users, achieving the purpose of traffic limiting for a single user. This meets the needs of project owners for traffic control in scenarios where users have multiple data streams downloading.

[0150] Figure 4 A flowchart of a data management method for a storage server according to an embodiment of this specification is shown, including steps 402 to 408.

[0151] Step 402: In response to the data download request submitted by the user client for the target data, send the data fragment information corresponding to the data download request to the business server.

[0152] Step 404: Receive the target rate limit information returned by the service server for the data fragmentation information.

[0153] Step 406: Adjust the download parameters corresponding to the data download request according to the target speed limit information.

[0154] Step 408: Transmit the target data to the user client according to the adjusted download parameters.

[0155] Optionally, the method includes: sending a download rate limiting request carrying data fragmentation information to the service server.

[0156] Optionally, the method includes: obtaining service configuration information of a user client; determining service identification information corresponding to the user client based on the service configuration information; parsing the data download request to obtain data identification information and data fragment information; generating a download rate limiting request based on the service identification information, the data identification information, and the data fragment information; and sending the download rate limiting request to the service server corresponding to the service identification information.

[0157] Optionally, the method further includes: generating an initial signature string based on the data identification information and the data fragmentation information, encrypting and encoding the initial signature string to obtain signature information, and writing the signature information and the initial signature string into the download speed limit request.

[0158] Optionally, the method includes: parsing the target speed limit information to obtain the download rate information, determining the download parameters corresponding to the data download request, adjusting the download speed limit value in the download parameters based on the download rate information, and continuing to transmit the target data to the user client according to the adjusted download parameters.

[0159] Optionally, the method includes: parsing the target speed limit information to obtain the download time information, obtaining the data transmission time corresponding to the target data, and when the data transmission time is detected to be greater than the download time information, adjusting the download speed limit value in the download parameters based on the basic speed limit information, and continuing to transmit the target data according to the adjusted download parameters.

[0160] This specification provides a data management method for a storage server. In response to a user client's data download request for target data, the method sends data fragmentation information corresponding to the data download request to the service server; receives target rate limiting information returned by the service server based on the data fragmentation information; adjusts the download parameters corresponding to the data download request according to the target rate limiting information; and transmits the target data to the user client according to the adjusted download parameters. By using the target rate limiting information returned by the service server, download parameters can be adjusted based on the target rate limiting information, and target data can be transmitted to the user client according to the download parameters, thus achieving rate limiting processing for the user client.

[0161] Figure 5 A flowchart of a data management method for a business server according to an embodiment of this specification is shown, including steps 502 to 506.

[0162] Step 502: Receive data fragment information of the associated data download request sent by the storage server, and query the initial rate limit information corresponding to the user client based on the data fragment information.

[0163] Step 504: Based on the initial speed limit information and the data fragmentation information, calculate the target speed limit information corresponding to the data download request.

[0164] Step 506: Send the target rate limit information to the storage server.

[0165] Optionally, the method includes: parsing the download rate limiting request to obtain the data fragment information, and performing calculation of the target rate limiting information corresponding to the data download request based on the initial rate limiting information and the data fragment information, and sending the target rate limiting information to the storage server.

[0166] Optionally, the method includes: parsing the download rate limiting request to obtain the initial signature string and the signature information; decrypting and encoding the signature information to obtain the target signature string; verifying the download rate limiting request based on the target signature string and the initial signature string; if the verification is successful, querying the initial rate limiting information corresponding to the user client based on the data identification information carried in the download rate limiting request; and continuing to calculate the target rate limiting information corresponding to the data download request based on the initial rate limiting information and the data fragmentation information.

[0167] Optionally, the method includes: if the data fragment information is detected to be empty, using the basic rate-limiting information as the target rate-limiting information corresponding to the data download request, and continuing to send the target rate-limiting information to the storage server; or,

[0168] The service server is configured to, when detecting that the data fragment information is not empty, calculate the download rate information and download time information based on the data fragment information, generate the target speed limit information based on the download rate information and the download time information, and continue to send the target speed limit information to the storage server.

[0169] Optionally, the method includes: when the data fragment information is detected to be non-empty, determining the token bucket corresponding to the data fragment information and the token capacity information of the token bucket, calculating the download rate information corresponding to the data download request based on the initial rate limit information and the token capacity information, and calculating the download time information corresponding to the data download request based on the download rate information and the data fragment information.

[0170] Optionally, the method further includes: receiving traffic statistics information from the user client; if the traffic statistics information exceeds a preset traffic threshold, determining that the user client is a rate-limited user client, and continuing to query the initial rate-limiting information corresponding to the user client.

[0171] This specification provides a data management method for a business server, comprising: receiving data fragment information of a related data download request sent by a storage server; querying initial rate-limiting information corresponding to the user client based on the data fragment information; calculating target rate-limiting information corresponding to the data download request based on the initial rate-limiting information and the data fragment information; and sending the target rate-limiting information to the storage server. By calculating the target rate-limiting information corresponding to the data download request using the data fragment information, the storage server can subsequently transmit data according to the target rate-limiting information, thus fulfilling the requirement to set corresponding target rate-limiting information for different data download requests from users and achieving the purpose of user-level rate-limiting processing.

[0172] The following is in conjunction with the appendix Figure 6 Taking the data management method provided in this specification as an example in resource downloading, the data management method will be further explained. Figure 6 A flowchart of a data management method according to an embodiment of this specification is shown, with specific steps including steps 602 to 614.

[0173] Step 602: The storage server receives a data download request submitted by the user client for the target data.

[0174] In one feasible approach, if a user wants to download target data A, they send a data download request for target data A to the storage server via their client.

[0175] Step 604: The storage server obtains the service configuration information of the user client and determines the service identification information corresponding to the user client based on the service configuration information.

[0176] In one feasible approach, the storage server obtains the user client's business configuration information, which includes information such as user ID and client ID. Based on the business configuration information, the storage server determines the business identification information corresponding to the user client, which is the URL address of the business server.

[0177] Step 606: The storage server parses the data download request to obtain data identification information and data fragment information, generates a download rate limiting request based on the business identification information, data identification information, and data fragment information, and sends the download rate limiting request to the business server corresponding to the business identification information.

[0178] In one feasible approach, the storage server parses the data download request to obtain data identification information and data fragment information. The data identification information includes user identification information and file identification information of target data A. The data fragment information includes the data size information of target data A. Based on the business identification information, data identification information, and data fragment information, a download rate limiting request is generated. The download rate limiting request is a callback request and is sent to the business server.

[0179] In one feasible approach, the storage server can also include signature information and an initial signature string in the download rate limiting request, which can then be used by the business server to verify the signature of the download rate limiting request based on the signature information and the initial signature string.

[0180] Step 608: The business server downloads the rate-limiting request, obtains the initial signature string and signature information, decrypts and encodes the signature information, and obtains the target signature string.

[0181] In one feasible approach, the business server parses the download rate-limiting request, obtains the initial signature string and signature information, decrypts and encodes the signature information, and obtains the target signature string.

[0182] Step 610: The business server verifies the download rate limiting request based on the target signature string and the initial signature string.

[0183] In one feasible approach, the download rate limit request is verified by comparing the target signature string with the initial signature string. If the verification passes, the initial rate limit information corresponding to the user client is retrieved based on the data identification information, and then the target rate limit information corresponding to the data download request is calculated based on the initial rate limit information and the data fragment information.

[0184] Step 612: The business server queries the initial rate limit information corresponding to the user client, calculates the target rate limit information corresponding to the data download request based on the initial rate limit information and data fragmentation information, and sends the target rate limit information to the storage server.

[0185] In one feasible approach, if the range parameter of the data fragmentation information is not empty, the download rate information and download time information are calculated according to a preset rate limiting algorithm, and the target rate limiting information is generated based on the download rate information and download time information and returned to the storage server.

[0186] Step 614: The storage server adjusts the download parameters corresponding to the data download request according to the target rate limit information, and transmits the target data to the user client according to the adjusted download parameters.

[0187] In one feasible approach, the storage server adjusts the download rate parameters corresponding to the download request based on the download rate information in the target rate limit information, and transmits the target data A to the user client according to the adjusted download rate parameters.

[0188] In another possible implementation, if the storage server detects that the data transfer time is longer than the download time, it adjusts the download rate limit in the download parameters to the minimum rate limit and continues to transfer the remaining target data according to the adjusted download parameters.

[0189] This specification provides a data management method in which a storage server sends data fragment information corresponding to a data download request submitted by a user client to a business server. The business server calculates the target rate limit information for this data download request based on the initial rate limit information and data fragment information corresponding to the user client, and sends it to the storage server. This allows the storage server to adjust the download parameters according to the target rate limit information and transmit the target data to the user client according to the download parameters. Thus, corresponding rate limit information can be set for different download requests from different users, achieving the purpose of traffic limiting for a single user. This also meets the needs of project owners for traffic control in scenarios where users have multiple data streams downloading.

[0190] Corresponding to the above method embodiments, this specification also provides embodiments of a data management device. Figure 7 A schematic diagram of the structure of a data management device according to an embodiment of this specification is shown. Figure 7 As shown, this device is applied to a storage system, wherein the system includes a business server and a storage server, wherein,

[0191] The sending module 702 is configured to send data fragment information corresponding to the data download request to the business server through the storage server when the storage server receives a data download request submitted by a user client for target data.

[0192] The calculation module 704 is configured to query the initial speed limit information corresponding to the user client through the business server, and calculate the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragmentation information.

[0193] The adjustment module 706 is configured to have the storage server adjust the download parameters corresponding to the data download request according to the target speed limit information, and transmit the target data to the user client according to the adjusted download parameters.

[0194] Optionally, the sending module 702 is further configured to send a download rate limiting request carrying data fragment information to the service server.

[0195] Optionally, the sending module 702 is further configured to: obtain service configuration information of the user client, determine the service identification information corresponding to the user client based on the service configuration information, parse the data download request to obtain data identification information and data fragment information, generate a download rate limiting request based on the service identification information, the data identification information and the data fragment information, and send the download rate limiting request to the service server corresponding to the service identification information.

[0196] Optionally, the device further includes a verification module configured to: generate an initial signature string based on the data identification information and the data fragmentation information, encrypt and encode the initial signature string to obtain signature information, and write the signature information and the initial signature string into the download speed limit request.

[0197] Optionally, the adjustment module 706 is further configured to: parse the target speed limit information to obtain the download speed information, determine the download parameters corresponding to the data download request, adjust the download speed limit value in the download parameters based on the download speed information, and continue to transmit the target data to the user client according to the adjusted download parameters.

[0198] Optionally, the calculation module 704 is further configured to: parse the download rate limiting request to obtain the data fragment information, and perform calculation of the target rate limiting information corresponding to the data download request based on the initial rate limiting information and the data fragment information, and send the target rate limiting information to the storage server.

[0199] Optionally, the calculation module 704 is further configured to: parse the download rate limiting request to obtain the initial signature string and the signature information; decrypt and encode the signature information to obtain the target signature string; verify the download rate limiting request based on the target signature string and the initial signature string; if the verification is successful, query the initial rate limiting information corresponding to the user client based on the data identification information carried in the download rate limiting request; and continue to calculate the target rate limiting information corresponding to the data download request based on the initial rate limiting information and the data fragmentation information.

[0200] Optionally, the calculation module 704 is further configured to: when the data fragment information is detected to be empty, use the basic rate limiting information as the target rate limiting information corresponding to the data download request, and continue to send the target rate limiting information to the storage server; or, the service server is configured to: when the data fragment information is detected to be not empty, calculate the download rate information and download time information based on the data fragment information, generate the target rate limiting information based on the download rate information and the download time information, and continue to send the target rate limiting information to the storage server.

[0201] Optionally, the calculation module 704 is further configured to: when the data fragment information is detected to be non-empty, determine the token bucket corresponding to the data fragment information and the token capacity information of the token bucket, calculate the download rate information corresponding to the data download request based on the initial rate limit information and the token capacity information, and calculate the download time information corresponding to the data download request based on the download rate information and the data fragment information.

[0202] Optionally, the device further includes a judgment module configured to: receive traffic statistics information from the user client; if the traffic statistics information exceeds a preset traffic threshold, determine that the user client is a rate-limited user client, and continue to query the initial rate-limiting information corresponding to the user client.

[0203] This specification provides a data management device applied to a storage system. The system includes a business server and a storage server. A sending module is configured to, upon receiving a data download request from a user client for target data, send data fragment information corresponding to the data download request to the business server via the storage server. A calculation module is configured to query the initial rate limit information corresponding to the user client via the business server, and calculate the target rate limit information corresponding to the data download request based on the initial rate limit information and the data fragment information. An adjustment module is configured to have the storage server adjust the download parameters corresponding to the data download request according to the target rate limit information, and transmit the target data to the user client according to the adjusted download parameters. By using the target rate limit information returned by the business server, download parameters can be adjusted based on the target rate limit information, and target data can be transmitted to the user client according to the download parameters, thus achieving rate limiting processing for the user client. Calculating the target rate limit information corresponding to the data download request using data fragment information allows the storage server to transmit data according to the target rate limit information, fulfilling the requirement to set corresponding target rate limit information for different user data download requests, achieving the purpose of user-level rate limiting processing.

[0204] The above is an illustrative scheme of a data management device according to this embodiment. It should be noted that the technical solution of this data management device and the technical solution of the data management method described above belong to the same concept. For details not described in detail in the technical solution of the data management device, please refer to the description of the technical solution of the data management method described above.

[0205] Corresponding to the above method embodiments, this specification also provides embodiments of a data management device. Figure 8 A schematic diagram of a data management device for a storage server, according to an embodiment of this specification, is shown. Figure 8 As shown, this device is used in a storage server and includes:

[0206] The sending module 802 is configured to send data fragment information corresponding to the data download request to the business server in response to a data download request submitted by a user client for target data.

[0207] The receiving module 804 is configured to receive the target rate limit information returned by the service server in response to the data fragmentation information.

[0208] The adjustment module 806 is configured to adjust the download parameters corresponding to the data download request based on the target speed limit information.

[0209] The transmission module 808 is configured to transmit the target data to the user client according to the adjusted download parameters.

[0210] Optionally, the sending module 802 is further configured to send a download rate limiting request carrying data fragment information to the service server.

[0211] Optionally, the sending module 802 is further configured to: obtain service configuration information of the user client, determine service identification information corresponding to the user client based on the service configuration information, parse the data download request to obtain data identification information and data fragment information, generate a download rate limiting request based on the service identification information, the data identification information and the data fragment information, and send the download rate limiting request to the service server corresponding to the service identification information.

[0212] Optionally, the device further includes a verification module configured to: generate an initial signature string based on the data identification information and the data fragmentation information, encrypt and encode the initial signature string to obtain signature information, and write the signature information and the initial signature string into the download speed limit request.

[0213] Optionally, the transmission module 808 is further configured to: parse the target speed limit information to obtain the download rate information, determine the download parameters corresponding to the data download request, adjust the download speed limit value in the download parameters based on the download rate information, and continue to transmit the target data to the user client according to the adjusted download parameters.

[0214] This specification provides a data management device applied to a storage server, comprising: a sending module configured to send data fragment information corresponding to a data download request submitted by a user client for target data to a service server; a receiving module configured to receive target rate limiting information returned by the service server for the data fragment information; an adjustment module configured to adjust the download parameters corresponding to the data download request according to the target rate limiting information; and a transmission module configured to transmit the target data to the user client according to the adjusted download parameters. By using the target rate limiting information returned by the service server, download parameters can be adjusted based on the target rate limiting information, and target data can be transmitted to the user client according to the download parameters, thereby achieving rate limiting processing for the user client.

[0215] The above is an illustrative scheme of a data management device according to this embodiment. It should be noted that the technical solution of this data management device and the technical solution of the data management method described above belong to the same concept. For details not described in detail in the technical solution of the data management device, please refer to the description of the technical solution of the data management method described above.

[0216] Corresponding to the above method embodiments, this specification also provides embodiments of a data management device. Figure 9 This specification illustrates a schematic diagram of a data management device applied to a business server, according to an embodiment of this specification. Figure 9 As shown, this device is used in a business server and includes:

[0217] The query module 902 is configured to receive data fragment information of the associated data download request sent by the storage server, and query the initial rate limit information corresponding to the user client based on the data fragment information.

[0218] The calculation module 904 is configured to calculate the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragmentation information.

[0219] The sending module 906 is configured to send the target rate limit information to the storage server.

[0220] Optionally, the sending module 906 is further configured to: parse the download rate limiting request to obtain the data fragment information, and perform calculation of the target rate limiting information corresponding to the data download request based on the initial rate limiting information and the data fragment information, and send the target rate limiting information to the storage server.

[0221] Optionally, the calculation module 904 is further configured to: parse the download speed limit request to obtain the initial signature string and the signature information; decrypt and encode the signature information to obtain the target signature string; verify the download speed limit request based on the target signature string and the initial signature string; if the verification is successful, query the initial speed limit information corresponding to the user client according to the data identification information carried in the download speed limit request; and continue to calculate the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragmentation information.

[0222] Optionally, the calculation module 904 is further configured to: when the data fragment information is detected to be empty, use the basic rate limiting information as the target rate limiting information corresponding to the data download request, and continue to send the target rate limiting information to the storage server; or, the service server is configured to: when the data fragment information is detected to be not empty, calculate the download rate information and download time information based on the data fragment information, generate the target rate limiting information based on the download rate information and the download time information, and continue to send the target rate limiting information to the storage server.

[0223] Optionally, the calculation module 904 is further configured to: when the data fragment information is detected to be non-empty, determine the token bucket corresponding to the data fragment information and the token capacity information of the token bucket, calculate the download rate information corresponding to the data download request based on the initial rate limit information and the token capacity information, and calculate the download time information corresponding to the data download request based on the download rate information and the data fragment information.

[0224] Optionally, the device further includes a judgment module configured to: receive traffic statistics information from the user client; if the traffic statistics information exceeds a preset traffic threshold, determine that the user client is a rate-limited user client, and continue to query the initial rate-limiting information corresponding to the user client.

[0225] This specification provides a data management device applied to a business server, comprising: a query module configured to receive data fragment information of a related data download request sent by a storage server, and query initial rate limit information corresponding to the user client based on the data fragment information; a calculation module configured to calculate target rate limit information corresponding to the data download request based on the initial rate limit information and the data fragment information; and a sending module configured to send the target rate limit information to the storage server. By calculating the target rate limit information corresponding to the data download request using the data fragment information, the storage server can subsequently transmit data according to the target rate limit information, thus fulfilling the requirement to set corresponding target rate limit information for different data download requests from users and achieving the purpose of user-level rate limiting processing.

[0226] The above is an illustrative scheme of a data management device according to this embodiment. It should be noted that the technical solution of this data management device and the technical solution of the data management method described above belong to the same concept. For details not described in detail in the technical solution of the data management device, please refer to the description of the technical solution of the data management method described above.

[0227] Figure 10 A structural block diagram of a computing device 1000 according to an embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0228] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0229] In one embodiment of this specification, the above-described components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0230] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 1000 can also be a mobile or stationary server.

[0231] The processor 1020 implements the data management method when executing the computer instructions.

[0232] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data management method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data management method described above.

[0233] An embodiment of this specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the data management method as described above.

[0234] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data management method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data management method described above.

[0235] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data management method.

[0236] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the data management method described above belong to the same concept. Details not described in detail in the computer program's technical solution can be found in the description of the data management method's technical solution.

[0237] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0238] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0239] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0240] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0241] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data management method, said method being applied to a storage system, said system comprising a business server and a storage server, wherein, When the storage server receives a data download request submitted by a user client for target data, the storage server sends data fragment information corresponding to the data download request to the business server. Here, the user client is a rate-limited user client, the data download request is a download request from the user client for the target data in a concurrent manner to download fragments, and the data fragment information is the size of the data file to be downloaded in the data download request. The service server queries the initial speed limit information corresponding to the user client. Based on the initial speed limit information and the data fragmentation information, the target speed limit information corresponding to the data download request is calculated. The calculation of the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragmentation information is used to set different speed limit values ​​for different data download requests from the same user for the same target data, based on the data fragmentation information of each data download request. Specifically, based on the initial speed limit information and the data fragmentation information, the target speed limit information corresponding to the data download request is calculated, including: When the service server detects that the data fragment information is empty, it uses the basic rate limit information as the target rate limit information corresponding to the data download request, wherein the basic rate limit information is the minimum rate limit value set in the initial rate limit information; or, When the business server detects that the data fragment information is not empty, it determines the token bucket corresponding to the data fragment information and the token capacity information of the token bucket. Based on the initial rate limit information and the token capacity information, it calculates the download rate information corresponding to the data download request. Based on the download rate information and the data fragment information, it calculates the download time information corresponding to the data download request. Based on the download rate information and the download time information, it generates the target rate limit information. Wherein, the token bucket corresponding to the data fragmentation information is the token bucket for processing the data download request; The download rate information corresponding to the data download request is calculated based on the initial rate limit information and the token capacity information, including: The initial speed limit information includes an initial speed limit value; the remaining total rate is calculated by subtracting the sum of the rates already distributed through the token bucket from the initial speed limit value, and the download rate information is equal to the remaining total rate multiplied by a calculation factor; The storage server adjusts the download parameters corresponding to the data download request based on the target rate limit information, and transmits the target data to the user client according to the adjusted download parameters.

2. The method as described in claim 1, wherein sending data fragment information corresponding to the data download request to the business server through the storage server includes: The storage server sends a download rate limiting request carrying data fragmentation information to the business server. The service server calculates the target rate limit information corresponding to the data download request based on the initial rate limit information and the data fragmentation information, including: The service server parses the download rate limit request to obtain the data fragment information, and calculates the target rate limit information corresponding to the data download request based on the initial rate limit information and the data fragment information, and sends the target rate limit information to the storage server.

3. The method as described in claim 2, wherein sending a download rate limiting request carrying data fragmentation information to the service server through the storage server includes: The system obtains the user client's service configuration information from the storage server, determines the service identification information corresponding to the user client based on the service configuration information, parses the data download request to obtain data identification information and data fragment information, generates a download rate limiting request based on the service identification information, the data identification information, and the data fragment information, and sends the download rate limiting request to the service server corresponding to the service identification information.

4. The method of claim 3, further comprising: The storage server generates an initial signature string based on the data identification information and the data fragmentation information, encrypts and encodes the initial signature string to obtain signature information, and writes the signature information and the initial signature string into the download speed limit request. The service server parses the download rate limit request to obtain the data fragment information, and then calculates the target rate limit information corresponding to the data download request based on the initial rate limit information and the data fragment information, including: The service server parses the download rate limit request to obtain the initial signature string and the signature information. The signature information is decrypted and encoded to obtain the target signature string. The download rate limit request is verified based on the target signature string and the initial signature string. If the verification is successful, the initial rate limit information corresponding to the user client is queried based on the data identification information carried in the download rate limit request. Then, the target rate limit information corresponding to the data download request is calculated based on the initial rate limit information and the data fragmentation information.

5. The method as described in claim 1, wherein the storage server adjusts the download parameters corresponding to the data download request according to the target rate limit information, including: The storage server parses the target speed limit information to obtain the download rate information, determines the download parameters corresponding to the data download request, adjusts the download speed limit value in the download parameters based on the download rate information, and continues to transmit the target data to the user client according to the adjusted download parameters.

6. The method of claim 5, further comprising: The storage server parses the target speed limit information to obtain the download time information, obtains the data transmission time corresponding to the target data, and if the data transmission time is detected to be greater than the download time information, the download speed limit value in the download parameters is adjusted based on the basic speed limit information, and the target data continues to be transmitted according to the adjusted download parameters.

7. The method of claim 1, further comprising: The service server receives traffic statistics information from the user client. If the traffic statistics information exceeds a preset traffic threshold, the service server determines that the user client is a rate-limited user client and continues to query the initial rate-limit information corresponding to the user client.

8. The method according to any one of claims 1-7, wherein the method further comprises: When the business server receives the address retrieval request submitted by the user client for the target data, the business server determines the storage server in the storage node cluster and the server address information corresponding to the storage server, and returns the server address information as the data download address to the user client, wherein the server address information is used by the user client to download the target data from the storage server.

9. The method according to any one of claims 1-7, wherein the method further comprises: When the business server receives a data storage request submitted by the user client for the target data, the business server determines the storage server in the storage node cluster according to a preset load balancing strategy, and the server address information corresponding to the storage server, and returns the server address information to the user client. The server address information is used by the user client to upload the target data to the storage server.

10. A storage system, the system comprising a service server and a storage server, wherein, The storage server is configured to respond to a data download request submitted by a user client for target data by sending data fragment information corresponding to the data download request to the business server, wherein the user client is a rate-limited user client, the data download request is a download request by the user client for the target data in a concurrent manner to download fragments, and the data fragment information is the size of the data file to be downloaded in the data download request; The service server is used to query the initial speed limit information corresponding to the user client, calculate the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragment information, and send the target speed limit information to the storage server. The calculation of the target speed limit information corresponding to the data download request based on the initial speed limit information and the data fragment information is used to set different speed limit values ​​for different data download requests from the same user for the same target data based on the data fragment information of each data download request. Specifically, based on the initial speed limit information and the data fragmentation information, the target speed limit information corresponding to the data download request is calculated, including: When the service server detects that the data fragment information is empty, it uses the basic rate limit information as the target rate limit information corresponding to the data download request, wherein the basic rate limit information is the minimum rate limit value set in the initial rate limit information; or, When the business server detects that the data fragment information is not empty, it determines the token bucket corresponding to the data fragment information and the token capacity information of the token bucket. Based on the initial rate limit information and the token capacity information, it calculates the download rate information corresponding to the data download request. Based on the download rate information and the data fragment information, it calculates the download time information corresponding to the data download request. Based on the download rate information and the download time information, it generates the target rate limit information. Wherein, the token bucket corresponding to the data fragmentation information is the token bucket for processing the data download request; The download rate information corresponding to the data download request is calculated based on the initial rate limit information and the token capacity information, including: The initial speed limit information includes an initial speed limit value; the remaining total rate is calculated by subtracting the sum of the rates already distributed through the token bucket from the initial speed limit value, and the download rate information is equal to the remaining total rate multiplied by a calculation factor; The storage server is used to adjust the download parameters corresponding to the data download request according to the target speed limit information, and transmit the target data to the user client according to the adjusted download parameters.

11. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor, when executing the computer instructions, performs the steps of the method according to any one of claims 1-9.

12. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1-9.

Images