Flow control method and system, storage medium and computer program product

Abstract

The invention discloses a flow control method and system, a storage medium and a computer program product, and the method comprises the steps: receiving a plurality of transmission requests on a target transmission link, and enabling the transmission requests to comprise transmission description information; the target transmission link is a link for transmitting data from a target source end to a target destination end; obtaining bandwidth configuration information corresponding to each transmission request according to the transmission description information contained in each transmission request; and based on the bandwidth configuration information corresponding to each transmission request, respectively allocating bandwidth limits to the plurality of transmission requests on the target transmission link. The bandwidth limit of the transmission link of the data transmission request is adjusted based on the bandwidth configuration information, differentiated bandwidth allocation modes for different users are achieved, bandwidth allocation of the transmission requests on the whole transmission link is more reasonable, and the method and the device have the advantages of being flexible and practical.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing, and in particular to a traffic control method and system, a storage medium and a computer program product. BACKGROUND

[0002] With the development of cloud computing technology, data is increasingly uploaded to the cloud and the amount of user data is growing explosively. In specific applications, it is often necessary to transfer a large amount of data, such as data migration, data backup and recovery, etc. Data migration refers to the process of moving data from one storage system to another. For example, as technology continues to update, new storage systems may be more efficient, secure and reliable, so data migration is needed to better utilize new technology. When performing data backup and recovery, data may need to be moved from one storage system or database to another storage system or database to ensure data security and recoverability. The goal of data transfer is to efficiently and quickly move data to avoid or minimize interference with applications.

[0003] Data transfer functions are usually hosted on cloud servers, which provide cloud services. As a public service, cloud services have the attribute of multi-tenancy, and different tenants have very different data transfer needs. In data transfer services, bandwidth is an important resource for data transfer. Currently, bandwidth is usually provided based on storage capacity, for example, if a tenant has 20 TB of storage in a storage system, the tenant is provided with a default bandwidth of 100 MBps. However, this fixed configuration of bandwidth cannot meet the flexible and variable bandwidth needs of tenants, and often results in redundant and wasteful bandwidth provided to tenants or insufficient bandwidth affecting data transfer for tenants. How to provide differentiated data transfer services under limited resources to meet different data transfer needs is a problem that needs to be solved. SUMMARY

[0004] The present application aims to provide a traffic control method, system, storage medium and computer program product to address the deficiencies of the prior art, which provides differentiated data transfer services through traffic control to meet different data transfer needs. The purpose is achieved through the following technical solutions.

[0005] The first aspect of the present application provides a traffic control method, the method comprising:

[0006] receiving a plurality of transmission requests on a target transmission link, the transmission request containing transmission description information; the target transmission link is a link for transmitting data from a target source end to a target destination end;

[0007] obtaining bandwidth configuration information corresponding to each transmission request according to the transmission description information contained in each transmission request;

[0008] allocate bandwidth quota for the multiple transmission requests on the target transmission link based on the bandwidth configuration information corresponding to each of the transmission requests.

[0009] The method of the first aspect has at least the following beneficial effects or advantages:

[0010] According to the transmission description information contained in the data transmission request, the bandwidth configuration information of the data transmission request when performing bandwidth allocation of the data transmission link is obtained, and the bandwidth quota of the transmission link of the data transmission request is adjusted based on the bandwidth configuration information, so that the differentiated bandwidth allocation mode for different data transmission requests is realized, and the bandwidth allocation of each transmission request on the entire transmission link is more reasonable.

[0011] The second aspect of the application provides a flow control system, which comprises a cloud storage management node and a cloud storage agent node; the cloud storage agent node comprises a task classifier and a scheduler:

[0012] The cloud storage management node is configured to receive multiple replication requests on a target transmission link, wherein the replication requests contain transmission description information; the target transmission link is a link for transmitting data from a target source end to a target destination end; and the replication requests are sent to the cloud storage agent node.

[0013] The task classifier is configured to obtain bandwidth configuration information corresponding to each of the replication requests according to the transmission description information contained in each of the replication requests.

[0014] The scheduler is configured to allocate bandwidth quota for the multiple replication requests on the target transmission link based on the bandwidth configuration information corresponding to each of the replication requests.

[0015] The flow control system of the second aspect has at least the following beneficial effects or advantages:

[0016] The flow control system of the embodiment of the application comprises a cloud storage management node and a cloud storage agent node, and the bandwidth configuration information of the data transmission request when performing bandwidth allocation of the data transmission link is obtained by the cloud storage management node according to the transmission description information contained in the replication request, and the bandwidth quota of the transmission link of the replication request is adjusted by the cloud storage agent node based on the bandwidth configuration information. Therefore, the bandwidth configuration information of the data replication request can be automatically determined based on the data replication request in the embodiment of the application, and then the bandwidth allocation is performed, so that the differentiated bandwidth allocation mode for different data replication requests is realized, and the bandwidth allocation of each data replication request on the entire transmission link is more reasonable.

[0017] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method according to the first aspect.

[0018] The fourth aspect of the present application provides a computer readable storage medium, having stored thereon a computer program, wherein the program is executed by a processor to implement the method according to the first aspect.

[0019] The fifth aspect of the present application provides a computer program product, comprising a computer program, wherein the computer program is executed by a processor to implement the method according to the first aspect.

[0020] The above description is only a summary of the technical solutions of the present application, in order to make the technical means of the present application more clearly understood, the present application can be implemented according to the content of the description, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 A flow chart of a flow control method according to an exemplary embodiment;

[0023] Figure 2 A structural schematic diagram of a flow control system according to an exemplary embodiment;

[0024] Figure 3 A structural schematic diagram of a cloud storage proxy node according to an exemplary embodiment;

[0025] Figure 4 A hardware structural schematic diagram of an electronic device according to an exemplary embodiment;

[0026] Figure 5 A structural schematic diagram of a storage medium according to an exemplary embodiment. DETAILED DESCRIPTION

[0027] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description of the exemplary embodiments is intended to apply to any embodiment of the application, unless specifically stated otherwise. It should be understood that the following description is not meant to limit the application to any particular kind of embodiment. Rather, the application is meant to encompass all embodiments that are within the scope of the claims.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] It should be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order or hierarchy. These terms are used only to distinguish one from another. For example, a first information can be termed a second information, and, similarly, a second information can be termed a first information, without departing from the scope of the present application. As used herein, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" taking into account the context in which the term is used.

[0030] Embodiments of the present application provide a flow control method, system, storage medium and computer program product. According to the transmission description information contained in the data transmission request, the bandwidth configuration information corresponding to the data transmission request is obtained, and the transmission link bandwidth quota of the data transmission request is adjusted based on the bandwidth configuration information. Thus, the bandwidth allocation of the data transmission request can be performed according to the bandwidth configuration information, and a differentiated bandwidth allocation mode for different users is realized, so that the bandwidth allocation of each data replication request on the entire transmission link is more reasonable. The flow control method and system provided by the present application can determine the weight value of the transmission request based on the bandwidth configuration information corresponding to the transmission request when the transmission link bandwidth is insufficient, and allocate bandwidth according to the weight value, thereby guaranteeing the implementation of the key data transmission request. When there is surplus bandwidth in the transmission link, the surplus bandwidth can be allocated completely, and bandwidth resource waste does not occur.

[0031] The technical solutions of the present application and how the technical solutions of the present application solve the foregoing technical problems will be described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described in detail below with reference to the drawings.

[0032] Figure 1 For a flow chart of a flow control method according to an exemplary embodiment, the method comprises the following steps:

[0033] Step 101: receiving a plurality of transmission requests on a target transmission link, wherein each of the transmission requests comprises transmission description information; the target transmission link is a link for transmitting data from a target source end to a target destination end;

[0034] Step 102: obtaining bandwidth configuration information corresponding to each of the transmission requests according to the transmission description information comprised in each of the transmission requests;

[0035] Step 103: allocating bandwidth quota to each of the plurality of transmission requests on the target transmission link based on the bandwidth configuration information corresponding to each of the transmission requests.

[0036] In some embodiments of the present application, the target source end and the target destination end can be different devices with data processing function or data storage function, which can be physical devices or virtual devices. For example, the target source end and the target destination end can be servers, storages or databases. In the embodiments of the present application, data transmission is a process of transmitting data from one place to another, which is commonly used in scenarios such as data backup, data migration, data synchronization or incremental data update after data migration. For example, in a specific application scenario, the embodiments of the present application need to migrate user data from a data center in city A to a data center in city B. The data transmission request can be initiated by a cloud service user based on a client or by an administrator based on a management device. The transmission description information comprised in the data transmission request can be a submission account of the data transmission request, a submission time, and the target source end information and the target destination end information. The target source end information can be source cluster information of data transmission, and the target destination end information can be destination cluster information of data transmission. The cluster can be a server cluster, a storage cluster or a database cluster. Of course, the above transmission description information is only an example, and other transmission description information can also be used in specific applications, which are not limited in the embodiments of the present application. For example, when data transmission is performed between storage nodes, the transmission description information can also be a source storage node of data transmission and a destination storage node of data transmission. Alternatively, the transmission description information can also include a source database of data transmission and a destination database of data transmission, which are not limited in the embodiments of the present application.

[0037] The bandwidth configuration information is used to represent the demand of the transmission request for bandwidth. The bandwidth configuration information can include one or more of a bandwidth size required to be used by the transmission request, a bandwidth proportion required to be used by the transmission request, a used bandwidth type of the transmission request, a priority of the transmission request, and user intervention information. The bandwidth size required to be used by the transmission request can be a demand bandwidth quota pre-set for the transmission request. The bandwidth proportion required to be used by the transmission request can be a ratio of the bandwidth size required to be used by the transmission request to a preset total bandwidth of the target transmission link. The used bandwidth type of the transmission request can include one or more of an uplink bandwidth, a downlink bandwidth, a shared bandwidth, and a dedicated bandwidth. The shared bandwidth refers to a bandwidth shared by multiple users using a same network resource. The dedicated bandwidth refers to a bandwidth with a fixed data transmission rate provided for a single user. The priority of the transmission request can be pre-set for the transmission request, and is used to represent a priority of the transmission request in using the bandwidth in the target transmission link. The user intervention information is used to represent whether the transmission request has an emergency degree set by a user in an emergency scenario. For example, if a machine room fails, a huge risk of data loss will occur if the storage data of the machine room is not migrated in an emergency. In this scenario, the user can set a high emergency degree for the transmission request.

[0038] In the embodiments of the present application, the bandwidth configuration information is used to determine a weight value or a weight proportion value of the bandwidth that can be used by the transmission request in the target transmission link. The weight value is used to represent a bandwidth size or a bandwidth proportion that can be used by the transmission request. The weight proportion value is used to represent a proportion of the weight value of the transmission request in a total weight value of the transmission requests in the target transmission link.

[0039] In some embodiments of the present application, the bandwidth configuration information corresponding to the transmission request can be obtained based on transmission description information included in the transmission request. Specifically, before a user initiates the transmission request, the user can set one or more of a bandwidth size, a bandwidth proportion, a used bandwidth type, a priority, and user intervention information required by the transmission request based on application demand. The transmission description information included in the transmission request can carry the information set by the user. After the system receives the transmission request, the system parses the transmission description information included in the transmission request, and extracts the information from the transmission description information as the bandwidth configuration information corresponding to the transmission request.

[0040] In other embodiments, a mapping relationship between a task type and bandwidth configuration information corresponding to the task can also be pre-configured in the system based on a demand of the task. After the user initiates a transmission request of a certain task type, the system receives the transmission request, determines the task type to which the transmission request belongs, and obtains the configuration information corresponding to the transmission request from the pre-configured mapping relationship between the task type and the bandwidth configuration information.

[0041] After obtaining the bandwidth configuration information corresponding to the transmission request in the manner described above, the weight value corresponding to the transmission request can be determined based on the bandwidth configuration information, so as to allocate bandwidth quota to the transmission request according to the weight value.

[0042] In some embodiments, the weight value of the transmission request can be calculated based on the bandwidth configuration information corresponding to the transmission request by a preset calculation rule. The preset calculation rule is used to specify the calculation method for calculating the weight value based on the information in the bandwidth configuration information, such as the required bandwidth size of the transmission request, the bandwidth proportion, the used bandwidth type, the priority, and the user intervention information. For example, if the user intervention information indicates that the transmission request has a very high emergency level, the weight value of the transmission request can be determined as a default higher weight value by the preset calculation rule. If the user intervention information indicates that the transmission request does not have a very high emergency level, the weight value of the transmission request can be obtained by normalizing each information included in the bandwidth configuration information, and then performing weighted summation or weighted average calculation processing on the values obtained by normalizing each information.

[0043] In the above manner, the weight value of the transmission request can be automatically determined based on the bandwidth configuration information, and then bandwidth allocation is performed according to the weight value, thereby realizing differentiated bandwidth allocation for different transmission requests, making the bandwidth allocation on the entire transmission link more reasonable. Moreover, the bandwidth configuration information comprehensively considers the various information of the transmission request, so that the determined weight value can be more matched with the bandwidth demand of the transmission request, improving the accuracy of allocating bandwidth for the transmission request.

[0044] In some embodiments of the present application, the weight value corresponding to each transmission request is obtained according to the bandwidth configuration information of each transmission request, which can also include: obtaining a task type corresponding to the transmission description information according to the transmission description information; and obtaining the weight value corresponding to each transmission request based on the task type to which each transmission request belongs.

[0045] For example, the task type corresponding to the submission account of the data transmission request is obtained. In this embodiment, the task type can be set differently according to application requirements. For example, in some application scenarios of the present application, the task type is set according to the priority of the data transmission request. For example, different submission accounts of different data transmission requests correspond to different priorities, and different priorities correspond to different task types. In this application scenario, the task type is: high-priority task and low-priority task. Alternatively, the task type can be distinguished according to the task source. In this scenario, the task type includes foreground task and background task. Alternatively, the task type can be distinguished according to the task emergency level. In this scenario, the task type includes emergency task and ordinary task. Of course, the task type can be set and adjusted differently according to actual application requirements.

[0046] In some application scenarios of the present application, a higher priority and a corresponding task type are assigned to a data transmission request initiated by a client, so that the traffic is preferentially allocated according to the task type; and another task type is used for a data transmission request initiated by a management device based on a manager, and the remaining traffic is used. In addition, in some emergency scenarios, the traffic priority of some data transmission requests can also be adjusted at any time for emergency operations. For example, when a machine room fails, if the storage data of the machine room is not migrated urgently, there will be a huge risk of data loss, so the data transmission request of the machine room can be adjusted to a high-level task type, and the bandwidth is preferentially allocated to the data migration operation of the machine room for this task type.

[0047] In the embodiments of the present application, the task type is determined according to the bandwidth configuration information of the data transmission request, the weight value of the transmission request is determined according to the task type, and then the bandwidth is allocated, so that differentiated data transmission services can be provided, and different data transmission requirements can be met.

[0048] In some embodiments of the present application, different tasks can be divided into different task types according to different application requirements, and a first mapping relationship between bandwidth configuration information and task types is pre-configured in the system. Different bandwidth configuration information is also set for each task type, and a second mapping relationship between the task type and the bandwidth configuration information is pre-configured in the system. When a user initiates a corresponding transmission request for a certain task, the system receives the transmission request, determines the bandwidth configuration information corresponding to the transmission request based on the transmission description information included in the transmission request, then obtains the task type corresponding to the transmission request from the first mapping relationship pre-configured above, and queries the bandwidth configuration information corresponding to the transmission request from the second mapping relationship based on the obtained task type.

[0049] In other embodiments, the task type to which the transmission request belongs can be automatically determined based on a pre-configured rule, which can be a comprehensive judgment based on various information included in the transmission description information and the bandwidth configuration information.

[0050] By the above manner, the task type to which the transmission request belongs is determined first, then the weight value corresponding to the task type is obtained, and then the bandwidth configuration is performed. The transmission request is classified based on the corresponding information of the transmission request, the transmission request of different task types has different weight values, and thus the calculation amount of the weight value corresponding to each transmission request is simplified by the classification of the transmission request, and the calculation efficiency is improved. The bandwidth is allocated based on the weight value corresponding to the task type, and the differentiated bandwidth allocation mode for different requests is realized, so that the bandwidth allocation on the entire transmission link is more reasonable. Moreover, when the task type is determined, the various information of the transmission request is comprehensively considered, so that the determined weight value can be more matched with the bandwidth demand of the transmission request, and the accuracy of allocating bandwidth for the transmission request is improved.

[0051] In some embodiments of the present application, the bandwidth quota is allocated to each of the plurality of transmission requests on the target transmission link based on the bandwidth configuration information corresponding to each transmission request, including:

[0052] In the case where the preset total bandwidth of the target transmission link is less than or equal to the total demand bandwidth of the plurality of transmission requests, the weight value corresponding to each transmission request is determined based on the bandwidth configuration information corresponding to each transmission request, and the bandwidth configuration information includes one or more of the bandwidth size required by the transmission request, the bandwidth proportion, the bandwidth type used, the priority of the transmission request, and the user intervention information; the bandwidth quota is allocated to each of the plurality of transmission requests on the target transmission link based on the preset total bandwidth and the weight value corresponding to each transmission request.

[0053] For example, in one specific application scenario of the present application, the priority of the data transmission request is set to include a first priority, a second priority, and a third priority, different task types are set according to different priorities, such as a first task type, a second task type, and a third task type, and different weight values corresponding to different task types are set, and the bandwidth of different data transmission requests is configured according to the weight value corresponding to the task type. In one specific application scenario of the present application, for data transmission requests on the same data transmission link, the data transmission requests are distinguished according to the submission account, different submission accounts correspond to different task types, and different task types correspond to different weight values, such as a submission account of the first task type corresponding to a weight value of 10, a submission account of the second task type corresponding to a weight value of 5, and a submission account of the third task type corresponding to a weight value of 1, wherein the weight value is an integer between 1 and 100, and the value between 0 and 1 is taken after normalization processing. In this application scenario, when it is judged according to the submission account of the data transmission request that the task type of the submission account is the first task type, the bandwidth quota is allocated to the data transmission request according to the weight value 10 when the bandwidth quota is allocated based on the preset total bandwidth of the transmission link.

[0054] In the embodiments of the present application, the preset total bandwidth of the transmission link is determined based on the transmission link determined based on a three-tuple of a target source end, a target destination end and a data transmission direction. Transmission links with the same target source end, target destination end and data transmission direction are the same transmission link, and the preset total bandwidth of the transmission link is allocated among multiple data transmission requests. For example, data is transmitted from a data center in city A to a data center in city B, and a data transmission link is determined. Similarly, data is transmitted from the data center in city A to a data center in city C, which is also a transmission link.

[0055] In the embodiments of the present application, a set of different weight values of 100 are set according to the task types of the data transmission requests, which are used to evaluate the allocation amount of the preset total bandwidth of the transmission link for each data transmission request. The higher the weight value, the more bandwidth can be used, so as to realize fine allocation of bandwidth. Through such fine and adaptive bandwidth allocation, the utilization rate of bandwidth can be improved. In the embodiments of the present application, the weight value is used to control the bandwidth amount allocated by the data transmission request. For example, when the weight value is 0, it means that no bandwidth is allocated. The weight value of important data transmission requests is larger, which differentially meets the various needs of users.

[0056] In the embodiments of the present application, bandwidth allocation is realized according to the task types and weight values of the data transmission requests, so that differentiated data transmission services can be provided, and different data transmission needs can be met. Moreover, the present application allocates bandwidth according to the weight value, so that fine management of the traffic of different users can be realized, and the application needs of different users can be met.

[0057] In some embodiments of the present application, the method further includes:

[0058] obtaining a plurality of product results of the preset total bandwidth and the weight values corresponding to the plurality of transmission requests, respectively;

[0059] if the product result corresponding to the transmission request is greater than or equal to the demand bandwidth amount corresponding to the transmission request, the demand bandwidth amount corresponding to the transmission request is taken as the bandwidth amount allocated to the transmission request;

[0060] if the product result corresponding to the transmission request is less than the demand bandwidth amount corresponding to the transmission request, the product result corresponding to the transmission request is taken as the bandwidth amount allocated to the transmission request.

[0061] In the embodiment, the bandwidth is allocated based on the product of the preset total bandwidth and the weight value corresponding to each of the plurality of transmission requests. In some cases, the bandwidth allocated to some transmission requests can be more than the required bandwidth of the transmission requests. In this case, the required bandwidth of the transmission request is used as the bandwidth allocated to the transmission request, so as to avoid waste of bandwidth resources. In other cases, the bandwidth allocated to some transmission requests can be less than the required bandwidth of the transmission requests. In this case, the weight value is used to determine that the importance of the transmission request is low, and the bandwidth allocated to the transmission request is less than the required bandwidth of the transmission request.

[0062] In the embodiment, the comparison and adjustment of the bandwidth allocated to the transmission request based on the weight value and the required bandwidth of the transmission request can ensure that the transmission request with high importance is given priority to obtain bandwidth satisfaction. In the case where the allocated bandwidth is more than the required bandwidth, the bandwidth is allocated based on the required bandwidth, so as to avoid waste of bandwidth. The application realizes fine management of traffic of different users.

[0063] In some embodiments of the application, in the case where the target transmission link has remaining bandwidth, a target transmission request with allocated bandwidth less than required bandwidth is determined from the plurality of transmission requests;

[0064] Based on the remaining bandwidth and the weight value corresponding to each of the target transmission requests, a supplementary bandwidth allocated to each of the target transmission requests is obtained;

[0065] For each of the target transmission requests, the supplementary bandwidth and the allocated bandwidth of the target transmission request are added to obtain an accumulated amount;

[0066] If the accumulated amount is greater than or equal to the required bandwidth corresponding to the target transmission request, the required bandwidth corresponding to the target transmission request is used as the bandwidth of the target transmission request;

[0067] If the accumulated amount is less than the required bandwidth corresponding to the target transmission request, the accumulated amount is used as the bandwidth of the target transmission request;

[0068] In the case where the target transmission link still has remaining bandwidth, the step of determining the target transmission request from the plurality of transmission requests is executed in a loop until the target transmission link has no remaining bandwidth.

[0069] In the embodiments of the present application, if there is idle bandwidth after the bandwidth of the user demand is allocated, the remaining bandwidth is allocated to the transmission request which is not satisfied with the bandwidth, so as to ensure that the key transmission request is allocated bandwidth resources preferentially and the bandwidth resources are fully utilized.

[0070] In some embodiments of the present application, the bandwidth configuration information corresponding to each transmission request is used to allocate bandwidth quota to each of the plurality of transmission requests on the target transmission link, including:

[0071] In the case that the preset total bandwidth of the target transmission link is greater than the total bandwidth demand of the plurality of transmission requests, the weight ratio value corresponding to each transmission request is determined based on the bandwidth configuration information corresponding to each transmission request, the weight ratio value is used to represent the proportion of the weight value of the transmission request in the total weight value of the transmission requests; the difference between the preset total bandwidth and the total bandwidth demand of the plurality of transmission requests is calculated; and the bandwidth quota is allocated to each of the plurality of transmission requests on the target transmission link based on the product of the difference and the weight ratio value corresponding to each transmission request.

[0072] In the embodiments of the present application, the remaining bandwidth can be allocated according to the weight ratio value in the case that there is remaining bandwidth before bandwidth allocation, which ensures that the task type with high priority obtains bandwidth preferentially and takes into account the fairness of bandwidth allocation, so that the bandwidth waste is avoided. For example, if all users are limited to use 100M bandwidth, but the actual transmission link has 200M bandwidth, the actual transmission link bandwidth is only 100M, and the other 100M bandwidth is wasted. The method of the embodiments of the present application can dynamically allocate the remaining bandwidth according to the weight ratio value, so that the bandwidth waste is avoided.

[0073] The embodiments of the present application can allocate bandwidth to the data transmission request according to the bandwidth configuration information, so as to realize the differentiated bandwidth allocation mode for different users and make the bandwidth allocation of each transmission request on the entire transmission link more reasonable. The flow control method and the flow control system provided by the present application can allocate bandwidth according to the weight value when the bandwidth of the transmission link is insufficient, so as to guarantee the implementation of the key data transmission request, and the remaining bandwidth can be allocated when the bandwidth of the transmission link is sufficient, so that the bandwidth resource waste is avoided. In the embodiments of the present application, the weight value and the weight ratio value are used to quantify whether to allocate bandwidth and how much bandwidth to allocate. The embodiments of the present application realize the fine bandwidth allocation, which better meets the application demand.

[0074] Reference Figure 2Fig. 1 shows a schematic diagram of a flow control system according to an exemplary embodiment. The present embodiment provides a flow control system, which comprises a cloud storage management node and a cloud storage proxy node; the cloud storage proxy node comprises a task classifier and a scheduler:

[0075] The cloud storage management node is configured to receive a plurality of replication requests on a target transmission link, wherein each replication request comprises transmission description information; the target transmission link is a link for transmitting data from a target source end to a target destination end; and the cloud storage management node is configured to send the replication requests to the cloud storage proxy node.

[0076] The task classifier is configured to obtain bandwidth configuration information corresponding to each replication request according to the transmission description information comprised in each replication request.

[0077] The scheduler is configured to allocate bandwidth quota to each replication request on the target transmission link based on the bandwidth configuration information corresponding to each replication request.

[0078] In some embodiments of the present application, the target source end and the target destination end are storage units arranged on a cloud storage system. The cloud storage system of the present embodiment is a hardware and software system for storing data, which comprises a plurality of storage clusters, and each storage cluster comprises a plurality of storage nodes. The plurality of storage nodes are configured to provide storage space for the data allocated thereto, so that a user can save a large amount of data. The cloud storage system of the present embodiment faces the storage access and sharing demand of massive data, and provides high-performance, high-reliability and scalable data storage and transmission capabilities based on multiple storage nodes, so as to realize the data transmission sharing of multiple users on the storage nodes.

[0079] In the present embodiment, the target source end can be a source storage cluster or a source storage node for data transmission, and the target destination end can be a destination storage cluster or a destination storage node for data transmission. That is, the target source end and the target destination end are different physical storage spaces.

[0080] In the embodiments of the present application, a cluster is a computer system that highly tightly cooperates to complete computing work through a set of loosely integrated computer software and / or hardware connection. In a certain sense, they can be regarded as a computer. The single computer in the cluster system is usually called a node, and is usually connected through a local area network, but other possible connection modes also exist. Cluster computers are usually used to improve the computing speed and / or reliability of single computers. Generally, the performance price ratio of cluster computers is much higher than that of single computers, workstations or supercomputers. A cluster is a set of independent computers that form a computer system through a high-speed network, and each cluster node is an independent server running its own process. For network users, the website backend is a single system that cooperates to provide system resources and system services to users.

[0081] The cloud storage management node receives the data replication request, stores the configuration information and the data replication request information into a database, splits the data replication request into subtasks, and invokes the cloud storage proxy node to perform corresponding operations according to a preset time. The configuration information includes bandwidth configuration information corresponding to the data transmission request, a task type corresponding to the bandwidth configuration information, and a weight value corresponding to the task type.

[0082] The cloud storage proxy node refers to an entity that performs a specific task in a computer system or network, and can be a software program or a hardware device.

[0083] In an application scenario of the present application, when there are multiple data replication requests of users, the multiple data replication requests have many differences, such as different users initiating different data replication requests, different urgency levels of the data replication requests, different data volumes required to be replicated, and the like. If the bandwidth quota is allocated to the multiple data replication requests of users on average, it is obviously impossible to meet the various situations and needs of users. Therefore, in the embodiments of the present application, the bandwidth configuration information of the data replication request can be determined according to the transmission description information of the multiple data replication requests of users, and the weight value corresponding to the data replication request is determined based on the bandwidth configuration information.

[0084] Specifically, the task type to which the data replication request belongs can be comprehensively judged based on the multiple aspects of information included in the bandwidth configuration information, the weight value pre-configured for the judged task type is acquired based on the judged task type, and the weight value is taken as the weight value corresponding to the data replication request. If the task type to which the data replication request belongs is not determined through the above manner, a default weight value pre-configured is acquired, and the default weight value is taken as the weight value corresponding to the data replication request.

[0085] In the embodiments of the present application, the data transmission request can be a data replication request, such as backup of a cloud disk, or migration of the entire cloud disk.

[0086] Referring to Figure 3 Fig. 1 shows a cloud storage proxy node structure diagram according to an exemplary embodiment. In this embodiment, the cloud storage proxy node includes a task classifier and a scheduler. The task classifier is configured to obtain bandwidth configuration information corresponding to each replication request based on the transmission description information contained in each replication request. The scheduler is configured to allocate bandwidth quota to each replication request on the target transmission link based on the bandwidth configuration information corresponding to each replication request.

[0087] In one specific application scenario of the embodiments, the priority of the replication request includes a first priority, a second priority, and a third priority. Different task types are set according to different priorities, such as a first task type, a second task type, and a third task type. The task classifier is configured to determine the bandwidth configuration information of each replication request based on the transmission description information contained in each replication request, and then determine the task type to which each replication request belongs based on the bandwidth configuration information of each replication request.

[0088] In some application scenarios, for replication requests on the same transmission link, the replication requests are distinguished according to the submission account. Different submission accounts correspond to different task types, and different task types correspond to different weight values. For example, if the replication request is determined to be of the first task type according to the submission account, the weight value corresponding to the replication request is 10. If the replication request is determined to be of the second task type according to the submission account, the weight value corresponding to the replication request is 5. If the replication request is determined to be of the third task type according to the submission account, the weight value corresponding to the replication request is 1. The weight value is an integer between 1 and 100, and after normalization, the value is between 0 and 1.

[0089] In this application scenario, the scheduler is configured to allocate bandwidth to different replication requests according to the weight value. If the task classifier determines that the submission account task type is the first task type according to the submission account of the replication request, the scheduler allocates a first bandwidth quota to the replication request according to the weight value 10 when allocating bandwidth quota based on the total bandwidth of the preset transmission link. If the task classifier determines that the submission account task type is the second task type according to the submission account of the replication request, the scheduler allocates a second bandwidth quota to the replication request according to the weight value 5 when allocating bandwidth quota based on the total bandwidth of the preset transmission link. If the task classifier determines that the submission account task type is the third task type according to the submission account of the replication request, the scheduler allocates a third bandwidth quota to the replication request according to the weight value 1 when allocating bandwidth quota based on the total bandwidth of the preset transmission link.

[0090] In the embodiments of the present application, the specific setting of the priority can be determined according to the application requirements, such as setting the first type of data transmission request as the first priority, the second type of data transmission request as the second priority, or setting the high priority, the medium priority, and the low priority. In the embodiments of the present application, the data replication request with the high priority is preferentially guaranteed to be allocated with sufficient bandwidth.

[0091] The traffic control system of the embodiments of the present application includes a cloud storage management node and a cloud storage proxy node. The cloud storage management node determines the bandwidth configuration information of the replication request according to the transmission description information contained in the replication request, and determines the weight value corresponding to the replication request based on the bandwidth configuration information. Alternatively, the task type to which the replication request belongs is determined based on the bandwidth configuration information, and the weight value corresponding to the replication request is determined based on the task type. The cloud storage proxy node adjusts the transmission link bandwidth quota of the replication request based on the weight value. Thus, the embodiments of the present application can allocate bandwidth to the replication request according to the bandwidth configuration information, and realize the differentiated bandwidth allocation mode for different users, so that the bandwidth allocation of each transmission request on the entire transmission link is more reasonable.

[0092] In the embodiments of the present application, the priority of the data replication request can be dynamically regulated, and can be dynamically adjusted during the running process of the cloud storage service end. The priority adjustment command can be generated according to the actual running condition without changing the program. Thus, the dynamic adjustment of the bandwidth allocation rule is realized, and the task type of the bandwidth allocation and the bandwidth configuration information corresponding to each task type are adjusted in real time according to the adjusted bandwidth allocation rule. The embodiments have the characteristics of convenience and flexibility, and can better meet the application requirements.

[0093] In another application scenario of the present application, if the task type is a data replication request initiated by a user based on a client, the traffic is preferentially allocated, and if the task type is a data replication request of an internal operator of the cloud device service end, the remaining traffic is used. It also includes adjusting the type of some tasks in an emergency scenario for emergency operation, such as adjusting the data replication request of a machine room to a high-level task when the machine room fails, so as to avoid the risk of data loss in an emergency.

[0094] In the embodiments of the present application, the cloud storage management node is used to manage the user replication request of an independent physical isolation unit, receives a replication request, generates a data replication task according to the transmission description information, divides the data replication task into a plurality of subtasks, and calls a plurality of cloud storage proxy nodes to perform corresponding operations when a preset time arrives. The management node is provided with a database, and the transmission description information and other task-related information, such as the execution time of the task, can be stored in the database. In the case of a small-scale private cloud application, only one cloud storage management node is independently operated. In the environment without a controller, that is, in the private cloud, the cloud storage management node is used to generate and issue configuration information.

[0095] In the embodiments of the present application, the number of cloud storage proxy nodes is set according to the number of storage nodes contained in the corresponding storage cluster. The cloud storage proxy node is internally provided with a task classifier and a scheduler, which are used to allocate bandwidth for different types of tasks. The replication request that can be processed by the cloud storage proxy node is derived from the information carried by the request and the public information saved by other terminals and can be used by all data replication requests. The cloud storage proxy node obtains and applies the latest configuration from the cloud storage management node each time the process is started. The cloud storage proxy node is deployed according to the storage cluster dimension, and each storage cluster contains a plurality of storage nodes. The number of cloud storage proxy nodes can be from 1 to N, and N is the number of storage nodes contained in the storage cluster.

[0096] In some embodiments of the present application, the bandwidth quota is allocated to each of the plurality of replication requests on the target transmission link based on the bandwidth configuration information corresponding to each of the replication requests, including:

[0097] In the case where the preset total bandwidth of the target transmission link is less than or equal to the total demand bandwidth of the plurality of replication requests, the weight value corresponding to each transmission request is determined based on the bandwidth configuration information corresponding to each transmission request, and the bandwidth configuration information includes one or more of the bandwidth size required by the transmission request, the bandwidth proportion, the bandwidth type used, the priority of the transmission request, and the user intervention information; and the bandwidth quota is allocated to each of the plurality of replication requests on the target transmission link based on the preset total bandwidth and the weight value corresponding to each of the replication requests.

[0098] In some embodiments of the present application, the scheduler is used to allocate the bandwidth quota to each of the plurality of replication requests on the target transmission link based on the preset total bandwidth and the weight value corresponding to each of the replication requests, including:

[0099] obtaining a plurality of product results of the preset total bandwidth and the weight value corresponding to each of the plurality of replication requests, respectively;

[0100] if the product result corresponding to the replication request is greater than or equal to the demand bandwidth quota corresponding to the replication request, taking the demand bandwidth quota corresponding to the replication request as the bandwidth quota allocated to the replication request;

[0101] if the product result corresponding to the replication request is less than the demand bandwidth quota corresponding to the replication request, taking the product result corresponding to the replication request as the bandwidth quota allocated to the replication request.

[0102] In some embodiments of the present application, the scheduler is further configured to, in the case that there is a remaining bandwidth quota in the target transmission link, determine a target replication request from the plurality of replication requests, the target replication request having an allocated bandwidth quota less than a demand bandwidth quota;

[0103] based on the remaining bandwidth quota and the weight value corresponding to each target replication request, respectively obtaining a supplementary bandwidth quota allocated to each target replication request;

[0104] for each target replication request, adding the supplementary bandwidth quota of the target replication request to the allocated bandwidth quota to obtain an accumulated quota;

[0105] if the accumulated quota is greater than or equal to the demand bandwidth quota corresponding to the target replication request, taking the demand bandwidth quota corresponding to the target replication request as the bandwidth quota of the target replication request;

[0106] if the accumulated quota is less than the demand bandwidth quota corresponding to the target replication request, taking the accumulated quota as the bandwidth quota of the target replication request;

[0107] In the case that there is still a remaining bandwidth quota in the target transmission link, the step of determining the target replication request from the plurality of replication requests is executed in a loop until there is no remaining bandwidth quota in the target transmission link.

[0108] In some embodiments of the present application, the scheduler is further configured to, based on the bandwidth configuration information corresponding to each replication request, respectively allocate a bandwidth quota to the plurality of replication requests on the target transmission link, comprising:

[0109] in the case that the preset total bandwidth of the target transmission link is greater than the total demand bandwidth of the plurality of replication requests, based on the bandwidth configuration information corresponding to each transmission request, respectively determining a weight proportion value corresponding to each transmission request, the weight proportion value being used to represent the proportion of the weight value of the transmission request in the total sum of the weight values of the transmission requests; calculating the difference between the preset total bandwidth and the total sum of the demand bandwidths of the replication requests;

[0110] The bandwidth quota is allocated to each of the plurality of replication requests on the target transmission link based on a product of the difference and a weight ratio value corresponding to each of the replication requests.

[0111] In some embodiments of the present application, when the cloud storage management nodes are multiple, the system further comprises:

[0112] a controller configured to send a flow control configuration file to the cloud storage management nodes, and configured to acquire running information of the cloud storage management nodes and the cloud storage proxy nodes, and generate a configuration instruction for adjusting the task type or the bandwidth configuration information according to the running information, and send the configuration instruction to the cloud storage management nodes;

[0113] the cloud storage management nodes are configured to receive the flow control configuration file sent by the controller, and persistently store the flow control configuration file, and update the task type or the bandwidth configuration information according to the configuration instruction.

[0114] When the cloud storage management nodes are multiple, each cloud storage management node has a partition, which can be distinguished by an availability zone.

[0115] a controller configured to receive a flow control configuration file, and persistently store the flow control configuration file in a preset database, and configured to send the updated flow control configuration file to each online cloud storage management node when the flow control configuration file is updated, and send the updated flow control configuration file to the online cloud storage management node when detecting that the offline target cloud storage management node is online.

[0116] The flow control configuration file includes setting information of the cloud storage management nodes and the cloud storage proxy nodes in the system, such as setting information of a task classifier and a scheduler in the cloud storage proxy nodes, which can include a node control strategy, a data processing strategy, a state reporting strategy, etc. The data processing strategy includes a strategy of the task classifier determining a task type of a transmission request, and a strategy of the scheduler allocating bandwidth for each transmission request, etc.

[0117] The controller sends the flow control configuration file to each online cloud storage management node, and sends the flow control configuration file when detecting that the offline cloud storage management node is online again, which can ensure that the flow control configuration file is sent to each cloud storage management node.

[0118] When the traffic management system of the embodiments of the present application further comprises a controller, the cloud storage management node provides a northbound interface for the controller to issue a configuration, and the configuration comprises task classifier settings and scheduler settings, etc. The configuration is stored in a remote database. Specifically, the cloud storage management node is configured to receive a flow control configuration file sent by the controller and persistently store the flow control configuration file to ensure that the configuration is not lost after the cloud storage management node fails.

[0119] In some other embodiments, the controller described above can not be deployed in the traffic control system, but the flow control configuration file can be configured and / or modified in the cloud storage management node by an operation and maintenance tool. Specifically, the cloud storage management node is configured to receive a configuration modification instruction submitted by the operation and maintenance tool and update the locally stored flow control configuration file based on the configuration modification instruction.

[0120] In some embodiments of the present application, the cloud storage proxy node is configured to, at startup, if the latest version of the flow control configuration file is obtained from the cloud storage management node, use the latest version of the flow control configuration file to allocate bandwidth quota for the plurality of replication requests; if the operation of obtaining the latest version of the flow control configuration file fails, retry the operation of obtaining the latest version of the flow control configuration file until the latest version of the flow control configuration file is obtained, and use the locally cached flow control configuration file in the retry process. In this way, it can be ensured that the cloud storage proxy node always has a flow control configuration file to use, so that the bandwidth can be allocated to the user's transmission request in time.

[0121] In the case where the controller is deployed in the traffic control system, the controller serves as a configuration center, centrally manages and issues configurations to the cloud storage management node. At the same time, the controller collects the running information of the cloud storage management node and the cloud storage proxy node, which is used to realize the observability and running function. The running information comprises: how much bandwidth is currently consumed by the cloud storage proxy node, how much available bandwidth is available, the current number of tasks, the current workload, the available bandwidth and the used bandwidth of the user level in the current available area of the cloud storage management node, and the current number of tasks, so as to realize the change of the task type and the weight information of the bandwidth allocation according to the system running condition. All information is persisted to an external remote database. The controller serves as a control center, and there are three nodes in total, one of which is a master node. The master node provides services externally, and the other two nodes are backup nodes. If the master node has a problem, the backup nodes will automatically take over, thereby ensuring the continuity and stability of the service.

[0122] The flow control system provided by the above embodiments can increase or decrease the number of cloud storage management nodes and / or cloud storage proxy nodes as needed, and has good universality and scalability. The flow control system can be applied to data replication scenarios, and can also be extended to data migration, data transmission, data recovery and other scenarios, has service robustness, can tolerate multiple levels of service failures, and has self-healing capability.

[0123] Other aspects of the flow control system of the embodiments of the present application can refer to the corresponding detailed description of the embodiments of the flow control method, which will not be repeated here.

[0124] The flow control system provided by the embodiments of the present application does not affect other nodes when a single cloud storage management node or a single cloud storage proxy node fails, reduces the probability of single storage cluster unavailability, and enhances service robustness. In the embodiments of the present application, the task type and bandwidth configuration information can be adjusted in real time, so that the embodiments of the present application can dynamically adjust the bandwidth of the data transmission service according to the application requirements, and can meet different data transmission requirements.

[0125] The working process of the flow control system in one specific application scenario of the embodiments of the present application will be described below.

[0126] The cloud storage management node uses a combination of edge triggering and horizontal triggering to provide a reliable and fast configuration effective channel. That is, after the administrator modifies the global configuration through the controller, the edge triggering configuration is issued to the cloud storage management node, and all online cloud storage management nodes can update the configuration. In addition, the background starts the horizontal triggering process to continuously observe the offline cloud storage management node, and if it is found to be online, the latest configuration is immediately issued. The specific execution of this process is as follows:

[0127] Step 201: The administrator modifies the global configuration through the controller.

[0128] Step 202: The controller synchronously persists the information to the remote database.

[0129] Step 203: The controller immediately issues the latest configuration to the cloud storage management node. The controller iterates through all cloud storage management nodes and issues the configuration in turn. If the issuance fails or times out, the cloud storage management node is skipped and a flag bit is recorded to indicate that the background horizontal triggering process is started.

[0130] When the horizontal triggering process is started, the controller executes step 203 at an interval of 1 minute. The interval time can also be set to other values, such as 1-10 minutes. Until all cloud storage management nodes successfully issue the configuration. The flag bit is removed.

[0131] The traffic control system in this application scenario can continue to run with the current configuration when the controller is unavailable or even completely offline. The cloud storage management node is offline for a short period of time, and the cloud storage proxy node can still run continuously with the configuration in the memory. The single-node cloud storage proxy node is offline, and the task completion is not affected. Therefore, the single cluster is unavailable: all cloud storage proxy nodes are offline and the corresponding cloud storage management node is offline.

[0132] In this application scenario, the cloud storage proxy node configuration information is only stored in the memory. After restarting, the latest configuration is obtained from the cloud storage management node side. If the cloud storage management node cannot be connected, the default value in the configuration file is used for running. The cloud storage proxy node has two key modules built-in: task classifier and scheduler.

[0133] The task classifier is used to identify the task type according to the characteristics of the data transmission request, and finally confirm the bandwidth allocation weight information. The characteristics of the data transmission request can include: the submission account of the data replication, the submission time, the source cluster of the data replication, the destination cluster of the data replication, and other information for comprehensive judgment. Its output is a weight value, and the higher the weight value, the more transmission link bandwidth can be used.

[0134] In this application scenario, the cloud storage proxy node workflow is as follows:

[0135] When the cloud storage proxy node starts, the configuration file is read as the default configuration.

[0136] The cloud storage proxy node calls the cloud storage management node to obtain the latest configuration information, which overrides the default configuration content. If the call to the storage management node fails, a background compensation thread is started, and the process is continuously retried until it is successful. The default configuration is used during the retry process.

[0137] When there is a new data transmission request, all task types are traversed, and when a task type matches the data transmission request, the traversal is stopped. The corresponding bandwidth allocation method in the task type is executed to obtain the weight value of the data transmission request.

[0138] After the traversal is completed, no task type is matched, and the default weight 0 is used.

[0139] The scheduler is used to determine how to allocate traffic between different data transmission requests. The scheduling algorithm supports dynamic configuration, and does not need to be recompiled or restarted to take effect.

[0140] The built-in algorithm of the scheduler works according to the following principles:

[0141] When the traffic is insufficient, the bandwidth is allocated according to the weight value until the bandwidth is exhausted.

[0142] When there is surplus traffic, the remaining bandwidth is allocated in proportion to the weight value.

[0143] The upper limit of bandwidth on the replication link is B(total), the weight of data transfer request i is represented as W(i), the bandwidth demand is represented as D(i), and the allocated bandwidth is represented as G(i).

[0144] The allocation algorithm when there is insufficient traffic is as follows:

[0145] Step 301: Calculate the set E of all data transfer requests whose bandwidths are not satisfied: such that D(e)-G(e)>0.

[0146] Wherein e is any one of the data transfer requests whose bandwidths are not satisfied, G(e) is the allocated bandwidth of the data transfer request, and D(e) is the bandwidth demand;

[0147] Step 302: The temporarily allocated bandwidth of data transfer request i is represented as T(i),

[0148]

[0149] Wherein bandwidth=B(total)

[0150] Step 303: The finally allocated bandwidth of data transfer request is G(i),

[0151]

[0152] Step 304: Calculate the excess bandwidth U(i)=T(i)-D(i).

[0153] After one round of allocation, the remaining bandwidth B(unused) is calculated:

[0154]

[0155] When the value is 0, the allocation algorithm ends.

[0156] When the remaining bandwidth B(unused)>0, steps 301-304 are continued to be executed, wherein bandwidth=B(unused).

[0157] It is known that when , the transmission link bandwidth has excess bandwidth before the allocation. At this time, the excess bandwidth B(extra) is allocated as follows:

[0158] The allocated bandwidth E(i) of data transfer request i:

[0159]

[0160] Wherein, i represents an arbitrary data transmission request, and n represents the number of all data transmission requests.

[0161] The traffic management system provided by the embodiments of the present application can dynamically configure the task type of the data transmission request and the corresponding bandwidth configuration information, and has the characteristic of good flexibility. The traffic management system provided by the embodiments of the present application has strong robustness and can tolerate multi-level faults of services. The traffic control system provided by the embodiments of the present application can adjust the priority of the data transmission request in an emergency, can handle the emergency, and thus has the characteristic of strong data transmission request guarantee capability. The traffic control system provided by the embodiments of the present application dynamically allocates bandwidth according to the weight value of the data transmission request when there is idle bandwidth in the data transmission link, and will not waste bandwidth resources, and has the characteristic of high traffic utilization rate. In the present application, fine management of the traffic of different users can be implemented, the needs of different users are met, and the stability and fairness of the cloud service are ensured.

[0162] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the related data need to comply with the relevant laws, regulations and standards of the country and region, and provide corresponding operation portal for the user to choose authorization or refusal.

[0163] The embodiments of the present application also provide an electronic device corresponding to the method provided by the preceding embodiments, to execute the above method.

[0164] Figure 4 A hardware structure diagram of an electronic device according to an exemplary embodiment is shown, which includes a communication interface 601, a processor 602, a memory 603 and a bus 604; wherein the communication interface 601, the processor 602 and the memory 603 complete the communication among each other through the bus 604. The processor 602 can execute the method described above by reading and executing the machine executable instructions in the memory 603 corresponding to the control logic of the method described above, and the specific content of the method is described in the above embodiments, which will not be repeated here.

[0165] The memory 603 mentioned in the present application can be any electronic, magnetic, optical or other physical storage system, and can contain stored information such as executable instructions, data, etc. Specifically, the memory 603 can be a RAM (Random Access Memory), a flash memory, a storage drive (such as a hard disk drive), any type of storage disk (such as an optical disk, a DVD, etc.), or similar storage medium, or a combination thereof. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 601 (which can be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. can be used.

[0166] The bus 604 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 603 is used to store programs, and the processor 602 executes the programs after receiving execution instructions.

[0167] The processor 602 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instructions in the form of software in the processor 602. The processor 602 mentioned above can be a general-purpose processor, including a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The method steps disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution.

[0168] The electronic device provided by the embodiments of the present application and the method provided by the embodiments of the present application have the same beneficial effects as the method they adopt, run or implement.

[0169] The present application also provides a computer readable storage medium corresponding to the method provided by the preceding embodiments, please refer to Figure 5 As shown in the figure, the computer readable storage medium is an optical disk 30, and a computer program (i.e. program product) is stored on the optical disk 30, and the computer program will execute the method provided by any of the preceding embodiments when run by the processor.

[0170] It should be noted that examples of the computer-readable storage medium can also include, but are not limited to, phase-change memories (PRAM), static random access memories (SRAM), dynamic random access memories (DRAM), other types of random access memories (RAM), read-only memories (ROM), electrically erasable programmable read-only memories (EEPROMs), flash memories or other optical, magnetic storage mediums, without limitation, which are not listed one by one here.

[0171] The computer-readable storage medium provided by the above-mentioned embodiments of the present application has the same beneficial effects as the method adopted, run or implemented by the application program stored therein, based on the same inventive concept as the method provided by the embodiments of the present application.

[0172] The embodiments of the present application also provide a computer program product corresponding to the method provided by the above-mentioned embodiments, which comprises a computer program executed by a processor to implement the method provided by the above-mentioned embodiments.

[0173] The computer program product provided by the above-mentioned embodiments of the present application has the same beneficial effects as the method adopted, run or implemented by the application program stored therein, based on the same inventive concept as the method provided by the embodiments of the present application.

[0174] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present application cover any and all variations of the application that come within the scope of the claims and their equivalents. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the application indicated by the following claims.

[0175] It should also be noted that the terms "comprising", "comprises", "including", "includes" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article or apparatus. Without limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article or apparatus that comprises the identified element.

[0176] The above-mentioned embodiments are only preferred embodiments of the present application, and are not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A flow control method, characterized by, The method comprises: receiving a plurality of transmission requests on a target transmission link, the transmission requests containing transmission description information; the target transmission link being a link for transmitting data from a target source end to a target destination end; obtaining bandwidth configuration information corresponding to each of the transmission requests according to the transmission description information contained in each of the transmission requests; allocating bandwidth quotas to the plurality of transmission requests on the target transmission link based on the bandwidth configuration information corresponding to each of the transmission requests.

2. The flow control method of claim 1, wherein, The step of allocating bandwidth quotas to the plurality of transmission requests on the target transmission link based on the bandwidth configuration information corresponding to each of the transmission requests comprises: in a case where a preset total bandwidth of the target transmission link is less than or equal to a total demand bandwidth of the plurality of transmission requests, determining a weight value corresponding to each of the transmission requests based on the bandwidth configuration information corresponding to each of the transmission requests, the bandwidth configuration information including one or more of a bandwidth size required to be used by the transmission request, a bandwidth proportion, a bandwidth type used, a priority of the transmission request, and user intervention information; allocating bandwidth quotas to the plurality of transmission requests on the target transmission link based on the preset total bandwidth and the weight value corresponding to each of the transmission requests.

3. The flow control method of claim 2, wherein, The step of allocating bandwidth quotas to the plurality of transmission requests on the target transmission link based on the preset total bandwidth and the weight value corresponding to each of the transmission requests comprises: obtaining a plurality of product results of the preset total bandwidth and the weight value corresponding to each of the transmission requests; if the product result corresponding to the transmission request is greater than or equal to a demand bandwidth quota corresponding to the transmission request, taking the demand bandwidth quota corresponding to the transmission request as the bandwidth quota allocated to the transmission request; if the product result corresponding to the transmission request is less than the demand bandwidth quota corresponding to the transmission request, taking the product result corresponding to the transmission request as the bandwidth quota allocated to the transmission request.

4. The flow control method of claim 3, wherein, The method further comprises: in a case where there is a remaining bandwidth quota on the target transmission link, determining a target transmission request from the plurality of transmission requests, the target transmission request having a demand bandwidth quota less than an allocated bandwidth quota; obtaining a supplementary bandwidth quota allocated to each of the target transmission requests based on the remaining bandwidth quota and the weight value corresponding to each of the target transmission requests; for each of the target transmission requests, accumulating the supplementary bandwidth quota and the allocated bandwidth quota of the target transmission request to obtain an accumulated quota; if the accumulated quota is greater than or equal to a demand bandwidth quota corresponding to the target transmission request, taking the demand bandwidth quota corresponding to the target transmission request as the bandwidth quota of the target transmission request; if the accumulated quota is less than the demand bandwidth quota corresponding to the target transmission request, taking the accumulated quota as the bandwidth quota of the target transmission request; in a case where there is still a remaining bandwidth quota on the target transmission link, returning to the step of determining the target transmission request from the plurality of transmission requests and performing the step cyclically until there is no remaining bandwidth quota on the target transmission link.

5. The flow control method according to any one of claims 1 to 4, characterized in that, The bandwidth quota is allocated to each of the multiple transmission requests on the target transmission link based on the bandwidth configuration information corresponding to each of the multiple transmission requests, including: In a case where the preset total bandwidth of the target transmission link is greater than the total bandwidth required by the multiple transmission requests, a weight ratio value corresponding to each of the multiple transmission requests is determined based on the bandwidth configuration information corresponding to each of the multiple transmission requests, and the weight ratio value is used to represent a proportion of a weight value of the transmission request in a total sum of weight values of the multiple transmission requests; A difference between the preset total bandwidth and the total bandwidth required by the multiple transmission requests is calculated; The bandwidth quota is allocated to each of the multiple transmission requests on the target transmission link based on a product of the difference and the weight ratio value corresponding to each of the multiple transmission requests.

6. A flow control system characterized by, The system comprises a cloud storage management node and a cloud storage agent node; the cloud storage agent node comprises a task classifier and a scheduler: The cloud storage management node is configured to receive multiple replication requests on a target transmission link, wherein the replication requests comprise transmission description information; the target transmission link is a link for transmitting data from a target source end to a target destination end; and the cloud storage management node is configured to send the replication requests to the cloud storage agent node. The task classifier is configured to obtain bandwidth configuration information corresponding to each of the replication requests according to the transmission description information included in each of the replication requests. The scheduler is configured to allocate a bandwidth quota to each of the multiple replication requests on the target transmission link based on the bandwidth configuration information corresponding to each of the replication requests.

7. The flow control system of claim 6, wherein, The bandwidth quota is allocated to each of the multiple transmission requests on the target transmission link based on the bandwidth configuration information corresponding to each of the multiple transmission requests, including: In a case where the preset total bandwidth of the target transmission link is less than or equal to the total bandwidth required by the multiple replication requests, a weight value corresponding to each of the multiple transmission requests is determined based on the bandwidth configuration information corresponding to each of the multiple transmission requests, and the bandwidth configuration information comprises one or more of a bandwidth size required by the transmission request, a bandwidth proportion, a bandwidth type used, a priority of the transmission request, and user intervention information. The bandwidth quota is allocated to each of the multiple replication requests on the target transmission link based on the preset total bandwidth and the weight value corresponding to each of the replication requests.

8. The flow control system of claim 7, wherein, The scheduler is configured to allocate a bandwidth quota to each of the multiple replication requests on the target transmission link based on the preset total bandwidth and the weight value corresponding to each of the replication requests, including: A plurality of product results of the preset total bandwidth and the weight value corresponding to each of the multiple replication requests are obtained; If the product result corresponding to the replication request is greater than or equal to a required bandwidth quota of the replication request, the required bandwidth quota of the replication request is taken as the bandwidth quota allocated to the replication request; If the product result corresponding to the replication request is less than the required bandwidth quota of the replication request, the product result corresponding to the replication request is taken as the bandwidth quota allocated to the replication request.

9. The flow control system of claim 8, wherein, The scheduler is further configured to determine, from the multiple replication requests, a target replication request whose allocated bandwidth quota is less than a required bandwidth quota, if there is a remaining bandwidth quota in the target transmission link; obtain, based on the remaining bandwidth quota and a weight value corresponding to each of the target replication requests, a supplementary bandwidth quota allocated to each of the target replication requests respectively; accumulate the supplementary bandwidth quota and the allocated bandwidth quota of the target replication request to obtain an accumulated quota for each of the target replication requests; if the accumulated quota is greater than or equal to the required bandwidth quota corresponding to the target replication request, take the required bandwidth quota corresponding to the target replication request as the bandwidth quota of the target replication request; if the accumulated quota is less than the required bandwidth quota corresponding to the target replication request, take the accumulated quota as the bandwidth quota of the target replication request; if there is still a remaining bandwidth quota in the target transmission link, perform the step of determining the target replication request from the multiple replication requests in a loop until there is no remaining bandwidth quota in the target transmission link.

10. The flow control system of any of claims 6-9, wherein, The scheduler is further configured to allocate, based on bandwidth configuration information corresponding to each of the replication requests, a bandwidth quota to each of the multiple replication requests on the target transmission link, including: if a preset total bandwidth of the target transmission link is greater than a total required bandwidth of the multiple replication requests, determine, based on the bandwidth configuration information corresponding to each of the transmission requests, a weight proportion value corresponding to each of the transmission requests, the weight proportion value being used to represent a proportion of a weight value of the transmission request in a total sum of weight values of the transmission requests; calculate a difference between the preset total bandwidth and the total required bandwidth of the replication requests; allocate, based on a product of the difference and the weight proportion value corresponding to each of the replication requests, a bandwidth quota to each of the multiple replication requests on the target transmission link.

11. The flow control system of any of claims 6-9, wherein, When there are multiple cloud storage management nodes, the system further comprises: a controller configured to send a flow control configuration file to the cloud storage management nodes, and to obtain running information of the multiple cloud storage management nodes and the cloud storage proxy nodes, and generate a configuration instruction for adjusting the bandwidth configuration information according to the running information, and send the configuration instruction to the cloud storage management nodes; the cloud storage management nodes are configured to receive the flow control configuration file sent by the controller, and to persistently store the flow control configuration file, and to update the bandwidth configuration information according to the configuration instruction.

12. The flow control system of claim 11, wherein the controller is configured to receive a flow control configuration file, and to persistently store the flow control configuration file in a preset database, and to send an updated flow control configuration file to each of the online cloud storage management nodes if the flow control configuration file is updated, and to send the updated flow control configuration file to the cloud storage management node that is online if a target cloud storage management node that is offline is detected to be online.

13. The flow control system of any one of claims 6-9, wherein The cloud storage proxy node is configured to, at startup, if the latest version of the flow control configuration file is obtained from the cloud storage management node, use the latest version of the flow control configuration file to allocate bandwidth quota for the plurality of replication requests; if the operation of obtaining the latest version of the flow control configuration file from the cloud storage management node fails, retry the operation of obtaining the latest version of the flow control configuration file until the latest version of the flow control configuration file is obtained, and use the locally cached flow control configuration file in the retry process.

14. The flow control system of any of claims 6-9, wherein, The cloud storage management node is configured to receive a configuration modification instruction submitted through an operation and maintenance tool, and update the locally stored flow control configuration file based on the configuration modification instruction.

15. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method of any of claims 1-5.

16. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any of claims 1-5.

17. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any of claims 1-5.

Images