Data sending method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a data sending method and device, a computer readable storage medium and electronic equipment, and relates to the field of computers.The method comprises the steps that the number of first tokens needed for sending a to-be-sent data packet is determined according to the data size of the to-be-sent data packet and a target corresponding relation, the target corresponding relationship is a corresponding relationship between a preset data volume and a token number; obtaining a second token number of the tokens in a preset token bucket; under the condition that the number of the second tokens is smaller than the number of the first tokens, the pre-borrowing tokens are supplemented into the token bucket, so that the number of the second tokens is larger than or equal to the number of the first tokens; and sending the to-be-sent data packet under the condition that it is determined that the tokens with the first number of tokens are deducted from the token bucket. According to the method provided by the invention, by designing a token pre-borrowing mechanism, the situation of data transmission interruption caused by insufficient tokens in the token bucket can be avoided, and the continuity of data transmission is ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a method and device for sending data, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the development of cloud computing and big data technology, distributed storage systems are widely used in various data centers due to their advantages such as high scalability, high availability and high concurrency. In distributed storage systems, data migration is often required between different storage pools, which consumes network bandwidth resources. In order to avoid data migration occupying too much network resources, it is necessary to limit the network traffic of data migration.

[0003] In the related art, the token bucket algorithm is usually used to control network traffic. The token bucket algorithm is a time-based traffic control algorithm that periodically adds a certain number of tokens to the bucket by presetting a fixed time interval (i.e., the token generation rate). When a data packet needs to be sent, if there are enough tokens in the bucket, the data packet can be sent, otherwise it needs to wait for a period of time and try again. This effectively limits network traffic.

[0004] However, the related technical approach is prone to data transmission interruption when performing large-scale data migration. Summary of the invention

[0005] The embodiments of the present application provide a method, device, computer-readable storage medium and electronic device for sending data, which can improve the continuity of data transmission.

[0006] According to one embodiment of the present application, a method for data sending is provided, the method comprising: determining a first token quantity required to send the data packet to be sent based on the data volume of the data packet to be sent and a target correspondence relationship, wherein the target correspondence relationship is a correspondence relationship between a preset data volume and a token quantity; obtaining a second token quantity of tokens in a preset token bucket; when the second token quantity is less than the first token quantity, replenishing pre-borrowed tokens into the token bucket so that the second token quantity is greater than or equal to the first token quantity; and when it is determined that the tokens of the first token quantity have been deducted from the token bucket, sending the data packet to be sent.

[0007] In an exemplary embodiment, when the second token number is less than the first token number, replenishing the pre-borrowed tokens into the token bucket includes: determining the token difference based on the first token number and the second token number; and replenishing the pre-borrowed tokens into the token bucket when the token difference is less than or equal to the maximum pre-borrowed token number.

[0008] In an exemplary embodiment, before obtaining the second number of tokens in a preset token bucket, the data sending method also includes: initializing relevant parameters of the token bucket, wherein the relevant parameters of the token bucket include the capacity of the token bucket, the token generation rate of the token bucket, and the maximum pre-borrowed number of the pre-borrowed tokens.

[0009] In an exemplary embodiment, the method for sending data further includes:

[0010] Acquire performance parameters of a transmission network, wherein the transmission network is used to transmit data between a source storage volume and a target storage volume, and the performance parameters include network available bandwidth, network delay, and network packet loss rate;

[0011] According to the performance parameter, evaluating the state of the transmission network, wherein the state of the transmission network includes at least one of the following: a good state, a medium state, and a congested state;

[0012] In a case where the network available bandwidth is greater than a bandwidth threshold, the network delay is lower than a delay threshold, and the network packet loss rate is lower than a packet loss rate threshold, determining that the state of the transmission network is a good state;

[0013] When determining that the state of the transmission network is in a good state, increasing the token generation rate of the token bucket based on the current token generation rate of the token bucket;

[0014] In a case where the network available bandwidth is less than the bandwidth threshold, or the network delay is lower than the delay threshold, or the network packet loss rate is lower than the packet loss rate threshold, determining that the state of the transmission network is a medium state;

[0015] When it is determined that the state of the transmission network is a medium state, adjusting the token generation rate of the token bucket within a preset range of the current token generation rate of the token bucket;

[0016] When the available bandwidth of the network is greater than or equal to the bandwidth threshold, the network delay is higher than the delay threshold, and the network packet loss rate is higher than the packet loss rate threshold, determining that the state of the transmission network is a congested state;

[0017] When it is determined that the state of the transmission network is a congested state, the token generation rate of the token bucket is reduced based on the current token generation rate of the token bucket.

[0018] In an exemplary embodiment, the method for sending data further includes:

[0019] Monitoring network traffic of the transmission network in real time;

[0020] When the network traffic is greater than a preset threshold, re-obtaining the performance parameters of the transmission network;

[0021] Based on the performance parameters of the transmission network, a status of the transmission network is re-evaluated.

[0022] In an exemplary embodiment, before determining the number of first tokens required to send the data packet to be sent according to the data volume of the data packet to be sent, the method for data sending further includes:

[0023] Get the parameters of the source storage volume and the target storage volume;

[0024] Determining configuration parameters of the data migration task based on parameters of the source storage volume and the target storage volume;

[0025] Establishing a data migration thread between the source storage volume and the target storage volume according to the configuration parameters of the data migration task;

[0026] The data migration thread is executed to read the data of the source storage volume as the data packet to be sent.

[0027] In an exemplary embodiment, when the second token quantity is less than the first token quantity, after replenishing the pre-borrowed tokens into the token bucket so that the second token quantity is greater than or equal to the first token quantity, the method for sending data further includes:

[0028] Recording the number of third tokens of pre-borrowed tokens that have been replenished in the token bucket;

[0029] When it is determined that the first number of tokens has been deducted from the token bucket, if the second number of tokens in the token bucket is greater than the third number of tokens, the third number of tokens is deducted from the token bucket.

[0030] According to another embodiment of the embodiments of the present application, a device for sending data is further provided, the device comprising:

[0031] A first quantity determination module, used to determine the first number of tokens required to send the data packet to be sent according to the data volume of the data packet to be sent and the preset data volume and target correspondence relationship, wherein the target correspondence relationship is the preset correspondence relationship between the data volume and the number of tokens;

[0032] A second quantity determination module, used to obtain a second token quantity of tokens in a preset token bucket;

[0033] a token pre-borrowing module, configured to replenish pre-borrowed tokens into the token bucket when the second token quantity is less than the first token quantity, so that the second token quantity is greater than or equal to the first token quantity;

[0034] The data sending module is used to send the data packet to be sent when it is determined that the tokens of the first token quantity have been deducted from the token bucket.

[0035] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0036] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0037] According to another embodiment of the present application, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0038] According to the data transmission method of the present application, first, the first token number required for sending the data packet to be sent is determined according to the data volume of the data packet to be sent and the target correspondence. Then, the second token number of the tokens in the preset token bucket is obtained to determine whether the second token number in the token bucket is sufficient to support the transmission of the data packet. In the case where the second token number is less than the first token number, it means that the token number in the token bucket is insufficient, but in order to ensure the continuity of data transmission, the token bucket is supplemented with pre-borrowed tokens so that the second token number is greater than or equal to the first token number, so that the data packet can be sent. Then, when it is determined that the token number of the first token has been deducted from the token bucket, the data packet to be sent is sent. In summary, the method of the present application can avoid the situation where data transmission is interrupted due to insufficient tokens in the token bucket by designing a pre-borrowed token mechanism, thereby ensuring the continuity of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 It is a hardware structure block diagram of a server device of a data sending method according to an embodiment of the present application;

[0041] Figure 2 is a flow chart of a method for sending data according to an embodiment of the present application;

[0042] Figure 3 is a flow chart of a method for replenishing pre-borrowed tokens according to an embodiment of the present application;

[0043] Figure 4 is a flow chart of a method for adjusting a token generation rate of a token bucket according to an embodiment of the present application;

[0044] Figure 5 is a flow chart of a method for evaluating the status of a transmission network according to an embodiment of the present application;

[0045] Figure 6 is a flow chart of a method for preparing data transmission according to an embodiment of the present application;

[0046] Figure 7 is a flow chart of a method for returning a pre-borrowed token according to an embodiment of the present application;

[0047] Figure 8 is a structural block diagram of a data sending device according to an embodiment of the present application;

[0048] Fig. 9 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0050] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0051] The data transmission method embodiment provided in the embodiment of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a data transmission method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0052] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for data transmission in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0053] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0054] In one embodiment, a method for sending data is provided, such as Figure 2 As shown, the method includes the following steps S200-230:

[0055] Step S200, determining the number of first tokens required to send the data packet to be sent according to the data volume of the data packet to be sent and the target correspondence relationship.

[0056] Among them, the target correspondence is the correspondence between the preset data volume and the number of tokens. The target correspondence can be pre-set. For example, it can be pre-designed that one token corresponds to 1 Byte of data (of course, one token can also correspond to other bytes of data, and this application does not impose any restrictions on this). Assuming that the data volume of the data packet to be sent is 4MB, the required tokens are 4*1048576. It is also possible to set a suitable target correspondence according to the actual situation and application scenario. Therefore, after obtaining the data packet to be sent, the data volume of the data packet to be sent can be substituted into the target correspondence to obtain the first number of tokens required to send the data packet to be sent.

[0057] It should be noted that the token bucket algorithm is an algorithm used to control network traffic. It is a time-based traffic control algorithm that periodically adds a certain number of tokens to the token bucket by presetting a fixed time interval. When a data packet needs to be sent, if there are enough tokens in the token bucket, the data packet can be sent. Otherwise, it is necessary to wait until enough tokens are generated in the token bucket before sending the data packet. This algorithm can effectively limit network traffic and avoid congestion.

[0058] Step S210, obtaining a second token quantity of tokens in a preset token bucket.

[0059] The preset token bucket can be initialized in advance, that is, the relevant parameters of the token bucket can be initialized, and the relevant parameters of the token bucket include the capacity of the token bucket, the token generation rate of the token bucket, and the maximum number of pre-borrowed tokens. The initial relevant parameters of the token bucket can be determined based on the parameters of the transmission network, such as the bandwidth, delay, packet loss rate, etc. of the transmission network.

[0060] Step S220: When the second token quantity is less than the first token quantity, add pre-borrowed tokens to the token bucket so that the second token quantity is greater than or equal to the first token quantity.

[0061] Specifically, when the number of second tokens is less than the number of first tokens, it means that the number of tokens in the token bucket is insufficient to send the data packets to be sent. In this case, in order to ensure that the data packets to be sent can be sent in a timely manner and to ensure the continuity of data transmission, pre-borrowed tokens are added to the token bucket so that the number of second tokens is greater than or equal to the number of first tokens.

[0062] Step S230: When it is determined that the first token quantity has been deducted from the token bucket, the data packet to be sent is sent.

[0063] It is understandable that in the token bucket algorithm, the token is a mark that controls whether data transmission is executed in the distributed storage system. Only when it is determined that the first number of tokens has been deducted from the token bucket can the operation of sending the data packet to be sent be executed.

[0064] In this embodiment, first, the first token number required for sending the data packet to be sent is determined based on the data volume of the data packet to be sent and the target correspondence. Then, the second token number of the tokens in the preset token bucket is obtained to determine whether the second token number in the token bucket is sufficient to support the sending of the data packet. In the case where the second token number is less than the first token number, it means that the number of tokens in the token bucket is insufficient, but in order to ensure the continuity of data transmission, pre-borrowed tokens are added to the token bucket so that the second token number is greater than or equal to the first token number, so that the data packet can be sent. Then, when it is determined that the tokens of the first token number have been deducted from the token bucket, the data packet to be sent is sent. In summary, the method of the present application can avoid the situation where data transmission is interrupted due to insufficient tokens in the token bucket by designing a pre-borrowed token mechanism, thereby ensuring the continuity of data transmission.

[0065] In one embodiment, Figure 3 As shown, step S220, when the second token quantity is less than the first token quantity, replenishing the pre-borrowed tokens into the token bucket includes: steps S300-S310. Among them:

[0066] Step S300, determining a token difference according to the first token quantity and the second token quantity.

[0067] The token difference may be determined according to the difference between the first token quantity and the second token quantity, and the token difference is the quantity of tokens that need to be added to the token bucket.

[0068] Step S310: When the token difference is less than or equal to the maximum pre-borrowed token quantity, the token bucket is replenished with pre-borrowed tokens.

[0069] Among them, the maximum number of pre-borrowed tokens is pre-defined, and the maximum number of pre-borrowed tokens can be determined based on the business volume of the distributed storage system. Because data migration involves multiple links such as data replication, transmission and recovery, it consumes a lot of network bandwidth resources. In the process of data migration, it is also necessary to ensure that other businesses in the distributed storage system are not affected or interrupted, so it is necessary to limit the traffic of data migration. This is also the significance of the token bucket algorithm. Therefore, the number of pre-borrowed tokens is not unlimited, and the maximum number of pre-borrowed tokens still needs to be set to avoid excessive network bandwidth resources occupied by data transmission and affecting other businesses.

[0070] Among them, when the token difference is greater than the maximum pre-borrowed number of pre-borrowed tokens, it is still necessary to wait for the token generation in the token bucket until the token difference is less than or equal to the maximum pre-borrowed number of pre-borrowed tokens, and then the pre-borrowed tokens can be added to the token bucket.

[0071] Among them, the maximum pre-borrowed quantity A of the pre-borrowed tokens is also updated in real time according to the actual situation. For example, when transmitting the previous data packet, a part of the pre-borrowed tokens B has been pre-borrowed. If pre-borrowed tokens are still needed when transmitting the next data packet, the maximum pre-borrowed quantity of the pre-borrowed tokens is A minus B.

[0072] In this embodiment, the token difference is first determined based on the number of first tokens and the number of second tokens, and then the pre-borrowed tokens are added to the token bucket when the token difference is less than or equal to the maximum pre-borrowed number of pre-borrowed tokens. The pre-borrowed token mechanism is thereby utilized to ensure the continuity of data migration, and data migration can be ensured even when network conditions are poor. This is suitable for scenarios with unstable network resources, such as data synchronization in a mobile network environment.

[0073] In one embodiment, Figure 4 As shown, the method for sending data further includes: steps S400-S470, wherein:

[0074] Step S400: acquiring performance parameters of the transmission network.

[0075] The transmission network is used to transmit data between the source storage volume and the target storage volume. The performance parameters include network available bandwidth, network latency, and network packet loss rate. In a distributed storage system, data migration is a complex and critical task that involves multiple links such as data replication, transmission, and recovery. Data transmission is carried out through the transmission network, transferring data from the source storage volume to the target storage volume.

[0076] Exemplarily, the current performance parameters of the transmission network can be monitored through the Simple Network Management Protocol (SNMP), which is a protocol for network management that can collect performance data of network devices (such as routers, switches, etc.), including but not limited to bandwidth utilization, network latency, and packet loss rate.

[0077] Step S410: Evaluate the state of the transmission network according to the performance parameters.

[0078] The state of the transmission network includes at least one of the following: a good state, a medium state, and a congested state. A good state indicates that the transmission network has sufficient bandwidth, low latency, low packet loss rate, and high network operation efficiency. A medium state indicates that the transmission network has average performance and may have certain bandwidth limitations or delays. A congested state indicates that the transmission network has tight bandwidth, high latency, increased packet loss rate, and decreased network performance.

[0079] Step S420: When the network available bandwidth is greater than the bandwidth threshold, the network delay is lower than the delay threshold, and the network packet loss rate is lower than the packet loss rate threshold, it is determined that the state of the transmission network is in a good state.

[0080] Among them, a bandwidth threshold can be designed, for example, 30% of the total bandwidth of the transmission network is used as the bandwidth threshold. When the network available bandwidth is greater than the bandwidth threshold, it means that the bandwidth is sufficient. A delay threshold can be set, for example, 100ms. When the network delay is lower than the delay threshold, it means that the delay is low. A packet loss rate threshold can be set, for example, 1%. When the network packet loss rate is lower than the packet loss rate threshold, it means that the network has a high data transmission success rate. Therefore, when the network available bandwidth is greater than the bandwidth threshold, the network delay is lower than the delay threshold, and the network packet loss rate is lower than the packet loss rate threshold, the transmission network is in good condition.

[0081] Step S430: when it is determined that the state of the transmission network is in a good state, the token generation rate of the token bucket is increased based on the current token generation rate of the token bucket.

[0082] Among them, the transmission network condition is evaluated as good, which means that the network bandwidth is sufficient, the delay is low, and the packet loss rate is low. At this time, the token generation rate can be increased. The token in the token bucket algorithm represents the amount of data allowed to be sent through the network. Increasing the token generation rate means allowing more data to be migrated, thereby speeding up data migration.

[0083] Step S440: When the network available bandwidth is less than the bandwidth threshold, or the network delay is lower than the delay threshold, or the network packet loss rate is lower than the packet loss rate threshold, it is determined that the state of the transmission network is a medium state.

[0084] Among them, when the network available bandwidth is less than the bandwidth threshold, or the network delay is lower than the delay threshold, or the network packet loss rate is lower than the packet loss rate threshold, it means that the performance of the transmission network is general, and there may be certain bandwidth limitations or delays, and the status of the transmission network is determined to be medium.

[0085] Step S450: When it is determined that the state of the transmission network is a medium state, the token generation rate of the token bucket is adjusted within a preset range of the current token generation rate of the token bucket.

[0086] The transmission network condition is evaluated as medium, and the storage system may maintain the current token generation rate or make small adjustments. This is because the network condition is not the best, but it is not the worst, and it is necessary to balance the data migration speed and network load.

[0087] Step S460: When the network available bandwidth is greater than or equal to the bandwidth threshold, the network delay is higher than the delay threshold, and the network packet loss rate is higher than the packet loss rate threshold, it is determined that the state of the transmission network is a congested state.

[0088] Among them, when the network available bandwidth is greater than or equal to the bandwidth threshold, the network delay is higher than the delay threshold, and the network packet loss rate is higher than the packet loss rate threshold, it means that the network bandwidth is tight, the delay is high, and the packet loss rate is increased, that is, the performance of the transmission network is poor, so the state of the transmission network is determined to be a congested state.

[0089] Step S470: When it is determined that the state of the transmission network is a congested state, the token generation rate of the token bucket is reduced based on the current token generation rate of the token bucket.

[0090] If the network condition is evaluated as congested, that is, the network bandwidth is tight, the delay is high, and the packet loss rate is high, the storage system will reduce the token generation rate. This is done to reduce the load on the network and prevent the network performance from further deteriorating due to excessive data migration.

[0091] In this embodiment, by monitoring the performance parameters of the network and then classifying the network conditions, it is possible to quickly respond to network changes, dynamically adjust the token generation rate, and more accurately control the rate of data migration. While ensuring the efficiency of data migration, it is possible to avoid excessive pressure on the network, thereby ensuring the stability of the network and the reliability of data migration.

[0092] In one embodiment, Figure 5 As shown, the method for sending data also includes: steps S500-S520. Among them:

[0093] Step S500, monitoring the network traffic of the transmission network in real time.

[0094] Among them, the real-time network traffic of the transmission network can indicate the current business volume of the transmission network, so real-time monitoring of the network traffic of the transmission network can determine the current business volume of the transmission network in real time.

[0095] Step S510: when the network traffic is greater than a preset threshold, reacquire the performance parameters of the transmission network.

[0096] Among them, when the network traffic is greater than the preset threshold, it means that the current business volume of the transmission network is too large, and the performance parameters of the transmission network need to be re-evaluated to balance the network resource allocation between data migration and other services of the transmission network.

[0097] Step S520: re-evaluate the state of the transmission network based on the performance parameters of the transmission network.

[0098] The status of the transmission network may be evaluated according to the method in the above embodiment, which will not be described in detail.

[0099] In this embodiment, the network traffic of the transmission network is monitored in real time, and the status of the transmission network is quickly responded to and evaluated based on the network traffic, so that the token generation rate can be quickly adjusted according to the actual situation of the transmission network to ensure the stability of the network and the reliability of data migration.

[0100] In one embodiment, Figure 6 As shown, in step S200, the number of first tokens required to send the data packet to be sent is determined according to the data volume of the data packet to be sent and the target correspondence relationship. Before, the method for data sending also includes: steps S600-S630. Among them:

[0101] Step S600: Obtain parameters of the source storage volume and the target storage volume.

[0102] Among them, the location, capacity, etc. of the source storage volume and the target storage volume may be obtained.

[0103] Step S610: determining configuration parameters of the data migration task based on parameters of the source storage volume and the target storage volume.

[0104] After obtaining the parameters of the source storage volume and the target storage volume, specific parameters of the migration task may be configured, which may include the size of the migrated data, the migration priority, and the like.

[0105] Step S620: establishing a data migration thread between the source storage volume and the target storage volume according to the configuration parameters of the data migration task.

[0106] Among them, one or more threads may be created for executing data migration in a block device service system (DBS) provided by a distributed storage system according to configuration parameters of the data migration task. A thread is the smallest unit that an operating system can schedule operations on, and each thread usually represents an independent execution process.

[0107] Step S630: execute the data migration thread to read the data of the source storage volume as the data packet to be sent.

[0108] Specifically, the data migration thread is executed to read the data blocks to be migrated from the source storage volume. A snapshot of the data blocks to be migrated may also be generated to record the state of the data blocks at a certain moment, so as to ensure the consistency of data migration.

[0109] In this embodiment, a data migration operation process is provided to facilitate reading of the data packet to be sent and to implement subsequent data migration.

[0110] In one embodiment, Figure 7 As shown, in step S220, when the second token quantity is less than the first token quantity, the token bucket is supplemented with pre-borrowed tokens so that the second token quantity is greater than or equal to the first token quantity. Afterwards, the method for data transmission further includes: steps S700-S710. Among them:

[0111] Step S700, recording the third token quantity of the pre-borrowed tokens that have been replenished in the token bucket.

[0112] Among them, after each pre-borrowed token is replenished in the token bucket, the third token quantity of the pre-borrowed token replenished in the token bucket is recorded. For example, when sending a data packet at the last moment, the number of pre-borrowed tokens replenished in the token bucket is C, then C is recorded.

[0113] Step S710, when it is determined that the first number of tokens has been deducted from the token bucket, if the second number of tokens in the token bucket is greater than the third number of tokens, then the third number of tokens is deducted from the token bucket.

[0114] Among them, when there are enough tokens in the token bucket, in addition to deducting the tokens of the first token quantity corresponding to the data packet when sending a data packet, it is also necessary to deduct the pre-borrowed tokens of the third token quantity. For example, the second token quantity in the token bucket is D, the first token quantity corresponding to the current data packet to be sent is E, and the third token quantity of the pre-borrowed tokens that have been replenished in the token bucket is C. If D>E+C, then when sending the data packet, E+C tokens are deducted from the token bucket, which means that the pre-borrowed tokens need to be returned when there are sufficient tokens.

[0115] In this embodiment, by recording the number of third tokens of the pre-borrowed tokens that have been replenished in the token bucket, the number of pre-borrowed tokens that have been pre-borrowed is recorded. Then, when there are enough tokens in the token bucket, when sending a data packet, in addition to deducting the tokens of the first token number corresponding to the data packet, it is also necessary to deduct the pre-borrowed tokens of the third token number, so that the pre-borrowed pre-borrowed tokens are returned, ensuring that the total traffic used for transmitting data remains unchanged. The pre-borrowed token mechanism is only a mechanism for maintaining the continuity of data transmission when the network status is poor, and will not affect the total traffic used for transmitting data, thereby ensuring that data migration will not affect the normal business of the storage system, and improving the stability and reliability of the storage system's business operations.

[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0117] In the present embodiment, a device for data transmission is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the modules described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0118] Figure 8 is a structural block diagram of a data transmission device according to an embodiment of the present application, the device comprising:

[0119] The first quantity determination module 801 is used to determine the first token quantity required to send the data packet to be sent according to the data volume of the data packet to be sent and the preset data volume and target correspondence, wherein the target correspondence is the correspondence between the preset data volume and the token quantity.

[0120] The second quantity determining module 802 is configured to obtain a second token quantity of tokens in a preset token bucket.

[0121] The token pre-borrowing module 803 is used to replenish the pre-borrowed tokens into the token bucket when the second token quantity is less than the first token quantity, so that the second token quantity is greater than or equal to the first token quantity.

[0122] The data sending module 804 is configured to send a data packet to be sent when it is determined that the first number of tokens has been deducted from the token bucket.

[0123] The above-mentioned device first determines the first token number required to send the data packet to be sent according to the data volume of the data packet to be sent and the target correspondence. Then, the second token number of the tokens in the preset token bucket is obtained to determine whether the second token number in the token bucket is sufficient to support the sending of the data packet. In the case where the second token number is less than the first token number, it means that the token number in the token bucket is insufficient, but in order to ensure the continuity of data transmission, the token bucket is supplemented with pre-borrowed tokens so that the second token number is greater than or equal to the first token number, so that the data packet can be sent. Then, when it is determined that the tokens of the first token number have been deducted from the token bucket, the data packet to be sent is sent. In summary, the method of the present application can avoid the situation where data transmission is interrupted due to insufficient tokens in the token bucket by designing a pre-borrowed token mechanism, thereby ensuring the continuity of data transmission.

[0124] In an exemplary embodiment, the token pre-borrowing module 803 includes:

[0125] a difference determination unit, configured to determine a token difference according to the first token quantity and the second token quantity;

[0126] The replenishing unit is used to replenish the pre-borrowed tokens into the token bucket when the token difference is less than the maximum pre-borrowed token quantity.

[0127] In an exemplary embodiment, the above-mentioned device is also used to initialize relevant parameters of the token bucket, wherein the relevant parameters of the token bucket include the capacity of the token bucket, the token generation rate of the token bucket, and the maximum pre-borrowed number of pre-borrowed tokens.

[0128] In an exemplary embodiment, the apparatus is further used to obtain performance parameters of a transmission network, wherein the transmission network is used to transmit data between a source storage volume and a target storage volume, and the performance parameters include network available bandwidth, network delay, and network packet loss rate;

[0129] According to the performance parameter, evaluating the state of the transmission network, wherein the state of the transmission network includes at least one of the following: a good state, a medium state, and a congested state;

[0130] When the network available bandwidth is greater than the bandwidth threshold, the network delay is lower than the delay threshold, and the network packet loss rate is lower than the packet loss rate threshold, determining that the state of the transmission network is in a good state;

[0131] When it is determined that the state of the transmission network is in a good state, based on the current token generation rate of the token bucket, the token generation rate of the token bucket is increased;

[0132] When the network available bandwidth is less than the bandwidth threshold, or the network delay is lower than the delay threshold, or the network packet loss rate is lower than the packet loss rate threshold, determining that the state of the transmission network is a medium state;

[0133] When it is determined that the state of the transmission network is a medium state, adjusting the token generation rate of the token bucket within a preset range of the current token generation rate of the token bucket;

[0134] When the network available bandwidth is greater than or equal to the bandwidth threshold, the network delay is higher than the delay threshold, and the network packet loss rate is higher than the packet loss rate threshold, determining that the state of the transmission network is a congested state;

[0135] When it is determined that the state of the transmission network is a congested state, the token generation rate of the token bucket is reduced based on the current token generation rate of the token bucket.

[0136] In an exemplary embodiment, the above-mentioned device is also used to monitor the network traffic of the transmission network in real time; when the network traffic is greater than a preset threshold, re-acquire the performance parameters of the transmission network; and re-evaluate the status of the transmission network based on the performance parameters of the transmission network.

[0137] In an exemplary embodiment, the above-mentioned device is also used to obtain parameters of the source storage volume and the target storage volume; determine the configuration parameters of the data migration task based on the parameters of the source storage volume and the target storage volume; establish a data migration thread between the source storage volume and the target storage volume according to the configuration parameters of the data migration task; and execute the data migration thread to read the data of the source storage volume as a data packet to be sent.

[0138] In an exemplary embodiment, the above-mentioned device is also used to record the third token quantity of pre-borrowed tokens that have been replenished in the token bucket; when it is determined that the first token quantity has been deducted from the token bucket, if the second token quantity in the token bucket is greater than the third token quantity, the third token quantity is deducted from the token bucket again.

[0139] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0140] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0141] Optionally, in this embodiment, the computer program may be configured to perform the following steps by means of the computer program:

[0142] S1, determining the first number of tokens required to send the data packet to be sent according to the data volume of the data packet to be sent and the target corresponding relationship, wherein the target corresponding relationship is a preset corresponding relationship between the data volume and the number of tokens.

[0143] S2, obtaining a second token quantity of tokens in a preset token bucket.

[0144] S3: When the second token quantity is less than the first token quantity, add pre-borrowed tokens to the token bucket so that the second token quantity is greater than or equal to the first token quantity.

[0145] S4: When it is determined that the first token quantity of tokens has been deducted from the token bucket, send the data packet to be sent.

[0146] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0147] The embodiment of the present application also provides an electronic device, such as Fig. 9 As shown, the electronic device includes a memory 902 and a processor 904. The memory 902 stores a computer program, and the processor 904 is configured to execute the steps in any of the above method embodiments through the computer program.

[0148] Optionally, in this embodiment, the processor 904 may be configured to perform the following steps through a computer program:

[0149] S1, determining the first number of tokens required to send the data packet to be sent according to the data volume of the data packet to be sent and the target corresponding relationship, wherein the target corresponding relationship is a preset corresponding relationship between the data volume and the number of tokens.

[0150] S2, obtaining a second token quantity of tokens in a preset token bucket.

[0151] S3: When the second token quantity is less than the first token quantity, add pre-borrowed tokens to the token bucket so that the second token quantity is greater than or equal to the first token quantity.

[0152] S4: When it is determined that the first token quantity of tokens has been deducted from the token bucket, send the data packet to be sent.

[0153] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0154] Alternatively, a person skilled in the art may understand that: Fig. 9 The structure shown is for illustration only. Fig. 9 The structure of the electronic device is not limited. For example, the electronic device may also include Fig. 9 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig. 9 Different configurations are shown.

[0155] Among them, the memory 902 can be used to store software programs and modules, such as the program instructions / modules corresponding to the method for sending data and the device for sending data in the embodiments of the present application. The processor 904 executes various functional applications and data processing by running the software programs and modules stored in the memory 902, that is, realizing the above-mentioned method for sending data. The memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 902 may further include a memory remotely located relative to the processor 904, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 902 may be specifically, but not limited to, used to store information such as system configuration files. As an example, such as Fig. 9 As shown, the memory 902 may include, but is not limited to, the first quantity determination module 801, the second quantity determination module 802, the token pre-borrowing module 803, and the data sending module 804 in the above-mentioned data sending device. In addition, other module units in the above-mentioned data sending device may also be included but are not limited to, which will not be repeated in this example.

[0156] Optionally, the transmission device 906 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one example, the transmission device 906 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 906 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0157] In addition, the electronic device further includes: a display 908; and a connection bus 910, which is used to connect various module components in the electronic device.

[0158] In other embodiments, the electronic device may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes in the form of network communication. Among them, the nodes may form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0159] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0160] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0161] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the above method embodiments.

[0162] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0163] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for sending data, characterized in that: The method comprises: Determine the first number of tokens required to send the data packet to be sent according to the data volume of the data packet to be sent and the target corresponding relationship, wherein the target corresponding relationship is a preset corresponding relationship between the data volume and the number of tokens; Obtain a second token quantity of tokens in a preset token bucket; When the second token quantity is less than the first token quantity, replenishing the pre-borrowed tokens into the token bucket so that the second token quantity is greater than or equal to the first token quantity; When it is determined that the first token quantity of tokens has been deducted from the token bucket, the to-be-sent data packet is sent.

2. The method for data transmission according to claim 1, characterized in that: When the number of the second tokens is less than the number of the first tokens, replenishing the pre-borrowed tokens into the token bucket includes: Determining a token difference based on the first token quantity and the second token quantity; When the token difference is less than or equal to the maximum pre-borrowed number of pre-borrowed tokens, the pre-borrowed tokens are added to the token bucket.

3. The method for data transmission according to claim 1, characterized in that: Before obtaining the second number of tokens in the preset token bucket, the method further includes: Initialize the relevant parameters of the token bucket, wherein the relevant parameters of the token bucket include the capacity of the token bucket, the token generation rate of the token bucket, and the maximum pre-borrowed number of the pre-borrowed tokens.

4. The method for data transmission according to claim 3, characterized in that: The method further comprises: Acquire performance parameters of a transmission network, wherein the transmission network is used to transmit data between a source storage volume and a target storage volume, and the performance parameters include network available bandwidth, network delay, and network packet loss rate; According to the performance parameter, evaluating the state of the transmission network, wherein the state of the transmission network includes at least one of the following: a good state, a medium state, and a congested state; In the case where the network available bandwidth is greater than a bandwidth threshold, the network delay is lower than a delay threshold, and the network packet loss rate is lower than a packet loss rate threshold, determining that the state of the transmission network is a good state; When determining that the state of the transmission network is in a good state, increasing the token generation rate of the token bucket based on the current token generation rate of the token bucket; In a case where the network available bandwidth is less than the bandwidth threshold, or the network delay is lower than the delay threshold, or the network packet loss rate is lower than the packet loss rate threshold, determining that the state of the transmission network is a medium state; When it is determined that the state of the transmission network is a medium state, adjusting the token generation rate of the token bucket within a preset range of the current token generation rate of the token bucket; When the available bandwidth of the network is greater than or equal to the bandwidth threshold, the network delay is higher than the delay threshold, and the network packet loss rate is higher than the packet loss rate threshold, determining that the state of the transmission network is a congested state; When it is determined that the state of the transmission network is a congested state, the token generation rate of the token bucket is reduced based on the current token generation rate of the token bucket.

5. The method for data transmission according to claim 4, characterized in that: The method further comprises: Monitoring network traffic of the transmission network in real time; When the network traffic is greater than a preset threshold, re-obtaining the performance parameters of the transmission network; Based on the performance parameters of the transmission network, a status of the transmission network is re-evaluated.

6. The method for transmitting data according to any one of claims 1 to 5, characterized in that: Before determining the number of first tokens required to send the data packet to be sent according to the data volume of the data packet to be sent, the method further includes: Get the parameters of the source storage volume and the target storage volume; Determining configuration parameters of the data migration task based on parameters of the source storage volume and the target storage volume; Establishing a data migration thread between the source storage volume and the target storage volume according to the configuration parameters of the data migration task; The data migration thread is executed to read the data of the source storage volume as the data packet to be sent.

7. The method for transmitting data according to any one of claims 1 to 5, characterized in that: In the case where the second token quantity is less than the first token quantity, after replenishing the pre-borrowed tokens into the token bucket so that the second token quantity is greater than or equal to the first token quantity, the method further includes: Recording the number of third tokens of pre-borrowed tokens that have been replenished in the token bucket; When it is determined that the first number of tokens has been deducted from the token bucket, if the second number of tokens in the token bucket is greater than the third number of tokens, the third number of tokens is deducted from the token bucket.

8. A device for transmitting data, characterized in that: The device comprises: A first quantity determination module, used to determine the first number of tokens required to send the data packet to be sent according to the data volume of the data packet to be sent and the preset data volume and target correspondence relationship, wherein the target correspondence relationship is the preset correspondence relationship between the data volume and the number of tokens; A second quantity determination module, used to obtain a second token quantity of tokens in a preset token bucket; a token pre-borrowing module, configured to replenish pre-borrowed tokens into the token bucket when the second token quantity is less than the first token quantity, so that the second token quantity is greater than or equal to the first token quantity; The data sending module is used to send the data packet to be sent when it is determined that the tokens of the first token quantity have been deducted from the token bucket.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Images