Data transmission method, device, electronic device and storage medium

By intelligently selecting the target channel according to the weight value of the data channel for file synchronization, the problem of low file transfer efficiency in big data scenarios is solved, and more efficient network bandwidth utilization and file transfer efficiency are achieved.

CN113810459BActive Publication Date: 2024-11-01QI AN XIN TECHNOLOGY GROUP INC +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110864819.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-29
Publication Date
2024-11-01
Estimated Expiration
2041-07-29

AI Technical Summary

Technical Problem

In big data scenarios, the existing single-channel transmission scheme is not sufficient to support file synchronization between server clusters, resulting in low file transfer efficiency.

Method used

By selecting the target data channel according to the weight value of each data channel, file events are assigned to the channel to achieve file synchronization. The weight value is determined by the file event to be processed, used to measure the pressure of the data channel and in load mode Adjust channel state to optimize transmission.

Benefits of technology

It achieves the shortening of overall transmission time, makes full use of network bandwidth, and improves file transfer efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113810459B_ABST
    Figure CN113810459B_ABST
Patent Text Reader

Abstract

The embodiments of the present invention provide a data transmission method, device, electronic device and storage medium. The data transmission method includes: receiving a file event pushed by a sending end, wherein the file event includes a file that needs to be synchronized; selecting a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels, and assigning the file event to the target data channel to synchronize the file to the receiving end through the target data channel, wherein the weight value of each data channel is determined by the file event to be processed in the data channel, and the weight value is used to measure the pressure of the file to be processed in the corresponding data channel. The embodiments of the present invention can minimize the overall transmission time, so as to fully utilize the network bandwidth and improve the efficiency of file transmission.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a data transmission method, device, electronic equipment and storage medium. Background Art

[0002] The security isolation network gate and data exchange platform support file-based business synchronization functions. In big data scenarios, a large amount of file data needs to be synchronized in the server cluster to enable devices to pull and push files between two networks. With the popularization of server disk arrays and 10G networks, the current single-channel transmission solution is not enough to support the above big data scenarios, and the file transmission efficiency is low. Summary of the invention

[0003] In view of the problems in the prior art, embodiments of the present invention provide a data transmission method, device, electronic device and storage medium.

[0004] Specifically, the embodiment of the present invention provides the following technical solutions:

[0005] In a first aspect, an embodiment of the present invention provides a data transmission method, including:

[0006] Receiving a file event pushed by a sending end, wherein the file event includes a file that needs to be synchronized;

[0007] selecting a target data channel from the multiple data channels based on a weight value of each data channel in the multiple data channels, and assigning the file event to the target data channel, so as to synchronize the file to the receiving end through the target data channel,

[0008] The weight value of each data channel is determined by the file events to be processed in the data channel. The weight value is used to measure the pressure of the files to be processed in the corresponding data channel.

[0009] In some examples, before selecting a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels and assigning the file event to the target data channel to synchronize the file to the receiving end through the target data channel, the method further includes:

[0010] Obtain multiple file events to be processed in the first data channel;

[0011] Obtaining a file event weight value of each file event in the plurality of file events;

[0012] Based on the file event weight value of each file event in the multiple file events, a weight value of the first data channel is obtained,

[0013] The first data channel is any one of the multiple data channels.

[0014] In some examples, obtaining the file event weight value of each file event in the plurality of file events includes:

[0015] Get the size of the file that needs to be synchronized for each file event;

[0016] Based on the size of the file that needs to be synchronized for each file event and a preset block size, a file event weight value of the corresponding file event is obtained.

[0017] In some examples, selecting a target data channel from the multiple data channels based on a weight value of each data channel in the multiple data channels, and assigning the file event to the target data channel to synchronize the file to the receiving end through the target data channel includes:

[0018] A data channel with the smallest weight value is selected from the multiple data channels as the target data channel, and the file event is assigned to the target data channel, so as to synchronize the file to the receiving end through the target data channel.

[0019] In some examples, the weight value is positively correlated with the pressure of the file to be processed in the corresponding data channel.

[0020] In some examples, before selecting a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels and assigning the file event to the target data channel to synchronize the file to the receiving end through the target data channel, the method further includes:

[0021] In the load mode, determining whether the weight value of each data channel in the plurality of data channels reaches a first preset value;

[0022] If the weight value of each data channel in the multiple data channels reaches a first preset value, the multiple data channels are set to a busy state, and before the size of the file to be synchronized in the file event reaches a second preset value, the file event is assigned to the control channel to synchronize the file to the receiving end through the control channel.

[0023] In some examples, after setting the multiple data channels to a busy state, the method further includes:

[0024] If the size of the file that needs to be synchronized in the file event reaches the second preset value, the first preset value is adjusted upward, and a target data channel is selected from the multiple data channels based on the weight value of each data channel in the multiple data channels, until a data channel whose weight value drops below the first preset value before the increase appears in the multiple data channels, the multiple data channels are restored to an idle state, and the first preset value before the increase is restored.

[0025] In some examples, the method further includes: if the low power consumption mode is used, the control channel is used as the target data channel, and the file event is assigned to the target data channel to synchronize the file to the receiving end through the target data channel.

[0026] In some examples, this also includes:

[0027] Receive control events pushed by the sender and process the control events through the control channel

[0028] In a second aspect, an embodiment of the present invention further provides a data transmission device, including:

[0029] A receiving module, used for receiving a file event pushed by a sending end, wherein the file event includes a file that needs to be synchronized;

[0030] an allocation module, configured to select a target data channel from the multiple data channels based on a weight value of each data channel in the multiple data channels, and allocate the file event to the target data channel, so as to synchronize the file to the receiving end through the target data channel,

[0031] The weight value of each data channel is determined by the file events to be processed in the data channel, and the weight value is used to measure the pressure of the files to be processed in the corresponding data channel.

[0032] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the data transmission method described in the first aspect are implemented.

[0033] In a fourth aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the data transmission method as described in the first aspect.

[0034] In a fifth aspect, an embodiment of the present invention further provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the data transmission method as described in the first aspect are implemented.

[0035] It can be seen from the above technical solutions that the data transmission method, device, electronic device and storage medium provided in the embodiments of the present invention intelligently select the target data channel according to the weight value of each data channel, and synchronize the file to the receiving end through the target data channel, which can minimize the overall transmission time and achieve the purpose of fully utilizing the network bandwidth and improving the file transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A flowchart of a data transmission method provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of a data transmission method provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of file event allocation of a data transmission method provided by an embodiment of the present invention;

[0040] Figure 4 A schematic diagram of a timeout retransmission mechanism of a data transmission method provided by an embodiment of the present invention;

[0041] Figure 5 A structural block diagram of a data transmission device provided by an embodiment of the present invention;

[0042] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Figure 1 FIG. 1 is a flow chart showing a data transmission method provided by an embodiment of the present invention. Figure 1 As shown, the data transmission method provided by the embodiment of the present invention includes the following steps:

[0045] Step 101: receiving a file event pushed by a sending end, where the file event includes a file that needs to be synchronized.

[0046] like Figure 2 As shown, the data transmission method of the embodiment of the present invention can be implemented by a multiplex transmission model, wherein the multiplex transmission model includes a distributor and multiple data channels. For example, the distributor receives file events sent by a sender, such as file events of synchronized file B, file events of synchronized file C, etc., wherein the sender can be a server processing file services, and the synchronized files can be file data, service data, etc.

[0047] In the above description, the data channel is a channel for file transmission, that is, a channel used to transmit files.

[0048] Step 102: Select a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels, and assign the file event to the target data channel to synchronize the file to the receiving end through the target data channel, wherein the weight value of each data channel is determined by the file event to be processed in the data channel, and the weight value is used to measure the pressure of the file to be processed in the corresponding data channel.

[0049] Combination Figure 2 As shown, when a channel (such as a data channel) is first started, it initiates registration information to the distributor. The distributor eventually obtains the channel identifier, such as UUID, and regularly maintains the channel to achieve the purpose of centralized management and unified allocation of channels.

[0050] Among them, event allocation refers to the sender pushing file events (such as new addition, query, etc.), and the distributor will allocate them to the data channel for processing.

[0051] In addition, in order to improve the efficiency of event processing, in an embodiment of the present invention, for file events pushed by the sender, if they are the same file events, such as: the file name, file size, and modification time are consistent, and the file is being transmitted, then they are considered to be the same file events, and the distributor will deduplicate the events to filter out redundant events, prevent repeated file synchronization, and improve bandwidth utilization.

[0052] In one embodiment of the present invention, when the sending end pushes a large number of file events, the distributor intelligently distributes the file events in combination with the pressure of the data channel, thereby achieving the purpose of synchronizing all file events in the shortest time. For example: when synchronizing medium-sized files, the data channel sends files evenly; in a scenario where large and small files are mixed, when the data channels are busy sending large files, small files can be quickly handed over to the control channel for sending; in the scenario of small files, the data channel and the control channel can send files evenly together. Among them, large files, medium-sized files and small files can be obtained by comparing with preset thresholds, for example, a first preset threshold and a second preset threshold are pre-set, wherein the first preset threshold is greater than the second preset threshold, and files greater than the first preset threshold are determined as large files, and files less than the second preset threshold are determined as small files, and files between the second preset threshold and the first preset threshold are medium-sized files.

[0053] In a specific example, a target data channel is selected from the multiple data channels based on the weight value of each data channel in the multiple data channels, and the file event is assigned to the target data channel before synchronizing the file to the receiving end through the target data channel, including: obtaining multiple file events to be processed in a first data channel; obtaining a file event weight value for each file event in the multiple file events; and obtaining a weight value of the first data channel based on the file event weight value for each file event in the multiple file events, wherein the first data channel is any data channel in the multiple data channels.

[0054] In this example, obtaining the file event weight value of each file event in the multiple file events includes: obtaining the size of the file that needs to be synchronized for each file event; obtaining the file event weight value of the corresponding file event based on the size of the file that needs to be synchronized for each file event and the preset block size, wherein the preset block size is used to identify the smallest unit of file storage. For example, if the preset block size is set to 4KB, then even if the file weight is smaller than 4KB, it will be calculated as 4KB. In most scenarios, the preset block size can be set to 4KB, and in other scenarios, the preset block size can also be set to 8KB, 16KB, etc. In the following embodiments, for the sake of convenience, the block size that appears in some locations has the same meaning as the block size preset here.

[0055] After obtaining the weight value of each data channel, a target data channel is selected from the multiple data channels based on the weight value of each data channel in the multiple data channels, and the file event is assigned to the target data channel, so as to synchronize the file to the receiving end through the target data channel, including: selecting a data channel with the smallest weight value from the multiple data channels as the target data channel, and assigning the file event to the target data channel, so as to synchronize the file to the receiving end through the target data channel. In this example, the weight value is positively correlated with the pressure of the file to be processed in the corresponding data channel.

[0056] That is: according to the file events pushed by the current sender, combined with the weight information of the data channel, the local optimal solution is solved. Looking at the overall file synchronization process, the effect of approaching the global optimal solution can be achieved, that is, the overall transmission time is the shortest. According to the business characteristics of cross-domain transmission files and considering the overhead of file events on the real file system, directory-related events with low overhead (directory additions, deletions, and file deletions) are uniformly processed on the control channel, and events with high file overhead (file query, addition) are assigned to the data channel or control channel for processing.

[0057] In the multi-channel transmission model, the file events that usually need to be assigned for processing are file query and file addition events. Therefore, the weight value of the data channel is set. The data channel with a low weight value indicates low business pressure. File events will be assigned to the data channel with a low weight value for processing. Figure 2 As shown, it is assumed that there are 1 control channel and N data channels, that is, C = {C0, D0, D1, D2...D N}, where C0 is the control channel, {D0, D1, D2…D N} is the data channel, the default weight W={W C ,W D0 ,W D1 ,W D2 ,…W DN}, and the data channel with the lowest weight value is selected for event allocation. The weight value of the directory event is Wec = {block size (blocks ize)}, and the weight value of the data event is Wed = {MAX (file size (byte), block size (blocks ize))}. After the event is allocated, it is added to the current weight value W DN, and then reduce the weight value after the event is completed. For example, the total number of pending and in-process file events allocated on a data channel is 5. Assuming the block size is 4K, the file sizes of these 5 file events are 2K, 1K, 8K, 10K and 15K, respectively. Then the file event weights of these 5 file events are 4K, 4K, 8K, 10K and 15K, respectively. Therefore, the weight value of this data channel is: 4K+4K+8K+10K+15K=41K.

[0058] like Figure 3 As shown, a schematic diagram of allocating file events using the data transmission method according to an embodiment of the present invention is shown, for example, files H, E and A are allocated to one data channel, files J, G and B are allocated to another data channel, and so on.

[0059] According to the data transmission method of the embodiment of the present invention, the target data channel is intelligently selected according to the weight value of each data channel, and the file is synchronized to the receiving end through the target data channel, which can shorten the overall transmission time and achieve the purpose of fully utilizing the network bandwidth and improving the file transmission efficiency.

[0060] In one embodiment of the present invention, a target data channel is selected from the multiple data channels based on the weight value of each data channel in the multiple data channels, and the file event is assigned to the target data channel, so as to synchronize the file to the receiving end through the target data channel, and it also includes: in a load mode, determining whether the weight value of each data channel in the multiple data channels reaches a first preset value; if the weight value of each data channel in the multiple data channels reaches the first preset value, setting the multiple data channels to a busy state, and before the size of the file to be synchronized in the file event reaches a second preset value, assigning the file event to a control channel, so as to synchronize the file to the receiving end through the control channel.

[0061] In this example, after setting the multiple data channels to a busy state, it also includes: if the size of the file that needs to be synchronized in the file event reaches the second preset value, the first preset value is increased, and a target data channel is selected from the multiple data channels based on the weight value of each data channel in the multiple data channels, until a data channel whose weight value drops below the first preset value before the increase appears in the multiple data channels, the multiple data channels are restored to an idle state, and the first preset value before the increase is restored.

[0062] Specifically, the control threshold T weight (ie: the first preset value) is used to identify the maximum weight value of the current transmission, the small file threshold T filesize(ie: the second preset value) is used to identify large and small files. When the weight value of each data channel reaches the control threshold, the distributor marks the data channel as busy. BUSY When all data channels are marked as busy, the control channel begins to take on the task of sending small files. When the overall busy situation occurs, the control threshold will be increased to amplify T weigh =T weight ×2, reset the threshold value T after restoring idle state weig =T weight ÷2.

[0063] In one embodiment of the present invention, a timeout retransmission mechanism is also included, that is, the data channel is required to periodically send keep-alive messages to the distributor, and the distributor collects the pressure values ​​and health status of each channel to ensure the robustness of the business. The timeout retransmission is intended to solve the problem of event loss caused by channel abnormalities. The file events being sent on the distributor's scoreboard are maintained in the form of a directory tree: Et = {Et0, Et1, Et2, ...Etn}, and the file nodes are colored and marked according to the data channel. A chain structure is used internally to string the events together E = {E0, E1, E2, ...EN}; when a channel is found to be abnormally offline, the colored nodes of the channel are recycled, and re-marked and allocated according to the intelligent allocation algorithm and dynamic threshold algorithm. Figure 4 As shown, channel 0 corresponds to file events of 1.txt, 2.jpg, and 7.exe. Channel 1 corresponds to file events of 3.txt and 5.jpg, and the rest correspond to file events of channel 2, and so on.

[0064] In one embodiment of the present invention, the method further includes: if the low power consumption mode is used, the control channel is used as the target data channel, and the file event is allocated to the target data channel, so as to synchronize the file to the receiving end through the target data channel. That is, the file event is directly processed by the control channel, so that there is no need to allocate the data channel, thus saving allocation overhead.

[0065] In one embodiment of the present invention, the method further includes: receiving a control event pushed by the sending end, and processing the control event through the control channel. That is, if the pushed event is a control event, such as a directory-related event with low overhead (directory addition, deletion, file deletion), it is processed uniformly on the control channel, thus saving the allocation overhead of the data channel.

[0066] Figure 5 FIG. 1 is a schematic diagram showing the structure of a data transmission device provided by an embodiment of the present invention. Figure 5 As shown, the data transmission device provided in this embodiment includes: a receiving module 510 and a distribution module 520, wherein:

[0067] A receiving module 510, configured to receive a file event pushed by a sending end, wherein the file event includes a file that needs to be synchronized;

[0068] The allocation module 520 is used to select a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels, and allocate the file event to the target data channel to synchronize the file to the receiving end through the target data channel, wherein the weight value of each data channel is determined by the file event to be processed in the data channel, and the weight value is positively correlated with the pressure of the file to be processed in the corresponding data channel.

[0069] Based on the contents of the above embodiments, in this embodiment, before selecting a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels and assigning the file event to the target data channel to synchronize the file to the receiving end through the target data channel, the method further includes:

[0070] Obtain multiple file events to be processed in the first data channel;

[0071] Obtaining a file event weight value of each file event in the plurality of file events;

[0072] Based on the file event weight value of each file event in the multiple file events, a weight value of the first data channel is obtained,

[0073] The first data channel is any one of the multiple data channels.

[0074] Based on the contents of the above embodiments, in this embodiment, obtaining the file event weight value of each file event in the multiple file events includes:

[0075] Get the size of the file that needs to be synchronized for each file event;

[0076] Based on the size of the file to be synchronized for each file event and a preset block size, a file event weight value of the corresponding file event is obtained, wherein the preset block size is used to identify the minimum unit of file storage.

[0077] Based on the contents of the above embodiments, in this embodiment, selecting a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels, and assigning the file event to the target data channel, so as to synchronize the file to the receiving end through the target data channel, includes:

[0078] A data channel with the smallest weight value is selected from the multiple data channels as the target data channel, and the file event is assigned to the target data channel, so as to synchronize the file to the receiving end through the target data channel.

[0079] Based on the contents of the above embodiments, in this embodiment, the weight value is positively correlated with the pressure of the file to be processed in the corresponding data channel.

[0080] Based on the contents of the above embodiments, in this embodiment, before selecting a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels and assigning the file event to the target data channel to synchronize the file to the receiving end through the target data channel, the method further includes:

[0081] In the load mode, determining whether the weight value of each data channel in the plurality of data channels reaches a first preset value;

[0082] If the weight value of each data channel in the multiple data channels reaches a first preset value, the multiple data channels are set to a busy state, and when the size of the file to be synchronized in the file event does not reach a second preset value, the file event is assigned to a control channel to synchronize the file to the receiving end through the control channel.

[0083] Based on the contents of the above embodiments, in this embodiment, after setting the multiple data channels to a busy state, the method further includes:

[0084] If the size of the file that needs to be synchronized in the file event reaches the second preset value, the first preset value is adjusted upward, and a target data channel is selected from the multiple data channels based on the weight value of each data channel in the multiple data channels, until a data channel whose weight value drops below the first preset value before the increase appears in the multiple data channels, the multiple data channels are restored to an idle state, and the first preset value before the increase is restored.

[0085] Based on the contents of the above embodiments, in this embodiment, it also includes: if it is a low power consumption mode, the control channel is used as the target data channel, and the file event is assigned to the target data channel to synchronize the file to the receiving end through the target data channel.

[0086] Based on the contents of the above embodiments, in this embodiment, it also includes:

[0087] Receive control events pushed by the sender and process the control events through the control channel

[0088] According to the data transmission device of the embodiment of the present invention, the target data channel is intelligently selected according to the weight value of each data channel, and the file is synchronized to the receiving end through the target data channel, so that the overall transmission time can be shortened, thereby achieving the purpose of fully utilizing the network bandwidth and improving the file transmission efficiency.

[0089] Since the data transmission device provided in the embodiment of the present invention can be used to execute the data transmission method described in the above embodiment, its working principle and beneficial effects are similar, so they are not described in detail here. For specific contents, please refer to the introduction of the above embodiment.

[0090] In this embodiment, it should be noted that the various modules in the device of the embodiment of the present invention can be integrated into one or deployed separately. The above modules can be combined into one module or further divided into multiple sub-modules.

[0091] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, see Figure 6 , the electronic device specifically includes the following contents: a processor 301, a memory 302, a communication interface 303 and a communication bus 304;

[0092] The processor 301, the memory 302, and the communication interface 303 communicate with each other via the communication bus 304;

[0093] The processor 301 is used to call the computer program in the memory 302. When the processor executes the computer program, all steps of the above-mentioned data transmission method are implemented. For example, when the processor executes the computer program, the following process is implemented: receiving a file event pushed by a sending end, wherein the file event includes a file that needs to be synchronized; selecting a target data channel from the multiple data channels based on a weight value of each data channel in the multiple data channels, and assigning the file event to the target data channel to synchronize the file to the receiving end through the target data channel, wherein the weight value of each data channel is determined by the file event to be processed in the data channel, and the weight value is used to measure the pressure of the file to be processed in the corresponding data channel.

[0094] It can be understood that the detailed functions and extended functions that can be executed by the computer program can refer to the description of the above embodiments.

[0095] Based on the same inventive concept, another embodiment of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, all the steps of the above-mentioned data transmission method are implemented. For example, when the processor executes the computer program, the following process is implemented: receiving a file event pushed by a sending end, the file event includes a file that needs to be synchronized; selecting a target data channel from the multiple data channels based on a weight value of each data channel in the multiple data channels, and assigning the file event to the target data channel to synchronize the file to the receiving end through the target data channel, wherein the weight value of each data channel is determined by the file event to be processed in the data channel, and the weight value is used to measure the pressure of the file to be processed in the corresponding data channel.

[0096] It can be understood that the detailed functions and extended functions that can be executed by the computer program can refer to the description of the above embodiments.

[0097] Based on the same inventive concept, another embodiment of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements all the steps of the above-mentioned data transmission method. For example, when the processor executes the computer program, it implements the following process: receiving a file event pushed by a sending end, and the file event includes a file that needs to be synchronized; selecting a target data channel from the multiple data channels based on a weight value of each data channel in the multiple data channels, and assigning the file event to the target data channel to synchronize the file to the receiving end through the target data channel, wherein the weight value of each data channel is determined by the file event to be processed in the data channel, and the weight value is used to measure the pressure of the file to be processed in the corresponding data channel.

[0098] It can be understood that the detailed functions and extended functions that can be executed by the computer program can refer to the description of the above embodiments.

[0099] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. Those of ordinary skill in the art may understand and implement them without creative effort.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the traffic audit method described in each embodiment or some parts of the embodiment.

[0102] In addition, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0103] In addition, in the present invention, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data transmission method, It is characterized in that include: Receiving a file event pushed by a sending end, wherein the file event includes a file that needs to be synchronized; selecting a target data channel from the multiple data channels based on a weight value of each data channel in the multiple data channels, and assigning the file event to the target data channel, so as to synchronize the file to the receiving end through the target data channel, The weight value of each data channel is determined by the file events to be processed in the data channel, and the weight value is used to measure the pressure of the files to be processed in the corresponding data channel; The selecting a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels comprises: In a case where all the files include files larger than a first preset threshold and files smaller than a second preset threshold, and when it is determined based on the weight value of each data channel that each of the data channels is a target data channel for sending files larger than the first preset threshold, the control channel is determined as a channel for sending files smaller than the second preset threshold; When the size of each of the files is smaller than the second preset threshold, determining a target data channel corresponding to each of the files from all of the data channels and the control channel based on a weight value of each data channel and a balancing rule; When the size of each of the files is smaller than the first preset threshold and larger than the second preset threshold, a target data channel corresponding to each of the files is determined from all the data channels based on a weight value of each data channel and a balancing rule.

2. The data transmission method according to claim 1, It is characterized in that Before selecting a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels and assigning the file event to the target data channel so as to synchronize the file to the receiving end through the target data channel, the method further includes: Obtain multiple file events to be processed in the first data channel; Obtaining a file event weight value of each file event in the plurality of file events; Based on the file event weight value of each file event in the multiple file events, a weight value of the first data channel is obtained, The first data channel is any one of the multiple data channels.

3. The data transmission method according to claim 2, It is characterized in that The obtaining of the file event weight value of each file event in the plurality of file events comprises: Get the size of the file that needs to be synchronized for each file event; Based on the size of the file to be synchronized for each file event and a preset block size, a file event weight value of the corresponding file event is obtained, wherein the preset block size is used to identify the minimum unit of file storage.

4. The data transmission method according to claim 1, It is characterized in that The weight value is positively correlated with the pressure of the file to be processed in the corresponding data channel.

5. The data transmission method according to claim 1, It is characterized in that Before selecting a target data channel from the multiple data channels based on the weight value of each data channel in the multiple data channels and assigning the file event to the target data channel so as to synchronize the file to the receiving end through the target data channel, the method further includes: In the load mode, determining whether the weight value of each data channel in the plurality of data channels reaches a first preset value; If the weight value of each data channel in the multiple data channels reaches a first preset value, the multiple data channels are set to a busy state, and before the size of the file to be synchronized in the file event reaches a second preset value, the file event is assigned to the control channel to synchronize the file to the receiving end through the control channel.

6. The data transmission method according to claim 5, It is characterized in that After the multiple data channels are set to a busy state, the method further comprises: If the size of the file that needs to be synchronized in the file event reaches the second preset value, the first preset value is adjusted upward, and a target data channel is selected from the multiple data channels based on the weight value of each data channel in the multiple data channels, until a data channel whose weight value drops below the first preset value before the increase appears in the multiple data channels, the multiple data channels are restored to an idle state, and the first preset value before the increase is restored.

7. The data transmission method according to claim 5, It is characterized in that Also includes: If it is a low power consumption mode, the control channel is used as a target data channel, and the file event is assigned to the target data channel, so as to synchronize the file to the receiving end through the target data channel.

8. The data transmission method according to claim 5, It is characterized in that Also includes: Receive the control event pushed by the sender, and process the control event through the control channel.

9. A data transmission device, It is characterized in that include: A receiving module, used for receiving a file event pushed by a sending end, wherein the file event includes a file that needs to be synchronized; an allocation module, configured to select a target data channel from the multiple data channels based on a weight value of each data channel in the multiple data channels, and allocate the file event to the target data channel, so as to synchronize the file to the receiving end through the target data channel, The weight value of each data channel is determined by the file events to be processed in the data channel, and the weight value is used to measure the pressure of the files to be processed in the corresponding data channel; The allocation module is specifically used for: In a case where all the files include files larger than a first preset threshold and files smaller than a second preset threshold, and when it is determined based on the weight value of each data channel that each of the data channels is a target data channel for sending files larger than the first preset threshold, the control channel is determined as a channel for sending files smaller than the second preset threshold; When the size of each of the files is smaller than the second preset threshold, determining a target data channel corresponding to each of the files from all of the data channels and the control channel based on a weight value of each data channel and a balancing rule; When the size of each of the files is smaller than the first preset threshold and larger than the second preset threshold, a target data channel corresponding to each of the files is determined from all the data channels based on a weight value of each data channel and a balancing rule.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps of the data transmission method according to any one of claims 1 to 8 are implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the data transmission method according to any one of claims 1 to 8 are implemented.

12. A computer program product, the computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the data transmission method according to any one of claims 1 to 8 are implemented.