Priority preemptive file transmission method and system oriented to edge cloud collaboration

Abstract

The invention provides a priority preemptive file transmission method and system oriented to edge cloud collaboration, and the method comprises the steps: dividing files into a plurality of priority queues, and sequentially transmitting the files in each priority queue from high priority to low priority; if the file in the low-priority queue is newly added in the high-priority queue when the file in the low-priority queue is transmitted, determining whether to start a preemption mode according to a user preset parameter, and if so, performing breakpoint transmission on the suspended file in the low-priority queue after the file in the high-priority queue is preferentially transmitted; if not, the files in the current low-priority queue are transmitted firstly, and then the newly added files in the high-priority queue are transmitted; priority transmission of the emergency files can be achieved, and compared with transmission of a single priority queue on all the files, the efficiency is higher; meanwhile, according to the preemption mode, the high-priority file can be transmitted to the destination node at the first time, a breakpoint transmission function is introduced, and the fault tolerance and transmission efficiency of file transmission are improved.

Description

Technical Field

[0001] The present invention relates to the field of file transfer technology, and in particular to a priority preemptive file transfer method and system for edge-cloud collaboration. Background Art

[0002] Due to the lack of vertical linkage capabilities between corporate headquarters and enterprise edges, the current stage mainly realizes vertical information linkage between corporate headquarters and enterprise edges through edge-cloud collaborative service subscription. Edge-cloud collaborative services need to support highly reliable access to multiple types of data, including structured, unstructured, and combined structures, as well as one-to-one and one-to-many distribution of file data, and many-to-one data backhaul aggregation.

[0003] When transferring files, it is necessary to consider the importance and urgency of the files. Conventional file transfers are performed based on the order of joining the nodes, which cannot meet the needs of transmitting urgent files as quickly as possible. Currently, there is a method of setting priorities for files, giving priority to files with high priorities, thus achieving priority transmission of urgent files in the case of resource competition.

[0004] However, in conventional priority-based file transmission, when a new high-priority file is added in an emergency, it is necessary to wait until the current low-priority file is transferred after it is completed before the new high-priority file can be transferred. There is no guarantee that the high-priority file will be transmitted to the destination node in the first place. Moreover, when the file transfer volume is too large, all files must be sorted according to the corresponding priority before transmission, which is inefficient. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention proposes a priority preemptive file transmission method for edge-cloud collaboration, comprising:

[0006] Divide files into multiple priority queues based on priority, and transfer files in each priority queue in order from high to low priority;

[0007] If a new file is added to the high-priority queue while the file in the low-priority queue is being transferred, the preemptive mode is determined based on the user's preset parameters. If it is enabled, the files in the high-priority queue are transferred first, and then the files in the suspended low-priority queue are transferred at breakpoints. If it is not enabled, the files in the current low-priority queue are transferred first, and then the new files in the high-priority queue are transferred.

[0008] Preferably, the transmitting of the files in each priority queue in order from high to low priority includes:

[0009] Store all files in the corresponding storage path according to the corresponding priority queue;

[0010] Scan each storage path in order from high to low priority, and transfer the files in the corresponding priority queue in each storage path in turn.

[0011] Preferably, scanning each storage path in order of priority from high to low, and sequentially transmitting files in the corresponding priority queue in each storage path, includes:

[0012] Scan each storage path in order from high to low priority, and transfer the files in the highest priority queue using full service rules;

[0013] Adopt limited service rules to transfer files in priority queues other than the highest priority queue in descending order.

[0014] Preferably, the adopting of the limited service rule to sequentially transmit the files in each priority queue other than the highest priority queue in order of priority from high to low includes:

[0015] File transfers are performed on priority queues other than the highest priority queue in descending order of priority;

[0016] Each time a file is transferred, all priority queues with a higher priority than the current queue are searched;

[0017] If the search result is empty, the next file in the current queue is transferred;

[0018] Otherwise, the files in the queue with a higher priority than the one in the current queue are transferred first, and after the transfer is completed, the next file in the current queue is transferred.

[0019] Preferably, after storing all files in corresponding storage paths according to corresponding priority queues, the method further includes:

[0020] Monitor each storage path at all times to determine whether there are files to be transferred until the transfer is complete.

[0021] Preferably, after each file transfer is completed, the file in the storage path is deleted according to user requirements.

[0022] Preferably, dividing the files into a plurality of priority queues based on priority includes:

[0023] Get the request parameters carried by the user when initiating a file distribution request;

[0024] Files are divided into multiple priority queues based on priority according to request parameters.

[0025] Preferably, the request parameters include a distribution destination and a priority of the file.

[0026] Preferably, if enabled, after the files in the high priority queue are transmitted first, the files in the suspended low priority queue are transmitted at breakpoints, including:

[0027] If the preemption mode is enabled, the files being transferred in the low-priority queue will be paused and recorded as interrupted files;

[0028] The transmitted content is saved at the target node of the interrupted file, and the byte offset of the interrupted file that has been transmitted is saved at the current node;

[0029] Switch to the high priority queue for file transfer. After the transfer is completed, read the byte offset of the interrupted file and continue to transfer the data after the byte offset of the interrupted file.

[0030] Based on the same inventive concept, the present invention provides a priority preemptive file transfer system for edge-cloud collaboration, including:

[0031] Priority transmission module: used to divide files into multiple priority queues based on priority, and transmit files in each priority queue in order from high to low priority;

[0032] Preemptive transmission module: If a new file is added to the high-priority queue while a file in the low-priority queue is being transmitted, the preemptive mode is determined based on the user's preset parameters. If it is enabled, the files in the high-priority queue are transmitted first, and then the files in the suspended low-priority queue are transmitted at breakpoints. If it is not enabled, the files in the current low-priority queue are transmitted first, and then the newly added files in the high-priority queue are transmitted.

[0033] Preferably, in the priority transmission module, files in each priority queue are transmitted in order from high to low priority, including:

[0034] Store all files in the corresponding storage path according to the corresponding priority queue;

[0035] Scan each storage path in order from high to low priority, and transfer the files in the corresponding priority queue in each storage path in turn.

[0036] Preferably, scanning each storage path in order of priority from high to low, and sequentially transmitting files in the corresponding priority queue in each storage path, includes:

[0037] Scan each storage path in order from high to low priority, and transfer the files in the highest priority queue using full service rules;

[0038] Adopt limited service rules to transfer files in priority queues other than the highest priority queue in descending order.

[0039] Preferably, the adopting of the limited service rule to sequentially transmit the files in each priority queue other than the highest priority queue in order of priority from high to low includes:

[0040] File transfers are performed on priority queues other than the highest priority queue in descending order of priority;

[0041] Each time a file is transferred, all priority queues with a higher priority than the current queue are searched;

[0042] If the search result is empty, the next file in the current queue is transferred;

[0043] Otherwise, the files in the queue with a higher priority than the one in the current queue are transferred first, and after the transfer is completed, the next file in the current queue is transferred.

[0044] Preferably, after storing all files in corresponding storage paths according to corresponding priority queues, the method further includes:

[0045] Monitor each storage path at all times to determine whether there are files to be transferred until the transfer is complete.

[0046] Preferably, after each file transfer is completed, the file in the storage path is deleted according to user requirements.

[0047] Preferably, in the priority transmission module, files are divided into multiple priority queues based on priority, including:

[0048] Get the request parameters carried by the user when initiating a file distribution request;

[0049] Files are divided into multiple priority queues based on priority according to request parameters.

[0050] Preferably, the request parameters include a distribution destination and a priority of the file.

[0051] Preferably, in the preemptive transmission module, if it is turned on, after the files in the high priority queue are transmitted first, the files in the suspended low priority queue are transmitted at breakpoints, including:

[0052] If the preemption mode is enabled, the files being transferred in the low-priority queue will be paused and recorded as interrupted files;

[0053] The transmitted content is saved at the target node of the interrupted file, and the byte offset of the interrupted file that has been transmitted is saved at the current node;

[0054] Switch to the high priority queue for file transfer. After the transfer is completed, read the byte offset of the interrupted file and continue to transfer the data after the byte offset of the interrupted file.

[0055] Compared with the closest prior art, the present invention has the following beneficial effects:

[0056] The present invention provides a priority preemptive file transmission method and system for edge-cloud collaboration, which divides files into multiple priority queues based on priority, and transmits files in each priority queue in order from high to low priority; if a new file is added to the high priority queue when transmitting a file in the low priority queue, it is determined whether to turn on the preemptive mode according to the user's preset parameters. If it is turned on, the files in the high priority queue are transmitted first, and then the files in the suspended low priority queue are transmitted at a breakpoint; if it is not turned on, the files in the current low priority queue are transmitted first, and then the new files in the high priority queue are transmitted. Using different rules to process multiple priority queues can not only realize the priority transmission of urgent files, but also be more efficient than the sorted transmission of all files in a single priority queue; preemptive transmission is performed for the newly added urgent high priority files, and after the transmission is completed, the files in the suspended low priority queue are transmitted at a breakpoint, which can realize the transmission of high priority files to the destination node as soon as possible, and the introduction of the breakpoint transmission function improves the fault tolerance and transmission efficiency of file transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flowchart of a priority preemptive file transfer method for edge-cloud collaboration provided by the present invention;

[0058] Figure 2 A schematic diagram of a priority-based scanning file storage path of the present invention;

[0059] Figure 3 A schematic diagram of transmission based on queue priority of the present invention;

[0060] Figure 4 A schematic diagram of implementing priority-based preemptive processing in the present invention;

[0061] Figure 5 It is a schematic diagram of the process of breakpoint resume transmission of the present invention;

[0062] Figure 6 A structural schematic diagram of a priority preemptive file transfer system for edge-cloud collaboration provided by the present invention. DETAILED DESCRIPTION

[0063] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0064] Embodiment 1:

[0065] The present invention provides a priority preemptive file transmission method for edge-cloud collaboration, such as Figure 1 As shown, including:

[0066] S1. Divide the files into multiple priority queues based on priority, and transfer the files in each priority queue in order from high to low priority;

[0067] S2. If a new file is added to the high priority queue while the file in the low priority queue is being transmitted, determine whether to enable the preemptive mode based on the user's preset parameters. If enabled, the files in the high priority queue are transmitted first, and then the files in the suspended low priority queue are transmitted at breakpoints. If not enabled, the files in the current low priority queue are transmitted first, and then the newly added files in the high priority queue are transmitted.

[0068] In step S1, the files are divided into multiple priority queues based on priority, and the files in each priority queue are transmitted in order from high to low priority, specifically including:

[0069] S11. Divide the files into multiple priority queues based on priority.

[0070] Specifically, Figure 2 As shown, the user initiates a file distribution request, giving the file to be distributed and request parameters, wherein the request parameters include the distribution destination and priority of the file.

[0071] After the system receives the user's file distribution request, it divides the files into multiple priority queues based on the priority according to the request parameters, and generates a storage path corresponding to each priority queue on the user side and the system's server side, that is, all files are stored in the corresponding storage path according to the corresponding priority queue.

[0072] S12. The files in each priority queue are transmitted in order from high to low priority.

[0073] Specifically, after all files are stored in the corresponding storage paths according to the corresponding priority queues, the system's monitoring program starts and constantly monitors the storage paths corresponding to each priority queue to determine whether there are files that need to be transferred until the entire transfer process is completed.

[0074] The system scans each storage path in order from high to low priority, and transfers the files in the corresponding priority queue in each storage path in turn. If there are files in the priority queue, they are transferred. If there are no files, the system processes the files in the lower queue, and so on.

[0075] like Figure 3 As shown, each storage path is scanned in order from high to low priority, and the full service rule is used to transfer files in the highest priority queue; the limited service rule is used to transfer files in each priority queue other than the highest priority queue in order from high to low priority.

[0076] When the full service rule is used to transfer files in the highest priority queue, all files in the highest priority queue are transferred completely, that is, the queue is empty, and then files in other priority queues are transferred.

[0077] When a priority queue other than the highest priority queue is transferred using a limited service rule, all priority queues with a higher priority than the current queue are searched each time a file is transferred;

[0078] If the search result is empty, the next file in the current queue is transferred; otherwise, the files in the queue with a higher priority than the current queue are transferred first, and the next file in the current queue is transferred after the transfer is completed.

[0079] When performing file transfer based on priority, all priority queues share the same queue scheduling system, that is, they share the same priority scheduling table. During scheduling, the priority level will be used to determine which queue will be scheduled first, thus ensuring fairness among multiple queues during transmission.

[0080] After each file transfer is completed, the file in the storage path is deleted according to user needs.

[0081] Corresponding rules are used for transmission for each priority queue, which can not only realize the priority transmission of urgent files, but also has higher transmission efficiency compared with the sorting transmission of all files in a single priority queue.

[0082] In step S2, if a new file is added to the high priority queue when the file in the low priority queue is being transmitted, it is determined whether to turn on the preemptive mode based on the user's preset parameters. If it is turned on, the files in the high priority queue are transmitted first, and then the files in the paused low priority queue are transmitted at breakpoints; if it is not turned on, the files in the current low priority queue are transmitted first, and then the newly added files in the high priority queue are transmitted.

[0083] Specifically, if a new file is added to the high priority queue while a file in the low priority queue is being transmitted, whether to turn on the preemptive mode is determined based on the user's preset parameters, that is, the switch of the preemptive mode is controlled by the preset parameters. If it is not turned on, the files in the current low priority queue are transmitted first, and then the new files in the high priority queue are transmitted.

[0084] If the preemptive mode is turned on, the file being transmitted in the low-priority queue is paused and recorded as an interrupted file; the transmitted content is saved in the transmission target node of the interrupted file, and the byte offset of the interrupted file that has been transmitted is saved in the current node; switch to the high-priority queue for file transmission. After the transmission is completed, read the byte offset of the interrupted file, and continue to transmit the data after the byte offset of the interrupted file.

[0085] Specifically, Figure 4-Figure 5As shown, if the preemptive mode is started, the file being transmitted in the low priority queue is directly suspended and recorded as an interrupted file;

[0086] ① Use the current node to initiate a data synchronization request to the superior node; ② After the superior node responds to the request data, the current node saves the byte offset of the interrupted file that has been transmitted to the local disk; ③ After the file transmission in the high priority queue is completed, the current node reads the byte offset of the interrupted file and reports the transmitted byte offset to the superior node; ④ The superior node continues to transmit the data after the byte offset of the interrupted file. After the transmission is completed, delete the file in the corresponding storage path according to user needs.

[0087] Through preemptive transmission, high-priority files can be transmitted to the destination node as soon as possible, and the breakpoint transmission function is introduced to improve the fault tolerance and transmission efficiency of file transmission. The system also introduces preemptive mode and priority queue transmission, which can well solve the compatibility of user needs in multi-system and multi-task scenarios, and configure the priority and whether to enable preemptive mode through user preset parameters.

[0088] In the present invention, the user's transmission logic (preemptive transmission and priority queue transmission) and the user's file transmission service (specific file content, file function, file format) are independent of each other and do not affect each other. There is no need to parse the file content before transmission, thus ensuring the file transmission efficiency.

[0089] Embodiment 2:

[0090] Based on the same inventive concept, the present invention also provides a priority preemptive file transfer system for edge-cloud collaboration, such as Figure 6 As shown, it includes: a priority transmission module and a preemptive transmission module.

[0091] Priority transmission module: used to divide files into multiple priority queues based on priority, and transmit the files in each priority queue in order from high to low priority.

[0092] Preemptive transmission module: If a new file is added to the high-priority queue while a file in the low-priority queue is being transmitted, the preemptive mode is determined based on the user's preset parameters. If it is enabled, the files in the high-priority queue are transmitted first, and then the files in the suspended low-priority queue are transmitted at breakpoints. If it is not enabled, the files in the current low-priority queue are transmitted first, and then the newly added files in the high-priority queue are transmitted.

[0093] Specifically, in the priority transmission module, files in each priority queue are transmitted in order from high to low priority, including:

[0094] Store all files in the corresponding storage path according to the corresponding priority queue;

[0095] Scan each storage path in order from high to low priority, and transfer the files in the corresponding priority queue in each storage path in turn.

[0096] Specifically, each storage path is scanned in descending order of priority, and the files in the corresponding priority queue in each storage path are transferred in sequence, including:

[0097] Scan each storage path in order from high to low priority, and transfer the files in the highest priority queue using full service rules;

[0098] Adopt limited service rules to transfer files in priority queues other than the highest priority queue in descending order.

[0099] Specifically, the limited service rules are adopted to sequentially transmit files in the priority queues other than the highest priority queue in descending order of priority, including:

[0100] File transfers are performed on priority queues other than the highest priority queue in descending order of priority;

[0101] Each time a file is transferred, all priority queues with a higher priority than the current queue are searched;

[0102] If the search result is empty, the next file in the current queue is transferred;

[0103] Otherwise, the files in the queue with a higher priority than the one in the current queue are transferred first, and after the transfer is completed, the next file in the current queue is transferred.

[0104] Specifically, after all files are stored in corresponding storage paths according to corresponding priority queues, the following steps are also included:

[0105] Monitor each storage path at all times to determine whether there are files to be transferred until the transfer is complete.

[0106] After each file transfer is completed, the file in the storage path is deleted according to user needs.

[0107] Specifically, in the priority transmission module, files are divided into multiple priority queues based on priority, including:

[0108] Get the request parameters carried by the user when initiating a file distribution request;

[0109] Files are divided into multiple priority queues based on priority according to request parameters.

[0110] The request parameters include the distribution destination and priority of the file.

[0111] Specifically, in the preemptive transmission module, if it is turned on, after the files in the high priority queue are transmitted first, the files in the suspended low priority queue are transmitted at breakpoints, including:

[0112] If the preemption mode is enabled, the files being transferred in the low-priority queue will be paused and recorded as interrupted files;

[0113] The transmitted content is saved at the target node of the interrupted file, and the byte offset of the interrupted file that has been transmitted is saved at the current node;

[0114] Switch to the high priority queue for file transfer. After the transfer is completed, read the byte offset of the interrupted file and continue to transfer the data after the byte offset of the interrupted file.

[0115] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the protection scope of the claims to be approved.

Claims

1. A priority preemptive file transfer method for edge-cloud collaboration, characterized in that: include: Divide files into multiple priority queues based on priority, and transfer files in each priority queue in order from high to low priority; If a new file is added to the high-priority queue while the file in the low-priority queue is being transferred, the preemptive mode is determined based on the user's preset parameters. If it is enabled, the files in the high-priority queue are transferred first, and then the files in the suspended low-priority queue are transferred at breakpoints. If it is not enabled, the files in the current low-priority queue are transferred first, and then the new files in the high-priority queue are transferred.

2. A priority preemptive file transfer method for edge-cloud collaboration as claimed in claim 1, characterized in that: The files in each priority queue are transmitted in order from high to low priority, including: Store all files in the corresponding storage path according to the corresponding priority queue; Scan each storage path in order from high to low priority, and transfer the files in the corresponding priority queue in each storage path in turn.

3. A priority preemptive file transfer method for edge-cloud collaboration as claimed in claim 2, characterized in that: The step of scanning each storage path in descending order of priority and sequentially transmitting files in the corresponding priority queue in each storage path includes: Scan each storage path in order from high to low priority, and transfer the files in the highest priority queue using full service rules; Adopt limited service rules to transfer files in priority queues other than the highest priority queue in descending order.

4. A priority preemptive file transfer method for edge-cloud collaboration as claimed in claim 3, characterized in that: The limited service rules are adopted to sequentially transmit files in priority queues other than the highest priority queue in descending order of priority, including: File transfers are performed on priority queues other than the highest priority queue in descending order of priority; Each time a file is transferred, all priority queues with a higher priority than the current queue are searched; If the search result is empty, the next file in the current queue is transferred; Otherwise, the files in the queue with a higher priority than the one in the current queue are transferred first, and after the transfer is completed, the next file in the current queue is transferred.

5. A priority preemptive file transfer method for edge-cloud collaboration as claimed in claim 2, characterized in that: After storing all files in corresponding storage paths according to corresponding priority queues, the method further includes: Monitor each storage path at all times to determine whether there are files to be transferred until the transfer is complete.

6. A priority preemptive file transfer method for edge-cloud collaboration as claimed in claim 2, characterized in that: After each file transfer is completed, the file in the storage path is deleted according to user needs.

7. A priority preemptive file transfer method for edge-cloud collaboration as claimed in claim 1, characterized in that: The files are divided into multiple priority queues based on priority, including: Get the request parameters carried by the user when initiating a file distribution request; Files are divided into multiple priority queues based on priority according to request parameters.

8. A priority preemptive file transfer method for edge-cloud collaboration as claimed in claim 7, characterized in that: The request parameters include the distribution destination and priority of the file.

9. A priority preemptive file transfer method for edge-cloud collaboration as claimed in claim 1, characterized in that: If enabled, files in the high-priority queue will be transferred first, and then files in the suspended low-priority queue will be transferred at breakpoints, including: If the preemption mode is enabled, the files being transferred in the low-priority queue will be paused and recorded as interrupted files; The transmitted content is saved at the target node of the interrupted file, and the byte offset of the interrupted file that has been transmitted is saved at the current node; Switch to the high priority queue for file transfer. After the transfer is completed, read the byte offset of the interrupted file and continue to transfer the data after the byte offset of the interrupted file.

10. A priority preemptive file transfer system for edge-cloud collaboration, characterized in that: include: Priority transmission module: used to divide files into multiple priority queues based on priority, and transmit files in each priority queue in order from high to low priority; Preemptive transmission module: If a new file is added to the high-priority queue while a file in the low-priority queue is being transmitted, the preemptive mode is determined based on the user's preset parameters. If it is enabled, the files in the high-priority queue are transmitted first, and then the files in the suspended low-priority queue are transmitted at breakpoints. If it is not enabled, the files in the current low-priority queue are transmitted first, and then the newly added files in the high-priority queue are transmitted.

Images