Distributed network file sharing downloading method, device and system and storage medium

Abstract

The invention discloses a distributed network file sharing and downloading method, device and system and a storage medium, and relates to the field of data sharing, and the method comprises the following steps: a first network node analyzes a magnetic link used for file sharing and downloading, and determines a second network node having an ownership relationship based on an analysis result; the second network node obtains identity information and a geographic information header of the first network node, and determines a neighborhood node of the first network node according to the geographic information header; the second network node divides a to-be-transmitted file into a plurality of file blocks, packs the plurality of file blocks and a file header into a data packet, and distributes the data packet to the first network node and a third network node; the first network node analyzes encryption information of a file header in the data packet, and when identity information of the first network node is consistent with identity information of a first network node in the file header, the first network node obtains a neighborhood node from a second network node; and the third network node completes file downloading with the first network node in combination with the file offset information of the file block.

Description

Technical Field

[0001] The present invention relates to the field of data sharing, and more specifically, to a distributed network file sharing downloading method, device, system and storage medium. Background Art

[0002] The existing large file data sharing and downloading methods are mainly distributed through server / client mode, magnet link or BT seed download mode. The server / client mode is conducive to controlling the scope or ownership of file sharing, but the download speed is easily affected by the server application performance and network bandwidth. The magnet link mode obtains the hash value through the visitor, searches the entire network through the hash value, and completes the file location through the file digital fingerprint.

[0003] In the field of satellite applications, remote sensing satellite data, like large file data, has the same problem of low data sharing download speed. Therefore, how to improve the sharing download speed of remote sensing satellite data files is an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a distributed network file sharing download method, device, system and storage medium, which solves the current problem of low sharing download speed of remote sensing satellite data files.

[0005] In a first aspect of the present invention, a distributed network file sharing and downloading method is provided, which is applied to a first network node and a second network node of a distributed network, wherein the first network node is a file downloading party and the second network node is a file owner, and the method comprises:

[0006] The first network node parses the magnetic link for file sharing and downloading, and determines the second network node with ownership relationship based on the parsing result; wherein the ownership relationship refers to the network node to which the file to be transferred belongs;

[0007] The first network node establishes a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and the geographic information header of the first network node, and determines a neighboring node of the first network node according to the geographic information header; wherein the neighboring node refers to a third network node that is close to the first network node in the distributed network;

[0008] The second network node divides the file to be transmitted into a plurality of file blocks according to the number of the third network nodes, packages the plurality of file blocks and the file header into a data packet, and distributes the data packet to the first network node and the third network node; wherein the file header includes encryption information of the file to be transmitted and the identity information of the first network node;

[0009] The first network node parses the encrypted information of the file header in the data packet, and when the identity information of the first network node is consistent with the identity information of the first network node in the file header, the first network node obtains the neighboring node from the second network node;

[0010] The first network node establishes a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block; wherein the file offset information of the file block is determined by the size of the file to be transmitted and the size of the file block.

[0011] In one implementation, the geographic information header consists of a prefix or code of a geographic area.

[0012] In an implementation, the first network node and the network node are each distributed with a data packet, each of which includes a file block and a file header.

[0013] In an implementation, the number of the file blocks is one more than the number of the third network nodes.

[0014] In an implementation, the method further includes: the first network node eliminating some network nodes in the third network nodes according to a preset distance value.

[0015] In one implementation, the method also includes: if one or more third network nodes are offline, the first network node re-establishes a communication link with the second network node, and after the communication link is established, re-determines a fourth network node to replace the offline third network node based on the geographic information header to complete the file download with the first network node.

[0016] In an implementation, if the fourth network node for replacing the offline third network node is not determined, the second network node is used as a replacement node for the offline third network node.

[0017] A second aspect of the present invention provides a distributed network file sharing and downloading system, which is applied to a first network node and a second network node of a distributed network, wherein the first network node is a file downloader and the second network node is a file owner, and the system comprises:

[0018] A second network node determination module, configured for the first network node to parse a magnetic link for file sharing and downloading, and to determine a second network node with an ownership relationship based on the parsing result; wherein the ownership relationship refers to the network node to which the file to be transferred belongs;

[0019] A neighbor node determination module, used for the first network node to establish a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and geographic information header of the first network node, and determines the neighbor node of the first network node according to the geographic information header; wherein the neighbor node refers to a third network node that is close to the first network node in the distributed network;

[0020] A file transmission module, used for the second network node to divide the file to be transmitted into multiple file blocks according to the number of the third network nodes, package the multiple file blocks and the file header into a data packet, and distribute it to the first network node and the third network node; wherein the file header includes encryption information of the file to be transmitted and the identity information of the first network node;

[0021] A neighbor node acquisition module, used for the first network node to parse the encrypted information of the file header in the data packet, and when the identity information of the first network node is consistent with the identity information of the first network node in the file header, the first network node acquires the neighbor node from the second network node;

[0022] The file download module is used for the first network node to establish a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block; wherein the file offset information of the file block is determined by the size of the file to be transmitted and the size of the file block.

[0023] The third aspect of the present invention provides a computer device, comprising: a memory for storing executable instructions; a processor for running the magnetic link-based downloading device to execute a distributed network file sharing downloading method provided in the first aspect of the present invention under the control of the executable instructions.

[0024] A fourth aspect of the present invention provides a readable storage medium, wherein the readable storage medium is used to store a computer program that can be read and executed by a computer, and the computer program is used to execute a distributed network file sharing download method provided in the first aspect of the present invention when read by the computer.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] The present invention first determines the second network node with the ownership relationship of the file to be transmitted through a magnetic link, then the first network node establishes a communication link with the second network node, after the communication link is established, the second network node obtains the identity information and geographic information header of the first network node, and determines the neighboring node of the first network node according to the geographic information header, and quickly determines the network nodes available in the distributed network based on the neighboring nodes; secondly, the second network node divides the file to be transmitted into multiple file blocks according to the number of third network nodes, packages the multiple file blocks and the file header into a data packet, and distributes them to the first network node and the third network node, so that the available distributed network nodes It also has a file block of a file to be transmitted; secondly, the first network node parses the encrypted information of the file header in the data packet, and when the identity information of the first network node is consistent with that of the first network node in the file header, the first network node obtains a neighboring node from the second network node, thereby obtaining a distributed network node that can realize file download, and finally, the first network node establishes a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block, thereby quickly completing the file download of the file to be transmitted through multiple network nodes in the neighboring nodes, thereby solving the problem of low shared download speed of remote sensing satellite data files. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0028] Figure 1 A schematic diagram of a flow chart of a distributed network file sharing and downloading method provided by an embodiment of the present invention;

[0029] Figure 2 A file transfer topology diagram of a distributed network provided by an embodiment of the present invention;

[0030] Figure 3 A principle block diagram of a distributed network file sharing and downloading system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0032] It should be noted that the terms "include" or "may include" used in various embodiments of the present application indicate the presence of the function, operation or element applied for, and do not limit the addition of one or more functions, operations or elements. In addition, as used in various embodiments of the present application, the terms "include", "have" and their cognates are only intended to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items.

[0033] In various embodiments of the present application, the expression "or" or "at least one of B or / and C" includes any combination or all combinations of the words listed at the same time. For example, the expression "B or C" or "at least one of B or / and C" may include B, may include C, or may include both B and C.

[0034] It should be understood that terms such as "first", "second", and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0035] In the field of satellite applications, remote sensing data has certain characteristics such as ownership attributes, remote sensing geographical characteristics, and limited sharing scope. Like large file data, it also has the same point of low sharing download speed. Therefore, how to improve the sharing download speed of remote sensing satellite data files is an urgent problem that needs to be solved.

[0036] In order to solve the problem of low shared download speed, an embodiment of the present invention provides a distributed network file sharing download method. The method first determines the second network node with the ownership relationship of the file to be transmitted through a magnetic link. Then, the first network node establishes a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and geographic information header of the first network node, and determines the neighboring node of the first network node according to the geographic information header, and quickly determines the network nodes available in the distributed network based on the neighboring node; secondly, the second network node divides the file to be transmitted into multiple file blocks according to the number of third network nodes, packages the multiple file blocks and the file header into a data packet, and distributes it to the first network node and the third network node. The invention discloses a method for transmitting the file to a third network node, wherein the first network node parses the encrypted information of the file header in the data packet, and when the identity information of the first network node is consistent with that of the first network node in the file header, the first network node obtains the neighboring node from the second network node, thereby obtaining the distributed network node that can realize file download, and finally, the first network node establishes a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block, thereby quickly completing the file download of the file to be transmitted through multiple network nodes in the neighboring nodes, thereby solving the problem of low shared download speed of remote sensing satellite data files.

[0037] The method provided by the embodiment of the present invention will be described in detail below with reference to specific examples. Figure 1 , Figure 1 The present invention provides a flow chart of a distributed network file sharing and downloading method, which is applied to a first network node and a second network node of a distributed network. The first network node is the file downloader, and the second network node is the file owner. Specifically, the distributed network is formed by interconnecting nodes with multiple terminals distributed in different locations. Any point in the network is connected to at least two lines. When any line fails, communication can be completed through other links, which has high reliability. Figure 1 As shown, the method includes:

[0038] S101, a first network node parses a magnetic link for file sharing and downloading, and determines a second network node with an ownership relationship based on the parsing result; wherein the ownership relationship refers to the network node to which the file to be transferred belongs.

[0039] In this embodiment, the encrypted data file is transmitted in a distributed network, mainly including three types of entities, namely: the first network node, which is the file downloader, defined as P2 in this embodiment. The second network node, which is the file owner, or the file initiator, defined as P1 in this embodiment. Secondly, it also includes neighboring nodes adjacent to the first network node, defined as {Q N |Q1……Q n Among them, the status of subjects is equal, and their structures are composed of geographic information header and data body.

[0040] Furthermore, a magnetic link is a link format that identifies a file through a hash function value, and is mainly used for file sharing in a peer-to-peer (P2P) network according to the Bit Torrent protocol. In this embodiment, the magnetic link does not include Tracker information, which includes the address of the Tracker server and related server configuration required for downloading in the Bit Torrent protocol. For magnetic links that do not include Tracker information, it is usually necessary to download seed files from the DHT network.

[0041] The magnetic link is initiated by the second network node, and the magnetic link includes a hash value generated by the file to be transmitted and a geographic information header.

[0042] The manner in which the first network node parses the magnetic link is a conventional technical means in the technical field, and no redundant description is given in this embodiment.

[0043] S102, the first network node establishes a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and geographic information header of the first network node, and determines the neighboring node of the first network node according to the geographic information header; wherein the neighboring node refers to a third network node that is close to the first network node in the distributed network.

[0044] In this embodiment, since the distributed network is formed by interconnecting node machines distributed in different locations and having multiple terminals, any point in the network is connected to at least two lines. When any line fails, communication can be completed via other links. Therefore, this embodiment does not provide redundant descriptions of the implementation method of the first network node establishing a communication link with the second network node.

[0045] Specifically, the geographic information header consists of the prefix or code of the geographic area, P1, P2, Q N The distance between them is recorded as D = P n+1 -P n , respectively, when the distance is 0, it is D0, and when the distance is within the local area network, it is D -1, the metropolitan area network and the provincial area network are denoted as D c , D p , where D p >D c >D0>D -1 , determine the neighboring nodes that are close to the first network node P2 and are online through these distances, please refer to Figure 2 ,The data file transmission nodes adopt a distributed network structure, and the ,distributed LAN distance D is limited to a certain geographical range, thereby ,increasing the file download rate.

[0046] S103, the second network node divides the file to be transmitted into multiple file blocks according to the number of third network nodes, packages the multiple file blocks and the file header into a data packet, and distributes it to the first network node and the third network node; wherein the file header includes encryption information of the file to be transmitted and the identity information of the first network node.

[0047] In this embodiment, the encrypted data file obtains its data block size and the hash value generated after the hash operation, so that its information is subsequently stored in the corresponding position of the file header data for encryption processing, that is, the header file includes the encryption information of the file to be transmitted. For the file to be transmitted, the second network node divides the data file into (N+1) file blocks by obtaining the number of online neighboring nodes N, and packages the file blocks with the file header. The first network node and the network node are each distributed a data packet, and each data packet includes a file block and a file header, and is sent to the first network node P2 and the neighboring node Q respectively. N The offset of the file block is determined by the file size and the data file block size, denoted as offsetN. The first network node P2 and the neighboring node Q N The data block size obtained at the beginning of data transmission is consistent with the offset offsetN.

[0048] S104, the first network node parses the encrypted information of the file header in the data packet, and when the identity information of the first network node is consistent with the identity information of the first network node in the file header, the first network node obtains the neighboring node from the second network node,

[0049] In this embodiment, the first network node P2 and the neighboring node Q NAfter receiving the data packet, the file header is decrypted and the data is compared. When the first network node P2 is consistent with the recipient in the file header, the first network node P2 obtains the information of the neighboring node from the second network node P1. The first network node P2 attempts to send communication message data to the second network node P1, where the communication message data contains the geographic information header and identity information of the first network node P2. If the second network node P1 monitors the data, it verifies through the magnetic link that the data usage rights of the first network node P2 are consistent with the user information, and then sends the complete file offset information to the first network node P2, and informs the neighboring node Q of the first network node P2 at the same time. N .

[0050] S105, the first network node establishes a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block; wherein the file offset information of the file block is determined by the size of the file to be transmitted and the size of the file block.

[0051] In this embodiment, the first network node P2 communicates with the domain node Q N A communication link is established, and the data download of the file block offsetN is completed in combination with the file offset information with the transmission file received by the first network node P2.

[0052] In a further embodiment, the method also includes: if one or more third network nodes are offline, the first network node re-establishes a communication link with the second network node, and after the communication link is established, re-determines a fourth network node to replace the offline third network node based on the geographic information header to complete the file download with the first network node.

[0053] Secondly, if the fourth network node for replacing the offline third network node is not determined, the second network node is used as a replacement node for the offline third network node.

[0054] In some embodiments, the method further includes: the first network node eliminating some network nodes in the third network nodes according to a preset distance value.

[0055] In this embodiment, the distance value is set to exclude some network nodes in the third network node in order to determine a valid neighborhood range to increase the download rate of the file.

[0056] Please refer to Figure 3 , Figure 3 A principle block diagram of a distributed network file sharing and downloading system provided by an embodiment of the present invention is shown in the figure. The system is applied to a first network node and a second network node of a distributed network, wherein the first network node is a file downloader and the second network node is a file owner. Figure 3 As shown, the system includes:

[0057] The second network node determination module 310 is used for the first network node to parse the magnetic link for file sharing and downloading, and determine the second network node with ownership relationship based on the parsing result; wherein the ownership relationship refers to the network node to which the file to be transferred belongs;

[0058] A neighbor node determination module 320 is used for the first network node to establish a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and the geographic information header of the first network node, and determines the neighbor node of the first network node according to the geographic information header; wherein the neighbor node refers to a third network node that is close to the first network node in the distributed network;

[0059] The file transmission module 330 is used for the second network node to divide the file to be transmitted into multiple file blocks according to the number of the third network nodes, package the multiple file blocks and the file header into a data packet, and distribute it to the first network node and the third network node; wherein the file header includes encryption information of the file to be transmitted and the identity information of the first network node;

[0060] A neighbor node acquisition module 340, configured for the first network node to parse the encrypted information of the file header in the data packet, and when the identity information of the first network node is consistent with the identity information of the first network node in the file header, the first network node acquires the neighbor node from the second network node;

[0061] The file download module 350 is used for the first network node to establish a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block; wherein the file offset information of the file block is determined by the size of the file to be transmitted and the size of the file block.

[0062] In this embodiment, a computer device is also provided, including: a memory for storing executable instructions; a processor for running the magnetic link-based downloading device to execute a distributed network file sharing downloading method provided in the first aspect of the present invention under the control of the executable instructions.

[0063] In this embodiment, a readable storage medium is also provided, wherein the readable storage medium is used to store a computer program that can be read and executed by a computer, and the computer program is used to execute a distributed network file sharing download method provided in the first aspect of the present invention when read by the computer.

[0064] A readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. The readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the above. The readable storage medium used herein is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0065] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A distributed network file sharing and downloading method, characterized in that: Applied to a first network node and a second network node of a distributed network, the first network node is a file downloader, and the second network node is a file owner, the method includes: The first network node parses the magnetic link for file sharing and downloading, and determines the second network node with ownership relationship based on the parsing result; wherein the ownership relationship refers to the network node to which the file to be transferred belongs; The first network node establishes a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and the geographic information header of the first network node, and determines a neighboring node of the first network node according to the geographic information header; wherein the neighboring node refers to a third network node that is close to the first network node in the distributed network; The second network node divides the file to be transmitted into a plurality of file blocks according to the number of the third network nodes, packages the plurality of file blocks and the file header into a data packet, and distributes the data packet to the first network node and the third network node; wherein the file header includes encryption information of the file to be transmitted and the identity information of the first network node; The first network node parses the encrypted information of the file header in the data packet, and when the identity information of the first network node is consistent with the identity information of the first network node in the file header, the first network node obtains the neighboring node from the second network node; The first network node establishes a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block; wherein the file offset information of the file block is determined by the size of the file to be transmitted and the size of the file block.

2. A distributed network file sharing and downloading method according to claim 1, characterized in that: The geographic information header consists of a prefix or code of a geographic area.

3. A distributed network file sharing and downloading method according to claim 1, characterized in that: The first network node and the network node are each distributed with a data packet, each of which includes a file block and a file header.

4. A distributed network file sharing and downloading method according to claim 1, characterized in that: The number of the file blocks is one more than the number of the third network nodes.

5. A distributed network file sharing and downloading method according to claim 1, characterized in that: The method further includes: the first network node eliminating some network nodes in the third network nodes according to a preset distance value.

6. A distributed network file sharing and downloading method according to claim 1, characterized in that: The method also includes: if one or more third network nodes are offline, the first network node re-establishes a communication link with the second network node, and after the communication link is established, re-determines a fourth network node to replace the offline third network node based on the geographic information header to complete the file download with the first network node.

7. A distributed network file sharing and downloading method according to claim 6, characterized in that: If the fourth network node for replacing the offline third network node is not determined, the second network node is used as a replacement node for the offline third network node.

8. A distributed network file sharing and downloading system, characterized in that: A first network node and a second network node applied to a distributed network, the first network node is a file downloader, and the second network node is a file owner, the system includes: A second network node determination module, configured for the first network node to parse a magnetic link for file sharing and downloading, and to determine a second network node with an ownership relationship based on the parsing result; wherein the ownership relationship refers to the network node to which the file to be transferred belongs; A neighbor node determination module, used for the first network node to establish a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and geographic information header of the first network node, and determines the neighbor node of the first network node according to the geographic information header; wherein the neighbor node refers to a third network node that is close to the first network node in the distributed network; A file transmission module, used for the second network node to divide the file to be transmitted into multiple file blocks according to the number of the third network nodes, package the multiple file blocks and the file header into a data packet, and distribute it to the first network node and the third network node; wherein the file header includes encryption information of the file to be transmitted and the identity information of the first network node; A neighbor node acquisition module, used for the first network node to parse the encrypted information of the file header in the data packet, and when the identity information of the first network node is consistent with the identity information of the first network node in the file header, the first network node acquires the neighbor node from the second network node; The file download module is used for the first network node to establish a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block; wherein the file offset information of the file block is determined by the size of the file to be transmitted and the size of the file block.

9. A computer device, characterized in that: include: A memory for storing executable instructions; A processor is used to run the magnetic link-based downloading device to execute a distributed network file sharing downloading method as described in any one of claims 1 to 7 under the control of the executable instructions.

10. A readable storage medium, wherein: The readable storage medium is used to store a computer program that can be read and executed by a computer, and the computer program is used to execute a distributed network file sharing and downloading method as described in any one of claims 1 to 7 when read by the computer.

Images