Distributed network file sharing and downloading method, device, system and storage medium

Abstract

The application discloses a distributed network file sharing download method, device, system and storage medium, relates to the field of data sharing, and the method comprises the following steps: a first network node analyzes a magnet link for file sharing download, determines a second network node with a right ownership relationship based on an analysis result; the second network node acquires identity information and geographic information of the first network node, and determines a neighborhood node of the first network node according to the geographic information; the second network node divides a to-be-transmitted file into a plurality of file blocks, packs the plurality of file blocks and a file header into a data packet, and distributes the data packet to the first network node and a third network node; the first network node analyzes encrypted information of the file header in the data packet, and when the identity information of the first network node in the file header is consistent with the first network node, the first network node acquires the neighborhood node from the second network node; and the third network node completes file downloading with the first network node in combination with file offset information of the file blocks.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data sharing, more particularly, it relates to a distributed network file sharing download method, device, system and storage medium. BACKGROUND

[0002] The existing large file data sharing download mode mainly carries out data sharing distribution through the server / client mode, the magnet link or the BT seed download mode. The server / client mode is beneficial to the control of the file sharing range or the ownership, but the download speed is easily affected by the server application performance and the network bandwidth, the magnet link mode obtains the hash value through the visitor, carries out the whole network retrieval in the distributed network through the hash value, and completes the file positioning through the file digital fingerprint.

[0003] In the satellite application field, the remote sensing satellite data and the large file data have the same point of low data sharing download speed, therefore, how to improve the sharing download speed of the remote sensing satellite data file is a problem to be solved at present. SUMMARY

[0004] The present application provides a distributed network file sharing download method, device, system and storage medium, which solves the problem of low sharing download speed of the remote sensing satellite data file at present.

[0005] The first aspect of the present application provides a distributed network file sharing download method, which is applied to a first network node and a second network node of a distributed network, the first network node is a file download party, and the second network node is a file owner party, and the method comprises the following steps:

[0006] The first network node parses a magnet link for file sharing download, and determines the second network node with the ownership relationship based on the parsing result; wherein the ownership relationship refers to the network node to which the to-be-transmitted file belongs;

[0007] The first network node establishes a communication link with the second network node, after the communication link is established, the second network node obtains the identity information and the geographic information header of the first network node, and determines the neighborhood nodes of the first network node according to the geographic information header; wherein the neighborhood nodes refer to the third network nodes which are close to the position of the first network node in the distributed network;

[0008] The second network node divides the to-be-transmitted file into a plurality of file blocks according to the number of the third network nodes, packs the plurality of file blocks and the file header into a data packet, and distributes the data packet to the first network node and the third network nodes; wherein the file header comprises the encrypted information of the to-be-transmitted file and the identity information of the first network node;

[0009] The first network node parses the encryption information of the file header in the data packet, and when the identity information of the first network node in the file header is consistent with the first network node, the first network node acquires the neighborhood nodes from the second network node.

[0010] The first network node establishes a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block; wherein the file offset information of the file block is determined by the size of the file to be transmitted and the size of the file block.

[0011] In an implementation scheme, the geographic information header is composed of a crown word or code of a geographic area.

[0012] In an implementation scheme, the first network node and the network node are each distributed a data packet, and each data packet includes a file block and a file header.

[0013] In an implementation scheme, the number of file blocks is one more than the number of third network nodes.

[0014] In an implementation scheme, the method further includes: the first network node eliminates part of the third network nodes according to a preset distance value.

[0015] In an implementation scheme, the method further includes: if one or more third network nodes are offline, the first network node re-establishes a communication link with the second network node, and after the communication link is established, a fourth network node is determined according to the geographic information header to replace the offline third network node, so as to complete the file download with the first network node.

[0016] In an implementation scheme, if the fourth network node to replace the offline third network node is not determined, the second network node is used as the replacement node of the offline third network node.

[0017] The second aspect of the application provides a distributed network file sharing download system applied to a first network node and a second network node of a distributed network, wherein the first network node is a file download party, and the second network node is a file owner party, and the system comprises:

[0018] A second network node determination module is configured to parse a magnet link for file sharing download by the first network node, and determine the second network node with a property right relationship based on the parsing result; wherein the property right relationship refers to a network node to which a file to be transmitted belongs.

[0019] The neighborhood node determining module is used for the first network node to establish a communication link with the second network node, and after the communication link is established, the second network node acquires the identity information and the geographic information header of the first network node, and determines the neighborhood node of the first network node according to the geographic information header; wherein the neighborhood node refers to a third network node which is close to the position of the first network node in the distributed network;

[0020] The file transmission module is used for the second network node to divide the to-be-transmitted file into a plurality of file blocks according to the number of the third network nodes, to pack the plurality of file blocks and the file header into data packets, and to distribute the data packets to the first network node and the third network nodes; wherein the file header comprises encrypted information of the to-be-transmitted file and the identity information of the first network node;

[0021] The neighborhood node acquiring module is used for the first network node to parse the encrypted information of the file header in the data packet, and when the identity information of the first network node in the file header is consistent with that of the first network node, the first network node acquires the neighborhood node from the second network node.

[0022] The file downloading module is used for the first network node to establish a communication link with the third network node, and the third network node to complete the file downloading from the first network node in combination with the file offset information of the file block; wherein the file offset information of the file block is determined by the size of the to-be-transmitted file and the size of the file block.

[0023] The third aspect of the present application provides a computer device, comprising: a memory for storing executable instructions; a processor for running the distributed network file sharing download method provided by the first aspect of the present application according to the control of the executable instructions.

[0024] The fourth aspect of the present application provides a readable storage medium, wherein the readable storage medium is used for storing a computer program which can be read and executed by a computer, and the computer program is used for executing the distributed network file sharing download method provided by the first aspect of the present application when being read by the computer.

[0025] Compared with the prior art, the present application has the following beneficial effects:

[0026] The application firstly determines the second network node of the file ownership relationship to be transmitted through magnetic link, and then the first network node establishes a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and geographic information header of the first network node, and determines the neighborhood nodes of the first network node according to the geographic information header, and quickly determines the available network nodes of the distributed network based on the neighborhood nodes. Secondly, the second network node divides the file to be transmitted into multiple file blocks according to the number of the third network nodes, packs the multiple file blocks and the file header into data packets, and distributes them to the first network node and the third network nodes, so that the available distributed network nodes also have the file blocks of the file to be transmitted. Thirdly, the first network node parses the encryption information of the file header in the data packet. When the identity information of the first network node in the file header is consistent with that of the first network node, the first network node obtains the neighborhood nodes from the second network node, so as to obtain the distributed network nodes that can realize file download. Finally, the first network node establishes a communication link with the third network node, and the third network node completes the file download with the first network node in combination with the file offset information of the file block. In this way, the file download of the file to be transmitted is quickly completed through the multiple network nodes in the neighborhood nodes, so as to solve the problem of low sharing and downloading speed of the remote sensing satellite data file. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings, which are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0028] Figure 1 A flowchart of a distributed network file sharing and downloading method provided by the embodiments of the application;

[0029] Figure 2 A file transmission topology diagram of a distributed network provided by the embodiments of the application;

[0030] Figure 3 A principle block diagram of a distributed network file sharing and downloading system provided by the embodiments of the application. DETAILED DESCRIPTION

[0031] In order to make the objects, technical solutions and advantages of the application clearer, the application will be further described in detail below with reference to the embodiments and drawings, and the illustrative embodiments of the application and the description thereof are only used to explain the application, and do not limit the application.

[0032] It is to be understood that the terms "including", "comprising", "consisting of", and "consisting essentially of", as used in various embodiments of the present application, indicate the presence of the stated function, operation, or element but do not preclude the presence or addition of one or more other features, numerals, steps, operations, elements, components, or combinations thereof. Also, as used in various embodiments of the present application, the terms "including", "containing", "having", and variations thereof mean the presence of the stated features, numerals, steps, operations, elements, components, or combinations thereof, but do not preclude the presence or addition of one or more other features, numerals, steps, operations, elements, components, or combinations thereof.

[0033] In various embodiments of the present application, the expression "or" or "at least one of B or / and C" includes any combination or all combinations of the listed terms. For example, the expression "B or C" or "at least one of B or / and C" can include B, can include C, or can include both B and C.

[0034] It is to be understood that terms such as "first", "second", "third", etc., are used only for descriptive purposes and are not to be construed as indicating or implying relative importance or a specific number of the technical features indicated. Thus, the features defined with "first", "second", "third" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0035] In the field of satellite applications, remote sensing data has certain ownership attributes, remote sensing geographical regional characteristics, and limited remote sensing data sharing range, and has the same points as large file data in terms of low sharing download speed. Therefore, how to improve the sharing download speed of remote sensing satellite data files is a problem that needs to be solved at present.

[0036] To solve the problem of low shared download speed, the embodiment of the application provides a distributed network file shared download method, which comprises the following steps: firstly, a second network node with file ownership relationship to be transmitted is determined through a magnetic link; then, a first network node establishes a communication link with the second network node; after the communication link is established, the second network node acquires identity information and geographic information of the first network node, and determines the neighborhood nodes of the first network node according to the geographic information, and based on the neighborhood nodes, the available network nodes of the distributed network are quickly determined; secondly, the second network node divides the file to be transmitted into multiple file blocks according to the number of third network nodes, packs the multiple file blocks and a file header into a data packet, and distributes the data packet to the first network node and the third network nodes, so that the available distributed network nodes also have the file blocks of the file to be transmitted; thirdly, the first network node analyzes the encryption information of the file header in the data packet, and when the identity information of the first network node in the file header is consistent with the identity information of the first network node, the first network node acquires the neighborhood nodes from the second network node, so as to acquire the distributed network nodes that can realize file download; finally, the first network node establishes a communication link with the third network nodes, and the third network nodes complete the file download from the first network nodes in combination with the file offset information of the file blocks, so that the file download of the file to be transmitted is quickly completed through the multiple network nodes in the neighborhood nodes, and the problem of low shared download speed of the current remote sensing satellite data file is solved.

[0037] The method provided by the embodiment of the application will be described in detail below with reference to specific examples. Figure 1 , Figure 1 A flowchart of a distributed network file shared download method provided by the embodiment of the application is shown in the figure, the method is applied to a first network node and a second network node of a distributed network, the first network node is a file download party, and the second network node is a file owner. Specifically, the distributed network is formed by nodes interconnected by nodes distributed in different places and having multiple terminals. Any point in the network is connected to at least two lines, when any one line fails, communication can be completed through other links, and the network has high reliability. Figure 1 The method comprises the following steps:

[0038] S101, the first network node analyzes a magnetic link for file shared download, and determines a second network node with file ownership relationship based on the analysis result; wherein the file ownership relationship refers to a network node to which a file to be transmitted belongs.

[0039] In the embodiment, the encrypted data file is transmitted in a distributed network, which mainly includes three types of subjects, i.e., a first network node, which is defined as P2 in the embodiment, as a file download party, a second network node, which is defined as P1 in the embodiment, as a file owner or a file initiator, and a neighborhood node adjacent to the first network node, which is defined as {Q N |Q1……Q n} in the embodiment. Wherein, the subjects are equal in status, and the structure of each subject is composed of a geographic information header and a data body.

[0040] Further, the magnet link is a link format for identifying a file through a hash function value, which is mainly used for file sharing in a peer-to-peer (P2P) network according to the Bit Torrent protocol. In the embodiment, the magnet link does not include Tracker information, and the Tracker information includes the address of the Tracker server and the related server configuration necessary for the Bit Torrent protocol download. For the magnet link without Tracker information, the seed file usually needs to be downloaded from the DHT network.

[0041] The magnet link is initiated by the second network node, and the magnet link includes the hash value and the geographic information header of the file to be transmitted.

[0042] The first network node analyzes the magnet link in a manner belonging to the conventional technical means in the technical field, and the embodiment does not make redundant description.

[0043] In S102, the first network node establishes a communication link with the second network node, and after the communication link is established, the second network node obtains the identity information and the geographic information header of the first network node, and determines the neighborhood nodes of the first network node according to the geographic information header. Wherein, the neighborhood nodes refer to the third network nodes close to the first network node in the distributed network.

[0044] In the embodiment, the distributed network is interconnected by nodes distributed in different places and having multiple terminals. Any point in the network is connected to at least two lines, and when any line fails, communication can be completed through other links. Therefore, the embodiment does not make redundant description on the implementation manner of establishing the communication link between the first network node and the second network node.

[0045] Specifically, the geographic information header is composed of the crown word or code of the geographic area, P1, P2, Q N The distance between them is D=P n+1 -P n , respectively, D0 is recorded when the distance is 0, and D -1, the metropolitan area network and the provincial area network are respectively denoted as D c , D p , wherein D p >D c >D0>D -1 The neighborhood nodes close to the first network node P2 and online are determined through the distances, please refer to Figure 2 The data file transmission node adopts a distributed network structure, and the distance D of the distributed local area network is limited within a certain geographical range, so as to improve the download rate of the file.

[0046] S103, the second network node divides the to-be-transmitted file into multiple file blocks according to the number of the third network nodes, packs the multiple file blocks and the file header into a data packet, and distributes the data packet to the first network node and the third network nodes; wherein the file header includes encrypted information of the to-be-transmitted file and the identity information of the first network node.

[0047] In this embodiment, the encrypted data file is generated by obtaining the data block size and the hash value generated by the hash operation, so as to facilitate subsequent saving of the information in the corresponding position of the header data of the file for encryption processing, that is, the header file includes the encrypted information of the to-be-transmitted file. For the to-be-transmitted file, the second network node divides the data file into (N+1) file blocks by obtaining the number N of the neighborhood nodes online, packs the file blocks and the file header, and distributes a data packet to the first network node and the network nodes, wherein each data packet includes a file block and a file header, and is respectively distributed to the first network node P2 and the neighborhood nodes Q N The offset of the file block is determined by the file size and the data block size, and is denoted as offsetN. The first network node P2 and the neighborhood nodes Q N The data block size and the offset offsetN are consistent at the beginning of the data transmission.

[0048] S104, the first network node parses the encrypted information of the file header in the data packet, and when the identity information of the first network node in the file header is consistent with that of the first network node, the first network node obtains the neighborhood nodes from the second network node,

[0049] In this embodiment, the first network node P2 and the neighborhood nodes Q NAfter obtaining the data packet, the file header is decrypted and data is compared. When the first network node P2 is consistent with the receiver in the file header, the first network node P2 obtains the information of the neighborhood node from the second network node P1. The first network node P2 attempts to send the communication message data to the second network node P1, wherein the communication message data contains the geographic information header and the identity information of the first network node P2. If the second network node P1 listens to the data, and the data usage permission of the first network node P2 is consistent with the user information after the magnetic link verification, the second network node P1 sends the complete file offset information to the first network node P2, and informs the neighborhood node Q of the first network node P2 N .

[0050] In S105, the first network node establishes a communication link with the third network node, and the third network node completes the file download from the first network node in combination with the file offset information of the file block. The file offset information of the file block is determined by the size of the file to be transmitted and the size of the file block.

[0051] In this embodiment, the first network node P2 establishes a communication link with the neighborhood node Q N , and completes the data download of the file block offsetN in combination with the file offset information of the file to be transmitted received by the first network node P2.

[0052] In a further embodiment, the method further comprises: if one or more third network nodes are offline, the first network node re-establishes a communication link with the second network node, and after establishing the communication link, a fourth network node is determined according to the geographic information header to replace the offline third network node, so as to complete the file download from the first network node.

[0053] Secondly, if the fourth network node is not determined to replace the offline third network node, the second network node serves as the replacement node of the offline third network node.

[0054] In some embodiments, the method further comprises: the first network node eliminates part of the third network nodes according to a preset distance value.

[0055] In this embodiment, the preset distance value is used to eliminate part of the third network nodes, so as to determine the effective neighborhood range and improve the download rate of the file.

[0056] Please refer to Figure 3 , Figure 3 A principle block diagram of a distributed network file sharing and downloading system provided by the embodiment of the present application is shown in the figure. The system is applied to the first network node and the second network node of the distributed network, wherein the first network node is the file downloading party, and the second network node is the file owner.Figure 3 As shown in the figure, the system comprises:

[0057] The second network node determining module 310 is configured to determine the second network node with the ownership relationship based on the result of the parsing of the first network node on the magnet link for the file sharing download, wherein the ownership relationship refers to the network node to which the file to be transmitted belongs.

[0058] The neighborhood node determining module 320 is configured to establish a communication link between the first network node and the second network node, and after the establishment of the communication link, the second network node acquires the identity information and the geographic information header of the first network node, and determines the neighborhood nodes of the first network node according to the geographic information header, wherein the neighborhood nodes refer to the third network nodes that are close to the first network node in the distributed network.

[0059] The file transmission module 330 is configured to divide the file to be transmitted into a plurality of file blocks according to the number of the third network nodes, pack the plurality of file blocks and the file header into data packets, and distribute the data packets to the first network node and the third network nodes, wherein the file header comprises encrypted information of the file to be transmitted and the identity information of the first network node.

[0060] The neighborhood node acquiring module 340 is configured to parse the encrypted information of the file header in the data packet, and when the identity information of the first network node in the file header is consistent with that of the first network node, the first network node acquires the neighborhood nodes from the second network node.

[0061] The file downloading module 350 is configured to establish a communication link between the first network node and the third network node, and the third network node completes the file downloading from the first network node in combination with the file offset information of the file block, wherein the file offset information of the file block is determined by the size of the file to be transmitted and the size of the file block.

[0062] In the embodiment, a computer device is also provided, comprising a memory for storing executable instructions, and a processor for running the executable instructions to control the downloading device based on the magnet link to execute the distributed network file sharing download method provided in the first aspect of the application.

[0063] In the embodiment, a readable storage medium is also provided, wherein the readable storage medium is used to store a computer program that can be read and executed by a computer, and the computer program is used to execute the distributed network file sharing download method provided in the first aspect of the application when being read by the computer.

[0064] The readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The readable storage medium can be, for example but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0065] The above description is provided as an enabling teaching of the application. It is to be understood that the above description is intended to be illustrative and not restrictive. Many other possible variations, modifications, and implementations can be readily devised by those skilled in the art with the benefit of the present disclosure, without departing from the scope of the application. Accordingly, the scope of the application should be determined not with reference to the above description, but instead with reference to the appended claims, along with their full scope of equivalents.

Claims

1. A distributed network file sharing and download method, characterized in that, This method applies to a distributed network where the first network node is the file downloader and the second network node is the file owner. The method includes: The first network node resolves the magnet link used for file sharing and downloading, and determines the second network node with ownership based on the resolution result; where the second network node refers to the network node to which the file to be transferred belongs; The first network node establishes a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and geographic information header of the first network node, and determines the neighboring nodes of the first network node based on the geographic information header. The neighboring nodes refer to the third network nodes that are located close to the first network node in the distributed network. The second network node divides the file to be transmitted into multiple file blocks based on the number of third network nodes. The multiple file blocks and the file header are packaged into a data packet and distributed to the first network node and the third network node. The file header includes encrypted information of the file to be transmitted and the identity information of the first network node. The first network node and the third network node are each distributed a data packet, and each data packet includes a file block and a file header. The first network node parses the encrypted information in the file header of the data packet. When the identity information of the first network node matches that of the first network node in the file header, the first network node obtains the neighboring nodes from the second network node. The first network node establishes a communication link with the third network node, and the third network node completes the file download from the first network node by combining the file offset information of the file blocks; wherein, the file offset information of the file blocks is determined by the size of the file to be transferred and the size of the file blocks.

2. The distributed network file sharing and download method according to claim 1, characterized in that, The geographic information header consists of a prefix or code for the geographic region.

3. The distributed network file sharing and download method according to claim 1, characterized in that, The number of file blocks is one more than the number of third network nodes.

4. The distributed network file sharing and download method according to claim 1, characterized in that, The method further includes: the first network node removing some network nodes from the third network node based on a pre-set distance value.

5. The distributed network file sharing and download method according to claim 1, characterized in that, The method further includes: if one or more third network nodes are offline, the first network node re-establishes a communication link with the second network node; after establishing the communication link, a fourth network node is re-determined based on the geographic information header to replace the offline third network node, so as to complete the file download with the first network node.

6. The distributed network file sharing and download method according to claim 5, characterized in that, If a fourth network node is not determined to replace the offline third network node, then the second network node shall be used as the replacement node for the offline third network node.

7. A distributed network file sharing and download system, characterized in that, The system utilizes a first network node and a second network node in a distributed network. The first network node is the file downloader, and the second network node is the file owner. The system includes: The second network node determination module is used by the first network node to parse the magnet link used for file sharing and downloading, and to determine the second network node with ownership relationship based on the parsing result; wherein, the second network node refers to the network node to which the file to be transferred belongs; The neighbor node determination module is used for the first network node to establish a communication link with the second network node. After the communication link is established, the second network node obtains the identity information and geographic information header of the first network node, and determines the neighbor nodes of the first network node based on the geographic information header. The neighbor node refers to a third network node that is geographically close to the first network node in the distributed network. The file transfer module is used by the second network node to divide the file to be transferred into multiple file blocks according to the number of third network nodes, and to package the multiple file blocks and file headers into a data packet, which is then distributed to the first network node and the third network node. The file header includes encrypted information of the file to be transferred and the identity information of the first network node. Both the first network node and the third network node are distributed a data packet, and each data packet includes a file block and a file header. The neighbor node acquisition module is used by the first network node to parse the encrypted information in the file header of the data packet. When the identity information of the first network node matches that of the first network node in the file header, the first network node obtains the neighbor node from the second network node. The file download module is used by the first network node to establish a communication link with the third network node. The third network node completes the file download from the first network node by combining the file offset information of the file block. The file offset information of the file block is determined by the size of the file to be transferred and the size of the file block.

8. A computer device, characterized in that, include: Memory is used to store executable instructions; A processor for executing program instructions to cause the computer device to perform a distributed network file sharing download method as described in any one of claims 1 to 6.

9. A readable storage medium, wherein, The readable storage medium is used to store a computer program that can be read and executed by a computer, the computer program being used to execute a distributed network file sharing and downloading method as described in any one of claims 1-6 when read by the computer.

Images