File acquisition method, device, node device and computer-readable storage medium

Abstract

The present invention relates to the technical field of peer-to-peer networks, and to a file acquisition method, device, node equipment, and computer-readable storage medium, the method comprising: receiving a file acquisition request sent by a requesting client; determining a first index node to which a partition to which a shared file to be acquired belongs according to a file identifier and first search information in the file acquisition request; sending the file identifier to the first index node, so that the first index node retrieves a second index node according to the file identifier and second search information, and returns the second index node to the requesting node when the search is successful; if the returned second index node is received, sending the file identifier to the second index node, so that the second index node acquires the shared file to be acquired according to the file identifier and third search information, and finally returns it to the requesting client. The present invention improves the acquisition efficiency of shared files in a P2P network.

Description

Technical Field

[0001] The present invention relates to the technical field of peer-to-peer networks, and in particular to a file acquisition method, device, node equipment and computer-readable storage medium. Background Art

[0002] In recent years, competition in the live broadcast and on-demand industry has intensified. High bandwidth costs have become a problem that all companies in the industry must face. In addition to live streaming, various scenarios such as gift packages, resource packages, and on-demand files will also incur huge bandwidth costs.

[0003] Peer-to-Peer (P2P) technology, also known as peer-to-peer Internet technology, relies on the computing power and bandwidth of participants in the network, rather than concentrating all the reliance on a few servers, providing a good solution for application scenarios with massive users and massive files.

[0004] How to improve the efficiency of obtaining shared files in a P2P network is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0005] The object of the present invention is to provide a file acquisition method, apparatus, node device and computer-readable storage medium, which can improve the acquisition efficiency of shared files in a P2P network.

[0006] The embodiments of the present invention can be implemented as follows:

[0007] In a first aspect, the present invention provides a file acquisition method, which is applied to a requesting node among multiple nodes in a peer-to-peer (P2P) network, wherein at least one of the nodes is connected in communication with at least one client, and at least one client stores a shared file that can be shared through the P2P network, and all the shared files stored by the clients are pre-divided into at least one partition, and each of the nodes stores first search information, second search information, and third search information, wherein the first search information is used to search for a node to which a partition belongs, the second search information is used to search for a node storing the third search information of a shared file in the partition of the node, and the third search information is used to search for a client to which the shared file stored by the node belongs, and the requesting node is connected in communication with the requesting client, and the method comprises:

[0008] Receiving a file acquisition request sent by the requesting client, wherein the file acquisition request includes a file identifier of a shared file to be acquired;

[0009] Determine, according to the file identifier and the local first search information, a first index node to which the partition to which the shared file to be obtained belongs;

[0010] Sending the file identifier to the first index node, so that the first index node searches for a second index node according to the file identifier and local second search information, and returns the second index node to the requesting node when the search is successful, wherein the second index node is a node storing the third search information of the shared file to be obtained;

[0011] If a returned second index node is received, a file identifier is sent to the second index node so that the second index node obtains the shared file to be obtained based on the file identifier and local third search information and returns the shared file to be obtained to the requesting node, so as to be returned to the requesting client through the requesting node.

[0012] In an optional embodiment, the method further comprises:

[0013] If the second index node is not received, randomly selecting a preset number of candidate nodes from the multiple nodes;

[0014] The file identifier is broadcast to the candidate node, so that the candidate node retrieves the target client storing the shared file to be obtained according to the third retrieval information stored locally, and obtains the shared file to be obtained when the retrieval is successful and returns the shared file to be obtained to the requesting node, so as to be returned to the requesting client through the requesting node.

[0015] In an optional implementation manner, the first index node retrieves the second index node according to the file identifier and the local second search information in the following manner:

[0016] If the first index node fails to read the local second search information, it is determined that the search fails;

[0017] If the first index node reads the local second search information successfully, the first index node determines the second index node according to the file identifier and the local second search information.

[0018] In an optional implementation manner, the second search information of the first index node is updated in the following manner:

[0019] When any of the nodes receives a first file synchronization request sent by a client, the synchronization node to which the partition to which the first file to be synchronized belongs is determined according to the first file identifier to be synchronized carried in the first file synchronization request and the local first search information;

[0020] If the synchronization node is the first index node, the first file to be synchronized and the node receiving the first file synchronization request are sent to the first index node, so that the first index node updates the second search information of the first index node according to the first file to be synchronized and the node receiving the first file synchronization request.

[0021] In an optional implementation manner, the third search information of the candidate node is updated in the following manner:

[0022] When any client sends a second file synchronization request to the candidate node, the third search information of the candidate node is updated according to the second to-be-synchronized file identifier carried in the second file synchronization request and the corresponding client.

[0023] In an optional implementation manner, the second index node acquires the to-be-acquired shared file in the following manner according to the file identifier and the local third search information:

[0024] Determine a target client storing the to-be-acquired shared file according to the file identifier and the local third search information;

[0025] Sending the file identifier to the target client;

[0026] The target client obtains the sharing priority of the shared file to be obtained according to the file identifier, and sends the shared file to be obtained to the second index node according to the sharing priority, wherein the sharing priority is set according to the popularity of the shared file to be obtained, and the popularity of the shared file to be obtained is obtained from a statistical node among the multiple nodes, and the second retrieval information stored in the statistical node includes a node storing the third retrieval information of the shared file to be obtained.

[0027] In an optional implementation manner, the statistical node counts the number of downloads of the to-be-acquired shared file, and the heat of the to-be-acquired shared file is calculated as follows:

[0028] The counting node counts the number of clients storing the to-be-acquired shared file according to the local second search information;

[0029] The statistical node calculates the popularity of the shared file to be obtained according to the number of downloads and the number.

[0030] In a second aspect, the present invention provides a file acquisition device, which is applied to a requesting node among multiple nodes in a peer-to-peer P2P network, at least one of the nodes is connected in communication with at least one client, at least one client stores a shared file that can be shared through the P2P network, and all the shared files stored by the clients are pre-divided into at least one partition, each of the nodes stores first search information, second search information, and third search information, the first search information is used to search for a node to which a partition belongs, the second search information is used to search for a node storing the third search information of the shared file in the partition of the node, and the third search information is used to search for a client to which the shared file stored by the node belongs, the requesting node is connected in communication with the requesting client, and the device comprises:

[0031] A receiving module, configured to receive a file acquisition request sent by the requesting client, wherein the file acquisition request includes a file identifier of a shared file to be acquired;

[0032] A determination module, configured to determine a first index node to which the partition to which the shared file to be obtained belongs belongs according to the file identifier and the local first search information;

[0033] a sending module, configured to send the file identifier to the first index node, so that the first index node retrieves the second index node according to the file identifier and local second search information, and returns the second index node to the requesting node when the search is successful, wherein the second index node is a node storing the third search information of the shared file to be obtained;

[0034] A return module is used to send a file identifier to a second index node if a returned second index node is received, so that the second index node obtains the shared file to be obtained according to the file identifier and local third search information and returns the shared file to be obtained to the requesting node, so as to be returned to the requesting client through the requesting node.

[0035] In a third aspect, the present invention provides a node device, including a processor and a memory, wherein the memory is used to store a program, and the processor is used to implement the file acquisition method described in any one of the aforementioned embodiments when executing the program.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the file acquisition method as described in any one of the aforementioned implementations.

[0037] Compared with the prior art, the embodiments of the present invention provide a file acquisition method, apparatus, node device and computer-readable storage medium, wherein a node of a P2P network is connected to at least one client in communication, at least one client stores a shared file that can be shared through the P2P network, all shared files are pre-divided into at least one partition, each node stores first search information, second search information and third search information, the first search information is used to retrieve a node to which the partition belongs, the second search information is used to retrieve a node to which the third search information of the shared file in the partition of the node is stored, and the third search information is used to retrieve a client to which the shared file stored in the node belongs, and when a requesting client needs to request a shared file to be obtained, a file acquisition request is sent to the requesting node, and the requesting node The first index node to which the partition to which the shared file to be obtained belongs is determined according to the file identifier in the file acquisition request and the local first search information, and then the file identifier is sent to the first index node. The first index node retrieves the second index node according to the file identifier and the local second search information, and returns the second index node to the requesting node when it is retrieved. The requesting node obtains the file to be obtained from the second index node and returns it to the requesting client. In the embodiment of the present invention, the shared files are managed according to the partitions, and the first search information, the second search information and the third search information are distributed on multiple nodes of the P2P network based on the partitions, so that when retrieving the shared files, the processing capabilities of all nodes in the P2P network can be fully utilized, thereby improving the efficiency of obtaining the shared files. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 An example diagram of an application scenario provided by an embodiment of the present invention.

[0040] Figure 2 An example block diagram of a node device provided in an embodiment of the present invention.

[0041] Figure 3 This is a diagram showing an example of updating the first search information provided by an embodiment of the present invention.

[0042] Figure 4 This is a diagram showing an example of updating the third retrieval information provided by an embodiment of the present invention.

[0043] Figure 5 This is a diagram showing an example of updating the second search information provided by an embodiment of the present invention.

[0044] Figure 6 This is a flowchart of an exemplary method for obtaining a file provided in an embodiment of the present invention.

[0045] Figure 7 This is a diagram illustrating an example of a process of performing a search based on first search information, second search information, and third search information provided by an embodiment of the present invention.

[0046] Figure 8 This is a diagram showing an example of a retrieval process using a broadcast method according to an embodiment of the present invention.

[0047] Fig. 9 This is a block diagram of an example of a file acquisition device provided in this embodiment.

[0048] Icons: 10 - node; 20 - client; 30 - node device; 31 - processor; 32 - memory; 33 - bus; 100 - file acquisition device; 110 - receiving module; 120 - determination module; 130 - sending module; 140 - return module. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0052] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0053] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0054] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0055] Solutions for quickly discovering clients with shared files in a P2P network are generally divided into two categories: centralized and decentralized. The centralized solution is simple, but has the risk of single point failure, and the central server has limited capacity, which makes it impossible to manage and schedule users and files in the P2P network, and cannot meet the scenario of massive users and massive files; the decentralized solution is complex, has high fault tolerance, and can break through capacity limitations, but the implementation of algorithms such as distributed hash tables has deficiencies in stability and latency, and is difficult to meet commercial requirements.

[0056] In view of this, embodiments of the present invention provide a file acquisition method, apparatus, node device, and computer-readable storage medium to improve the acquisition efficiency of shared files in a P2P network, which will be described in detail below.

[0057] Please refer to Figure 1 , Figure 1 An example diagram of an application scenario provided by an embodiment of the present invention, Figure 1In the embodiment, the P2P network includes a plurality of nodes 10, at least one node 10 is in communication connection with at least one client 20, at least one client 20 stores a shared file that can be shared through the P2P network, and a shared file stored in a client 20 can be shared with other clients 20 through the P2P network. The shared files stored in all clients 20 in the P2P network are divided into at least one partition. In order to reduce the amount of retrieved data and ensure retrieval efficiency, the number of partitions can be greater than the number of nodes. Each node is responsible for managing at least one partition. Each node stores first retrieval information, second retrieval information, and third retrieval information. Each shared file has a file identifier. The partition to which the shared file belongs can be obtained according to the file identifier. Through the first retrieval information, the node can retrieve the node to which the partition belongs. Each node stores second retrieval information for retrieving the node storing the third retrieval information of the shared file in the partition of the node. The third retrieval information is used to retrieve the client to which the shared file stored in the node belongs. For example, client A and node B are connected. Point 1 is connected in communication, client A sends a sharing request to node 1 to obtain shared file x, node 1 obtains that shared file x belongs to partition 1 based on shared file x, and partition 1 is managed by node 2, then node 1 sends the file identifier of shared file x to node 2, node 2 locally stores the second search information, node 2 obtains the third search information that stores shared file x based on the local second search information is node 3 and node 4, node 2 returns node 3 and node 4 to node 1, node 1 sends a request to node 3 and node 4 to find the nodes that own shared file x from node 3 and node 4 based on their respective locally stored third search information and return them to client A, at this point, client A can obtain shared file x from client B and client C.

[0058] The node 10 may be a server or a server group or server cluster consisting of multiple servers. The node 10 may be a virtual server or a physical server.

[0059] The client 20 includes, but is not limited to, a mobile terminal or a wireless terminal such as a mobile phone, a tablet computer, a wearable device, and a laptop computer.

[0060] exist Figure 1 Based on the above, an embodiment of the present invention provides a block diagram of a node device 30, and the node device 30 may be Figure 1 Any node 10 in communication connection with the client 20 that needs to request a shared file is also referred to as a requesting node in subsequent embodiments. It can be understood that Figure 1 Any node 10 in can be a requesting node. Figure 2 , Figure 2This is a block diagram of an exemplary node device 30 provided in an embodiment of the present invention. The node device 30 includes a processor 31 , a memory 32 and a bus 33 . The processor 31 , the memory 32 are connected via the bus 33 .

[0061] The processor 31 may be an integrated circuit chip with signal processing capability. In the implementation process, the file acquisition method of this embodiment, or each step of the file acquisition method of this embodiment, may be completed by an integrated logic circuit of hardware in the processor 31 or by instructions in the form of software. The above-mentioned processor 31 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0062] The memory 32 is used to store programs, such as the file acquisition device in this embodiment. The file acquisition device includes at least one software function module that can be stored in the memory 32 in the form of software or firmware or fixed in the operating system (OS) of the node device 30. After receiving the execution instruction, the processor 31 executes the program to implement the file acquisition device method disclosed in the above embodiment.

[0063] In order to more clearly illustrate the file acquisition method provided by this embodiment, this embodiment first describes the updating process of the first search information, the second search information and the third search information, and then describes the implementation process of the file acquisition method based on the first search information, the second search information and the third search information.

[0064] In the scenario of massive users and massive files, how to find a group of other users who own the file identifier through the file identifier is a very complicated matter. With the increase of users and files, the P2P network will face the problem of expansion, and the maintenance and expansion of the index information used to index the users who own the specified files will also become more complicated. On the other hand, the amount of index information is also very large. For example, for the application scenario of 100 million users and 1 billion files, the amount of index data even reaches TB level. In addition, the index data will be frequently changed and queried, so the maintenance of index data is quite complicated, which greatly affects the efficiency of file acquisition.

[0065] To this end, this embodiment adopts a three-level indexing mechanism of first search information, second search information and third search information to improve file acquisition efficiency. At the same time, the first search information, second search information and third search information are dispersed among nodes in the P2P network, simplifying the maintenance and expansion of index data.

[0066] In this embodiment, the first search information is used to retrieve the node to which the partition belongs. Each node in the P2P network stores the first search information. The first search information may simply be the correspondence between the partition and the node, and one node corresponds to at least one partition. As an implementation method, a server independent of the P2P network may be used to manage the first search information, that is, the server determines the correspondence between the partition and the node, and synchronizes the correspondence to each node in the P2P network. When a new node joins the P2P network, or an existing node is deleted from the P2P network, or a node goes online or offline, the server updates the first search information and synchronizes the updated first search information to each node in the P2P network in a timely manner. For example, if a P2P network has 10 nodes, when a new node joins, the server will update the first search information so that each node corresponds to at least one partition, and synchronize the updated first search information to each node, thereby making the second search information of the node used to retrieve the third search information of the shared file stored in the partition of the node roughly evenly distributed to all nodes. Please refer to Figure 3 , Figure 3 An example diagram of updating the first search information provided in an embodiment of the present invention, the P2P network originally has 3 nodes and 4 partitions, and the first search information determined by the server is: node 1-partition 1 and partition 2; node 2-partition 3; node 3-partition 4. Now, node 4 is added to the P2P network, and the server updates the first search information. The updated first search information is: node 1-partition; node 2-partition 3; node 3-partition 4, node 4-partition 2.

[0067] It should be noted that Figure 3 This is just an example of updating the first search information. In actual scenarios, the server that manages the first search information can update the correspondence between nodes and partitions according to preset rules, for example, correspond node 1 to partition 2, and correspond node 4 to partition 1, depending on the specific preset rules.

[0068] It should also be noted that Figure 3 Just to illustrate the synchronization of the first retrieved information between the server and the nodes in the P2P network, Figure 3 The client 20 in the P2P network is not shown, nor is the communication connection relationship between nodes shown, but this does not mean that these do not exist in actual application scenarios.

[0069] After determining the correspondence between nodes and partitions, each node needs to store the second retrieval information of the shared files in the partition corresponding to the node. The second retrieval information is used to retrieve the node storing the third retrieval information of the shared files in the partition of the node, that is, the second retrieval information represents the correspondence between the shared files in the partition of the node and the node storing the third retrieval information of the shared files.

[0070] Taking into account the readability of the description content, this embodiment first explains the third search information. The third search information is used to retrieve the client to which the shared file stored by the node belongs. Since users usually share files by logging into the client, the user who owns the shared file can be regarded as the client logged in by the user. At this time, the user and the client can be equated. The client synchronizes the locally stored shared files to the node connected to it for communication. The node associates the shared files with the client. At this time, the correspondence between the shared files and the client is the third search information. In addition, when multiple users log in to the same client, the client can synchronize the shared files and user identifiers to the node. At this time, the correspondence between the shared files and the client and user identifier is the third search information. When the shared files stored locally on the client change, the client needs to inform the node of the changed shared files. The node updates the local third search information. Please refer to Figure 4 , Figure 4 An example diagram of updating the third retrieval information provided in an embodiment of the present invention, client A is connected to node 1 for communication. Client A originally stores shared files x, y, and z. At this time, the third retrieval information of node 1 is: A-(x, y, z). Then, y is deleted, and client A informs node 1 that the shared files stored by itself are changed to x and z. At this time, node 1 updates the third retrieval information, and the updated third retrieval information is A-(x, z).

[0071] The node that has updated the third search information needs to synchronize information with the node storing the third search information to update the second search information in the node storing the third search information. Figure 5 , Figure 5 This is an example diagram of updating the second search information provided by an embodiment of the present invention. Figure 5 In the figure, the third retrieval information of node 1 is client A-shared file x. Node 1 determines the partition a to which the shared file belongs based on the file identifier of shared file x. According to the local first retrieval information of node 1, it is determined that the node to which partition a belongs is node 2. Node 1 synchronizes information with node 2 to enable node 2 to update the local second retrieval information: (a, x)-node 1.

[0072] When client B wants to obtain shared file x, client B sends a request to node 3 to which it is communicating. Node 3 determines the partition a to which the shared file belongs based on the file identifier of shared file x, and determines that the node to which partition a belongs is node 2 based on the first local search information of node 3. Node 3 sends the file identifier of shared file x to node 2. Node 2 determines that the third search information storing shared file x is node 1 based on the file identifier of shared file x and the second local search information of node 2, and returns node 1 to node 3. Node 3 sends a request to node 1, and node 1 obtains shared file x from client A and returns it to node 3, and then returns it to client B through node 3.

[0073] It should be noted that, in addition to the updates of the first search information, the second search information, and the third search information, the online and offline of a node will affect the migration of part of the second search information from one node to another, for example Figure 3 In the process, the second search information related to the shared file in partition 2 of node 1 will be migrated to node 4.

[0074] Based on the first search information, the second search information and the third search information, this embodiment also provides a file acquisition method. Figure 6 , Figure 6 The following is a flowchart of a method for obtaining a file provided by an embodiment of the present invention, wherein the method comprises the following steps:

[0075] Step S101: receiving a file acquisition request sent by a requesting client, wherein the file acquisition request includes a file identifier of a shared file to be acquired.

[0076] In this embodiment, the requesting client can be Figure 1 Any client 20 that needs to obtain a shared file, the requesting node can be Figure 1 The node 10 in communication connection with the requesting client. The file identifier is also called a file ID, which is used to uniquely identify a file entity. The file identifier of the shared file to be obtained is used to uniquely identify the shared file to be obtained.

[0077] Step S102: determining the first index node to which the partition to which the shared file to be obtained belongs according to the file identifier and the local first search information.

[0078] In this embodiment, as an implementation, a hash value is calculated according to the file identifier and a preset hash function, the partition to which the shared file to be obtained belongs is determined according to the hash value, and the first index node to which the partition to which the shared file to be obtained belongs is determined according to the partition to which the shared file to be obtained belongs and the first search information. As another implementation, the partition to which the shared file to be obtained belongs can also be determined according to the file identifier and a preset mapping function, wherein the preset mapping function represents the correspondence between the file identifier and the partition.

[0079] Step S103, sending the file identifier to the first index node, so that the first index node retrieves the second index node according to the file identifier and the local second search information, and returns the second index node to the requesting node when the search is successful. The second index node is a node storing the third search information of the shared file to be obtained.

[0080] In this embodiment, the retrieval may succeed or fail. For example, when the first index node is abnormal, the retrieval will fail. When the first index node is normal but the reading of the second retrieval information locally stored in the first index node is abnormal, the retrieval will also fail. When the network between the requesting node and the first index node is abnormal, the retrieval will also fail. The second index node can only be obtained when the retrieval is successful.

[0081] Step S104, if a returned second index node is received, a file identifier is sent to the second index node, so that the second index node obtains the shared file to be obtained according to the file identifier and the local third search information and returns the shared file to be obtained to the requesting node, so as to be returned to the requesting client through the requesting node.

[0082] In this embodiment, the requesting node sends the file identifier to the second indexing node. As an implementation method, the second indexing node obtains the file to be obtained based on the received file identifier and the third retrieval information stored locally and returns it to the requesting node, so that the file to be obtained is returned to the requesting client through the requesting node; as another implementation method, the second indexing node may also first send a message to the requesting node whether it is ready to receive, and then send the shared file to be obtained to the requesting node after the requesting node returns a confirmation message of being ready. In addition, when there are multiple second indexing nodes, the requesting node may also select a preset number of nodes from the multiple second indexing nodes according to its own bandwidth or load conditions to obtain the file to be shared from them.

[0083] The above method provided in this embodiment manages shared files according to partitions, and then distributes the first search information, the second search information and the third search information on multiple nodes of the P2P network based on the partitions, so that when searching for shared files, the processing capabilities of all nodes in the P2P network can be fully utilized, thereby improving the efficiency of obtaining shared files.

[0084] In an optional implementation, a method for retrieving the second index node may be:

[0085] If the first index node fails to read the local second search information, it is determined that the search has failed;

[0086] If the first index node reads the local second search information successfully, the first index node determines the second index node according to the file identifier and the local second search information.

[0087] For a clearer explanation of the above retrieval process, please refer to Figure 7 , Figure 7 An example diagram of a process of performing a search based on first search information, second search information, and third search information provided by an embodiment of the present invention, Figure 7 In the example, the request client sends a request to node a. Node a determines that the second search information of the partition to which the file belongs is stored in node x according to the file identifier in the request and the local first search information. Node a sends the file identifier to node x. Figure 7 As shown by the dotted line numbered 1, node x determines that the nodes storing the third node search information of the file are node b and node c according to the file identifier and the local second search information, such as Figure 7 As shown by the dotted line numbered 2, node a sends a file identifier to node b and node c, such as Figure 7 As shown by the dotted line numbered 3, node b obtains the file based on the local third search information and returns it to node a, such as Figure 7 As shown by the solid line numbered 4, node a returns the file to the requesting client. Node c finds that its current load is high, so it responds to node a with overload current limiting feedback to notify node a that it is currently unable to respond to node a's request normally, as shown in the figure below. Figure 7 As shown by the dotted line with number 5.

[0088] It should be noted that if both node b and node c can respond to node a normally, node a can also receive information from both node b and node c at the same time or only need to receive information from one of node b and node c according to its current load situation, thereby achieving a better flow control effect.

[0089] In an optional implementation, when the retrieval of the second index node fails or the second retrieval information in the second index node is temporarily unavailable during the migration process, in order to minimize the impact on obtaining the file, this embodiment also provides an implementation method for obtaining the file in this scenario:

[0090] If the returned second index node is not received, a preset number of candidate nodes are randomly selected from the multiple nodes;

[0091] The file identifier is broadcast to the candidate node, so that the candidate node retrieves the shared file to be obtained according to the third search information stored locally, and obtains the shared file to be obtained when the retrieval is successful and returns the shared file to the requesting node, so as to be returned to the requesting client through the requesting node.

[0092] In this embodiment, since broadcasting has great uncertainty, in order to minimize the impact of broadcasting on network communication bandwidth, this embodiment randomly selects a preset number of candidate nodes from multiple nodes. The preset number can be pre-set according to the bearing capacity of the network communication bandwidth.

[0093] For a clearer explanation of the above retrieval process, please refer to Figure 8 , Figure 8 An example diagram of a process of performing a search in a broadcast manner provided by an embodiment of the present invention, Figure 8 In the example, node a is a requesting node. The requesting client sends a request to node a, which includes a file identifier. Node a determines the node storing the second search information based on the file identifier and the first search information stored locally at node a. As a result, it is found that the second search information is invalid. Node a determines the node to be broadcasted from the P2P network, that is, node be, and sends the file identifier to node be in a broadcasting manner, such as Figure 8 As shown by the dotted line with label 1, node d finds that there is no third search information of the file identifier locally based on the file identifier, and does not respond to node a. Nodes b, c, and e all find that they have the third search information of the file identifier stored locally. Nodes b and c find that they can respond normally, so they send a request to node a whether it can respond, as shown in Figure 1. Figure 8 As shown by the dotted line numbered 2, node e finds that its current load is high, so it responds to node a with overload current limiting feedback to inform node a that it is currently unable to respond to node a's request normally, as shown in Figure 8 As shown by the dotted line numbered 5, node a receives the request sent by nodes b and c to see if it can respond. Based on its own load situation, it determines that it only needs to obtain information from node b, so it responds to node b to confirm that it can receive the information. Figure 8 As shown by the dotted line numbered 3, node b obtains the corresponding file according to the file identifier and the third search information stored locally and sends it to node a, such as Figure 8 As shown by the solid line numbered 4, node a then returns it to the requesting client.

[0094] As described above in this embodiment, the first search information, the second search information and the third search information will be partially updated along with the online and offline nodes in the P2P network or the update of the files stored in the client. Therefore, this embodiment also provides a method for updating the second search information of the first index node:

[0095] When any node receives the first file synchronization request sent by the client, it determines the synchronization node to which the partition to which the first file to be synchronized belongs belongs according to the first file identifier to be synchronized carried in the first file synchronization request and the local first search information;

[0096] If the synchronization node is the first index node, the first file to be synchronized and the node receiving the first file synchronization request are sent to the first index node, so that the first index node updates the second search information of the first index node according to the first file to be synchronized and the node receiving the first file synchronization request.

[0097] This embodiment also provides a method for updating the third search information of the candidate node:

[0098] When any client sends a second file synchronization request to the candidate node, the third search information of the candidate node is updated according to the second to-be-synchronized file identifier carried in the second file synchronization request and the corresponding client.

[0099] Since the updating method of the second search information and the third search information of any node has been described above, it will not be repeated here.

[0100] In this embodiment, in order to ensure that the shared files to be obtained with strong demand can be shared to the required clients in time through the P2P network, this embodiment also provides a method for the second index node to obtain the shared files to be obtained according to the file identifier and the local third search information:

[0101] First, determine the target client storing the shared file to be obtained according to the file identifier and the local third search information;

[0102] Secondly, the target client obtains the sharing priority of the shared file to be obtained according to the file identifier, and sends the shared file to be obtained to the second index node according to the sharing priority. The sharing priority is set according to the popularity of the shared file to be obtained. The popularity of the shared file to be obtained is obtained from a statistical node among multiple nodes. The second retrieval information stored in the statistical node includes the node storing the third retrieval information of the shared file to be obtained.

[0103] In this embodiment, the higher the popularity, the higher the sharing priority. In addition to scheduling shared files according to the sharing priority, you can also set the files to be downloaded first, uploaded first, and kept in the cache first, so as to ensure that the most popular files are shared efficiently first.

[0104] In this embodiment, when the statistics node receives an acquisition request for a shared file to be acquired, it can count the number of downloads of the shared file to be acquired. As a statistical method, the number of downloads of the shared file to be acquired within a preset time period can be counted.

[0105] In an optional implementation, a method for calculating the heat of a shared file to be obtained is:

[0106] First, the counting node counts the number of clients that need to obtain the shared file based on the local second search information;

[0107] Secondly, the statistical node calculates the popularity of the shared file to be obtained based on the number and number of downloads.

[0108] In this embodiment, as an implementation method, the popularity of a shared file = the number of downloads of the shared file 2 / The number of clients storing shared files.

[0109] In order to execute the corresponding steps in the above embodiment and each possible implementation mode, the following provides a method for implementing the file acquisition device 100, which is applied to Figure 1 Node 10 and Figure 2 Node device 30 in. Please refer to Fig. 9 , Fig. 9 This is a block diagram of an example file acquisition device 100 provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the file acquisition device 100 provided in this embodiment are the same as those of the corresponding above-mentioned embodiments, which are not mentioned in this embodiment for the sake of brief description.

[0110] The file acquisition device 100 includes a receiving module 110 , a determining module 120 , a sending module 130 and a returning module 140 .

[0111] The receiving module 110 is used to receive a file acquisition request sent by a requesting client, where the file acquisition request includes a file identifier of a shared file to be acquired;

[0112] A determination module 120, configured to determine a first index node to which the partition to which the shared file to be obtained belongs belongs according to the file identifier and the local first search information;

[0113] The sending module 130 is used to send the file identifier to the first index node, so that the first index node searches for the second index node according to the file identifier and the local second search information, and returns the second index node to the requesting node when the search is successful, wherein the second index node is a node storing the third search information of the shared file to be obtained;

[0114] The return module 140 is used to send a file identifier to the second index node if a returned second index node is received, so that the second index node obtains the shared file to be obtained according to the file identifier and the local third search information and returns the shared file to be obtained to the requesting node, so as to return it to the requesting client through the requesting node.

[0115] In an optional implementation manner, the sending module 130 is further configured to:

[0116] If the returned second index node is not received, a preset number of candidate nodes are randomly selected from the multiple nodes;

[0117] The file identifier is broadcast to the candidate node, so that the candidate node retrieves the shared file to be obtained according to the third search information stored locally, and obtains the shared file to be obtained when the retrieval is successful and returns the shared file to the requesting node, so as to be returned to the requesting client through the requesting node.

[0118] In an optional implementation manner, the first index node searches for the second index node according to the file identifier and the local second search information in the following manner:

[0119] If the first index node fails to read the local second search information, it is determined that the search has failed;

[0120] If the first index node reads the local second search information successfully, the first index node determines the second index node according to the file identifier and the local second search information.

[0121] In an optional implementation manner, the second search information of the first index node is updated in the following manner:

[0122] When any node receives the first file synchronization request sent by the client, it determines the synchronization node to which the partition to which the first file to be synchronized belongs belongs according to the first file identifier to be synchronized carried in the first file synchronization request and the local first search information;

[0123] If the synchronization node is the first index node, the first file to be synchronized and the node receiving the first file synchronization request are sent to the first index node, so that the first index node updates the second search information of the first index node according to the first file to be synchronized and the node receiving the first file synchronization request.

[0124] In an optional implementation manner, the third search information of the candidate node is updated in the following manner:

[0125] When any client sends a second file synchronization request to the candidate node, the third search information of the candidate node is updated according to the second to-be-synchronized file identifier carried in the second file synchronization request and the corresponding client.

[0126] In an optional implementation manner, the second index node acquires the shared file to be acquired according to the file identifier and the local third search information in the following manner:

[0127] Determine a target client storing the shared file to be obtained according to the file identifier and the local third search information;

[0128] Sending the file identifier to the target client;

[0129] The target client obtains the sharing priority of the shared file to be obtained according to the file identifier, and sends the shared file to be obtained to the second index node according to the sharing priority. The sharing priority is set according to the popularity of the shared file to be obtained. The popularity of the shared file to be obtained is obtained from a statistical node among multiple nodes. The second retrieval information stored in the statistical node includes a node storing third retrieval information of the shared file to be obtained.

[0130] In an optional implementation manner, the statistics node counts the number of downloads of the shared file to be obtained, and the heat of the shared file to be obtained is calculated as follows:

[0131] The counting node counts the number of clients that need to obtain the shared file according to the local second search information;

[0132] The statistical node calculates the popularity of the shared file to be obtained based on the download times and number.

[0133] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the file acquisition method as described in any one of the aforementioned implementations is implemented.

[0134] In summary, the embodiments of the present invention provide a file acquisition method, apparatus, node device and computer-readable storage medium, wherein a node of a P2P network is communicatively connected to at least one client, at least one client stores a shared file that can be shared through the P2P network, all shared files are pre-divided into at least one partition, each node stores first search information, second search information and third search information, the first search information is used to retrieve a node to which a partition belongs, the second search information is used to retrieve a node storing the third search information of a shared file in the partition of the node, the third search information is used to retrieve a client to which the shared file stored in the node belongs, when a requesting client needs to request a shared file to be obtained, a file acquisition request is sent to a requesting node, the requesting node determines a first index node to which the partition to which the shared file to be obtained belongs according to a file identifier in the file acquisition request and the local first search information, and then sends the file identifier to the first index node, the first index node retrieves the second index node according to the file identifier and the local second search information, and after retrieving the second index node, The index node obtains the shared file from the second index node and returns it to the requesting node, which then returns it to the requesting client. Compared with the prior art, the embodiments of the present invention have at least the following advantages: (1) By managing the shared files according to partitions, and then distributing the first search information, the second search information and the third search information on multiple nodes of the P2P network based on the partitions, the processing capabilities of all nodes in the P2P network can be fully utilized when retrieving shared files, thereby improving the efficiency of obtaining shared files; (2) When the node is offline or the second search information is unavailable, the file is obtained through the broadcast mechanism, thereby increasing the availability of obtaining shared files through the P2P network; (3) When the broadcast mechanism is adopted, the number of broadcast nodes is determined, thereby minimizing the impact of the broadcast mechanism on the P2P network bandwidth; (4) When obtaining shared files, the flow control mechanism is used to avoid unnecessary bandwidth occupation; (5) The priority of the shared file in downloading, uploading and temporary caching is determined by the popularity of the shared file, thereby ensuring that the shared files with high popularity are shared through the P2P network first.

[0135] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A file acquisition method, characterized in that: A requesting node is applied to a plurality of nodes in a peer-to-peer (P2P) network, at least one of the nodes is connected in communication with at least one client, at least one client stores a shared file that can be shared through the P2P network, all the shared files stored by the clients are pre-divided into at least one partition, each of the nodes stores first search information, second search information and third search information, the first search information is used to search for a node to which a partition belongs, the second search information is used to search for a node storing the third search information of a shared file in the partition of the node, the third search information is used to search for a client to which the shared file stored by the node belongs, the requesting node is connected in communication with the requesting client, and the method comprises: Receiving a file acquisition request sent by the requesting client, wherein the file acquisition request includes a file identifier of a shared file to be acquired; Determine, according to the file identifier and the local first search information, a first index node to which the partition to which the shared file to be obtained belongs; Sending the file identifier to the first index node, so that the first index node searches for a second index node according to the file identifier and local second search information, and returns the second index node to the requesting node when the search is successful, wherein the second index node is a node storing the third search information of the shared file to be obtained; If the returned second index node is received, a file identifier is sent to the second index node, so that the second index node obtains the shared file to be obtained and returns the shared file to be obtained to the requesting node, so as to be returned to the requesting client through the requesting node. The method for obtaining the shared file to be obtained is: Determine a target client storing the to-be-acquired shared file according to the file identifier and the local third search information; send the file identifier to the target client; The target client obtains the sharing priority of the shared file to be obtained according to the file identifier and is set according to the popularity of the shared file to be obtained, and sends the shared file to be obtained to the second index node according to the sharing priority, the popularity is obtained from a statistical node among the multiple nodes, and the second retrieval information stored in the statistical node includes a node storing third retrieval information of the shared file to be obtained.

2. The file acquisition method according to claim 1, characterized in that: The method further comprises: If the second index node is not received, randomly selecting a preset number of candidate nodes from the multiple nodes; The file identifier is broadcast to the candidate node so that the candidate node retrieves the shared file to be obtained according to the third search information stored locally, and obtains the shared file to be obtained when the retrieval is successful and returns the shared file to be obtained to the requesting node, so as to be returned to the requesting client through the requesting node.

3. The file acquisition method according to claim 1, characterized in that: The first index node searches for the second index node according to the file identifier and the local second search information in the following manner: If the first index node fails to read the local second search information, it is determined that the search fails; If the first index node reads the local second search information successfully, the first index node determines the second index node according to the file identifier and the local second search information.

4. The file acquisition method according to claim 1, characterized in that: The second search information of the first index node is updated in the following manner: When any of the nodes receives a first file synchronization request sent by a client, the synchronization node to which the partition to which the first file to be synchronized belongs is determined according to the first file identifier to be synchronized carried in the first file synchronization request and the local first search information; If the synchronization node is the first index node, the first file to be synchronized and the node receiving the first file synchronization request are sent to the first index node, so that the first index node updates the second search information of the first index node according to the first file to be synchronized and the node receiving the first file synchronization request.

5. The file acquisition method according to claim 2, characterized in that: The third search information of the candidate node is updated in the following manner: When any client sends a second file synchronization request to the candidate node, the third search information of the candidate node is updated according to the second to-be-synchronized file identifier carried in the second file synchronization request and the corresponding client.

6. The file acquisition method according to claim 1, characterized in that: The statistical node counts the number of downloads of the shared file to be obtained, and the heat of the shared file to be obtained is calculated as follows: The counting node counts the number of clients storing the to-be-acquired shared file according to the local second search information; The statistical node calculates the popularity of the shared file to be obtained according to the number of downloads and the number.

7. A file acquisition device, characterized in that: A requesting node is applied to a plurality of nodes in a peer-to-peer (P2P) network, at least one of the nodes is connected to at least one client in communication, at least one client stores a shared file that can be shared through the P2P network, all the shared files stored by the clients are pre-divided into at least one partition, each of the nodes stores first search information, second search information and third search information, the first search information is used to search for a node to which a partition belongs, the second search information is used to search for a node storing the third search information of a shared file in the partition of the node, the third search information is used to search for a client to which the shared file stored by the node belongs, the requesting node is connected to the requesting client in communication, and the device comprises: A receiving module, configured to receive a file acquisition request sent by the requesting client, wherein the file acquisition request includes a file identifier of a shared file to be acquired; A determination module, configured to determine a first index node to which the partition to which the shared file to be obtained belongs belongs according to the file identifier and the local first search information; a sending module, configured to send the file identifier to the first index node, so that the first index node retrieves the second index node according to the file identifier and local second search information, and returns the second index node to the requesting node when the search is successful, wherein the second index node is a node storing the third search information of the shared file to be obtained; The return module is used for sending a file identifier to the second index node if a returned second index node is received, so that the second index node obtains the shared file to be obtained and returns the shared file to be obtained to the requesting node, so as to return it to the requesting client through the requesting node. The method of obtaining the shared file to be obtained is: Determine a target client storing the to-be-acquired shared file according to the file identifier and the local third search information; send the file identifier to the target client; The target client obtains the sharing priority of the shared file to be obtained according to the file identifier and is set according to the popularity of the shared file to be obtained, and sends the shared file to be obtained to the second index node according to the sharing priority, the popularity is obtained from a statistical node among the multiple nodes, and the second retrieval information stored in the statistical node includes a node storing third retrieval information of the shared file to be obtained.

8. A node device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a program, and the processor is used to implement the file acquisition method according to any one of claims 1 to 6 when executing the program.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the file acquisition method according to any one of claims 1 to 6 is implemented.

Images