Data acquisition method, device and system, computer equipment and storage medium

Abstract

The invention relates to a data acquisition method, device and system, computer equipment and a storage medium. The method comprises the following steps: a client initiates a data request to a first edge node to request to obtain target data, the first edge node responds to the data request to query the target data in a local cache, and if the query fails, a target networking center is determined from a plurality of networking centers according to a hash result corresponding to a resource positioning identifier of the target data, and initiating a node query request to the target networking center to obtain a node address of the second edge node, providing the node address of the second edge node for the first edge node through the target networking center, and returning a 302 redirection address indicating the node address of the second edge node to the client by the first edge node, and enabling the client to access the second edge node according to the 302 redirection address so as to obtain the target data. In this way, the cache hit rate can be increased without reducing the number of service nodes, and the disaster recovery risk can be reduced through a plurality of networking centers.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data acquisition method, apparatus, system, computer equipment and storage medium. Background Art

[0002] The existing CDN (Content Delivery Network) access mechanism is that the client first initiates a data request to the edge node. If the edge node cache hits the data request, it will directly return the content corresponding to the data request to the client. If it misses, it needs to go back to the parent node to query the content corresponding to the data request. The bandwidth cost between the parent node and the edge node is the transmission cost within the CDN. In order to reduce costs and improve service quality, it is necessary to improve the hit rate of edge nodes for data requests, that is, the cache hit rate.

[0003] At present, the method used to improve cache hit rate is node coverage contraction. This method is to make most users resolve to the same node to obtain the requested data content during DNS resolution, thereby increasing the cache hit rate of each node by reducing the number of service nodes. However, this method increases the risk of disaster recovery due to too few nodes, and also causes the node accessed by the user to be far away from the user, affecting the data transmission speed. Summary of the invention

[0004] The present application provides a data acquisition method, apparatus, system, computer equipment and storage medium to solve the problem of how to provide a data acquisition method that can improve cache hit rate without increasing disaster recovery risk and affecting data transmission speed.

[0005] In a first aspect, the present application provides a data acquisition system, the system comprising:

[0006] A client is used to initiate a data request, wherein the data request is used to request to obtain target data;

[0007] The first edge node is used to query the target data in the local cache in response to the data request, and when the target data is not found, determine the target networking center from multiple networking centers according to the hash result corresponding to the resource location identifier of the target data, and initiate a node query request to the target networking center to request to obtain the node address of the second edge node, and when receiving the node address of the second edge node, send a 302 redirection address indicating the node address of the second edge node to the client, so that the client accesses the second edge node according to the 302 redirection address to obtain the target data, and the first edge node and the second edge node are any different edge nodes;

[0008] The target networking center is used to respond to the node query request and query the second edge node corresponding to the resource location identifier of the target data in the data storage records of each edge node, and return the node address of the second edge node to the first edge node when the second edge node is queried and the usage status of the second edge node is available.

[0009] In a second aspect, the present application provides a data acquisition method, which is applied to an edge node, and the method includes:

[0010] Receiving a data request from a client, wherein the data request is used to request to obtain target data;

[0011] When the target data is not found in the local cache of the first edge node, determining the target networking center from multiple networking centers according to the hash result corresponding to the resource location identifier of the target data, wherein each of the networking centers stores data storage records of multiple edge nodes;

[0012] Sending a corresponding node query request to the target networking center according to the data request, wherein the node query request is used to request the target networking center to provide a node address of a second edge node storing the target data according to a data storage record;

[0013] When the node address of the second edge node is included in the feedback information returned by the target networking center, the 302 redirection address indicating the node address of the second edge node is returned to the client, so that the client can access the second edge node according to the 302 redirection address to obtain the target data.

[0014] In a third aspect, the present application provides a data acquisition method, which is applied to a networking center, and the method includes:

[0015] Receiving a node query request from a first edge node, wherein the node query request is used to request a query for an edge node storing the target data;

[0016] According to the data storage records of each edge node, query the second edge node corresponding to the resource location identifier of the target data in the node query request;

[0017] When the second edge node is found and the usage status of the second edge node is available, the node address of the second edge node is returned to the first edge node as feedback information.

[0018] In a fourth aspect, the present application provides a data acquisition device, the device comprising:

[0019] A data query module, configured to query the local cache for target data corresponding to the data request upon receiving a data request initiated by a client, wherein the data request is used to request to obtain the target data;

[0020] A determination module, configured to determine a target networking center from a plurality of networking centers according to a hash result corresponding to a resource location identifier of the target data when the target data is not found in the local cache, wherein each of the networking centers stores data storage records of a plurality of edge nodes;

[0021] A sending module, used for sending a corresponding node query request to the target networking center according to the data request, wherein the node query request is used to request the target networking center to provide a node address of a second edge node storing the target data according to a data storage record, and the second edge node is an edge node storing the target data;

[0022] A feedback module is used to feed back the node address of the available second edge node to the client when the feedback information returned by the target networking center contains the node address of the available second edge node, so that the client accesses the available second edge node according to the node address to obtain the target data.

[0023] In a fifth aspect, the present application provides a data acquisition device, the device comprising:

[0024] A receiving module, configured to receive a node query request from a first edge node, wherein the node query request is used to request a query for an edge node storing the target data;

[0025] A node query module, used to query the second edge node corresponding to the resource location identifier of the target data in the node query request according to the data storage records of each edge node;

[0026] The second feedback module is configured to return the node address of the second edge node as feedback information to the first edge node when the second edge node is found and the usage status of the second edge node is available.

[0027] In a sixth aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned data acquisition method when executing the computer program.

[0028] In a seventh aspect, the present application also provides a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the above-mentioned data acquisition method.

[0029] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: in the data acquisition system provided by the embodiment of the present application, the client initiates a data request to the first edge node to request to obtain the target data, and the first edge node responds to the data request to query the target data in the local cache, and if the query fails, the target networking center is determined from multiple networking centers according to the hash result corresponding to the resource location identifier of the target data, and a node query request is initiated to the target networking center to obtain the node address of the second edge node, and the second edge node storing the target data is queried through the target networking center, and the target networking center queries the second edge node corresponding to the resource location identifier of the target data in the data storage records of each edge node, and when the target networking center queries the second edge node and the usage status of the second edge node is available, the second edge node is The node address of the node is returned to the first edge node, and the first edge node returns the 302 redirection address indicating the node address of the second edge node to the client, so that the client can access the second edge node according to the 302 redirection address to obtain the target data. Based on the above data acquisition system, there is no need to reduce the number of service nodes, but the target networking center in multiple networking centers provides the client with an edge node storing the target data, thereby improving the cache hit rate, and multiple networking centers provide services to the client to reduce the disaster recovery risk. Without reducing the number of nodes, providing data download services to the client through more distributed edge nodes will not affect the data transmission speed, thereby solving the problem of how to provide a data acquisition method that can improve the cache hit rate without increasing the disaster recovery risk and affecting the data transmission speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0032] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0033] Figure 1 An application environment diagram of a data acquisition method provided in an embodiment of the present application;

[0034] Figure 2 A flowchart of a data acquisition method provided in an embodiment of the present application;

[0035] Figure 3 A flowchart of a data acquisition method provided in an embodiment of the present application;

[0036] Figure 4 A flowchart of a data acquisition method provided in an embodiment of the present application;

[0037] Figure 5 A flowchart of a data acquisition method provided in an embodiment of the present application;

[0038] Figure 6 A structural block diagram of a data acquisition device provided in an embodiment of the present application;

[0039] Figure 7 A structural block diagram of a data acquisition device provided in an embodiment of the present application;

[0040] Figure 8 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0042] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0043] Reference Figure 1 The data acquisition method is applied to a data acquisition system. The data acquisition system comprises:

[0044] The client 110 is used to initiate a data request, wherein the data request is used to request to obtain target data;

[0045] A first edge node is used to query the target data in a local cache in response to the data request, and when the target data is not found, determine a target networking center 130 from multiple networking centers 130 according to a hash result corresponding to a resource location identifier of the target data, and initiate a node query request to the target networking center 130 to request to obtain a node address of a second edge node, and upon receiving the node address of the second edge node, send a 302 redirection address indicating the node address of the second edge node to the client 110, so that the client 110 accesses the second edge node according to the 302 redirection address to obtain the target data, wherein the first edge node and the second edge node are any different edge nodes 120;

[0046] The target networking center 130 is used to respond to the node query request and query the second edge node corresponding to the resource location identifier of the target data in the data storage record of each edge node 120, and return the node address of the second edge node to the first edge node when the second edge node is queried and the usage status of the second edge node is available.

[0047] Specifically, the client 110, the edge node 120, and the networking center 130 are connected to each other through a network. The client 110 can be implemented by a desktop terminal or a mobile terminal, and the mobile terminal can be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The edge node 120 can be a CDN node or a home node. The home node is a router or set-top box that uses idle bandwidth to provide download services for ordinary user families. The CDN node is a node in the content distribution network (CDN network), wherein the CDN node refers to a network node with fewer intermediate links from the end user access, and has better response capabilities and connection speeds for the end access users. The CDN node can be used as a cache server to provide data services. Each networking center 130 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0048] Before the client 110 accesses the first edge node, it first obtains the node address of the first edge node through the domain name resolution service (DNS resolution service). Then, the client 110 accesses the first edge node according to the node address provided by the domain name resolution service. The first edge node responds to the data request of the client 110 and provides it with the target data or a 302 redirection address. When the target data is not found in the local cache of the first edge node, the networking center 130 provides the first edge node with the node address of other edge nodes 120 storing the target data, and allows the first edge node to send the 302 redirection address indicating the node address of the second edge node storing the target data to the client 110, so that the client 110 re-accesses the second edge node according to the 302 redirection address to obtain the target data.

[0049] Based on the above data acquisition system, there is no need to reduce the number of service nodes. Instead, the target networking center 130 among multiple networking centers 130 provides the client 110 with an edge node 120 storing target data, thereby improving the cache hit rate. In addition, multiple networking centers 130 provide services to the client 110 to reduce the disaster recovery risk. Without reducing the number of nodes, providing data download services to the client 110 through the edge node 120 that is closer to the client 110 will not affect the data transmission speed. This solves the problem of how to provide a data acquisition method that can improve the cache hit rate without increasing the disaster recovery risk and affecting the data transmission speed.

[0050] In one embodiment, Figure 2 A flow chart of a data acquisition method in an embodiment is shown in FIG. Figure 2 and Figure 5 , provides a data acquisition method. This embodiment mainly applies this method to the above Figure 1 Taking the application to the edge node 120 as an example, the data acquisition method specifically includes the following steps:

[0051] Step S210: receiving a data request from the client 110, wherein the data request is used to request to obtain target data.

[0052] Specifically, the data request includes a resource location identifier of the target data, which is the resource file that the client 110 in the Internet wants to request to obtain, and the resource location identifier is storeurl, which is used to indicate the location and access method of the resource file and is the address of a standard resource on the Internet. Each resource file in the Internet has a unique resource location identifier.

[0053] In step S210, when the target data is not found in the local cache of the first edge node, the target networking center 130 is determined from multiple networking centers 130 according to the hash result corresponding to the resource location identifier of the target data, wherein each of the networking centers 130 stores data storage records of multiple edge nodes 120.

[0054] Specifically, when the first edge node receives the data request from the client 110, it first searches its own local cache for the target data requested by the data request. The target data is used to indicate the resource file that the client 110 wants to request. If the target data is found in the local cache, the target data is fed back to the client 110. If the target data is not found in the local cache, it is necessary to request the networking center 130 to query other second edge nodes that store the target data, but it is necessary to first filter out the target networking center 130 from the multiple networking centers 130. Specifically, the target networking center 130 is determined from the multiple networking centers 130 according to the hash result corresponding to the resource location identifier of the target data, that is, each networking center 130 corresponds to multiple The corresponding hash result of the resource location identifier of each resource file is used to indicate which resource storage locations of the resource files are recorded in the networking center 130, and the networking center 130 also stores the data storage records of multiple edge nodes 120, and the data storage records include the resource location identifier of the resource file stored in the corresponding edge node 120, which is used to indicate which data is stored in the corresponding edge node 120. Multiple networking centers 130 record the data storage records of each edge node 120, which can not only reduce the disaster recovery risk of a single networking center 130, but also determine the target networking center 130 associated with the hash result corresponding to the resource location identifier of the target data based on the data storage records stored in each networking center 130.

[0055] Step S210, sending a corresponding node query request to the target networking center 130 according to the data request, wherein the node query request is used to request the target networking center 130 to provide a node address of a second edge node storing the target data based on a data storage record.

[0056] Specifically, after determining the target networking center 130, the first edge node sends a corresponding node query request to the target networking center 130 according to the data request. The node query request includes a resource location identifier of the target data to request the target networking center 130 to query and feedback the node address of the second edge node that stores the target data.

[0057] In step S210, when the feedback information returned by the target networking center 130 includes the node address of the second edge node, the redirection address 302 indicating the node address of the second edge node is returned to the client 110, so that the client 110 can access the second edge node according to the redirection address 302 to obtain the target data.

[0058] Specifically, the first edge node receives feedback information from the target networking center 130 regarding the node query request. When the feedback information includes the node address of the second edge node, the first edge node returns a 302 redirection address indicating the node address of the second edge node to the client 110, causing the client 110 to switch from accessing the first edge node to accessing the second edge node according to the 302 redirection address, so as to obtain target data from the second edge node.

[0059] Based on the above data acquisition method, there is no need to reduce the number of service nodes. Instead, the networking center 130 provides the first edge node with the node addresses of other second edge nodes storing target data, so that the client 110 obtains the target data from the second edge node, thereby improving the cache hit rate. In addition, without reducing the number of nodes, the edge node 120 that is closer to the client 110 provides data services to the client 110, and does not affect the data download speed. This reduces the disaster recovery risk and ensures the data service quality while improving the cache hit rate.

[0060] In one embodiment, referring to Figure 5 After sending a corresponding node query request to the target networking center 130 according to the data request, the method further includes:

[0061] When the feedback information returned by the target networking center 130 indicates that the node query fails, forwarding the data request to the parent node of the first edge node;

[0062] When the target data returned by the parent node is received, the target data is returned to the client 110 .

[0063] Specifically, the feedback information received by the first edge node indicates that the node query failed, which means that the target networking center 130 has not found the edge node 120 storing the target data, or has queried the edge node 120 storing the target data but the edge node 120 is unavailable, that is, the target networking center 130 cannot provide the first edge node with the node address of other second edge nodes that store the target data and are available. At this time, the first edge node forwards the data request to its parent node. The parent node of the first edge node can be the upper-level CDN node that is communicated with the first edge node or the service source station that publishes the target data, that is, the parent node of the first edge node is farther away from the client 110.

[0064] The first edge node obtains the target data through its parent node, and the first edge node returns the obtained target data to the client 110 and keeps it in the local cache, so that the target data can be directly provided to the client 110 later, thereby improving the cache hit rate. The first edge node will request the target data from the parent node only after the first edge node fails to query the target data in the local cache and fails to obtain the node address of the second edge node from the networking center 130. Compared with the method of directly requesting the target data from the parent node after the first edge node fails to query the target data in the local cache, the number of interactions between the first edge node and the parent node can be reduced. Therefore, the cache hit rate can also be improved by increasing the interaction process between the first edge node and the networking center 130.

[0065] In one embodiment, referring to Figure 3 and Figure 5 , provides a data acquisition method, which is applied to the networking center 130, and the method includes:

[0066] Step S310: receiving a node query request from a first edge node, wherein the node query request is used to request a query for an edge node 120 storing the target data.

[0067] Specifically, when the networking center 130 receives the node query request from the first edge node, it indicates that the local cache of the first edge node does not store the target data, and the networking center 130 needs to query other edge nodes 120 that store the target data.

[0068] Step S320 , querying the second edge node corresponding to the resource location identifier of the target data in the node query request according to the data storage records of each edge node 120 .

[0069] Specifically, the networking center 130 stores data storage records of multiple edge nodes 120, and the data storage records include the location resource identifier of each edge node 120 storing data. The second edge node storing the target data is queried based on the data storage record and the resource location identifier of the target data. The networking center 130 records the resource files stored on each edge node 120, and will not be affected by node changes and cause subsequent redirection access confusion, which can ensure that the client 110 accurately accesses the second edge node through redirection to obtain the target data, thereby achieving a stable improvement in the cache hit rate.

[0070] Step S330: When the second edge node is found and the usage status of the second edge node is available, the node address of the second edge node is returned to the first edge node as feedback information.

[0071] Specifically, when the networking center 130 queries the second edge node and the usage status of the second edge node is available, the node address of the second edge node is fed back to the first edge node, and the first edge node returns the 302 redirection address indicating the node address of the second edge node to the client 110 to ensure that the client 110 can access the available second edge node according to the 302 redirection address, thereby obtaining the target data from the second edge node, that is, the networking center 130 provides accurate and reliable feedback information to the first edge node.

[0072] In one embodiment, referring to Figure 4 , when the second edge node is found and the usage status of the second edge node is available, returning the node address of the second edge node as feedback information to the first edge node, comprises:

[0073] Step S3301: when multiple second edge nodes are found and the usage status of each second edge node is available, weighted sum of each operating parameter of the second edge node is performed according to a preset scoring scheme to obtain a node score of the second edge node, wherein the preset scoring scheme includes weighted coefficients corresponding to each of the operating parameters;

[0074] Step S3302: Return the node address of the second edge node corresponding to the node score with the largest value to the first edge node as feedback information.

[0075] Specifically, if the networking center 130 queries multiple available second edge nodes that store target data, it is necessary to select the node address of one of the second edge nodes and feed it back to the first edge node, and the screening rule is to perform weighted summation on the various operating parameters of each second edge node according to the preset scoring scheme to obtain the corresponding node score. The operating parameter can be any one of the parameters such as delay time, download speed, packet loss rate, return rate, node utilization rate, node transmission bandwidth, etc. The preset scoring scheme includes weighted coefficients of various operating parameters corresponding to business needs. The node score of the second edge node can be obtained by weighted summing the parameter values ​​of each operating parameter according to the weighted coefficients of each operating parameter. The larger the value of the node score, the more it meets the business requirements corresponding to the preset scoring scheme, and is also used to indicate the second edge node with the best service quality under the business requirements. Therefore, the node address of the second edge node corresponding to the node score with the largest value is returned to the first edge node, and the first edge node feeds back the 302 redirection address indicating the node address of the second edge node with the best service quality to the client 110, so that the client 110 accesses the second edge node with the best service quality according to the 302 redirection address to obtain the target data, that is, the second edge node with the best service quality provides the target data download service for the client 110, thereby improving the service quality for the client 110.

[0076] In one embodiment, performing weighted summation on each operating parameter of the second edge node according to a preset scoring scheme to obtain a node score of the second edge node includes:

[0077] The node utilization rate and the node transmission bandwidth of the second edge node are weighted and summed according to a preset scoring scheme to obtain a node score of the second edge node.

[0078] Specifically, only the corresponding values ​​of the node utilization and the node transmission bandwidth of the second edge node are weighted and summed according to a preset scoring scheme. The node utilization is used to indicate the operating resource occupancy of the second edge node. The higher the node utilization, the busier the second edge node is and the less likely it is that the client 110 is to be provided with the data download service for the target data in a timely manner. The client 110 needs to wait for a long time before the data download service for the target data is provided to the client 110. The lower the node utilization, the idler the second edge node is and the more likely it is that the target data download service can be provided to the client 110 in a timely manner.

[0079] The node transmission bandwidth is used to indicate the transmission bandwidth when the second edge node transmits data. The larger the node transmission bandwidth, the faster the data transmission speed of the second edge node. The smaller the node transmission bandwidth, the slower the data transmission speed of the second edge node.

[0080] In this embodiment, only the node utilization and node transmission bandwidth of the second edge nodes are focused on. Therefore, the corresponding values ​​of the node utilization and the corresponding values ​​of the node transmission bandwidth of each second edge node are weighted and summed to obtain the node score of each second edge node. The second edge node with a higher node score is an edge node 120 that is in an idle state and has a larger transmission bandwidth. It can provide target data to the client 110 in a timely manner and at a faster transmission speed to improve the data service quality for the client 110.

[0081] In one embodiment, after querying the second edge node corresponding to the resource location identifier of the target data in the node query request, the method further includes:

[0082] When the second edge node is found and the usage status of the second edge node is unavailable, or when the second edge node is not found, feedback information indicating that the node query fails is returned to the first edge node.

[0083] Specifically, when the second edge node is queried but the usage status of the second edge node is unavailable, it means that the second edge node is abnormal and cannot provide the data download service of the target data for the client 110. The failure to query the second edge node is likely due to the fact that the target data is the latest published data, and the networking center 130 has not yet recorded the storage location of the target data. Based on the above two situations, the networking center 130 cannot feedback the node address of the available second edge node to the first edge node, and then returns the feedback information indicating the node query failure to the first edge node. Figure 5 , and the first edge node pulls the target data from its parent node and then provides it to the client 110. The networking center 130 will return a feedback message to the first edge node regardless of whether the query result of the second edge node is successful or not, so that the first edge node can continue to provide the client 110 with a 302 redirection address or the target data pulled from the parent node according to the feedback information of the networking center 130.

[0084] In summary, only when the target data is not found in the local cache of the first edge node and the target data is not newly released data, and the node address of the second edge node can be obtained from the target networking center 130, the 302 redirection technology will be used to allow the client 110 to access the second edge node according to the 302 redirection address. Compared with the first edge node, when the target data is not found in the local cache and the target data is newly released data, the second edge node storing the target data can still be queried through the scheduling system, which reduces the number of 302 redirections when the target data is newly released data, thereby reducing the 302 redirection ratio to reduce the impact of a high 302 redirection ratio on the loading speed. In particular, for cold data, the access frequency of cold data is very low, and many cold data may have only one access record. Based on the above data acquisition method, the acquisition method for cold data is to download it from the parent node of the first edge node, and no 302 redirection jump will be performed, thereby ensuring that the data loading speed is not affected by the 302 redirection.

[0085] Figure 2-5 FIG. 1 is a flow chart of a data acquisition method in one embodiment. It should be understood that although Figure 2-5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-5 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0086] In one embodiment, Figure 6 As shown, a data acquisition device is provided, comprising:

[0087] The data query module 410 is used to query the target data corresponding to the data request in the local cache when receiving the data request initiated by the client 110, wherein the data request is used to request to obtain the target data;

[0088] A determination module 420 is used to determine a target networking center 130 from among multiple networking centers 130 when the target data is not found in the local cache, according to a hash result corresponding to a resource location identifier of the target data, wherein each of the networking centers 130 stores data storage records of multiple edge nodes 120;

[0089] A sending module 430 is used to send a corresponding node query request to the target networking center 130 according to the data request, wherein the node query request is used to request the target networking center 130 to provide a node address of a second edge node storing the target data according to a data storage record, and the second edge node is the edge node 120 storing the target data;

[0090] A feedback module is used to feed back the node address of the available second edge node to the client 110 when the feedback information returned by the target networking center 130 contains the node address of the available second edge node, so that the client 110 accesses the available second edge node according to the node address to obtain the target data.

[0091] In one embodiment, the feedback module is further used for:

[0092] When the feedback information returned by the target networking center 130 indicates that the node query fails, forwarding the data request to the parent node of the first edge node;

[0093] When the target data returned by the parent node is received, the target data is returned to the client 110 .

[0094] In one embodiment, Figure 7 As shown, a data acquisition device is provided, the device comprising:

[0095] The receiving module 510 is configured to receive a node query request from a first edge node, wherein the node query request is used to request a query for an edge node 120 storing the target data;

[0096] A node query module 520, configured to query the second edge node corresponding to the resource location identifier of the target data in the node query request according to the data storage records of each edge node 120;

[0097] The second feedback module 530 is configured to return the node address of the second edge node as feedback information to the first edge node when the second edge node is found and the usage status of the second edge node is available.

[0098] In one embodiment, the second feedback module 530 is further configured to:

[0099] When multiple second edge nodes are queried and the usage status of each second edge node is available, weighted summing each operating parameter of the second edge node according to a preset scoring scheme to obtain a node score of the second edge node, wherein the preset scoring scheme includes weighted coefficients corresponding to each of the operating parameters;

[0100] The node address of the second edge node corresponding to the node score with the largest value is returned to the first edge node as feedback information.

[0101] In one embodiment, the second feedback module 530 is further configured to:

[0102] The node utilization rate and the node transmission bandwidth of the second edge node are weighted and summed according to a preset scoring scheme to obtain a node score of the second edge node.

[0103] In one embodiment, the second feedback module 530 is further configured to:

[0104] When the second edge node is found and the usage status of the second edge node is unavailable, or when the second edge node is not found, feedback information indicating that the node query fails is returned to the first edge node.

[0105] like Figure 8 As shown, an embodiment of the present application provides a computer device, including a processor 711, a communication interface 712, a memory 713 and a communication bus 714, wherein the processor 711, the communication interface 712, and the memory 713 communicate with each other through the communication bus 714;

[0106] Memory 713, used for storing computer programs;

[0107] In one embodiment of the present application, the processor 711 is used to implement the data acquisition method provided by any one of the aforementioned method embodiments when executing the program stored in the memory 713.

[0108] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] In one embodiment, the data acquisition device provided by the present application can be implemented in the form of a computer program. The computer program can be Figure 8 The computer device shown in the figure is run. The memory of the computer device can store various program modules constituting the data acquisition device, such as: Figure 6 The data query module 410, the determination module 420, the sending module 430 and the first feedback module 440 are shown. The computer program composed of various program modules enables the processor to execute the steps in the data acquisition method of each embodiment of the present application described in this specification.

[0110] Figure 8 The computer device shown can be Figure 6 The data query module 410 in the data acquisition device shown executes to query the target data corresponding to the data request in the local cache when receiving the data request initiated by the client 110, wherein the data request is used to request to obtain the target data. The computer device can determine the target networking center 130 among multiple networking centers 130 according to the hash result corresponding to the resource location identifier of the target data when the target data is not queried in the local cache through the determination module 420, wherein each of the networking centers 130 stores data storage records of multiple edge nodes 120. The computer device can execute to send a corresponding node query request to the target networking center 130 according to the data request through the sending module 430, wherein the node query request is used to request the target networking center 130 to provide the node address of the second edge node storing the target data according to the data storage record, and the second edge node is the edge node 120 storing the target data. The computer device can execute through the first feedback module 440 that when the node address of the available second edge node is included in the feedback information returned by the target networking center 130, the node address of the available second edge node is fed back to the client 110, so that the client 110 accesses the available second edge node according to the node address to obtain the target data.

[0111] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the data acquisition method provided in any of the aforementioned method embodiments are implemented.

[0112] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0114] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0115] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data acquisition system, characterized in that: The system comprises: A client is used to initiate a data request, wherein the data request is used to request to obtain target data; The first edge node is used to query the target data in the local cache in response to the data request, and when the target data is not found, determine the target networking center from multiple networking centers according to the hash result corresponding to the resource location identifier of the target data, and initiate a node query request to the target networking center to request to obtain the node address of the second edge node, and when receiving the node address of the second edge node, send a 302 redirection address indicating the node address of the second edge node to the client, so that the client accesses the second edge node according to the 302 redirection address to obtain the target data, and the first edge node and the second edge node are any different edge nodes; The target networking center is used to respond to the node query request and query the second edge node corresponding to the resource location identifier of the target data in the data storage record of each edge node, and return the node address of the second edge node to the first edge node when the second edge node is queried and the usage status of the second edge node is available.

2. A data acquisition method, characterized in that: Applied to an edge node, the method comprises: Receiving a data request from a client, wherein the data request is used to request to obtain target data; When the target data is not found in the local cache of the first edge node, a target networking center is determined from multiple networking centers according to a hash result corresponding to a resource location identifier of the target data, wherein each of the networking centers stores data storage records of multiple edge nodes; Sending a corresponding node query request to the target networking center according to the data request, wherein the node query request is used to request the target networking center to provide a node address of a second edge node storing the target data according to a data storage record; When the node address of the second edge node is included in the feedback information returned by the target networking center, the 302 redirection address indicating the node address of the second edge node is returned to the client, so that the client can access the second edge node according to the 302 redirection address to obtain the target data.

3. The method according to claim 2, characterized in that After sending a corresponding node query request to the target networking center according to the data request, the method further includes: When the feedback information returned by the target networking center indicates that the node query fails, forwarding the data request to a parent node of the first edge node; When the target data returned by the parent node is received, the target data is returned to the client.

4. A data acquisition method, characterized in that: Applied to a networking center, the method includes: Receiving a node query request from a first edge node, wherein the node query request is used to request a query for an edge node storing the target data; According to the data storage records of each edge node, query the second edge node corresponding to the resource location identifier of the target data in the node query request; When the second edge node is found and the usage status of the second edge node is available, the node address of the second edge node is returned to the first edge node as feedback information.

5. The method according to claim 4, characterized in that The step of returning the node address of the second edge node as feedback information to the first edge node when the second edge node is found and the usage status of the second edge node is available includes: When multiple second edge nodes are queried and the usage status of each second edge node is available, weighted summing each operating parameter of the second edge node according to a preset scoring scheme to obtain a node score of the second edge node, wherein the preset scoring scheme includes weighted coefficients corresponding to each of the operating parameters; The node address of the second edge node corresponding to the node score with the largest value is returned to the first edge node as feedback information.

6. The method according to claim 5, characterized in that The weighted summation of the operating parameters of the second edge node is performed according to a preset scoring scheme to obtain a node score of the second edge node, including: The node utilization rate and the node transmission bandwidth of the second edge node are weighted and summed according to a preset scoring scheme to obtain a node score of the second edge node.

7. The method according to claim 4, characterized in that The method includes: after querying the second edge node corresponding to the resource location identifier of the target data in the node query request, the method also includes: When the second edge node is found and the usage status of the second edge node is unavailable, or when the second edge node is not found, feedback information indicating that the node query fails is returned to the first edge node.

8. A data acquisition device, characterized in that: The device comprises: A data query module, configured to query the local cache for target data corresponding to the data request upon receiving a data request initiated by a client, wherein the data request is used to request to obtain the target data; A determination module, configured to determine a target networking center from a plurality of networking centers according to a hash result corresponding to a resource location identifier of the target data when the target data is not found in the local cache, wherein each of the networking centers stores data storage records of a plurality of edge nodes; A sending module, used for sending a corresponding node query request to the target networking center according to the data request, wherein the node query request is used to request the target networking center to provide a node address of a second edge node storing the target data according to a data storage record, and the second edge node is an edge node storing the target data; The first feedback module is used to feed back the node address of the available second edge node to the client when the feedback information returned by the target networking center includes the node address of the available second edge node, so that the client accesses the available second edge node according to the node address to obtain the target data.

9. A data acquisition device, characterized in that: The device comprises: A receiving module, configured to receive a node query request from a first edge node, wherein the node query request is used to request a query for an edge node storing the target data; A node query module, used to query the second edge node corresponding to the resource location identifier of the target data in the node query request according to the data storage records of each edge node; The second feedback module is configured to return the node address of the second edge node as feedback information to the first edge node when the second edge node is found and the usage status of the second edge node is available.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 2 to 7 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 2 to 7 are implemented.

Images