Distributed storage accelerated access method, device, equipment and storage medium

Abstract

The present disclosure relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for accelerating access to distributed storage. The present disclosure allocates a target storage node to a client, stores the files uploaded by the client to the target storage node and synchronizes them to a central storage node, synchronizes the files on demand when the files need to be downloaded, does not need to synchronize the files to each storage node, and can directly download the files from the central storage node when downloading files, thus saving storage resources, and not requiring the user to bear the time cost of caching from the central storage node to the target storage node, ensuring the real-time nature of file synchronization and downloading, and improving the efficiency of file acquisition.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for accelerating access to distributed storage. Background Art

[0002] With the continuous development of computer technology, more and more companies are using distributed storage to manage their internal information.

[0003] Normally, each storage node in a distributed storage system usually needs to be in the same network. Based on the distributed characteristics, the files in each storage node are synchronized to other storage nodes respectively; if a file in a storage node is not synchronized to other storage nodes, users of other storage nodes cannot successfully obtain the file.

[0004] Therefore, the existing file acquisition method has high network cost and storage cost, and has the problem of synchronization delay. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a method, device, equipment and storage medium for accelerating access to distributed storage to reduce storage costs and reduce synchronization delays.

[0006] In a first aspect, an embodiment of the present disclosure provides a method for accelerating access to distributed storage, the method being applied to a distributed storage system based on a region, comprising:

[0007] Determine a target storage node corresponding to the client from multiple candidate storage nodes of the storage system, wherein the multiple candidate storage nodes include a central storage node and an edge storage node;

[0008] According to the file upload request initiated by the client, the to-be-uploaded file corresponding to the file upload request is saved in the target storage node;

[0009] If the target storage node is an edge storage node, synchronizing the file to be uploaded to the central storage node;

[0010] Determine, according to the file download request initiated by the client, a target file corresponding to the file download request;

[0011] If the target file exists in the storage system and the upload path of the target file is not the target storage node, the target file is downloaded from the central storage node to the client, and the target file in the central storage node is cached in the target storage node.

[0012] In some embodiments, determining a target storage node corresponding to the client from a plurality of candidate storage nodes of the storage system includes:

[0013] Obtain the public IP address of each of the candidate storage nodes and the public IP address of the client;

[0014] The candidate storage node and the client are matched according to the public network IP address, and the candidate storage node in the same network as the client is used as the target storage node of the client.

[0015] In some embodiments, the method further comprises:

[0016] If there is no candidate storage node in the same network as the client, then obtaining the intranet detection address of the candidate storage node, so that the client can access each candidate storage node through the intranet detection address, and taking the successfully accessed candidate storage node as the target storage node of the client; or,

[0017] If there is no candidate storage node that is successfully accessed through the intranet detection address, the central storage node is allocated to the client.

[0018] In some embodiments, according to the file upload request initiated by the client, saving the to-be-uploaded file corresponding to the file upload request to the target storage node includes:

[0019] Obtaining a hash value of the file to be uploaded;

[0020] If it is determined based on the hash value that the file to be uploaded does not exist in the storage system, the file to be uploaded is stored in the target storage node.

[0021] In some embodiments, storing the to-be-uploaded file in the target storage node includes:

[0022] Uploading multiple slice data of the file to be uploaded to the target storage node in sequence;

[0023] Merging the multiple fragmented data to obtain a merged file;

[0024] If the hash value of the merged file is consistent with the hash value of the file to be uploaded, it is determined that the uploading of the file to be uploaded is completed.

[0025] In some embodiments, after determining the target file corresponding to the file download request initiated by the client, the method further includes:

[0026] Acquire file storage information of the target file according to the file identifier of the target file;

[0027] Based on the file storage information, determine the upload path of the target file.

[0028] In some embodiments, the method further comprises:

[0029] When it is determined based on the file storage information that the target file is in a state to be synchronized, querying the file identifier in the synchronization queue;

[0030] If the file identifier does not exist in the synchronization queue, adding the synchronization task of the target file to the synchronization queue; or,

[0031] If the file identifier exists in the synchronization queue, the file storage information is obtained again after a preset waiting time.

[0032] In a second aspect, an embodiment of the present disclosure provides a distributed storage access acceleration device, including:

[0033] A first determination module, configured to determine a target storage node corresponding to a client from a plurality of candidate storage nodes of a storage system, wherein the plurality of candidate storage nodes include a central storage node and an edge storage node;

[0034] The upload module is used to save the to-be-uploaded file corresponding to the file upload request to the target storage node according to the file upload request initiated by the client;

[0035] A synchronization module, configured to synchronize the file to be uploaded to the central storage node if the target storage node is an edge storage node;

[0036] A second determination module, configured to determine a target file corresponding to a file download request initiated by a client;

[0037] A download module is used to download the target file from the central storage node to the client if the target file exists in the storage system and the upload path of the target file is not the target storage node, and cache the target file in the central storage node to the target storage node.

[0038] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0039] Memory;

[0040] Processor; and

[0041] Computer programs;

[0042] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect.

[0043] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method described in the first aspect.

[0044] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the accelerated access method of distributed storage as described above is implemented.

[0045] The accelerated access method, apparatus, device and storage medium for distributed storage provided by the embodiments of the present disclosure allocate a target storage node to the client, store the files uploaded by the client to the target storage node and synchronize them to the central storage node, synchronize the files on demand when they need to be downloaded, and there is no need to synchronize the files to each storage node. When downloading files, they can be downloaded directly from the central storage node, thus saving storage resources and eliminating the need for users to bear the time cost of caching from the central storage node to the target storage node, thereby ensuring the real-time synchronization and downloading of files and improving the efficiency of file acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 A flow chart of a method for accelerating access to distributed storage provided by an embodiment of the present disclosure;

[0049] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0050] Figure 3 A schematic diagram of a storage node allocation process provided by an embodiment of the present disclosure;

[0051] Figure 4 A schematic diagram of a query flow before uploading a file provided in an embodiment of the present disclosure;

[0052] Figure 5A schematic diagram of a file upload process provided by an embodiment of the present disclosure;

[0053] Figure 6 A schematic diagram of a target file pre-download judgment process provided by an embodiment of the present disclosure;

[0054] Figure 7 A schematic diagram of a central storage node file download process provided by an embodiment of the present disclosure;

[0055] Figure 8 A schematic diagram of a file download process for an edge storage node provided in an embodiment of the present disclosure;

[0056] Fig. 9 A schematic diagram of the structure of a distributed storage accelerated access device provided in an embodiment of the present disclosure;

[0057] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0059] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0060] The embodiments of the present disclosure provide a method for accelerating access to distributed storage, which will be described below in conjunction with specific embodiments.

[0061] Figure 1 A flow chart of a method for accelerating access to distributed storage provided by an embodiment of the present disclosure. The method can be applied to Figure 2 In the region-based distributed storage system shown, the storage system application scenario includes various storage nodes connected via the Internet, including edge storage nodes and central storage nodes in the data center. The central computing node in the data center is used to manage each storage node and the corresponding client.

[0062] There is no narrow concept of network cluster between storage nodes. In fact, the storage system is built on the Internet (wide area) network, and the storage devices of each edge storage node are deployed in the branch offices in various regions, connected to the branch office LAN, and connected to the storage center through the branch office's Internet exit. Figure 2In the region-based distributed storage system shown, branch A, branch B, and branch C are located in different regions, and the branches in each region correspond to different edge storage nodes, for example, branch A corresponds to edge storage node a; branch B corresponds to edge storage node b; branch C corresponds to edge storage node c.

[0063] It is understandable that the method for accelerating access to distributed storage provided by the embodiments of the present disclosure may also be applied in other scenarios.

[0064] Combine the following Figure 2 The application scenario shown is Figure 1 The method for accelerating access to distributed storage shown in the figure is introduced, and the specific steps of the method are as follows:

[0065] S101. Determine a target storage node corresponding to a client from a plurality of candidate storage nodes of a storage system, where the plurality of candidate storage nodes include a central storage node and an edge storage node.

[0066] The storage system includes multiple storage nodes, which are divided into central storage nodes and edge storage nodes.

[0067] Edge storage nodes are storage terminals, usually deployed in the branch office LAN. Storage media can be configured on them, and disk column storage, NAS storage or cloud storage can be selected. They mainly provide storage functions and cache computing functions. When a branch office has a fixed IP, the edge storage node can be upgraded to a central storage node.

[0068] The central storage node is used to back up all files, which is essentially the same as the edge storage node, that is, it is equivalent to designating a qualified edge storage node as the central storage node.

[0069] All storage nodes in the storage system can be used as candidate storage nodes and assigned to corresponding clients. Specifically, when there is a central storage node, an edge storage node that meets the conditions is assigned to each client first; if there is no edge storage node that meets the conditions, a central storage node is assigned to the client.

[0070] Optionally, before assigning a candidate storage node to a client, the client needs to provide authorization information to complete client authentication to confirm the client's authority to access the storage system and ensure data security in the storage system.

[0071] The client also includes a client component (or upload component), which encapsulates the function of interacting with the business system. It needs to be initialized and participates in the allocation of storage nodes and the upload and download functions of files.

[0072] For example, when the client is located in a branch office network, it uses the branch office LAN to automatically connect to the branch office's corresponding edge storage node. In other cases, it connects to the central storage node by default to ensure that the function is always available.

[0073] S102: According to the file upload request initiated by the client, the to-be-uploaded file corresponding to the file upload request is saved in the target storage node.

[0074] S103: If the target storage node is an edge storage node, synchronize the file to be uploaded to the central storage node.

[0075] If the target storage node is an edge storage node, after the file to be uploaded is saved to the target storage node, a file synchronization operation will be triggered to submit the file to be uploaded to the synchronization queue and finally uploaded to the central storage node.

[0076] Correspondingly, if the target storage node is a central storage node, no file synchronization operation is required.

[0077] S104: Determine a target file corresponding to the file download request according to the file download request initiated by the client.

[0078] In the storage system, each file has its own file storage information, such as file ID and file Metadata, etc. The file download request carries relevant information for identifying the target file, usually the file ID, based on which the target file to be downloaded can be accurately identified in the storage system.

[0079] Optionally, if the corresponding target file is not found based on the file download request, it indicates that the target file may not exist in the storage system, and an error message is returned to the client.

[0080] S105. If the target file exists in the storage system and the upload path of the target file is not the target storage node, the target file is downloaded from the central storage node to the client, and the target file in the central storage node is cached in the target storage node.

[0081] The upload path refers to the storage node to which the target file is initially uploaded. For example, in the above step S102, the upload path of the file to be uploaded is the target storage node.

[0082] If the upload path of the target file is not the target storage node corresponding to the client, it means that the target file does not exist in the target storage node at this time. However, since the target file is directly uploaded to the central storage node or synchronized to the central storage node through a file synchronization operation, the target file is stored in the central storage node. At this time, the target file is downloaded from the central storage node to the client, and the target file in the central storage node is cached to the target storage node of the client.

[0083] The disclosed embodiment determines the target storage node corresponding to the client from multiple candidate storage nodes of the storage system, wherein the multiple candidate storage nodes include a central storage node and an edge storage node; according to a file upload request initiated by the client, saves the to-be-uploaded file corresponding to the file upload request in the target storage node; if the target storage node is an edge storage node, synchronizes the to-be-uploaded file to the central storage node; according to a file download request initiated by the client, determines the target file corresponding to the file download request; if the target file exists in the storage system and the upload path of the target file is not the target storage node, downloads the target file from the central storage node to the client, and caches the target file in the central storage node to the target storage node; by allocating the target storage node to the client, the file uploaded by the client is stored in the target storage node and synchronized to the central storage node; when the file needs to be downloaded, the file is synchronized on demand without synchronizing the file to each storage node; when the file is downloaded, it can be downloaded directly from the central storage node, thereby saving storage resources and eliminating the need for the user to bear the time cost of caching from the central storage node to the target storage node, thereby ensuring the real-time performance of file synchronization and downloading and improving the efficiency of file acquisition.

[0084] Based on the above embodiment, the method of determining the target storage node corresponding to the client from multiple candidate storage nodes of the storage system includes: obtaining the public IP address of each of the candidate storage nodes and the public IP address of the client; matching the candidate storage nodes and the client according to the public IP addresses, and using the candidate storage nodes in the same network as the client as the target storage node of the client.

[0085] Figure 3 A schematic diagram of a storage node allocation process provided by an embodiment of the present disclosure. Figure 3As shown, first establish a heartbeat mechanism for each edge storage node. Each edge storage node regularly reports the current service status, including the health status and network status of the edge storage node, as well as the public IP address of each edge storage node. The central computing node records the service status and saves the IP address to the routing table. When the client is initialized, the central computing node extracts the public IP of the client, matches it with the public IP of each edge storage node, and uses the public IP address of the client to query whether there is a corresponding edge storage node in the routing table to determine whether there is an edge storage node in the same network as the client.

[0086] If the public IP address matches successfully, the edge storage node in the same network as the client will be used as the target storage node for the client.

[0087] Furthermore, if there is no candidate storage node in the same network as the client, the intranet detection address of the candidate storage node is obtained, so that the client can access each of the candidate storage nodes through the intranet detection address, and the candidate storage node that is successfully accessed is used as the target storage node of the client; or, if there is no candidate storage node that is successfully accessed through the intranet detection address, the central storage node is allocated to the client.

[0088] If the public network IP address fails to match, the storage node needs to be allocated according to the intranet detection address. Specifically, the client accesses the intranet detection address of each edge storage node in turn, and uses the successfully accessed edge storage node as the client's target storage node; if there is no successfully accessed edge storage node, the central storage node is used as the client's target storage node.

[0089] Preferably, the same intranet detection address is configured for each edge storage node, and in this case, the client accesses the intranet detection address once to determine whether the access is successful.

[0090] The disclosed embodiment preferentially allocates edge detection nodes located in the branch office LAN to the client, relying on the network advantage of the 10 Gigabit LAN. The client accesses the storage node through the LAN to optimize the upload and download rates of files to the greatest extent.

[0091] In some embodiments, according to the file upload request initiated by the client, the file to be uploaded corresponding to the file upload request is saved in the target storage node, including: obtaining the hash value of the file to be uploaded; if it is determined based on the hash value that the file to be uploaded does not exist in the storage system, the file to be uploaded is stored in the target storage node.

[0092] Figure 4A schematic diagram of a pre-file upload query process provided in an embodiment of the present disclosure.

[0093] like Figure 4 As shown, the user can select the file to be uploaded on the client, and the upload component reads the hash value of the file to be uploaded, and updates the client's visual interface to display the UI interface ready to upload the file. The hash value of the file to be uploaded is a fixed-length value calculated by a hash function, which is used to represent the unique identifier of the content of the file to be uploaded.

[0094] The client sends a file upload request carrying the hash value of the file to be uploaded to the central computing node. The central computing node queries the storage system based on the hash value whether the file to be uploaded exists, that is, determines whether there is an uploaded file identical to the file to be uploaded in the storage system.

[0095] If the file to be uploaded exists in the storage system, it means that the file has been uploaded before. At this time, the file storage information of the file to be uploaded is read to determine whether the file has been uploaded. If the file has been uploaded, the corresponding upload record is saved and the file storage information is cached in the client for subsequent use.

[0096] If the file to be uploaded does not exist in the storage system, the file storage information of the file to be uploaded is constructed; or, if the file to be uploaded exists in the storage system but the upload has not been completed, the file status in the file storage information is modified to an incomplete status, the corresponding upload record is saved, and the file storage information is cached in the client, and then the file to be uploaded is uploaded.

[0097] Correspondingly, the upload component will update the client's visual interface to display the UI interface to be uploaded.

[0098] Specifically, storing the file to be uploaded to the target storage node includes: uploading multiple fragment data of the file to be uploaded to the target storage node in sequence; merging the multiple fragment data to obtain a merged file; if the hash value of the merged file is consistent with the hash value of the file to be uploaded, determining that the upload of the file to be uploaded is completed.

[0099] Figure 5 A schematic diagram of a file upload process provided for an embodiment of the present disclosure. It should be noted that the upload process is similar when the target storage node is an edge storage node or a central storage node, except that when the target storage node is an edge storage node, after the file to be uploaded is uploaded, it needs to be synchronized from the edge storage node to the central storage node, and the synchronization status of the file to be uploaded is sent to the central computing node for storage.

[0100] like Figure 5As shown, after the target storage node obtains the file storage information, it determines whether the file to be uploaded exists in its own disk. If so, the file storage information is marked as uploaded. If not, the fragment data of the file to be uploaded uploaded by the client is read.

[0101] Optionally, for a file to be uploaded that does not have a shard data set, the target storage node needs to first create a shard data set for it to store the read shard data.

[0102] In the process of reading shard data, if the shard data with null content is read, one possibility is that it means that the currently uploaded shard data has been read, and the client needs to continue uploading the shard data. The upload component in the client updates the client's visual interface accordingly to display the UI interface of the upload progress; another possibility is that an error occurs in the shard data. After correcting it, continue to read the shard data into the shard data collection.

[0103] Furthermore, all the shard data in the shard data set are merged to obtain a merged file. If the hash value of the merged file is consistent with the hash value of the file to be uploaded, it means that the merged file is consistent with the file to be uploaded, and the uploading of the file to be uploaded is complete; if the sum of the merged file and the file to be uploaded is inconsistent, it means that there is a deviation between the merged file and the file to be uploaded, and the file to be uploaded needs to be uploaded again.

[0104] After the uploaded file is uploaded, the storage status in the file storage information is changed to the upload completion status and cached to the central computing node and the client. The central computing node saves the file storage status and upload path; the upload component in the client updates the client's visual interface accordingly to display the UI interface of the upload completion.

[0105] Optionally, when the file to be uploaded needs to be synchronized to the central storage node, the storage status is modified to a pending synchronization status, and a corresponding synchronization task is created in the synchronization queue; after the synchronization is completed, the storage status is modified to an upload completion status.

[0106] The disclosed embodiment avoids synchronous uploading of files by comparing hash values ​​of files and verifies the completeness of file uploading, thereby further improving the efficiency and accuracy of distributed storage.

[0107] In some embodiments, after determining the target file corresponding to the file download request initiated by the client, the method further includes: obtaining file storage information of the target file according to a file identifier of the target file; and determining an upload path for the target file based on the file storage information.

[0108] Correspondingly, if the upload path of the target file is the target storage node of the client, the target file is downloaded from the target storage node to the client.

[0109] Specifically, the file storage data also includes the storage status of the file, which is at least divided into a pending upload status, a pending synchronization status, and an upload completion status.

[0110] Among them, when it is determined that the target file is in a state to be synchronized based on the file storage information, the file identifier is queried in the synchronization queue; if the file identifier does not exist in the synchronization queue, the synchronization task of the target file is added to the synchronization queue; or, if the file identifier exists in the synchronization queue, the file storage information is obtained again after a preset waiting time.

[0111] Figure 6 This is a schematic diagram of the target file pre-download judgment process provided by the embodiment of the present disclosure. Figure 6 As shown, the user selects a target file on the client, and the upload component determines the file identifier of the target file and requests the download address of the target file from the central computing node.

[0112] The central computing node reads the file storage information through the file identifier and determines the file mark of the target file according to the existence of the file storage information and the storage status therein. When the file storage information of the target file is queried and the storage status therein is in the pending upload state, the file mark is determined to be non-existent; otherwise, the file mark is determined to be successful and fed back to the client.

[0113] When the file is marked as successful, the client requests the target storage node for the download address of the target file; when the file is marked as non-existent, an error is reported to the user through the client, indicating that the target file download failed and the reason for the failure.

[0114] The following describes the specific process of downloading the target file when the target storage node is a central storage node and when the target storage node is an edge storage node.

[0115] Figure 7 A schematic diagram of a central storage node file download process provided in an embodiment of the present disclosure.

[0116] like Figure 7 As shown, the central storage node reads the file storage information of the target file from the central stage node according to the file identifier of the target file. If the file storage status in the file storage information is to be uploaded, it means that the target file has not been uploaded yet. At this time, the file is determined to be marked as non-existent, and an error is reported to the user through the client.

[0117] If the file status in the file storage information is the upload completion status, determine whether the target file exists in the central storage node disk. If so, construct the corresponding download address and send it to the client. The target file can be downloaded by opening the download address. Alternatively, if the target file does not exist in the central storage node disk, determine whether the upload path of the target file is the current central storage node. If so, determine that the file is marked as non-existent, and report an error to the user through the client.

[0118] If the file status in the file storage information is in the pending synchronization status, or the target file does not exist in the central storage node disk and the upload path of the target file is the edge storage node, the central computing node interface is called to query whether there is a synchronization task corresponding to the target file in the synchronization queue according to the file identifier of the target file. If not, the synchronization task of the target file is added to the synchronization queue, the target file is synchronized again, and the file is determined to be marked as pending synchronization. If so, the file is directly determined to be marked as pending synchronization, and the upload component sets the corresponding timed task to enable the central storage node to re-obtain the file storage information of the target file at a preset waiting time, and update the client's visual interface to display the downloading UI interface.

[0119] Figure 8 A schematic diagram of an edge storage node file download process provided in an embodiment of the present disclosure.

[0120] like Figure 8 As shown, the edge storage node reads the file storage information according to the file identifier. If the file storage information exists locally in the edge storage node, the file storage status therein is directly obtained; if the file storage information does not exist locally in the edge storage node, the file storage information is obtained from the central computing node according to the file identifier, and then the file storage status therein is determined.

[0121] If the file storage status in the file storage information is a pending upload status, it indicates that the target file has not been uploaded yet. In this case, the file is determined to be marked as non-existent, and an error is reported to the user through the client.

[0122] If the file status in the file storage information is the upload completion status, determine whether the target file exists in the edge storage node disk. If so, construct the corresponding download address and send it to the client. The target file can be downloaded by opening the download address. Alternatively, if the target file does not exist in the edge storage node disk, or the file storage status is to be synchronized, determine the upload path of the target file.

[0123] Specifically, if the upload path is the current edge storage node, the file is marked as not existing, and an error is reported to the user through the client.

[0124] Alternatively, if the upload path is other edge storage nodes, the central computing node interface is called to query whether there is a synchronization task corresponding to the target file in the synchronization queue according to the file identifier of the target file. If not, the synchronization task of the target file is added to the synchronization queue, the target file is synchronized again, and the file is marked as pending synchronization. If so, the file is directly marked as pending synchronization, and the upload component sets the corresponding timed task to enable the edge storage node to re-acquire the file storage information of the target file at a preset waiting time, and update the client's visual interface to display the downloading UI interface.

[0125] Alternatively, if the upload path is the central storage node, the central computing node interface is called to obtain the download address of the target file in the central storage node according to the file identifier, and the target file is added to the download queue according to the download address, so that the client can complete the download operation based on the download address. At the same time, the edge storage node caches the target file to the local edge storage node according to the download address, and updates the local file storage information to the completed state.

[0126] In the download process of the distributed storage system, the disclosed embodiment synchronizes and caches on demand based on the download requirements of the user, so the cached files must be files that will be used by the user, so as to ensure the efficiency of file synchronization and cache. At the same time, the disclosed embodiment can reduce abnormal conditions in the file download process as much as possible and ensure the stability of the file acquisition process by multiple verifications of the storage status and existence status of the target file during the download process.

[0127] Fig. 9 The schematic diagram of the structure of the accelerated access device for distributed storage provided by the embodiment of the present disclosure. The accelerated access device for distributed storage provided by the embodiment of the present disclosure can execute the processing flow provided by the accelerated access embodiment of distributed storage, such as Fig. 9As shown, the accelerated access device 90 for distributed storage includes: a first determination module 91, an upload module 92, a synchronization module 93, a second determination module 94, and a download module 95; wherein the first determination module 91 is used to determine the target storage node corresponding to the client from multiple candidate storage nodes of the storage system, and the multiple candidate storage nodes include a central storage node and an edge storage node; the upload module 92 is used to save the to-be-uploaded file corresponding to the file upload request to the target storage node according to the file upload request initiated by the client; the synchronization module 93 is used to synchronize the to-be-uploaded file to the central storage node if the target storage node is an edge storage node; the second determination module 94 is used to determine the target file corresponding to the file download request according to the file download request initiated by the client; the download module 95 is used to download the target file from the central storage node to the client if the target file exists in the storage system and the upload path of the target file is not the target storage node, and cache the target file in the central storage node to the target storage node.

[0128] Optionally, the first determination module 91 includes a first acquisition unit 911 and a matching unit 912; the first acquisition unit 911 is used to obtain the public IP address of each candidate storage node and the public IP address of the client; the matching unit 912 is used to match the candidate storage nodes and the client according to the public IP addresses, and use the candidate storage nodes in the same network as the client as the target storage nodes of the client.

[0129] Optionally, the first determination module 91 also includes an allocation unit 913, which is used to obtain the intranet detection address of the candidate storage node if there is no candidate storage node in the same network as the client, so that the client can access each of the candidate storage nodes through the intranet detection address, and use the successfully accessed candidate storage node as the target storage node of the client; or, if there is no candidate storage node that is successfully accessed through the intranet detection address, allocate the central storage node to the client.

[0130] Optionally, the upload module 92 includes a second acquisition unit 921 and an upload unit 922; the second acquisition unit 921 is used to obtain the hash value of the file to be uploaded; the upload unit 922 is used to store the file to be uploaded to the target storage node if it is determined based on the hash value that the file to be uploaded does not exist in the storage system.

[0131] Optionally, the uploading unit 922 is used to upload multiple fragment data of the file to be uploaded to the target storage node in sequence; merge the multiple fragment data to obtain a merged file; if the hash value of the merged file is consistent with the hash value of the file to be uploaded, it is determined that the upload of the file to be uploaded is completed.

[0132] Optionally, the download module 95 includes a determination unit 951, which is used to obtain file storage information of the target file according to the file identifier of the target file; and determine an upload path of the target file based on the file storage information.

[0133] Optionally, the download module 95 includes a query unit 952 and a third acquisition unit 953; the query unit 952 is used to query the file identifier in the synchronization queue when it is determined that the target file is in a state to be synchronized based on the file storage information; the third acquisition unit 953 is used to add the synchronization task of the target file to the synchronization queue if the file identifier does not exist in the synchronization queue; or, if the file identifier exists in the synchronization queue, obtain the file storage information again after a preset waiting time.

[0134] Fig. 9 The distributed storage accelerated access device of the illustrated embodiment can be used to execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar and will not be repeated here.

[0135] Fig.10 The electronic device provided by the embodiment of the present disclosure can execute the processing flow provided by the embodiment of the accelerated access method of distributed storage, such as Fig.10 As shown, the electronic device 100 includes: a memory 101, a processor 102, a computer program and a communication interface 103; wherein the computer program is stored in the memory 101 and is configured so that the processor 102 executes the accelerated access method of distributed storage as described above.

[0136] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the method for accelerating access to distributed storage described in the above embodiment.

[0137] In addition, an embodiment of the present disclosure also provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the accelerated access method of distributed storage as described above is implemented.

[0138] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0139] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for accelerating access to distributed storage, characterized in that: The method is applied to a distributed storage system based on a region, and the method includes: Determine a target storage node corresponding to the client from multiple candidate storage nodes of the storage system, wherein the multiple candidate storage nodes include a central storage node and an edge storage node; According to the file upload request initiated by the client, the to-be-uploaded file corresponding to the file upload request is saved in the target storage node; If the target storage node is an edge storage node, synchronizing the file to be uploaded to the central storage node; Determine, according to the file download request initiated by the client, a target file corresponding to the file download request; If the target file exists in the storage system, and the upload path of the target file is not the target storage node, download the target file from the central storage node to the client, and cache the target file in the central storage node to the target storage node; The step of determining a target storage node corresponding to the client from a plurality of candidate storage nodes of the storage system includes: Obtain the public IP address of each of the candidate storage nodes and the public IP address of the client; Matching the candidate storage node and the client according to the public network IP address, and taking the candidate storage node in the same network as the client as the target storage node of the client; If there is no candidate storage node in the same network as the client, then obtaining the intranet detection address of the candidate storage node, so that the client can access each candidate storage node through the intranet detection address, and taking the successfully accessed candidate storage node as the target storage node of the client; or, If there is no candidate storage node that is successfully accessed through the intranet detection address, the central storage node is allocated to the client.

2. The method according to claim 1, characterized in that The step of storing the to-be-uploaded file corresponding to the file upload request in the target storage node according to the file upload request initiated by the client includes: Obtaining a hash value of the file to be uploaded; If it is determined based on the hash value that the file to be uploaded does not exist in the storage system, the file to be uploaded is stored in the target storage node.

3. The method according to claim 2, characterized in that The step of storing the file to be uploaded to the target storage node includes: Uploading multiple slice data of the file to be uploaded to the target storage node in sequence; Merging the multiple fragmented data to obtain a merged file; If the hash value of the merged file is consistent with the hash value of the file to be uploaded, it is determined that the uploading of the file to be uploaded is completed.

4. The method according to claim 1, characterized in that: After determining the target file corresponding to the file download request according to the file download request initiated by the client, the method further includes: Acquire file storage information of the target file according to the file identifier of the target file; Based on the file storage information, determine the upload path of the target file.

5. The method according to claim 4, characterized in that The method further comprises: When it is determined based on the file storage information that the target file is in a state to be synchronized, querying the file identifier in the synchronization queue; If the file identifier does not exist in the synchronization queue, adding the synchronization task of the target file to the synchronization queue; or, If the file identifier exists in the synchronization queue, the file storage information is obtained again after a preset waiting time.

6. A distributed storage accelerated access device, characterized in that: include: A first determination module, configured to determine a target storage node corresponding to a client from a plurality of candidate storage nodes of a storage system, wherein the plurality of candidate storage nodes include a central storage node and an edge storage node; The upload module is used to save the to-be-uploaded file corresponding to the file upload request to the target storage node according to the file upload request initiated by the client; A synchronization module, configured to synchronize the file to be uploaded to the central storage node if the target storage node is an edge storage node; A second determination module, configured to determine a target file corresponding to a file download request initiated by a client; A download module, configured to download the target file from the central storage node to the client if the target file exists in the storage system and the upload path of the target file is not the target storage node, and cache the target file in the central storage node to the target storage node; The first determination module includes a first acquisition unit and a matching unit; the first acquisition unit is used to acquire the public IP address of each candidate storage node and the public IP address of the client; The matching unit is used to match the candidate storage node and the client according to the public network IP address, and use the candidate storage node in the same network as the client as the target storage node of the client; The first determination module also includes an allocation unit, which is used to obtain the intranet detection address of the candidate storage node if there is no candidate storage node in the same network as the client, so that the client can access each of the candidate storage nodes through the intranet detection address, and use the successfully accessed candidate storage node as the target storage node of the client; or, if there is no candidate storage node that is successfully accessed through the intranet detection address, allocate the central storage node to the client.

7. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images