Remote multi-network storage scheduling method and system based on zero trust

Abstract

The present invention belongs to the field of data storage technology, and specifically discloses a zero-trust-based off-site multi-network storage scheduling method and a zero-trust-based off-site multi-network storage scheduling system. The method includes: a client sends a file upload request to a central server to perform off-site processing of file upload, or a client sends a file download request to a central server to perform off-site processing of file download; the system includes a central server, an edge node, and an OSS source station. The present invention significantly improves the data transmission speed, especially in a cross-regional, multi-network environment; enhances data security and reduces potential security risks through a zero-trust architecture; reduces bandwidth costs and improves bandwidth utilization through intelligent scheduling and merging back to the source mechanism; improves system scalability and facilitates deployment of new edge nodes to cope with business growth; improves user experience and supports fast upload and download of large files and breakpoint resumption.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a zero-trust-based remote multi-network storage scheduling method and a zero-trust-based remote multi-network storage scheduling system. Background Art

[0002] With the development of cloud computing and big data, enterprises and organizations are facing increasing demands for data storage and transmission. Traditional centralized storage solutions often encounter bandwidth bottlenecks, high access latency and other issues when facing large-scale, cross-regional data access. At the same time, data security has become an increasingly important issue. Current data storage technologies have the following means:

[0003] Content Delivery Network (CDN): CDN deploys nodes at the edge of the network to cache content closer to users, improving access speed;

[0004] Distributed storage system: such as Hadoop HDFS, which improves data availability and read and write performance by distributing and storing data on multiple nodes;

[0005] Cloud storage acceleration: For example, Alibaba Cloud's OSS acceleration service optimizes data transmission paths by deploying nodes in different regions.

[0006] However, the above data storage technology still has the following problems:

[0007] Insufficient security: Traditional CDN and distributed storage systems do not take security into account comprehensively, especially in a multi-network environment;

[0008] Lack of flexibility: Existing systems are often deployed in a fixed manner and are difficult to dynamically adjust based on real-time network conditions and user needs;

[0009] Low efficiency: In cross-network and cross-region data transmission, network resources are often not fully utilized, resulting in low transmission efficiency;

[0010] High cost: Deploying and maintaining large-scale distributed nodes requires high hardware and operation and maintenance costs.

[0011] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0012] The purpose of the present invention is to solve the technical problems existing in the background technology. To this end, a zero-trust-based remote multi-network storage scheduling method and a zero-trust-based remote multi-network storage scheduling system are provided.

[0013] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0014] The off-site multi-network storage scheduling method based on zero trust includes the following steps:

[0015] Step S1: The client sends a file upload or download request to the central server. If the client sends a file upload request to the central server, the process jumps to steps S21-S27. If the client sends a file download request to the central server, the process jumps to steps S31-S36.

[0016] Step S21: The central server selects the optimal edge node according to the upload request information of the client and the heartbeat content continuously reported by the edge node, and caches the selection result;

[0017] Step S22: The central server returns the upload link of the selected edge node to the client, and adds a token to the upload link. The token is generated by the client's upload request information.

[0018] Step S23: The client uploads the file to the edge node through the received upload link;

[0019] Step S24: The edge node parses the TOKEN of the upload link. If the parse is invalid, the edge node refuses to receive the file uploaded by the client. If the parse is valid, the edge node allows to receive the file uploaded by the client.

[0020] Step S25: After the edge node receives the file, it performs a file integrity check;

[0021] Step S26: After the integrity check of the file passes, the edge node creates an upload task and uploads the file to the OSS source station for archiving and persistent storage;

[0022] Step S27: file upload is completed;

[0023] Step S31: The central server checks whether the corresponding file has a corresponding edge node cache result according to the download request information of the client;

[0024] Step S32: If yes, the central server directly returns the download link of the edge node of the cache result to the client, and adds a token to the download link. The token is generated by the download request information of the client;

[0025] If not, the central server selects the optimal edge node based on the client's download request information and the heartbeat content continuously reported by the edge node, caches the selection result, and returns the download link of the selected edge node to the client, and adds a token to the download link. The token is generated by the client's download request information;

[0026] Step S33: the client downloads the file from the edge node through the received download link;

[0027] Step S34: the edge node parses the TOKEN of the download link. If the parse is invalid, the client is denied from downloading the file from the edge node. If the parse is valid, the client is allowed to download the file from the edge node.

[0028] Step S35: After the analysis is valid, the edge node searches the local cache for the corresponding file ID;

[0029] If yes, the edge node directly returns the file content to the client;

[0030] If not, the edge node requests the OSS source station to download, and the OSS source station returns the file content to the edge node, and then the edge node returns the file content to the client;

[0031] Step S36: File downloading is completed.

[0032] The following is a technical solution further defined by the method in the present invention, wherein a zero-trust security channel is established between a central server and an edge node, including the following steps:

[0033] The edge node carries its own public key to request the central server;

[0034] After receiving the public key of the edge node, the central server returns the server public key to the edge node;

[0035] The edge node compares the server public key with the local server public key;

[0036] If the check is consistent, the establishment of the secure channel is completed and data transmission is carried out;

[0037] If the check is inconsistent, it means that there is a man-in-the-middle attack: the middleman intercepts the public key of the edge node and the server public key, and sends the middleman's public key to the central server and edge node. The central server and edge node record and discard the middleman's public key.

[0038] The following is a technical solution further defined by the method of the present invention, in which after the secure channel is established between the central server and the edge node, data transmission is performed, including the steps of:

[0039] The edge node uses its own private key to encrypt data and report the heartbeat content;

[0040] The central server uses the public key of the edge node to decrypt the data, then uses its own private key to encrypt the data, and returns the encrypted data to the edge node;

[0041] The edge node decrypts the data using the received server public key.

[0042] The following is a technical solution further defined by the method of the present invention, the heartbeat content continuously reported by the edge node to the central server includes: service area, its current CPU load, its current memory usage, and its disk IO status;

[0043] The client's upload request information includes: current network region, target archive storage type, file MD5 information and file name;

[0044] The client's download request information includes: the current network region and the download file ID.

[0045] The following is a technical solution further defined by the method of the present invention, wherein the central server selects the optimal edge node according to the upload request information of the client and the heartbeat content continuously reported by the edge node, including the steps of:

[0046] The edge node continuously initiates heartbeats to the central server, and the central server registers the heartbeat content of the corresponding edge node to the registry;

[0047] After receiving the upload request from the client, the central server retrieves the registry;

[0048] Match edge nodes based on the current network region and target archive storage type requested by the client;

[0049] Then the edge node is scored based on its CPU load, memory usage, and disk IO status. The edge node score = CPU load * 1 + memory usage * 1.5 + disk IO status * 2.

[0050] Select the edge node with the lowest score as the optimal edge node;

[0051] The central server selects the optimal edge node based on the client's download request information and the heartbeat content continuously reported by the edge node, including the following steps:

[0052] The edge node continuously initiates heartbeats to the central server, and the central server registers the heartbeat content of the corresponding edge node to the registry;

[0053] After receiving the download request from the client, the central server retrieves the registry;

[0054] Match edge nodes based on the current network region requested by the client and the storage type of the downloaded file ID;

[0055] Then the edge node is scored based on its CPU load, memory usage, and disk IO status. The edge node score = CPU load * 1 + memory usage * 1.5 + disk IO status * 2.

[0056] The edge node with the lowest score is selected as the optimal edge node.

[0057] The following is a technical solution further limited by the method of the present invention, the TOKEN of the upload link includes the MD5 information of the upload request, the file name and the file metadata; the TOKEN of the download link includes the file ID of the download request and the expiration time of the download link;

[0058] After the edge node receives the file, it verifies the file MD5 based on the MD5 information in the TOKEN of the upload link to confirm that the file is complete. If the file is incomplete, an error is reported.

[0059] The following is a technical solution further defined by the method of the present invention, wherein the edge node uploads the file to the OSS source station, including the steps of:

[0060] The edge node creates an upload task;

[0061] Serialize the file information, where the file information includes MD5 information, file ID, file name, and remote storage information;

[0062] Store files into a disk queue;

[0063] Take out the upload task and upload the file to the remote OSS source station according to the remote storage information. If the upload fails, put the file back into the disk queue.

[0064] The following is a technical solution further defined in the method of the present invention: when the edge node fails to find the corresponding file ID, the edge node queries whether there is already a local request for the source station to download;

[0065] If so, wait;

[0066] If not, create a download task and request the source site to download;

[0067] The remote OSS source station verifies the file MD5 information;

[0068] If the verification is successful, the remote OSS source station returns the file content to the edge node, and then the edge node returns the file content to the client;

[0069] If the verification fails, an alarm is sent to the edge node, which then sends an alarm to the client.

[0070] The following is a technical solution further limited by the method in the present invention. When multiple clients send file download requests to the central server and the edge node has no local cache, multiple edge nodes request the source station to download: when the front request comes, request the source station and mark it as being requested. If the subsequent request is the same as the current request, copy the download traffic of the source station to the subsequent request.

[0071] A zero-trust-based remote multi-network storage scheduling system is used to implement the above-mentioned zero-trust-based remote multi-network storage scheduling method, characterized in that it includes a central server, an edge node, and an OSS source station;

[0072] Central server for overall scheduling and control;

[0073] Edge nodes are deployed in the intranet computer room and serve as a transit station for data transmission;

[0074] OSS source station, as the final data storage location;

[0075] Set up a zero-trust security channel between the central server and the edge node to ensure communication security.

[0076] Compared with the prior art, the present invention has the following technical effects:

[0077] 1. Significantly improve data transmission speed, especially in cross-regional and multi-network environments;

[0078] 2. Enhance data security and reduce potential security risks through zero-trust architecture;

[0079] 3. Reduce bandwidth costs and improve bandwidth utilization through intelligent scheduling and merging back to the source mechanism;

[0080] 4. Improve system scalability and make it easier to deploy new edge nodes to cope with business growth;

[0081] 5. Improve user experience and support fast upload and download of large files and resume of breakpoint download.

[0082] The present invention is further described below in conjunction with the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0084] Figure 1 It is a schematic diagram of the file uploading process of the present invention;

[0085] Figure 2 It is a schematic diagram of the file downloading process of the present invention;

[0086] Figure 3 It is a schematic diagram of the process of establishing a secure channel between the edge node and the central server of the present invention;

[0087] Figure 4 It is a schematic diagram of the process of uploading files from the edge node of the present invention to the remote OSS source station;

[0088] Figure 5 It is a schematic diagram of the process of an edge node of the present invention downloading a file from a remote OSS source station. DETAILED DESCRIPTION

[0089] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0090] like Figure 1-5 As shown, this embodiment provides a zero-trust-based remote multi-network storage scheduling system for data storage and transmission in a remote multi-network environment. It mainly includes the following key components:

[0091] Central server: used for overall scheduling and control;

[0092] CDN edge node: deployed in the intranet computer room as a transit station for data transmission;

[0093] External OSS source station: as the final data storage location;

[0094] Zero Trust Channel: Connects the central server and edge nodes to ensure communication security.

[0095] The zero-trust-based off-site multi-network storage scheduling method includes: a client sends a file upload request to a central server for off-network file upload processing, or a client sends a file download request to a central server for off-network file download processing.

[0096] like Figure 1 As shown, file upload includes the following steps:

[0097] 1.1. The client sends a file upload request to the central server.

[0098] 1.2. The central server selects the optimal edge node based on the client's upload request information (the upload request information includes: the current network area, the target archive storage type, the file MD5 information and the file name) and the heartbeat content continuously reported by the edge node (the heartbeat content includes: the service area, its current CPU load, its current memory usage, and its disk IO status), and caches the selection result.

[0099] Specifically:

[0100] The edge node continuously initiates heartbeats to the central server, and the central server registers the heartbeat content of the corresponding edge node to the registry;

[0101] After receiving the upload request from the client, the central server retrieves the registry;

[0102] Match edge nodes according to the current network region and target archive storage type requested by the client. At this time, multiple edge nodes are usually matched, so the edge nodes are further screened;

[0103] Then the edge node is scored based on its CPU load, memory usage, and disk IO status. The edge node score = CPU load * 1 + memory usage * 1.5 + disk IO status * 2.

[0104] The edge node with the lowest score is selected as the optimal edge node.

[0105] 1.3. The central server returns the upload link of the selected edge node to the client, and adds a token to the upload link. The token is generated by the client's upload request information. The upload link token contains the MD5 information of the upload request, the file name, and the file metadata.

[0106] 1.4. The client uploads the file to the edge node through the received upload link;

[0107] 1.5. The edge node parses the TOKEN of the upload link. If the parse is invalid, the edge node refuses to receive the file uploaded by the client. If the parse is valid, the edge node allows the file uploaded by the client to be received.

[0108] 1.6. After the edge node receives the file, it performs a file integrity check: check the file MD5 according to the MD5 information in the TOKEN of the upload link to confirm that the file is complete. If the file is incomplete, an error is reported.

[0109] 1.7. After the file integrity check passes, the edge node creates an upload task and uploads the file to the OSS source site for archiving and persistent storage.

[0110] Specifically, Figure 4 As shown in the figure, the edge node uploads the file to the external OSS source station, including the following steps:

[0111] The edge node creates an upload task;

[0112] Serialize the file information, where the file information includes MD5 information, file ID, file name, and remote storage information;

[0113] Store files into a disk queue;

[0114] Take out the upload task and upload the file to the remote OSS source station according to the remote storage information. If the upload fails, put the file back into the disk queue and keep retrying until it succeeds. If it fails all the time, an alarm will be issued.

[0115] 1.8. File upload is completed.

[0116] like Figure 2 As shown, the file download includes the following steps:

[0117] 2.1. The client sends a file download request to the central server.

[0118] 2.2. The central server checks whether the corresponding file has a corresponding edge node cache result based on the client's download request information (the download request information includes: the current network area and the download file ID).

[0119] 2.3. If there is, the central server directly returns the download link of the edge node of the cache result to the client, and adds a token to the download link. The token is generated by the client's download request information. Among them, the download link token contains the file ID of the download request and the expiration time of the download link.

[0120] If not, the central server selects the optimal edge node based on the client's download request information (the download request information includes: the current network area and the download file ID) and the heartbeat content continuously reported by the edge node (the heartbeat content includes: the service area, its current CPU load, its current memory usage, and its disk IO status), and caches the selection result.

[0121] Specifically:

[0122] The edge node continuously initiates heartbeats to the central server, and the central server registers the heartbeat content of the corresponding edge node to the registry;

[0123] After receiving the download request from the client, the central server retrieves the registry;

[0124] Match edge nodes according to the current network region requested by the client and the storage type of the downloaded file ID. At this time, multiple edge nodes are usually matched, so the edge nodes are further screened;

[0125] Then the edge node is scored based on its CPU load, memory usage, and disk IO status. The edge node score = CPU load * 1 + memory usage * 1.5 + disk IO status * 2.

[0126] The edge node with the lowest score is selected as the optimal edge node.

[0127] The central server returns the download link of the selected edge node to the client and adds a token to the download link. The token is generated by the client's download request information. The download link token contains the file ID of the download request and the expiration time of the download link.

[0128] 2.4. The client downloads the file from the edge node through the received download link.

[0129] 2.5. The edge node parses the TOKEN of the download link. If the parsing is invalid, the client is denied from downloading the file from the edge node. If the parsing is valid, the client is allowed to download the file from the edge node.

[0130] 2.6. After the analysis is effective, if Figure 5 As shown, the edge node searches the local cache to see whether there is a corresponding file ID.

[0131] If yes, the edge node returns the file content directly to the client.

[0132] If not, the edge node queries whether there is already a local request for the source station to download;

[0133] If so, wait;

[0134] If not, create a download task and request the source site to download;

[0135] The remote OSS source station verifies the file MD5 information;

[0136] If the verification is successful, the remote OSS source station returns the file content to the edge node, and then the edge node returns the file content to the client;

[0137] If the verification fails, an alarm is sent to the edge node, which then sends an alarm to the client.

[0138] 2.7. File download is completed.

[0139] It should be noted that when multiple clients send file download requests to the central server and the edge node has no local cache, multiple edge nodes request the source site to download, that is, the merged back-to-source mode is adopted: when the front request comes, the source site is requested and marked as being requested. If the subsequent request is the same as the current request, the download traffic of the source site is copied to the subsequent request, thereby reducing CPU or memory usage, improving download performance, and reducing pressure on the source site.

[0140] like Figure 3 As shown in the figure, a zero-trust security channel is established between the central server and the edge node, including the following steps:

[0141] The edge node carries its own public key to request the central server;

[0142] After receiving the public key of the edge node, the central server returns the server public key to the edge node;

[0143] The edge node compares the server public key with the local server public key;

[0144] If the check is consistent, the establishment of the secure channel is completed and data transmission is carried out;

[0145] If the check is inconsistent, it means that there is a man-in-the-middle attack: the middleman intercepts the public key of the edge node and the server public key, and sends the middleman's public key to the central server and edge node. The central server and edge node record and discard the middleman's public key.

[0146] After the secure channel is established between the central server and the edge node, data transmission is performed, including the following steps:

[0147] The edge node uses its own private key to encrypt data and report the heartbeat content;

[0148] The central server uses the public key of the edge node to decrypt the data, then uses its own private key to encrypt the data, and returns the encrypted data to the edge node;

[0149] The edge node decrypts the data using the received server public key.

[0150] Edge nodes regularly report heartbeats, metadata, and load conditions, and the central server performs intelligent scheduling based on the reported content.

[0151] When the client requests a download, according to the HTTP protocol, a Range request header will be carried to mark the starting position of the current file download; when the download is interrupted and then re-downloaded, the browser will retrieve the size of the downloaded file and carry the Range request header to request the CDN edge node. The CDN edge node of the present invention will parse the Range field and offset the file pointer to the corresponding starting position, thereby realizing the breakpoint resume function.

[0152] Table 1 is an edge node calculation score analysis table for multiple regions and multiple storage types:

[0153] Node Name A B C D E F Service Area south south south south north north Service Storage Tencent Cloud Tencent Cloud Alibaba Cloud Alibaba Cloud Alibaba Cloud Tencent Cloud CPU Load 4 3 4 2 1 1 Memory usage 2 4 2 3 3 2 IO situation 3 1 4 1 5 2 Calculation score: load*1+memory*1.5+IO*2 13 11 15 8.5 15.5 8

[0154] As shown in Table 1, when a user in the south transfers file a to Alibaba Cloud for remote storage:

[0155] Filter out nodes C and D;

[0156] The evaluation shows that node D has a lower score, so node D is selected first;

[0157] And bind file a to node D.

[0158] When a user in the south downloads file b from Alibaba Cloud (which was previously transferred using node D):

[0159] According to the affinity principle, if node D is alive, node D is used for downloading first, because it does not need to go back to the source and the speed is faster.

[0160] When a user in the south downloads file c from Alibaba Cloud (it was previously transmitted using node B, but after background adjustments, node B is no longer responsible for Alibaba Cloud):

[0161] Because Node B is no longer responsible for Alibaba Cloud, even if there is affinity, there is no guarantee that the files cached in Node B still exist;

[0162] So we filter out nodes C and D;

[0163] The evaluation shows that node D has a lower score, so node D is selected first;

[0164] And bind file c to node D.

[0165] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Any technician familiar with the art can make many possible changes and modifications to the technical solution of the present invention by using the above disclosed methods and technical contents without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Therefore, all equivalent changes made according to the shape, structure and principle of the present invention without departing from the content of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. A remote multi-network storage scheduling method based on zero trust, characterized in that: The following steps are involved: Step S1: The client sends a file upload or download request to the central server. If the client sends a file upload request to the central server, the process jumps to steps S21-S27. If the client sends a file download request to the central server, the process jumps to steps S31-S36. Step S21: The central server selects the optimal edge node according to the upload request information of the client and the heartbeat content continuously reported by the edge node, and caches the selection result; Step S22: The central server returns the upload link of the selected edge node to the client, and adds a token to the upload link. The token is generated by the client's upload request information. Step S23: The client uploads the file to the edge node through the received upload link; Step S24: The edge node parses the TOKEN of the upload link. If the parse is invalid, the edge node refuses to receive the file uploaded by the client. If the parse is valid, the edge node allows to receive the file uploaded by the client. Step S25: After the edge node receives the file, it performs a file integrity check; Step S26: After the integrity check of the file passes, the edge node creates an upload task and uploads the file to the OSS source station for archiving and persistent storage; Step S27: file upload is completed; Step S31: The central server checks whether the corresponding file has a corresponding edge node cache result according to the download request information of the client; Step S32: If yes, the central server directly returns the download link of the edge node of the cache result to the client, and adds a token to the download link. The token is generated by the download request information of the client; If not, the central server selects the optimal edge node based on the client's download request information and the heartbeat content continuously reported by the edge node, caches the selection result, and returns the download link of the selected edge node to the client, and adds a token to the download link. The token is generated by the client's download request information; Step S33: the client downloads the file from the edge node through the received download link; Step S34: the edge node parses the TOKEN of the download link. If the parse is invalid, the client is denied from downloading the file from the edge node. If the parse is valid, the client is allowed to download the file from the edge node. Step S35: After the analysis is valid, the edge node searches the local cache for the corresponding file ID; If yes, the edge node directly returns the file content to the client; If not, the edge node requests the OSS source station to download, and the OSS source station returns the file content to the edge node, and then the edge node returns the file content to the client; Step S36: File downloading is completed.

2. The zero-trust-based remote multi-network storage scheduling method according to claim 1, characterized in that: Establishing a zero-trust secure channel between the central server and the edge node includes the following steps: The edge node carries its own public key to request the central server; After receiving the public key of the edge node, the central server returns the server public key to the edge node; The edge node compares the server public key with the local server public key; If the check is consistent, the establishment of the secure channel is completed and data transmission is carried out; If the check is inconsistent, it means that there is a man-in-the-middle attack: the middleman intercepts the public key of the edge node and the server public key, and sends the middleman's public key to the central server and edge node. The central server and edge node record and discard the middleman's public key.

3. The zero-trust-based remote multi-network storage scheduling method according to claim 2, characterized in that: After the secure channel is established between the central server and the edge node, data transmission is performed, including the following steps: The edge node uses its own private key to encrypt data and report the heartbeat content; The central server uses the public key of the edge node to decrypt the data, then uses its own private key to encrypt the data, and returns the encrypted data to the edge node; The edge node decrypts the data using the received server public key.

4. The zero-trust-based remote multi-network storage scheduling method according to claim 1, characterized in that: The heartbeat information that the edge node continuously reports to the central server includes: service area, its current CPU load, its current memory usage, and its disk IO status; The client's upload request information includes: current network region, target archive storage type, file MD5 information and file name; The client's download request information includes: the current network region and the download file ID.

5. The zero-trust-based remote multi-network storage scheduling method according to claim 4, characterized in that: The central server selects the optimal edge node based on the client's upload request information and the heartbeat content continuously reported by the edge node, including the following steps: The edge node continuously initiates heartbeats to the central server, and the central server registers the heartbeat content of the corresponding edge node to the registry; After receiving the upload request from the client, the central server retrieves the registry; Match edge nodes based on the current network region and target archive storage type requested by the client; Then the edge node is scored based on its CPU load, memory usage, and disk IO status. The edge node score = CPU load * 1 + memory usage * 1.5 + disk IO status * 2. Select the edge node with the lowest score as the optimal edge node; The central server selects the optimal edge node based on the client's download request information and the heartbeat content continuously reported by the edge node, including the following steps: The edge node continuously initiates heartbeats to the central server, and the central server registers the heartbeat content of the corresponding edge node to the registry; After receiving the download request from the client, the central server retrieves the registry; Match edge nodes based on the current network region requested by the client and the storage type of the downloaded file ID; Then the edge node is scored based on its CPU load, memory usage, and disk IO status. The edge node score = CPU load * 1 + memory usage * 1.5 + disk IO status * 2. The edge node with the lowest score is selected as the optimal edge node.

6. The zero-trust-based remote multi-network storage scheduling method according to claim 1, characterized in that: The upload link TOKEN contains the MD5 information of the upload request, the file name, and the file metadata; the download link TOKEN contains the file ID of the download request and the download link expiration time; After the edge node receives the file, it verifies the file MD5 based on the MD5 information in the TOKEN of the upload link to confirm that the file is complete. If the file is incomplete, an error is reported.

7. The zero-trust-based remote multi-network storage scheduling method according to claim 1, characterized in that: The edge node uploads the file to the OSS source station, including the following steps: The edge node creates an upload task; Serialize the file information, where the file information includes MD5 information, file ID, file name, and remote storage information; Store files into a disk queue; Take out the upload task and upload the file to the remote OSS source station according to the remote storage information. If the upload fails, put the file back into the disk queue.

8. The zero-trust-based remote multi-network storage scheduling method according to claim 1, characterized in that: When the edge node fails to find the corresponding file ID, it queries whether there is already a local request to download from the origin server. If so, wait; If not, create a download task and request the source site to download; The remote OSS source station verifies the file MD5 information; If the verification is successful, the remote OSS source station returns the file content to the edge node, and then the edge node returns the file content to the client; If the verification fails, an alarm is sent to the edge node, which then sends an alarm to the client.

9. The zero-trust-based remote multi-network storage scheduling method according to claim 8, characterized in that: When multiple clients send file download requests to the central server and the edge node has no local cache, multiple edge nodes request the source site to download: when the front request comes, the source site is requested and marked as being requested. If the subsequent request is the same as the current request, the download traffic of the source site is copied to the subsequent request.

10. A zero-trust-based remote multi-network storage scheduling system, used to implement the zero-trust-based remote multi-network storage scheduling method according to any one of claims 1 to 9, characterized in that: Including central servers, edge nodes, and OSS source stations; Central server for overall scheduling and control; Edge nodes are deployed in the intranet computer room and serve as a transit station for data transmission; OSS source station, as the final data storage location; Set up a zero-trust security channel between the central server and the edge node to ensure communication security.

Images