Cloud-edge collaborative distributed storage method and device based on double-layer Raft consensus protocol

Abstract

The embodiments of the present disclosure provide a cloud-edge collaborative distributed storage method and device based on a two-layer Raft consensus protocol, which is applied to the field of big data processing technology. The method includes receiving a data query request sent by a user terminal; the data query request includes target information; determining the target edge cluster information from a preset hash ring according to the target information; and returning the target edge cluster information to the user terminal, so that the user terminal obtains the data to be queried from the corresponding edge cluster. In this way, the data retrieval efficiency can be further improved while ensuring that the network nodes can be flexibly expanded.

Description

Technical Field

[0001] The present disclosure relates to the field of big data processing technology, and in particular to a cloud-edge collaborative distributed storage method and device based on a double-layer Raft consensus protocol. Background Art

[0002] In a big data environment, the volume of metadata is also very large, and the access performance of metadata is the key to the performance of the entire distributed file system. Common metadata management can be divided into centralized and distributed metadata management architectures. The centralized metadata management architecture uses a single metadata server, which is simple to implement. However, there are problems such as single point failure. The distributed metadata management architecture distributes metadata on multiple nodes, thereby solving problems such as the performance bottleneck of the metadata server. It also improves the scalability of the metadata management architecture, but the implementation is more complicated and introduces the problem of metadata consistency. In addition, there is a distributed architecture without a metadata server, which organizes data through online algorithms and does not require a dedicated metadata server. However, this architecture is difficult to ensure data consistency and is more complicated to implement. The file directory traversal operation is inefficient and lacks the global monitoring and management function of the file system.

[0003] Distributed storage system is to store data in multiple independent devices. Traditional network storage system uses centralized storage server to store all data. Storage server becomes the bottleneck of system performance and the focus of reliability and security, which cannot meet the needs of large-scale storage applications. Distributed network storage system adopts scalable system structure, uses multiple storage servers to share storage load, and uses location server to locate storage information. It not only improves the reliability, availability and access efficiency of the system, but also makes it easy to expand.

[0004] Therefore, how to further improve retrieval efficiency while ensuring that network nodes can be flexibly expanded is an issue that needs to be addressed urgently. Summary of the invention

[0005] The present disclosure provides a cloud-edge collaborative distributed storage method and device based on a double-layer Raft consensus protocol.

[0006] According to the first aspect of the present disclosure, a cloud-edge collaborative distributed storage method based on a double-layer Raft consensus protocol is provided, which is applied to a cloud server. The method comprises: receiving a data query request sent by a user end; the data query request comprises target information;

[0007] Determine the target edge cluster information from the preset hash ring according to the target information; wherein the preset hash ring is generated by the following steps: perform hash calculation on the network address of the edge server to obtain a corresponding hash value, map and store the hash value in a clockwise direction to form an initial hash ring, segment the initial hash ring according to a preset step size, divide the initial hash ring into multiple ring segments, detect the density of the edge cluster information in each ring segment, and in response to the density of the edge cluster information in the ring segment being greater than a preset threshold, store a preset number of edge cluster information in the corresponding ring segment at a corresponding position in a subsequent ring segment, and generate a corresponding mapping relationship;

[0008] The target edge cluster information is returned to the user terminal, so that the user terminal obtains the data to be queried from the corresponding edge cluster.

[0009] According to the above aspects and any possible implementation, there is further provided an implementation, wherein the cloud server is deployed with a meta information storage cluster based on the Raft consensus protocol;

[0010] The meta information storage cluster stores information of multiple edge clusters.

[0011] According to the above aspects and any possible implementation, there is further provided an implementation, wherein the cloud server corresponds to an edge server;

[0012] The edge server is deployed with a data storage cluster based on the Raft consistency protocol;

[0013] The data storage cluster stores a plurality of data.

[0014] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the density of edge cluster information in each ring segment is detected, and in response to the density of edge cluster information in the ring segment being greater than a preset threshold, a preset number of edge cluster information in the corresponding ring segment is stored at a corresponding position in a subsequent ring segment, including:

[0015] The density of edge cluster information in each ring segment is detected to determine whether there is a node in the current ring segment whose amount of edge cluster information is greater than a preset threshold. In response to the presence of a node in the current ring segment whose amount of edge cluster information is greater than the preset threshold, half of the edge cluster information in the node is stored in a node corresponding to the current node position in a subsequent ring segment.

[0016] According to the above aspects and any possible implementation manner, there is further provided an implementation manner, wherein the target edge cluster information includes a network address of a target edge server;

[0017] The determining target edge cluster information from a preset hash ring according to the target information includes:

[0018] Performing hash calculation on the target information to obtain a target value;

[0019] Searching for a consistent hash value in the hash ring according to the target value;

[0020] According to the hash value found, determine the network address of the corresponding edge server;

[0021] The network address of the edge server is used as the target edge cluster information.

[0022] According to the above aspects and any possible implementation manner, an implementation manner is further provided, wherein the method further includes:

[0023] The metadata storage cluster is updated according to the status of the edge cluster.

[0024] According to the above aspects and any possible implementation, an implementation is further provided, wherein the state of the edge cluster includes: registered or online, and offline;

[0025] The updating of the meta information storage cluster according to the state of the edge cluster includes:

[0026] When a new edge cluster is registered or launched,

[0027] According to the network address of the edge server corresponding to the edge cluster uploaded when registering or going online, the new edge cluster is added to the hash ring;

[0028] When an edge cluster goes offline,

[0029] According to the network address of the edge server corresponding to the offline edge cluster, the offline edge cluster is deleted from the hash ring.

[0030] According to a second aspect of the present disclosure, a cloud-edge collaborative distributed storage device based on a double-layer Raft consensus protocol is provided, which is applied to a cloud server. The device includes:

[0031] A request receiving module, used for receiving a data query request sent by a user terminal; the data query request includes target information;

[0032] An information search module, used to determine the target edge cluster information from a preset hash ring according to the target information; wherein the preset hash ring is generated by the following steps: performing hash calculation on the network address of the edge server to obtain a corresponding hash value, mapping and storing the hash value in a clockwise direction to form an initial hash ring, segmenting the initial hash ring according to a preset step size, dividing the initial hash ring into a plurality of ring segments, detecting the density of the edge cluster information in each ring segment, and in response to the density of the edge cluster information in the ring segment being greater than a preset threshold, storing a preset number of edge cluster information in the corresponding ring segment at a corresponding position in a subsequent ring segment, and generating a corresponding mapping relationship;

[0033] The information returning module is used to return the target edge cluster information to the user end, so that the user end obtains the data to be queried from the corresponding edge cluster.

[0034] According to a third aspect of the present disclosure, an electronic device is provided, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method described above is implemented.

[0035] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0036] The embodiments of the present disclosure provide a cloud-edge collaborative distributed storage method and device based on a two-layer Raft consistency protocol. By deploying a metadata storage cluster based on the Raft consistency protocol on the cloud, a hash ring is used to store the information of each edge storage cluster, and a data storage cluster based on the Raft consistency protocol is deployed on the edge, which is specifically responsible for the storage of actual data. In this way, the received customer query data requests can be quickly retrieved and customer needs can be responded to in a short time, thereby improving the user experience. In addition, since both the cloud and the edge are deployed based on the Raft consistency protocol, the flexibility of network nodes in expanding the entire storage system can be improved.

[0037] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0039] Figure 1 An interactive schematic diagram of a cloud-edge collaborative distributed storage method based on a double-layer Raft consensus protocol according to an embodiment of the present disclosure is shown;

[0040] Figure 2 A flow chart of a cloud-edge collaborative distributed storage method based on a double-layer Raft consensus protocol according to an embodiment of the present disclosure is shown;

[0041] Figure 3 A block diagram of a cloud-edge collaborative distributed storage device based on a double-layer Raft consensus protocol according to an embodiment of the present disclosure is shown;

[0042] Figure 4 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0044] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0045] Figure 1 An interactive schematic diagram of a cloud-edge collaborative distributed storage method based on a two-layer Raft consistency protocol according to an embodiment of the present disclosure is shown. A metadata storage cluster based on the Raft consistency protocol is deployed on the cloud, and the cluster stores relevant information of each edge storage cluster through a consistent hash ring with virtual nodes. A data storage cluster based on the Raft consistency protocol is deployed on the edge, and the cluster is responsible for storing actual data. Cross-LAN communication is achieved between edge terminals through cloud relay forwarding and UDP hole punching. When a client sends a request, it first accesses the metadata cluster on the cloud, calculates the position of the requested data on the consistent hash ring, finds the storage cluster that serves the data through the position, and returns the information to the client. Finally, the client sends a request to the target storage cluster to obtain the required data.

[0046] Figure 2A flow chart of a cloud-edge collaborative distributed storage method 200 based on a double-layer Raft consensus protocol according to an embodiment of the present disclosure is shown. Applied to a cloud server, the method 200 includes:

[0047] Step 210: receiving a data query request sent by a user terminal; the data query request includes target information.

[0048] In some embodiments, when a user has a data query request, the user needs to query which target, such as edge server A, for the required data. At this time, when the user initiates a data query request to the cloud server, it is necessary to tell the cloud server the network address of edge server A, that is, the target information.

[0049] In some embodiments, the target information may also be a unique code negotiated in advance between the user end and the cloud end for distinguishing each edge server.

[0050] In some embodiments, the cloud server is deployed with a meta information storage cluster based on the Raft consensus protocol; the meta information storage cluster stores multiple edge cluster information. The cloud server corresponds to an edge server; the edge server is deployed with a data storage cluster based on the Raft consensus protocol; the data storage cluster stores multiple data.

[0051] Among them, the Raft consensus protocol is a distributed protocol designed to solve the consistency problem in distributed systems, and it is a strong consistency. Its goal is to ensure that each node in the distributed system can reach a consistent state and maintain consistency in the event of node failure or network partition. Each node in the Raft protocol can be in three states: Leader, Follower, and Candidate. In the initial state, all nodes are Followers. At any time, a node can be a Follower or a Candidate, but there can only be one Leader, which avoids the occurrence of multiple Leaders. The process of Leader election is as follows: When a Follower does not receive a message from the Leader within a random time interval, it becomes a Candidate and starts a new round of elections. The Candidate sends a RequestVote request to other nodes, requesting other nodes to vote for itself. After receiving the request, if other nodes have not voted for other Candidates, they will vote for the Candidate and reset the election timeout. If a Candidate receives votes from most nodes within the election timeout, it will become the new Leader. If a candidate does not receive votes from the majority of nodes within the election timeout, its election fails and a new round of elections is restarted to avoid the system from not working properly after the leader fails. If a node receives legitimate log requests from the leader, it must execute these requests.

[0052] Since both the cloud and the edge are deployed based on the Raft consensus protocol, the flexibility of network nodes in expanding the entire storage system can be improved.

[0053] Step 220, determining the target edge cluster information from the preset hash ring according to the target information. The preset hash ring is generated by the following steps: performing hash calculation on the network address of the edge server to obtain a corresponding hash value, mapping and storing the hash value in a clockwise direction to form an initial hash ring, segmenting the initial hash ring according to a preset step length, dividing the initial hash ring into multiple ring segments, detecting the density of the edge cluster information in each ring segment, and in response to the density of the edge cluster information in the ring segment being greater than a preset threshold, storing a preset number of edge cluster information in the corresponding ring segment at a corresponding position in the subsequent ring segment, and generating a corresponding mapping relationship.

[0054] In some embodiments, the hash ring is pre-generated by the following steps: performing hash calculation on the network address of the edge server to obtain a corresponding hash value; mapping and storing the hash value in a clockwise direction to form an initial hash ring; segmenting the initial hash ring according to a preset step length, dividing the initial hash ring into multiple ring segments, detecting the density of edge cluster information in each ring segment, and in response to the density of edge cluster information in the ring segment being greater than a preset threshold, storing a preset number of edge cluster information in the corresponding ring segment at a corresponding position in a subsequent ring segment, and generating a corresponding mapping relationship. Wherein, detecting the density of edge cluster information in each ring segment, and in response to the density of edge cluster information in the ring segment being greater than a preset threshold, storing a preset number of edge cluster information in the corresponding ring segment at a corresponding position in a subsequent ring segment, includes: detecting the density of edge cluster information in each ring segment, determining whether there is a node in the current ring segment whose number of edge cluster information is greater than a preset threshold, and in response to the presence of a node in the current ring segment whose number of edge cluster information is greater than the preset threshold, storing half of the edge cluster information in the node in the node corresponding to the current node position in the subsequent ring segment. In this way, nodes with too dense storage information can be balanced to avoid too concentrated storage of edge cluster information on the hash ring. That is, the meta-information storage cluster stores the relevant information of each edge storage cluster through a consistent hash ring with virtual nodes.

[0055] In some embodiments, the target edge cluster information includes a network address of a target edge server; determining the target edge cluster information from a preset hash ring based on the target information includes: performing a hash calculation on the target information to obtain a target value; searching for a consistent hash value in the hash ring based on the target value; determining the network address of the corresponding edge server based on the found hash value; and using the network address of the edge server as the target edge cluster information.

[0056] In some embodiments, if the target information is a unique code that is negotiated in advance between the user end and the cloud to distinguish each edge server, then the unique code is hashed to form a hash ring, and then the location of the corresponding edge server is determined by the hash value, and the network address of the corresponding edge server is obtained according to the determined location.

[0057] Among them, the hash ring uses the consistent hashing algorithm to map the entire hash value space into a virtual ring, and the entire space is organized in a clockwise direction. Then the service request (such as the network address of the edge server mentioned above) is mapped, and the corresponding hash value of the service request is calculated using the hash algorithm. Then, according to the position of the hash value, it is searched clockwise along the ring. The first server encountered is the corresponding server that processes the request. When a new server is added, the only data affected is the data between the newly added server and the previous server in its ring space (that is, the first server encountered in the counterclockwise direction), and the others will not be affected. In summary, the consistent hashing algorithm only needs to relocate a small part of the data in the ring space for the addition and subtraction of nodes, and has good fault tolerance and scalability.

[0058] Step 230: Return the target edge cluster information to the user terminal, so that the user terminal obtains the data to be queried from the corresponding edge cluster.

[0059] In some embodiments, the target edge cluster information may be the network address of edge server A, or the network address of the edge cluster where edge server A is located, that is, the edge cluster is a collection of multiple edge servers. Then, this information is returned to the user end, and the user end sends a data query request to the target edge cluster based on this information to obtain the required data. Of course, since the cluster where edge server A is located may be offline or invaded, after data migration before going offline, the target edge cluster information determined based on the position determined by the hash ring may also be the network address of edge server B.

[0060] In this way, we can realize the received customer query data requests, achieve rapid retrieval and respond to customer needs in a short time, and improve the user experience.

[0061] Based on the above implementation, another implementation provided in the present disclosure includes: updating the meta information storage cluster according to the state of the edge cluster.

[0062] In some embodiments, the status of the edge cluster includes: registration or online, and offline; the updating of the meta-information storage cluster according to the status of the edge cluster includes: when a new edge cluster is registered or online, the new edge cluster is added to the hash ring according to the network address of the edge server corresponding to the edge cluster uploaded when registered or online; when an edge cluster is offline, the offline edge cluster is deleted from the hash ring according to the network address of the edge server corresponding to the offline edge cluster.

[0063] That is to say, only when the edge cluster is online will it be added to the hash ring for processing to improve retrieval efficiency.

[0064] Based on the above implementation, another implementation provided in the present disclosure includes: data migration of edge clusters: achieved through cloud relay forwarding and / or UDP hole punching.

[0065] In some embodiments, data migration when the edge cluster is offline or invaded is achieved through cloud relay forwarding and / or UDP hole punching. Cloud relay forwarding means that the data is forwarded by the cloud after being sent to the cloud. UDP hole punching is a technology that allows two devices behind NAT (network address translation) or firewall to establish a direct UDP connection. The basic idea is to use a public server to help the two devices negotiate their network address information, thereby opening a "hole" for direct communication. The following are the basic steps of UDP hole punching: Both machines (A and B) establish a UDP connection with the relay server (R). A sends a message containing its private address and port to the server. After receiving A's message, B sends a message containing its private address and port, as well as A's public address and port to the server. The server forwards B's message to A. A and B try to establish a connection with each other's public address and port respectively to achieve direct data migration. Timely data backup is achieved to avoid data loss. At the same time, it also enables users to respond quickly when querying data, avoiding the edge cluster being offline and causing user needs to be unfulfilled.

[0066] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0067] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.

[0068] Figure 3 FIG. 3 is a block diagram of a cloud-edge collaborative distributed storage device 300 based on a double-layer Raft consensus protocol according to an embodiment of the present disclosure, which is applied to a cloud server. Figure 3 As shown, the device 300 includes:

[0069] The request receiving module 310 is used to receive a data query request sent by a user terminal; the data query request includes target information.

[0070] The information search module 320 is used to determine the target edge cluster information from a preset hash ring according to the target information; wherein the preset hash ring is generated by the following steps: performing hash calculation on the network address of the edge server to obtain a corresponding hash value, mapping and storing the hash value in a clockwise direction to form an initial hash ring, segmenting the initial hash ring according to a preset step size, dividing the initial hash ring into multiple ring segments, detecting the density of the edge cluster information in each ring segment, and in response to the density of the edge cluster information in the ring segment being greater than a preset threshold, storing a preset number of edge cluster information in the corresponding ring segment at corresponding positions in subsequent ring segments, and generating a corresponding mapping relationship.

[0071] The information returning module 330 is used to return the target edge cluster information to the user terminal, so that the user terminal obtains the data to be queried from the corresponding edge cluster.

[0072] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0073] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0074] Figure 4 A schematic block diagram of an electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0075] The electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in the ROM 402 or a computer program loaded from the storage unit 408 into the RAM 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O interface 405 is also connected to the bus 404.

[0076] Multiple components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0077] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as a cloud-edge collaborative distributed storage method based on a double-layer Raft consensus protocol. For example, in some embodiments, the cloud-edge collaborative distributed storage method based on a double-layer Raft consensus protocol may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or the communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of the cloud-edge collaborative distributed storage method based on the double-layer Raft consensus protocol described above may be executed. Alternatively, in other embodiments, the computing unit 401 may be configured in any other appropriate manner (eg, by means of firmware) to execute a cloud-edge collaborative distributed storage method based on a two-layer Raft consensus protocol.

[0078] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0079] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0080] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0081] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0082] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0083] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0084] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0085] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A cloud-edge collaborative distributed storage method based on a two-layer Raft consensus protocol, applied to a cloud server, characterized in that: include: Receive a data query request sent by a user; The data query request includes target information; Determine target edge cluster information from a preset hash ring according to the target information; The preset hash ring is generated by the following steps: Hash the network address of the edge server to get the corresponding hash value. The hash values ​​are mapped and stored in a clockwise direction to form an initial hash ring. Segmenting the initial hash ring according to a preset step size, dividing the initial hash ring into multiple ring segments, Detect the density of edge cluster information in each ring segment. In response to the density of the edge cluster information in the ring segment being greater than a preset threshold, storing a preset amount of edge cluster information in the corresponding ring segment at a corresponding position in a subsequent ring segment, And generate the corresponding mapping relationship; Returning the target edge cluster information to the user terminal, so that the user terminal obtains the data to be queried from the corresponding edge cluster; The cloud server is deployed with a meta information storage cluster based on the Raft consensus protocol; The meta information storage cluster stores information of multiple edge clusters; The target edge cluster information includes the network address of the target edge server; The determining target edge cluster information from a preset hash ring according to the target information includes: Performing hash calculation on the target information to obtain a target value; Searching for a consistent hash value in the hash ring according to the target value; According to the hash value found, determine the network address of the corresponding edge server; Using the network address of the edge server as target edge cluster information; The detecting the density of the edge cluster information in each ring segment, and in response to the density of the edge cluster information in the ring segment being greater than a preset threshold, storing a preset amount of edge cluster information in the corresponding ring segment at a corresponding position in a subsequent ring segment, comprises: The density of edge cluster information in each ring segment is detected to determine whether there is a node in the current ring segment whose amount of edge cluster information is greater than a preset threshold. In response to the presence of a node in the current ring segment whose amount of edge cluster information is greater than the preset threshold, half of the edge cluster information in the node is stored in a node corresponding to the current node position in a subsequent ring segment.

2. The method according to claim 1, characterized in that The cloud server corresponds to an edge server; The edge server is deployed with a data storage cluster based on the Raft consistency protocol; The data storage cluster stores a plurality of data.

3. The method according to claim 1, characterized in that The method further comprises: The metadata storage cluster is updated according to the status of the edge cluster.

4. The method according to claim 3, characterized in that The status of the edge cluster includes: registered or online, and offline; The updating of the meta information storage cluster according to the state of the edge cluster includes: When a new edge cluster is registered or launched, According to the network address of the edge server corresponding to the edge cluster uploaded when registering or going online, the new edge cluster is added to the hash ring; When an edge cluster goes offline, According to the network address of the edge server corresponding to the offline edge cluster, the offline edge cluster is deleted from the hash ring.

5. The device used in the cloud-edge collaborative distributed storage method based on the double-layer Raft consensus protocol according to any one of claims 1 to 4 is applied to a cloud server, characterized in that: include: A request receiving module, used for receiving a data query request sent by a user terminal; The data query request includes target information; An information search module, used to determine the target edge cluster information from a preset hash ring according to the target information; wherein the preset hash ring is generated by the following steps: performing hash calculation on the network address of the edge server to obtain a corresponding hash value, mapping and storing the hash value in a clockwise direction to form an initial hash ring, segmenting the initial hash ring according to a preset step size, dividing the initial hash ring into a plurality of ring segments, detecting the density of the edge cluster information in each ring segment, and in response to the density of the edge cluster information in the ring segment being greater than a preset threshold, storing a preset number of edge cluster information in the corresponding ring segment at a corresponding position in a subsequent ring segment, and generating a corresponding mapping relationship; The information returning module is used to return the target edge cluster information to the user end, so that the user end obtains the data to be queried from the corresponding edge cluster.

6. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

Images