Distributed storage system, view management method, and computing device cluster

Abstract

The present invention provides a distributed storage system, a view management method, and a computing device cluster. The distributed storage system comprises N storage nodes and computing nodes, N being a positive integer, and the N storage nodes comprising first view generation nodes, wherein the first view generation nodes are used for determining M view push nodes from among the N storage nodes, M being a positive integer less than or equal to N; the first view generation nodes are used for sending view information to the M view push nodes, the view information being used for indicating a rule for distributing data to the N storage nodes; and the computing nodes are used for determining a first view push node from among the M view push nodes, requesting the view information from the first view push node, and performing data writing or reading on the N storage nodes on the basis of the requested view information. View generation nodes are used for view generation and distribution, and computing nodes request views from view push nodes, thereby reducing the load of the view generation nodes and minimizing the risk of becoming a performance bottleneck.

Description

A distributed storage system, view management method and computing device cluster

[0001] The present application claims priority to Chinese Patent Application No. 202410725311.8, filed on June 5, 2024, entitled "A View Management Method and Distributed Storage Cluster", and Chinese Patent Application No. 202410938986.0, filed on July 12, 2024, entitled "A Distributed Storage System, View Management Method and Computing Device Cluster", the contents of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of storage, and in particular to a distributed storage system, view management method and computing device cluster. BACKGROUND

[0003] In a distributed storage system, the system maintains a globally consistent view information, which records the rules of distributing data to the storage nodes in the system. The storage nodes and computing nodes in the distributed storage system determine the data storage and reading location by querying the view. Among them, the distributed storage system will perform view management: when a storage node joins or exits, or a storage node temporarily fails, the view information is automatically updated, and the accuracy and consistency of the view in the distributed storage system are ensured through passive query or active update. View management mainly includes centralized management and decentralized management. Centralized management refers to a process of a management role independently deployed to specifically handle view updates and distribution. The decentralized management way is that each storage node is responsible for managing part of the view, and synchronizes the view information on the node to other storage nodes through some propagation protocol.

[0004] For the centralized method, as the number of storage nodes in the distributed storage system increases, the load of the management node increases, which may make the view management a performance bottleneck of the distributed storage system. SUMMARY

[0005] The application provides a distributed storage system, a view management method and a computing device cluster, by dividing storage nodes into view production nodes and view push nodes, the view production nodes determine the view push nodes and push views of the distributed storage system to the view push nodes, and when a computing node needs a view, the computing node requests view information from any view push node; on the one hand, the view production nodes are used for view generation and view delivery, the computing node requests a view from the view push node, the load of the view production node in view management is reduced, the storage node has strong expansibility, the scale of the storage node can be flexibly expanded, and the possibility that view management becomes a performance bottleneck is reduced; on the other hand, view synchronization between multiple storage nodes is not needed, and the view synchronization cost is reduced; on the other hand, in the scenario that the storage nodes in the distributed storage system frequently change, the view production nodes push views to the view push nodes, the view push path and time are predictable, and therefore the view distribution efficiency is improved.

[0006] In a first aspect, the application provides a distributed storage system, the distributed storage system comprising: N storage nodes and a computing node, N being a positive integer, the N storage nodes comprising a first view generation node;

[0007] The first view generation node is configured to determine M view push nodes from the N storage nodes, M being a positive integer less than or equal to N;

[0008] The first view generation node is configured to send view information to the M view push nodes, the view information being used to indicate a rule of distributing data to the N storage nodes;

[0009] The computing node is configured to determine a first view push node from the M view push nodes, request view information from the first view push node, and perform data writing or data reading on the N storage nodes based on the requested view information.

[0010] In the scheme provided by the application, on the one hand, the view production nodes are used for view generation and view delivery, the computing node requests a view from the view push node, the load of the view production node in view management is reduced, the storage node has strong expansibility, the scale of the storage node can be flexibly expanded, and the possibility that view management becomes a performance bottleneck is reduced; on the other hand, view synchronization between multiple storage nodes is not needed, and the view synchronization cost is reduced; on the other hand, in the scenario that the storage nodes in the distributed storage system frequently change, the view production nodes push views to the view push nodes, the view push path and time are predictable, and therefore the view distribution efficiency is improved.

[0011] In some possible implementation manners, the first view generation node is configured to determine an association relationship between the second view push node and H view consumption nodes, the H view consumption nodes being storage nodes other than the first view generation node and the M view push nodes in the N storage nodes, H being a positive integer less than N; and the second view push node being any one of the M view push nodes.

[0012] The first view generation node is configured to send the association relationship to the second view push node.

[0013] The second view push node is configured to determine, based on the association relationship, a view consumption node associated with the second view push node, and send view information to the associated view consumption node.

[0014] In the scheme provided in the application, the view push node is associated with the view consumption node, the view push node needs to synchronize the view to the associated view consumption node, the view producer does not need to synchronize the view to the view consumer, the load of the view production node in the view management is reduced, the storage node has strong expansibility, and the size of the storage node can be flexibly expanded without affecting the view management.

[0015] In some possible implementation manners, the distributed storage system further includes a service node, the service node being in communication with the N storage nodes.

[0016] The first view generation node is configured to determine view push node information, the view push node information being used to indicate the M view push nodes.

[0017] The first view generation node is configured to send the view push node information to the service node.

[0018] The computing node is configured to request the view push node information from the service node.

[0019] The computing node is configured to determine the first view push node from the M view push nodes based on the view push node information.

[0020] In the scheme provided in the application, the computing node can select the view push node from a large number of view push nodes to request the view, without requesting the view generation node, the load of the view production node in the view management is reduced, the storage node has strong expansibility, the size of the storage node can be flexibly expanded, and the possibility that the view management becomes a performance bottleneck is reduced.

[0021] In one possible example of the implementation manner, the service node is configured to send node states of the N storage nodes to the first view generation node.

[0022] The first view generation node is configured to determine the view information based on the node states of the N storage nodes and a data distribution rule.

[0023] In the scheme provided by the application, the node state monitoring of the storage node is undertaken by the service node, without the view generation node monitoring the state of the storage node, thereby reducing the load of the view production node in view management, making the storage node have strong scalability, and enabling the scale of the storage node to be flexibly expanded, and reducing the possibility of view management becoming a performance bottleneck. In addition, in the scenario of frequent changes of the storage node in the distributed storage system, since the view production node realizes view updating by monitoring the node state through the service node, the speed of view updating can be improved.

[0024] In another possible example of the implementation mode, the service node is configured to send the updated node state of the target storage node to the first view generation node after determining the node state update of the target storage node in the N storage nodes.

[0025] In the scheme provided by the application, the service node monitors the node state of the storage node, and sends the updated node state to the view production node after the state is updated, and the view production node updates the view.

[0026] In another possible example of the implementation mode, the first view generation node is configured to send view information to the service node, and the service node is configured to store the view information.

[0027] In the scheme provided by the application, the view production node sends the view information to the service node for persistence, and in the case of failure of any management node in the distributed storage system, the influence on view management is reduced.

[0028] In some possible implementation modes, the first view generation node is configured to determine that a third view push node in M view push nodes fails, select a fourth view push node from storage nodes other than the first view generation node and the M view push nodes in N storage nodes to replace the third view push node, and obtain updated view push node information.

[0029] The first view generation node is configured to send the updated view push node information to the service node.

[0030] The first view generation node is configured to send the view information to the fourth view push node.

[0031] In the scheme provided by the application, the view production node replaces the failed view push node after the view push node fails, and realizes fault handling.

[0032] In some possible implementation modes, the distributed storage system further includes a second view production node.

[0033] In a case where the first view generation node fails, a second view generation node is configured to read view information and view push node information from the service node.

[0034] In the scheme, after the view generation node fails, other view generation nodes take over view management, read view information and view push node information from the service node, and improve the efficiency of view generation node failure processing.

[0035] In some possible implementation manners, the computing node is configured to determine that the request for the view information fails, determine a fifth view push node based on the view push node information, and request the view information from the fifth view push node.

[0036] In an example of the implementation manner, the computing node is configured to determine that the request for the view information fails, request the latest view push node information from the service node, determine a sixth view push node based on the latest view push node information, and request the latest view information from the sixth view push node.

[0037] In the scheme, when the computing node determines that the view information cannot be requested, the computing node requests the latest view push node information from the service node, and re-determines the view push node to request the view, so that the view is updated quickly in a case where the view expires.

[0038] For example, the computing node is configured to determine that the request for the view information fails in a case where a number of times of the request for the view information is greater than or equal to a preset number of times, or in a case where the view information is not requested within a preset time length.

[0039] In some possible implementation manners, the first view generation node is a management node in the N storage nodes.

[0040] In some possible implementation manners, the M view push nodes are storage nodes other than the management node in the N storage nodes.

[0041] In a second aspect, the present application provides a view management method, which is implemented by a distributed storage system. The steps performed by the method and the beneficial effects achieved by the method are described with reference to the functions of the distributed storage system provided in the first aspect of the present application, and will not be described here.

[0042] Specifically, the method is applied to a distributed storage system, and the distributed storage system includes N storage nodes and a computing node, N is a positive integer, the N storage nodes include a first view generation node, and the method includes the following steps.

[0043] The first view generation node determines M view push nodes from the N storage nodes, and M is a positive integer less than or equal to N.

[0044] The first view generation node sends view information to the M view push nodes, the view information being used to indicate a rule of distributing data to the N storage nodes;

[0045] The computing node determines the first view push node from the M view push nodes, requests the view information from the first view push node, and performs data writing or data reading on the N storage nodes based on the requested view information.

[0046] In some possible implementation manners, the method further includes:

[0047] The first view generation node determines an association relationship between the second view push node and the H view consumption nodes, the H view consumption nodes being storage nodes in the N storage nodes other than the first view generation node and the M view push nodes, H being a positive integer less than N; and the second view push node being any one of the M view push nodes;

[0048] The first view generation node sends the association relationship to the second view push node;

[0049] The second view push node determines, based on the association relationship, a view consumption node associated with the second view push node, and sends the view information to the associated view consumption node.

[0050] In some possible implementation manners, the distributed storage system further includes a service node, the service node being in communication with the N storage nodes;

[0051] The first view generation node determines view push node information, the view push node information being used to indicate the M view push nodes;

[0052] The first view generation node sends the view push node information to the service node;

[0053] The computing node determines the first view push node from the M view push nodes, including:

[0054] The computing node requests the view push node information from the service node;

[0055] The computing node determines the first view push node from the M view push nodes based on the view push node information.

[0056] In some possible implementation manners, the method further includes:

[0057] The service node sends, after determining a node state update of a target storage node in the N storage nodes, the updated node state of the target storage node to the first view generation node.

[0058] In some possible implementation manners, the method further includes:

[0059] The first view generation node sends the view information to the service node;

[0060] The service node is configured to store the view information.

[0061] In some possible implementation manners, the method further includes:

[0062] The first view generation node determines that a third view push node in the M view push nodes is faulty, selects a fourth view push node from the N storage nodes other than the first view generation node and the M view push nodes to replace the third view push node, and obtains updated view push node information;

[0063] The first view generation node sends the updated view push node information to the service node.

[0064] The first view generation node sends the view information to the fourth view push node.

[0065] In some possible implementation manners, the distributed storage system further includes a second view generation node, and the method further includes:

[0066] In a case where the first view generation node is faulty, the second view generation node reads the view information and the view push node information from the service node.

[0067] In some possible implementation manners, the method further includes:

[0068] The computing node determines that the request for the view information fails, determines a fifth view push node based on the view push node information, and requests the view information from the fifth view push node.

[0069] In some possible implementation manners, the method further includes:

[0070] The computing node determines that the request for the view information fails, requests the latest view push node information from the service node, determines a sixth view push node based on the latest view push node information, and requests the latest view information from the sixth view push node.

[0071] In some possible implementation manners, the method further includes:

[0072] The computing node determines that the request for the view information fails in a case where a number of times of failure in the request for the view information is greater than or equal to a preset number of times, or in a case where the view information is not requested within a preset time length.

[0073] In some possible implementation manners, the first view generation node is a management node in the N storage nodes.

[0074] In some possible implementation manners, the M view pushing nodes are for the storage nodes other than the management node in the N storage nodes.

[0075] In a third aspect, the present application provides a computing device, comprising a memory and a processor, the memory storing program instructions, and the processor executing the program instructions to perform the method provided in the second aspect and any possible implementation manner thereof.

[0076] In a fourth aspect, the present application provides a computing device cluster, comprising a plurality of computing devices, the plurality of computing devices comprising a plurality of processors and a plurality of memories, the plurality of memories storing program instructions, and the plurality of processors executing the program instructions so that the computing device cluster performs the method provided in the second aspect and any possible implementation manner thereof.

[0077] In a fifth aspect, an embodiment of the present application provides a computer storage medium, the computer storage medium storing instructions, when the instructions are executed on a computer, causing the computer to perform the method provided in the second aspect and any possible implementation manner thereof.

[0078] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, when the instructions are executed on a computer, causing the computer to perform the method provided in the second aspect and any possible implementation manner thereof. BRIEF DESCRIPTION OF DRAWINGS

[0079] FIG. 1a is a schematic diagram of an architecture of a distributed storage system according to an embodiment of the present application;

[0080] FIG. 1b is a schematic diagram of an architecture of a distributed storage system according to an embodiment of the present application;

[0081] FIG. 2a is a schematic diagram of a structure of a distributed storage system according to an embodiment of the present application;

[0082] FIG. 2b is a schematic diagram of a structure of a distributed storage system according to an embodiment of the present application;

[0083] FIG. 3 is a schematic diagram of a view management scheme of a distributed storage system according to an embodiment of the present application;

[0084] FIG. 4 is a schematic diagram of a view management method according to an embodiment of the present application;

[0085] FIG. 5 is a schematic diagram of a view management method according to an embodiment of the present application;

[0086] FIG. 6 is a schematic diagram of a view management scheme of a distributed storage system according to an embodiment of the present application;

[0087] FIG. 7 is a schematic diagram of a view management method according to an embodiment of the present application;

[0088] Fig. 8 is a schematic diagram of a view management method according to an embodiment of the present application;

[0089] Fig. 9 is a schematic diagram of a view management method according to an embodiment of the present application;

[0090] Fig. 10 is a schematic diagram of a view management method according to an embodiment of the present application;

[0091] Fig. 11 is a schematic diagram of a view management method according to an embodiment of the present application;

[0092] Fig. 12 is a schematic diagram of a view management method according to an embodiment of the present application;

[0093] Fig. 13 is a schematic diagram of a view management method according to an embodiment of the present application;

[0094] Fig. 14 is a schematic diagram of a view management scenario according to an embodiment of the present application;

[0095] Fig. 15 is a schematic diagram of a view distribution scenario of the view management according to Fig. 14;

[0096] Fig. 16 is a schematic diagram of a failure handling scenario of the view management according to Fig. 14;

[0097] Fig. 17 is a schematic diagram of a structure of a distributed storage system according to an embodiment of the present application;

[0098] Fig. 18 is a schematic diagram of a structure of a computing device according to an embodiment of the present application;

[0099] Fig. 19 is a schematic diagram of a structure of a computing device cluster according to an embodiment of the present application;

[0100] Fig. 20 is a schematic diagram of a network connection of computing devices in a computing device cluster according to an embodiment of the present application. DETAILED DESCRIPTION

[0101] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings, obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0102] In this document, reference to“an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase“in an embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. Those skilled in the art will appreciate that embodiments described herein can be combined with other embodiments unless clearly indicated otherwise.

[0103] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in a various embodiment" or "in some embodiments" or "in other embodiments" in various places throughout this specification are not necessarily all referring to the same embodiment, but can refer to one or more, but not all, embodiments, unless otherwise specifically stated. The terms "including," "comprising," "having" and variations thereof herein are meant to be broad and encompass the terms "consisting of" and "consisting essentially of" unless otherwise noted.

[0104] FIG. 1a shows an example diagram of an architecture of a distributed storage system to which embodiments of the present application are applied. As shown in FIG. 1a, the distributed storage system 100 includes a computing node cluster, a storage node cluster and a service node 103, the computing node cluster including one or more computing nodes 101, each of the computing nodes 101 being capable of communicating with each other. The computing node 101 can be a virtual machine or a computing device, which can be, for example, a server, a desktop computer or a controller of a storage array. Any of the computing nodes 101 can access any of the storage nodes 102 in the storage node cluster through a network. The storage node cluster includes a plurality of storage nodes 102. The service node 103 communicates with any of the storage nodes 102 in the storage node cluster to determine a node status of each of the storage nodes 102 in the storage node cluster, which can be normal operation or abnormal. It is to be noted that embodiments of the present application are not intended to limit the number of the computing nodes 101, the storage nodes 102 and the service node 103, which can be determined according to actual needs.

[0105] In the embodiment of the present application, the storage node 102 stores actual data and provides data IO service externally. In actual application, a user accesses the distributed storage system through the computing node 101. In some possible scenarios, the user accesses the computing node 101 through a terminal device, and in other possible scenarios, the distributed storage system 100 is deployed in the cloud, and the user accesses the computing node 101 through a terminal device accessing a cloud management platform. The terminal device can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The exemplary embodiments of the terminal device involved in the embodiment of the present application include, but are not limited to, electronic devices running iOS, android, Windows, Harmony OS or other operating systems. The embodiment of the present application does not specifically limit the type of electronic device. The cloud management platform can be a separate electronic device or a device integrated on an electronic device, and the embodiment of the present application does not limit the specific deployment manner and deployment location of the cloud management platform.

[0106] In a possible scenario, the computing node 101 and the storage node 102 are deployed integrally. In this scenario, the distributed storage system 100 includes an application server cluster. As shown in FIG. 2a, the application server cluster includes one or more application servers 110 (two application servers 110 are shown in FIG. 2a, but the number of application servers 110 is not limited to two). The application servers 110 can communicate with each other. The application server 110 is a device with both computing and storage capabilities, such as a server, a desktop computer, etc. On the software side, each application server 110 has an operating system. A virtual machine 115 can be created on the application server 110, and the computing resources required by the virtual machine 115 are sourced from the local processor 111 and the memory 112 of the application server 110, and the storage resources required by the virtual machine 115 can be sourced from the local hard disk 114 of the application server 110 or from the hard disk 114 of another application server 110. In addition, various application programs can be run in the virtual machine 115, and the user can trigger read / write data requests through the application programs in the virtual machine 115. In the embodiment of the present application, the virtual machine 115 can serve as the computing node 101, and the hard disk 103 can serve as the storage node 102.

[0107] On the hardware, as shown in FIG. 2a, the application server 110 at least includes a processor 111, a memory 112, a network card 113 and a hard disk 114. The processor 111, the memory 112, the network card 113 and the hard disk 114 are connected through a bus. Among them, the processor 111 and the memory 112 are used to provide computing resources. Among them, the hard disk 114 is used to provide storage resources, such as storing data. It can be a magnetic disk or other types of storage media, such as a solid state disk (SSD) or a shingled magnetic recording hard disk, etc. It should be noted that the hard disk 114 is only an example of a non-volatile memory and does not constitute a specific limitation. In actual applications, other non-volatile memories can also be used, and the embodiments of the present application take the hard disk 114 as an example to describe the scheme. Specifically, the processor 111 is a central processing unit (CPU) for processing data access requests from outside the application server 110 or requests generated inside the application server 110. For example, when the processor 111 receives a user's write data request, it will temporarily save the data in the write data request in the memory 112. When the total amount of data in the memory 112 reaches a certain threshold, the processor 111 sends the data stored in the memory 112 to the hard disk 114 for persistent storage. In addition, the processor 111 is also used for data calculation or processing, such as metadata management, duplicate data deletion, data compression, data verification, virtual storage space and address translation, etc. Only one processor 111 is shown in FIG. 2a, but in actual applications, the number of processors 111 is often more than one, and one processor 111 has one or more CPU cores. The number of CPUs and the number of CPU cores are not limited in this embodiment.

[0108] The memory 112 refers to an internal memory that exchanges data directly with the processor. It can read and write data at any time and at a very fast speed, and serves as temporary data storage for the operating system or other programs that are running. The memory 112 includes at least two types of memories, for example, the memory can be a random access memory or a read-only memory (ROM). For example, the random access memory is a dynamic random access memory (DRAM) or a storage class memory (SCM). The DRAM is a semiconductor memory, which, like most random access memories (RAM), is a type of volatile memory device. The SCM is a hybrid storage technology that combines the characteristics of traditional storage devices and memories. The storage class memory can provide faster read and write speeds than hard drives, but slower computing speeds than DRAM, and is more cost-effective than DRAM. However, the DRAM and the SCM are only exemplary in the present embodiment, and the memory 112 can also include other random access memories, such as static random access memories (SRAM) and the like. For the read-only memory, for example, it can be a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), and the like. In addition, the memory 113 can also be a dual in-line memory module (DIMM), i.e., a module composed of dynamic random access memories (DRAM), and can also be a solid state drive (SSD). In practical applications, multiple memories 112 and different types of memories 112 can be configured in the computing node 101. The present embodiment does not limit the number and type of the memory 112. In addition, the memory 112 can be configured to have a power retention function. The power retention function refers to that when the system is powered off and then powered on again, the data stored in the memory 112 will not be lost. The memory with the power retention function is called a non-volatile memory.

[0109] The network card 113 is used for communication with other application servers 110.

[0110] In one possible scenario, the computing nodes 101 and the storage nodes 102 are deployed separately. As shown in FIG. 2b, the computing node cluster includes one or more computing nodes 101 (two computing nodes 101 are shown in FIG. 2b, but the number of computing nodes 101 is not limited to two). In hardware, as shown in FIG. 2b, the computing node 101 at least includes a processor 121, a memory 122, and a network card 123. The details of the processor 121 are described above in the description of the processor 111 and will not be repeated here. The details of the memory 122 are described above in the description of the memory 112 and will not be repeated here.

[0111] The network card 123 is used to communicate with the storage node 102. For example, when the total amount of data in the memory 122 reaches a certain threshold, the computing node 101 can send a request to the storage node 102 through the network card 123 to store the data persistently. In addition, the computing node 101 can also include a bus for communication between the components inside the computing node 101. In terms of function, since the main function of the computing node 101 in FIG. 2b is to compute services, when storing data, it can use remote memory to achieve persistent storage, so it has less local memory than a conventional server, thereby achieving cost and space savings. However, this does not mean that the computing node 101 cannot have local memory. In actual implementation, the computing node 101 can also have a small amount of built-in hard disk or externally connected hard disk.

[0112] Any computing node 101 can access any storage node 102 in the storage node cluster through the network. The storage node cluster includes a plurality of storage nodes 102 (two storage nodes 102 are shown in FIG. 2b, but the number of storage nodes 102 is not limited to two). One storage node 102 includes one or more controllers 131 (deployed with the storage node 102), a network card 132, and a plurality of hard disks 133. The network card 132 is used to communicate with the computing node 101. The hard disk 133 is used to store data. The controller 131 is used to write data into the hard disk 133, or read or delete data from the hard disk 133 according to the read / write data request sent by the computing node 101. In the process of reading and writing data, the controller 131 needs to convert the address carried in the read / write data request into an address that can be recognized by the hard disk. As can be seen, the controller 131 also has some simple computing functions.

[0113] The network can be a wired network or a wireless network. For example, the wired network can be a cable network, a fiber network, a digital data network (DDN), etc., and the wireless network can be a telecommunication network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a ZigBee network, a Global System for Mobile Communications (GSM) network, etc., or any combination thereof. It can be understood that the network can use any known network communication protocol to enable communication between the different client layers and the gateway. The network communication protocol can be various wired or wireless communication protocols, such as Ethernet, universal serial bus (USB), firewire, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), new radio (NR), etc.

[0114] In practical applications, the control unit 131 can have various forms. In one case, the control unit 131 includes a CPU and a memory. The CPU is used to perform address conversion and read / write data operations. The memory is used to temporarily store data to be written into the hard disk 133 or data read from the hard disk 133 to be sent to the computing node 101. In another case, the control unit 131 is a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of the CPU, but is more specialized and can efficiently operate on network packets, storage requests, or analysis requests. The DPU is distinguished from the CPU by a greater degree of parallelism (a large number of requests need to be processed). Optionally, the DPU here can also be replaced by a graphics processing unit (GPU), an embedded neural-network processing unit (NPU), or other processing chips. Generally, the number of control units 131 can be one, two, or more. When the storage node 102 contains at least two control units 131, the hard disk 133 has an ownership relationship with the control unit 131, and each controller can only access the hard disk belonging to it, so this often involves forwarding read / write data requests between the control units 131, resulting in a longer data access path. In addition, if the storage space is insufficient, when a new hard disk 133 is added to the storage node 102, the ownership relationship between the hard disk 133 and the controller 131 needs to be re-bound, which is complex and results in poor scalability of the storage space.

[0115] Therefore, in another embodiment, the functions of the control unit 131 can be offloaded to the network card 132. In other words, the storage node 102 does not have a controller 131 inside, but the data read / write, address conversion, and other computing functions are completed by the network card 132. At this time, the network card 132 is a smart network card. It can include a CPU and a memory. The CPU is used to perform address conversion and read / write data operations. The memory is used to temporarily store data to be written into the hard disk 133 or data read from the hard disk 133 to be sent to the computing node 101. It can also be a programmable electronic component, such as a data processing unit (DPU). There is no ownership relationship between the network card 132 and the hard disk 133 in the storage node 102, and the network card 132 can access any hard disk 133 in the storage node 102, so it is more convenient to expand the hard disk when the storage space is insufficient.

[0116] It should be understood that the distributed storage system to which the embodiments of the present application are applicable can also be applicable to other distributed storage systems other than the distributed storage system shown in FIG. 1a, and the embodiments of the present application do not limit this.

[0117] In some possible scenarios, as shown in FIG. 1b, the storage node cluster can further include a management node 104, the number of the management node 104 can be determined in combination with actual needs, and embodiments of the present application do not make specific limitation thereon; in a specific implementation, the nodes in the storage node cluster can be artificially divided into the management node 104 and the storage node 102; in another possible implementation, the nodes in the storage node cluster negotiate to obtain the management node 104 and the storage node 102. It is worth noting that the management node 104 is only a possible naming manner, and different naming manners can be used in different companies or different distributed storage systems, for example, the Master node.

[0118] It should be noted that the management node 104 can be multiple. In an example, a master-backup mode can be used for deployment, that is, the management node 104 is given several copies, and the information of the backup management node 104 is kept consistent with the master management node 104 at all times, and once the master management node 104 hangs up, the backup management node 104 immediately follows. In another example, a consensus algorithm can be used among the multiple management nodes 104 to select a master, and once the master management node 104 hangs up, the other management nodes 104 use the consensus algorithm to reselect a master.

[0119] In the related art, the distributed storage system 100 maintains a globally consistent view information, which is used to describe the rule of distributing data to the storage node cluster, and in a specific work, the storage node 102 and the computing node 101 determine the physical location of data storage and reading by querying the view information. Exemplarily, the view information is used to record the data storage of each storage node 102 in the storage node cluster, for example, the view information includes the data characteristics corresponding to the storage node 102 and the size of the data characteristics in the storage space of the storage node 102. For example, the view information can include: how many storage nodes 102 the storage node cluster has, how much remaining capacity each storage node 102 has, how the load is, which storage node 102 stores which data, and the like.

[0120] Among them, the sources of the view information mainly include the following modes:

[0121] 1. Configuration. It can be understood as the static information of the storage node cluster, such as how many storage nodes 102 the storage node cluster initially has, the physical topology of the storage node 102, the capacity of each storage node 102, and the like. Under the architecture of FIG. 1a, the storage node 102 can load these configuration information at startup. Under the architecture of FIG. 1b, the management node 104 can load these configuration information at startup.

[0122] 2. Reporting. Under the architecture of FIG. 1b, the storage node 102 actively reports its state to the management node 104 at runtime, such as whether the storage node 102 is alive, load information of the storage node 102, which data is stored in the storage node 102, and the like. During the running of the distributed storage system 100, the storage node 102 will continuously report to the management node 104 through a heartbeat or the like at a regular time.

[0123] With the view information, the computing node 101 can have a grasp of the situation of the entire storage node cluster, and thus make a series of decisions, as follows:

[0124] 1. Schedule. A new data write request comes, which storage node 102 is responsible for allocation, usually a small load is selected.

[0125] 2. Balance. As the storage node 102 changes, data is added or deleted, the distribution of data in different machines may no longer be uniform, forming read-write hotspots in some machines, and wasting resources in other machines, thereby affecting the overall performance of the system. Therefore, real-time monitoring and timely migration are needed.

[0126] In the related art, the distributed storage system 100 will perform view management: when a storage node 102 joins or exits, or the storage node 102 temporarily fails, the view information is automatically updated, and the accuracy and consistency of the view information in the distributed storage system 100 are ensured through passive query or active update.

[0127] Among them, the view management mainly includes centralized management and decentralized management. The centralized management means that a process of a type of management role is independently deployed to specially process view update and distribution, which is called a management node 104 in the embodiment of the application, and the management node 104 can be deployed in a master-slave manner, and when the master management node 104 fails, the standby management node 104 can be promoted to the master to realize failover. The decentralized management means that each storage node 102 is responsible for managing part of the view, and synchronizes the view information on the storage node 102 to other storage nodes 102 through some propagation protocol such as the Gossip protocol.

[0128] However, the view management has the following problems:

[0129] One, for the centralized management mode, the management node 104 becomes the weak link of the system, once the management node 104 fails, the view management function of the entire distributed storage system 100 can be affected. When the management node 104 is deployed in a master-slave manner, when the master management node 104 fails, the standby management node 104 can be upgraded to the master during the master upgrade period, and the view update of the distributed storage system 100 can be affected.

[0130] Two, for the centralized management mode, as the size of the distributed storage system 100 increases, the load of the management node 104 increases rapidly, and the view management can become a bottleneck of the distributed storage system 100.

[0131] Three, for the decentralized management mode, when the view information of the distributed storage system 100 is propagated through a propagation protocol such as the Gossip protocol, the time and space path are uncertain. In the environment where the storage nodes 102 frequently change, the view update speed can not keep up with the speed of the storage nodes 102 changing, so that the view information held by part of the storage nodes 102 can be outdated.

[0132] In order to solve the above problems, based on this, the embodiment of the application provides the following technical scheme of view management.

[0133] Aspect 1: View hierarchy management. As shown in FIG. 3, the present application divides view management into three layers according to responsibilities, namely view generator, view pusher and view consumer. The view generator is used to generate view information of the distributed storage system 100, and sends the generated view information to the view pusher. The view pusher sends the view information to the view consumer. In the architecture shown in FIG. 1a, the view generator can be directly pre-configured. In the architecture shown in FIG. 1b, the management node 104 acts as the view generator. The view pusher can be a pre-configured storage node 102, or can be flexibly configured by the view generator according to actual conditions. The other storage nodes 102 and computing nodes 101 except the view generator and the view pusher act as the view consumer. It should be noted that when the number of storage nodes 102 of the storage cluster is small, all storage nodes 102 except the view generator can be used as the view pusher. In this case, the computing nodes 101 act as the view consumer. The computing nodes 101 select any one of the view pushers to request the view information of the distributed storage system 100, and perform data IO on the storage nodes 102 based on the requested view information of the distributed storage system 100. Data IO means data readout and data write-in. In some possible implementation manners, the computing nodes 101 can obtain information (for the sake of description and distinction, it can be referred to as view pusher information) indicating all view pushers from the view generator. In other possible implementation manners, the view generator sends the view information of the distributed storage system 100 and the information indicating all view pushers to the service node 103 for persistence, and the view consumer requests the view pusher information indicating all view pushers from the service node 103.

[0134] Aspect 2: Fault handling mechanism. In the case that any node in the storage node cluster fails to cause the view request to fail, the view information can be pulled from other storage nodes 102 or the service node 103. In specific implementation, if the primary management node 104 fails, the standby management node 104 quickly becomes the primary management node. The new management node 104 can obtain the latest view information from the service node 103, and the view pusher directly queries the view information from the new management node 104. If the view pusher fails, the view consumer can request the view information from other view pushers.

[0135] For the above technical solutions, the beneficial effects that can be achieved are as follows:

[0136] 1. The view generator is used to recommend the view to the view pusher, so that view synchronization between multiple storage nodes is not needed, and the overhead of view synchronization is reduced.

[0137] 2. The view generator is used for view generation and view distribution, the computing node requests the view from the view pusher, which reduces the load of the view generator in view management, makes the storage node have strong scalability, can flexibly expand the scale of the storage node, and reduces the possibility that the view management becomes the performance bottleneck of the distributed storage system 100.

[0138] 3. In the scenario that the storage nodes in the distributed storage system 100 frequently change, the view is pushed from the view generator to the view pusher, the view push path and time have predictability, and synchronization between the storage nodes is not needed, thereby improving the view distribution efficiency.

[0139] Next, a view management method provided by an embodiment of the application is described in detail in combination with the distributed storage system 100 provided above. The application takes the view generator as the management node 104, and takes the view generator, the view pusher, and the view consumer as the view production node, the view push node, and the view consumption node as an example to describe the scheme of the application. It is assumed that the storage node cluster includes N storage nodes 102, the N storage nodes 102 include a first view generator, the first view generator can be directly pre-configured under the architecture shown in FIG. 1a, or the N storage nodes 102 negotiate the first view generator; under the architecture shown in FIG. 1b, the management node 104 acts as the first view generator.

[0140] FIG. 4 is a flowchart of the view management method provided by the embodiment of the application. The embodiment can be applied to the distributed storage system 100. As shown in FIG. 4, the view management method provided by the embodiment of the application at least includes the following steps:

[0141] In step 401, the first view generator determines M view push nodes from the N storage nodes 102.

[0142] In some possible implementation manners, the first view generator can take each storage node 102 except the first view generator as a view push node.

[0143] In some possible implementation manners, the first view generator can take part of the M storage nodes 102 as the view push node. The rule of selecting the view push node can be as follows:

[0144] 1. Random selection;

[0145] 2. Selection according to a proportion, for example, selecting a preset proportion of the M storage nodes 102 as the view push node;

[0146] 3. According to the hardware configuration of the storage node 102, the storage node 102 with a high hardware configuration is selected as the view pushing node as much as possible, so that the view pushing node can more efficiently implement the task.

[0147] 4. According to the reliability and stability, the storage node 102 in different regions is selected as much as possible, so that the reliability and stability of the view pushing node are ensured.

[0148] Step 402, the first view generating node sends view information to the M view pushing nodes, and the view information is used to indicate the rule of distributing data to the N storage nodes.

[0149] The view information is used to describe the rule of distributing data to the storage node cluster, and the view information is used to record the data storage condition of each storage node 102 in the storage node cluster, for example, the view information includes the data characteristics corresponding to the storage node 102 and the size of the data characteristics in the storage space of the storage node 102.

[0150] In the embodiment of the application, the data distribution can consider data uniformity, data stability, node heterogeneity, and can also consider factors such as isolation of fault domain and performance stability.

[0151] The data uniformity mainly includes two aspects: the data stored in different storage nodes 102 should be as balanced as possible, so as to avoid that one or several storage nodes 102 have too much storage pressure, while other storage nodes 102 have almost no data. For example, there are 100G data and 4 storage nodes 102 of the same type, and it is usually desired that the data storage is as balanced as possible, for example, each storage node 102 stores 25G data. In addition, user access should also be balanced, so as to avoid that the access amount of one or several storage nodes 102 is large, while other storage nodes 102 are not accessed. For example, there are 1000 requests, and for the above-mentioned 4 storage nodes 102, the user access requests are processed as balanced as possible, for example, each storage node 102 processes 250 requests, and in actual situation, the difference between each storage node 102 is not large.

[0152] When the storage node 102 needs to be removed or expanded due to failure, the data migration should not be large-scale. For example, there are 100G data, and there are 4 storage nodes 102 of the same type (storage nodes 1-4) at the beginning, each storage node 102 stores 25G data, and now the storage node 2 fails, and the data on the node 2 is migrated to other nodes as much as possible, without the need to migrate and store all the data, so as to ensure the stability of the data.

[0153] Among them, from the dimension of node heterogeneity, the hardware configuration of different storage nodes 102 can be very different. For example, some storage nodes 102 have high hardware configuration and can store a large amount of data and can bear more requests; but some storage nodes 102 have poor hardware configuration and cannot store too much data and cannot access too much user. For such storage nodes 102 with large hardware differences, the amount of data and user access allocated to them are similar, which is essentially an imbalance.

[0154] Among them, from the dimension of isolated fault domain, it is to ensure the availability and reliability of data. For example, data reliability is usually achieved through backup. But if each data and its backup are distributed to the same hard disk or storage node 102, the availability and reliability of data cannot be guaranteed. Therefore, a set of storage nodes 102 should be mapped for each data, and these storage nodes 102 should be as different as possible in different fault domains, such as different machine rooms, different racks, etc., so as to ensure the availability and reliability of data.

[0155] Among them, from the dimension of performance stability, the efficiency of data storage and query needs to be guaranteed, and the addition or removal of storage nodes 102 cannot cause serious decline in storage or access performance.

[0156] Step 403, the computing node 101 determines a first view push node from the M view push nodes.

[0157] In some possible implementation manners, the computing node 101 can randomly select a view push node from the view push nodes indicated by the view push node information as the first view push node.

[0158] In another possible implementation manner, the computing node 101 can select the nearest view push node as the first view push node according to the geographical affinity relationship.

[0159] In still another possible implementation manner, the computing node 101 can select the view push node with the lowest load as the first view push node.

[0160] Step 404, the computing node 101 requests view information from the first view push node.

[0161] In specific implementation, the computing node 101 sends a view information request to the first view push node, and the first view push node sends the view information to the computing node 101; it should be noted that the view information sent by the first view push node to the computing node 101 is the latest, and the latest view information is the view information sent by the first view generating node last received by the first view push node.

[0162] Step 405, the computing node 101 performs data writing or data reading on the N storage nodes 102 based on the requested view information.

[0163] In a specific implementation, the computing node 101 generates a data operation request through the view information and sends the data operation request to the storage node 102. The data operation request can be generated for processing a data access request from outside or for processing a request generated inside the distributed storage system 100. For example, the processing can include metadata management, data deduplication, data compression, data verification, virtual storage space, address conversion, etc. The storage node 102 is configured to store data, receive the data operation request from the computing node 101, and perform data operation based on the data operation request. For example, the data operation request is configured to instruct writing of current to-be-written data to a target storage node 102 or reading data from the target storage node 102.

[0164] In one example, if data needs to be written, the computing node 101 queries the view information according to the current to-be-written data, determines a target storage node 102 to which the current to-be-written data needs to be written, and generates a data operation request based on the current to-be-written data and the target storage node 102.

[0165] In one example, if data needs to be read, the view information is queried according to the characteristics of the current to-be-read data, a target storage node 102 in which the current to-be-read data is located is determined, and a data operation request is generated according to the characteristics of the current to-be-read data and the target storage node 102.

[0166] In the scheme provided by the present application, on the one hand, the view production node is configured to generate and distribute the view, and the computing node requests the view from the view push node, which reduces the load of the view production node in view management, makes the storage node have strong scalability, can flexibly expand the scale of the storage node, and reduces the possibility that the view management becomes a performance bottleneck. On the other hand, the view synchronization between multiple storage nodes is not required, which reduces the overhead of view synchronization. On the other hand, in the scenario that the storage nodes in the distributed storage system frequently change, the view production node pushes the view to the view push node, the view push path and time are predictable, and thus the view distribution efficiency is improved.

[0167] In some possible embodiments, as shown in FIG. 3 and FIG. 6, the view push node sends the view information to the view consumption node.

[0168] FIG. 5 shows a flowchart of another view management method provided by an embodiment of the present application.

[0169] As shown in FIG. 5, based on the steps 401 to 405 shown in FIG. 4, in the embodiment of the present application, at least the following steps are further included.

[0170] In step 501, the first view generating node determines the association relationship between the second view pushing node and the H view consuming nodes, the H view consuming nodes being the storage nodes 102 other than the first view generating node and the M view pushing nodes among the N storage nodes 102, and H being a positive integer less than N; the second view pushing node being any one of the M view pushing nodes.

[0171] In some possible implementation manners, the first view generating node selects part of the storage nodes 102 among the N storage nodes 102 as the view pushing nodes, and selects the H storage nodes 102 that are not selected as the view consuming nodes. In the embodiment of the application, H is less than or equal to N-M-number of the view generating nodes.

[0172] In a specific implementation, for the second view pushing node (any one of the M view pushing nodes), the first view generating node determines the association relationship between the second view pushing node and the H view consuming nodes, the association relationship being used to indicate the view consuming nodes associated with the second view pushing node, the view consuming nodes associated with the second view pushing node being part of the H view consuming nodes. In a possible case, the first view generating node can determine the view consuming nodes associated with the second view pushing node by considering the geographical affinity relationship. Specifically, for the second view pushing node, the second view pushing node and the view consuming nodes associated with the second view pushing node have the geographical affinity relationship. For example, the view consuming nodes and the second view pushing node are located in the same machine room, or the view consuming nodes and the second view pushing node are located in different machine rooms, and the distance between the different machine rooms does not exceed a preset threshold.

[0173] In step 502, the first view generating node sends the association relationship to the second view pushing node.

[0174] In step 503, the second view pushing node determines the view consuming nodes associated with the second view pushing node based on the association relationship, and sends the view information to the associated view consuming nodes.

[0175] In a specific implementation, the view information sent by the second view pushing node to the associated view consuming nodes is the latest, and the latest view information is the view information sent by the first view generating node last time received by the second view pushing node. For example, the second view pushing node can push the latest view information to the associated view consuming nodes, and the view consuming nodes can also pull the latest view information from the associated second view pushing node.

[0176] It is worth noting that in actual application, the first view generation node needs to determine the association between each of the M view push nodes and the H view consumption nodes, for example, the H view consumption nodes can be equally distributed to each of the view push nodes; subsequently, each of the view push nodes is responsible for pushing the latest view information to the associated storage node 102.

[0177] In the scheme provided by the application, the view push node is associated with the view consumption node, and the view push node needs to synchronize the view to the associated view consumption node, without the need for the view generation node to synchronize the view to the view consumption node, thereby reducing the load of the view production node in view management, and making the storage node have strong expansibility, so that the scale of the storage node can be flexibly expanded without affecting the view management.

[0178] In some possible embodiments, as shown in FIG. 6, the view generation node can send the view push node information (used to indicate all the view push nodes) to the service node 103 for storage, and the view consumption node can request the view push node information from the service node 103, and select the view push node to request the view information based on the requested view push node information. It should be noted that the view consumption node obtains the view push node information from the service node 103 only as an example, and does not constitute a specific limitation, and in some possible implementation manners, the view consumption node can also obtain the view push node information from the view generation node.

[0179] FIG. 7 shows a flow diagram of another view management method provided by an embodiment of the application.

[0180] As shown in FIG. 7, on the basis of steps 401 to 405 shown in FIG. 4, in the embodiment of the application, at least the following steps are further included:

[0181] Step 701: The first view generation node determines view push node information, and the view push node information is used to indicate M view push nodes.

[0182] The view push node information can be an identifier such as a number of each view push node. For example, the view push node information can be a list, and each item in the list is an identifier such as a number of a view push node.

[0183] Step 702: The first view generation node sends the view push node information to the service node 103.

[0184] Step 703: The service node 103 stores the view push node information.

[0185] In specific implementation, the service node 103 will persistently store the view push node information, so as to ensure that the view push node information will not be lost.

[0186] Step 403 comprises:

[0187] Step 4031, the computing node 101 requests the view push node information from the service node 103.

[0188] In a specific implementation, the computing node 101 sends a view push node information request to the service node 103, and the service node 103 sends the view push node information to the computing node 101; it should be noted that the view push node information sent by the service node 103 to the computing node 101 is the latest, and the latest view push node information is the view push node information sent by the first view generation node last received by the service node 103.

[0189] Step 4032, the computing node 101 determines the first view push node from the M view push nodes based on the request to the view push node information.

[0190] In the scheme provided by the application, the computing node can select the view push node from a large number of view push nodes to request the view, without requesting the view generation node, which reduces the load of the view production node in view management, makes the storage node have strong expansibility, can flexibly expand the scale of the storage node, and reduces the possibility that the view management becomes a performance bottleneck.

[0191] In some possible embodiments, as shown in FIG. 6, the view generation node monitors the node state of each storage node 102 in the storage cluster through the service node 103, dynamically updates the view information based on the node state of each storage node 102 in the storage cluster, and sends it to the view push node, which is then sent to the view consumer node, and thus to all storage nodes 102.

[0192] FIG. 8 shows a flowchart of another view management method provided by an embodiment of the application. As shown in FIG. 7, based on steps 401 to 405 shown in FIG. 4, in the embodiment of the application, at least the following steps are further included:

[0193] Step 801, the service node 103 sends the node state of the N storage nodes to the first view generation node.

[0194] Exemplarily, the node state can be normal operation or failure.

[0195] Step 802, the first view generation node determines the view information based on the node state of the N storage nodes and the data distribution rule.

[0196] In a specific implementation, the management node 104 can deploy an algorithm for describing the data distribution rule, input the node state of each storage node 102 in the storage node cluster into the algorithm, and output the view information.

[0197] It should be noted that the view information is the latest view information, and the latest view information is the view information determined by the management node 104 last time. Wherein, when the distributed storage system 100 is initially run, the view information obtained for the first time is the latest view information, and subsequently, when the node state of any storage node 102 in the storage node cluster changes, the view information needs to be updated, and the updated view information is the latest view information. For details, refer to the description of step 402 above, which will not be repeated here.

[0198] Step 803, the service node 103 determines that the node state of the target storage node 102 in the N storage nodes 102 changes.

[0199] In some possible implementation manners, the case where the node state changes can be that the node state of the storage node 102 changes from normal to failure, or from failure to normal, and can also be newly added. Wherein, in the case of newly added node state, the target storage node 102 is the newly added storage node 102.

[0200] It should be noted that the target storage node 102 is a storage node 102 other than the view generation node, and the number can be one or more.

[0201] Step 804, the service node 103 sends the updated node state of the target storage node 102 to the first view generation node.

[0202] Step 805, the first view generation node updates the view information based on the updated node state of the target storage node 102 and the data distribution rule, and obtains the updated view information.

[0203] In a specific implementation, the management node 104 updates the view information based on the updated node state of the target storage node 102, the node state of the other storage nodes 102, and the data distribution rule.

[0204] Step 806, the first view generation node sends the updated view information to the M view pushing nodes.

[0205] Subsequently, if the computing node 101 sends a view information request to the view pushing node, the view pushing node will feedback the updated view information.

[0206] In the scheme provided by the application, the node state monitoring of the storage node is undertaken by the service node, and the view generation node does not need to monitor the state of the storage node, thereby reducing the load of the view generation node in view management, making the storage node have strong scalability, and enabling the scale of the storage node to be flexibly expanded, and reducing the possibility that the view management becomes a performance bottleneck. In addition, in the scenario of frequent changes of the storage node in the distributed storage system, since the view generation node realizes view updating by monitoring the node state through the service node, the speed of view updating can be improved.

[0207] In some possible embodiments, as shown in FIG. 6, the view generation node sends the view information to the service node 103, and the service node 103 is configured to permanently store the view information, thereby reducing the impact on view management in the case of failure of any management node in the distributed storage system.

[0208] FIG. 9 shows a flowchart of another view management method provided by an embodiment of the application.

[0209] As shown in FIG. 9, on the basis of steps 401 to 405 shown in FIG. 4, in the embodiment of the application, at least the following steps are further included:

[0210] Step 901: The first view generation node sends the view information to the service node 103.

[0211] It should be noted that the view information is the latest view information, and the latest view information is the view information determined by the management node 104 last time. Wherein, when the distributed storage system 100 is initially run, the view information obtained for the first time is the latest view information, and subsequently, when the node state of any storage node 102 in the storage node cluster changes, the view information needs to be updated, and the updated view information is the latest view information. For details, refer to the description of step 402 above, which will not be repeated here.

[0212] Step 902: The service node 103 stores the view information.

[0213] In the specific implementation, the service node 103 will persistently store the view information, thereby ensuring that the view information will not be lost, and further reducing the impact on view management in the case of failure of any management node in the distributed storage system 100.

[0214] In some possible embodiments, if the view pushing node fails, the view generation node can select the storage node 102 to replace the failed view pushing node, and update the view pushing node information.

[0215] FIG. 10 shows a flowchart of another view management method provided by an embodiment of the application.

[0216] As shown in FIG. 10, on the basis of steps 401 to 405 shown in FIG. 4, in the embodiment of the application, at least the following steps are further included:

[0217] In step 1001, the first view generation node determines that the third view push node in the M view push nodes is faulty, selects a fourth view push node from the N storage nodes 102 other than the first view generation node and the M view push nodes to replace the third view push node, and obtains updated view push node information.

[0218] It should be noted that the service node 103 sends the node state of the third view push node to the first view generation node in the case where the node state of the third view push node is faulty, and the first view generation node determines that the third view push node is faulty based on the node state of the third view push node sent by the service node 103. It should be noted that the third view push node being faulty represents that the node state of the storage node 102 in the distributed storage system 100 has changed, and at this time, the first view generation node needs to update the view information.

[0219] In step 1002, the first view generation node sends the view information to the fourth view push node.

[0220] In step 1003, the first view generation node sends the updated view push node information to the service node 103.

[0221] In step 1004, the service node 103 stores the updated view push node information.

[0222] Subsequently, if the computing node 101 sends a view push node information request to the service node 103, the service node 103 will feed back the updated push node information request.

[0223] It should be noted that in the case where the target storage node 102 in step 804 is the third view push node and the node state of the target storage node 102 is faulty, the embodiment of the application does not intend to limit the execution order of steps 805 and 1001 in FIG. 8, for example, steps 805 and 1001 can be executed in parallel.

[0224] In the scheme provided by the application, the view production node replaces the faulty view push node after the view push node is faulty, and implements fault processing.

[0225] In some possible embodiments, the distributed storage system 100 further includes a second view generation node, which can be a backup node of the first view generation node. The first view generation node can synchronize the latest view information and view push node information to the second view generation node. After the first view generation node fails, the second view generation node takes over, and replaces the failed first view generation node to obtain the latest view information and view push node information from the service node 103, so as to implement view management. FIG. 11 shows a flowchart of another view management method provided by an embodiment of the present application.

[0226] As shown in FIG. 11, on the basis of steps 401 to 405 shown in FIG. 4, in the embodiment of the present application, the following steps are further included:

[0227] In step 1101, the second view generation node sends a view-related information request to the service node 103 when the first view generation node fails.

[0228] It should be noted that the first view generation node sends heartbeat information to the second view generation node, and the second view generation node can determine whether the first view generation node fails based on the received heartbeat information.

[0229] In a specific implementation, if there is only one second view generation node in the distributed storage system 100, which is a slave node of the first view generation node, the second view generation node takes over the first view generation node. If there are multiple second view generation nodes in the distributed storage system 100, the second view generation nodes can perform an election to select a second view generation node to take over.

[0230] In step 1102, the service node 103 sends the latest view information and the latest view push node information to the second view generation node.

[0231] It should be noted that the service node 103 sends the view information and the view push node information last received from the first view generation node as the latest view information and the latest view push node information to the second view generation node.

[0232] In the scheme provided by the present application, after the view generation node fails, other view generation nodes take over the view management, read the view information and the view push node information from the service node, and improve the efficiency of processing the view generation node failure.

[0233] In some possible embodiments, if the view push node fails, the view consumption node can request the view information from other view push nodes.

[0234] FIG. 12 shows a flowchart of another view management method provided by an embodiment of the present application.

[0235] As shown in FIG. 12, on the basis of steps 401 to 405 shown in FIG. 4, in the embodiment of the application, at least the following steps are further included:

[0236] In step 1201, the computing node 101 determines that the request for view information to the first view push node fails, and determines the fifth view push node based on the requested view push node information.

[0237] It should be noted that, in the case of failure of the first view push node, if the computing node 101 requests view information to the first view push node, at this time, the first view push node will feedback an error code, indicating that the request for view information fails. It should be noted that the failure of the first view push node represents that the node state of the storage node 102 in the distributed storage system 100 has changed, at this time, the first view generation node needs to update the view information, and update the view push node information.

[0238] In step 1202, the computing node 101 requests view information to the fifth view push node.

[0239] In step 1203, the computing node 101 requests view push node information to the service node 103 when it is determined that the request for view information fails.

[0240] In the embodiment of the application, when the number of times of requesting view information by the computing node 101 reaches a preset threshold, or when the view information is not requested within a preset time length, it is indicated that the view push node has a large-scale failure, and it is determined that the request for view information fails; in the case of failure of the request for view information, it is indicated that the view push node information may have expired, at this time, the view push node information needs to be requested again.

[0241] In the specific implementation, the computing node 101 sends a view push node information request to the service node 103, and the service node 103 sends the view push node information to the computing node 101; it should be noted that the view push node information sent by the service node 103 to the computing node 101 is the latest, and the latest view push node information is the view push node information sent by the first view generation node last received by the service node 103.

[0242] In step 1204, the computing node 101 determines the sixth view push node based on the requested latest view push node information.

[0243] In step 1205, the computing node 101 requests view information to the sixth view push node.

[0244] In a specific implementation, the computing node 101 sends a view information request to the sixth view push node, and the sixth view push node sends the view information to the computing node 101; it should be noted that the view information sent by the sixth view push node to the computing node 101 is the latest, and the latest view information is the view information determined by the first view generation node last time.

[0245] In step 1206, the computing node 101 performs data writing or data reading on the N storage nodes 102 based on the requested latest view information.

[0246] For details, refer to the description of step 405 above, which will not be repeated here.

[0247] In the scheme provided by the present application, when the computing node determines that the view information cannot be requested, the computing node requests the latest view push node information from the service node, and re-determines the view push node to request the view, so that the view update is quickly realized in the case of view expiration.

[0248] FIGS. 4 to 12 only show the basic embodiment of the method of the embodiment of the present application, and certain optimization and expansion can be made on the basis thereof, and other preferred embodiments of the method can also be obtained.

[0249] In one embodiment, the present embodiment is based on the foregoing embodiments, and the view management process is described in more detail and optimized to a certain extent. The method in the present embodiment comprises:

[0250] FIG. 13 is a flowchart of a view management method provided by an embodiment of the present application. The present embodiment can be applied to the distributed storage system 100, and the management node 104 acts as a view generation node. As shown in FIG. 13, the view management method provided by the present embodiment comprises at least the following steps:

[0251] In step 1301, each storage node 102 in the storage node cluster sends a heartbeat signal to the service node 103.

[0252] In the present embodiment, the storage node 102 can send the heartbeat signal to the service node 103 according to a preset period.

[0253] In step 1302, the service node 103 determines the node state of each storage node 102 in the storage node cluster based on the received heartbeat signal.

[0254] It should be noted that the node state of the storage node 102 is the latest node state.

[0255] In step 1303, the service node 103 sends the node state of each storage node 102 in the storage node cluster to the management node 104.

[0256] It should be noted that the service node 103 needs to send the initial node state of each storage node 102 in the storage node cluster to the management node 104 when the distributed storage system 100 is initially run, and subsequently, the service node 103 discovers the node state update of the storage node 102 in the storage node cluster and sends the updated node state to the management node 104.

[0257] Step 1304, the management node 104 determines the view information based on the node state of each storage node 102 in the storage node cluster and the data distribution rule.

[0258] It should be noted that the view information is the latest view information, and the latest view information is the view information determined by the management node 104 last time. Wherein, when the distributed storage system 100 is initially run, the view information obtained for the first time is the latest view information, and subsequently, when the node state of any storage node 102 in the storage node cluster changes, the view information needs to be updated, and the updated view information is the latest view information. For details, see the description of step 402 above, which will not be repeated.

[0259] Step 1305, the management node 104 sends the view information to the service node 103.

[0260] Step 1306, the service node 103 stores the view information.

[0261] In specific implementation, the service node 103 will persistently store the view information, so as to ensure that the view information will not be lost, and further reduce the impact on view management in the case of any management node failure in the distributed storage system 100.

[0262] Step 1307, the management node 104 determines the view push node information and the association information, the view push node information indicates M view push nodes selected from the storage node cluster, and the association information indicates the information of the view consumer nodes associated with the M view push nodes respectively.

[0263] For details, see the description of step 401, which will not be repeated.

[0264] Step 1308, the management node 104 sends the view push node information to the service node 103.

[0265] Step 1309, the service node 103 stores the view push node information.

[0266] In specific implementation, the service node 103 will persistently store the view push node information, so as to ensure that the view push node information will not be lost.

[0267] Step 1310, the management node 104 sends view information and information of associated view consumer nodes to M view push nodes respectively.

[0268] In a specific implementation, the view information sent by the management node 104 to the view push node is the latest, and the latest view information is the view information determined by the management node 104 last time; for example, the management node 104 can push the latest view information to the M view push nodes at the same time, and the view push node also pulls the latest view information from the management node 104.

[0269] Step 1311, the M view push nodes send the view information to the associated view consumer nodes respectively.

[0270] Step 1312, the computing node 101 sends a view push node information request to the service node 103.

[0271] In some possible implementations, the computing node 101 can start to send the view push node information request to the service node 103 after starting until receiving the view push node information sent by the service node 103.

[0272] Step 1313, the service node 103 sends the view push node information to the computing node 101.

[0273] In a specific implementation, the view push node information sent by the service node 103 to the computing node 101 is the latest, and the latest view push node is the view push node information sent by the management node 104 last time and received by the service node 103.

[0274] Step 1314, the computing node 101 determines a first view push node based on the requested view push node information.

[0275] For details, refer to the description of step 403 above, which will not be repeated here.

[0276] Step 1315, the computing node 101 sends a view information request to the first view push node.

[0277] Step 1316, the first view push node sends the view information to the computing node 101.

[0278] In a specific implementation, the view information sent by the first view push node to the computing node 101 is the latest, and the latest view information is the view information sent by the management node 104 last time and received by the first view push node.

[0279] Step 1317, the computing node 101 performs data writing or data reading on the storage node 102 in the storage node cluster based on the requested view information.

[0280] In the scheme, on the one hand, the view production node is used for view generation and view delivery, the node state monitoring of the storage node is undertaken by the service node, and the computing node requests the view from the view push node, which reduces the load of the view production node in view management, makes the storage node have strong scalability, can flexibly expand the scale of the storage node, and reduces the possibility that the view management becomes a performance bottleneck; on the other hand, view synchronization is not required between multiple storage nodes, which reduces the overhead of view synchronization; on the other hand, in the scenario that the storage nodes in the distributed storage system frequently change, the view production node realizes view update by monitoring the node state through the service node, thereby improving the speed of view update, in addition, the view push node is pushed by the view production node, the view push path and time are predictable, thereby improving the view distribution efficiency; on the other hand, the view generation node sends the view information to the service node for storage, and in the case that any management node in the distributed storage system 100 fails, the influence on view management can be reduced.

[0281] It should be noted that in actual application, the distributed storage system 100 is deployed with multiple processes, which can include a master management process (Master), a storage node working process (store worker), a storage client (client), and a zookeeper, wherein the master management process (Master) is located at the management node 104, the storage node working process (store worker) is located at the storage node 102, the storage client (client) is located at the computing node 101, and the zookeeper is located at the service node 103. In the embodiment of the application, the master management process (Master) can be used as the view production node, each storage node working process (store worker) can be used as the view push node, and the storage client (client) can be used as the view consumption node, or the master management process (Master) can be used as the view production node, part of the storage node working process (store worker) can be used as the view push node, and the storage client (client) and another part of the storage node working process (store worker) can be used as the view consumption node.

[0282] In summary, in combination with FIGS. 3 to 13, in the embodiment of the application, the service node 103 can have the following functions:

[0283] 1. Monitoring the state of the distributed storage system 100, including the joining, exiting and other changes of the nodes, and maintaining the node information of the distributed storage system 100. Specifically, the service node 103 can deploy a zookeeper service to realize the monitoring of the node state of the distributed storage system 100 through the zookeeper service.

[0284] 2. The view information and the view push node information of the persistent storage distributed storage system 100 are stored.

[0285] The view generation node has the following functions:

[0286] 1. The view generation node monitors the node state of the entire distributed storage system 100 through the zookeeper service deployed by the service node 103, including the joining, exiting and other changes of the nodes, and maintains the node information of the distributed storage system 100.

[0287] 2. The view generation node generates the view information of the distributed storage system 100; specifically, the view generation node generates the view information of the distributed storage system 100 through the node state and data distribution rules of the current storage nodes 102 of the distributed storage system 100, and sends the view information to the service node 103 for persistent storage.

[0288] 3. The view generation node elects a batch of view push nodes from the storage nodes 102. For example, the view generation node selects a batch of nodes from all the storage nodes 102 to serve as view push nodes, and associates the unselected storage nodes 102 to each view push node, and each view push node is responsible for synchronizing the view information to the storage nodes 102 associated therewith. The rules for the view generation node to select the view push nodes are: ① when the distributed storage system 100 is initialized, a batch of storage nodes 102 are randomly selected; ② when the view push node fails, the failed view push node is removed, and an unfailed storage node 102 is randomly selected from the remaining storage nodes 102 as a new view push node.

[0289] 4. The view generation node sends the information of the view push node to the service node 103 for persistent storage.

[0290] 5. The view generation node pushes the view information of the distributed storage system 100 and the association relationship between the view push node and the view consumption node to each view push node, and the view push node also pulls the latest view information from the view generation node.

[0291] 6. The view generation node updates and pushes the view information in real time. The view generation node monitors the node state of each storage node 102 of the distributed storage system 100 through the zookeeper, and when the storage node 102 joins or exits, the view information is recalculated and pushed to the view push node.

[0292] 7. When the view generation node fails, other view generation nodes take over, the view generation node that takes over pulls the latest view information and the latest view push node information from the service node 103, the view generation node that takes over pushes the latest view information to the view push node, and the view push node also pulls the latest view information from the view generation node that takes over. For example, when the view generation node is deployed in the manner of master-slave management node 104, when the master management node 104 fails, the standby management node 104 quickly becomes the master to realize failover, and the view push node directly pulls the latest view information from the standby management node 104.

[0293] The view push node can have the following functions:

[0294] Pull the latest view information from the view generation node; when receiving the view information request of the computing node 101, send the latest view information to the computing node 101; when it is necessary to send the view information of the distributed storage system 100 to other storage nodes 102, send the view information to the associated view consumption node based on the association relationship of the view consumption node sent by the management node 104, and the view consumption node also pulls the latest view information from the view push node. After the view information is updated, send the updated view information to the associated view consumption node.

[0295] For the storage node 102 as the view consumption node, the following functions can be provided:

[0296] Pull the latest view information from the view push node, and receive the latest view information sent by the view push node.

[0297] The computing node 101 can have the following functions:

[0298] 1. The computing node 101 writes or reads data after obtaining the view information. The computing node 101 first obtains the view push node information from the service node 103, randomly determines a view push node as the source node of the view, and actively pulls the view information from the source node.

[0299] 2. When the computing node 101 fails to request the view information from the view push node, the computing node 101 requests the view information from the remaining view push nodes, and if the request fails more than a certain number of times or more than a certain time length, the computing node 101 obtains the latest view push node information from the service node 103, determines a view push node as the source node of the view, and actively pulls the view information from the source node.

[0300] Based on the view management method provided above, the specific application of the view management method is described. FIG. 14 is a flowchart of a specific application of a view management method provided by the embodiment of the application. As shown in FIG. 14, the specific content includes:

[0301] As shown in FIG. 14, the distributed storage system 100 includes a master management node 1, a slave management node 2, a plurality of storage nodes 102, a plurality of computing nodes 101, a service node 103, and the service node 103 is deployed with a cluster view persistence component (zookeeper). The plurality of computing nodes 101 are view consumer nodes, denoted as computing node C0, …, computing node C p . The plurality of storage nodes 102 are denoted as storage node A0, …, storage node A M , storage node B0, …, storage node B H .

[0302] On the basis of FIG. 14, as shown in FIG. 15, in the distributed storage system 100, the system view is managed by using the following flow.

[0303] (1) The master management node 1 generates view information of the distributed storage system 100, and elects a batch of view push nodes from the storage nodes 102: storage node A0, …, storage node A M , and the remaining storage nodes B0, …, storage node B H are view consumer nodes, and the view information and the view push node list of the distributed storage system 100 are persisted to the distributed coordination service.

[0304] (2) The master management node 1 synchronizes the view information of the distributed storage system 100 to the slave management node 2.

[0305] (3) The master management node 1 pushes the latest view information to the view push nodes: storage node A0, …, storage node A M , and the view push nodes: storage node A0, …, storage node A M pull the latest view information from the master management node 1.

[0306] (4) The view push nodes: storage node A0, …, storage node A M push the latest view information to the view consumer nodes: storage node B0, …, storage node B H , and the view consumer nodes: storage node B0, …, storage node B H pull the latest view information from the view push nodes: storage node A0, …, storage node A M .

[0307] (5) The computing node C0 queries the latest view push node list from the distributed coordination service.

[0308] (6) The computing node C0 randomly selects a view push node based on the latest view push node list, for example, storage node A M , and the storage node AM Request the latest view information.

[0309] Based on Figure 15, as shown in Figure 16, when a node in the distributed storage system 100 fails, the following fault handling mechanism can be adopted.

[0310] (1) When the primary management node 1 fails, the backup management node 2 takes over and becomes the new cluster manager.

[0311] (2) Backup management node 2 reads the latest view information and view push node list from the distributed coordination service.

[0312] (3) Backup management node 2 pushes nodes to the view: storage node A0, ..., storage node A M Push the latest view information; view push nodes: storage node A0, ..., storage node A M Retrieve the latest view information from backup management node 2.

[0313] (4) If storage node A0 fails, compute node C0 will first push nodes to other views (e.g., storage node A). M Request the latest view information; if that still fails, continue trying other view push nodes.

[0314] (5) When a view request fails more than a certain number of times or after a certain period of time, the compute node C0 obtains the latest list of view push nodes from the distributed coordination service, and then continues to select view push nodes to obtain the latest view information based on the latest list of view push nodes.

[0315] This invention also provides a view management system. Please refer to Figure 17, which is a schematic diagram of the structure of a view management system provided in this invention. Details are as follows:

[0316] A view generation device is used to determine M view push nodes from N storage nodes, where M is a positive integer less than or equal to N;

[0317] A view generation device is used to send view information to M view push nodes. The view information is used to indicate the rules for distributing data to N storage nodes.

[0318] A computing device is configured to determine a first view push node from M view push nodes, request view information from the first view push node, and write or read data from N storage nodes based on the requested view information.

[0319] In some possible implementation manners, the view generation apparatus is configured to determine an association relationship between the second view push node and H view consumption nodes, the H view consumption nodes being storage nodes other than the first view generation node and the M view push nodes in the N storage nodes, H being a positive integer less than N; and the second view push node being any one of the M view push nodes.

[0320] The view generation apparatus is configured to send the association relationship to the second view push node, so that the second view push node determines, based on the association relationship, a view consumption node associated with the second view push node, and sends view information to the associated view consumption node.

[0321] In some possible implementation manners, the distributed storage system 100 can further include a service apparatus.

[0322] In an embodiment, the service apparatus is configured to send, to the view generation apparatus, node states of the N storage nodes.

[0323] The view generation apparatus is configured to determine the view information based on the node states of the N storage nodes and a data distribution rule, and send the view information to the M view push nodes.

[0324] In an embodiment, the service apparatus is configured to send, to the view generation apparatus, a node state of a target storage node in the N storage nodes after determining that the node state of the target storage node is updated.

[0325] In an embodiment, the view generation apparatus is configured to determine view push node information, the view push node information being used to indicate the M view push nodes, and send the view push node information to the service apparatus.

[0326] The computing apparatus is configured to request, from the service apparatus, the view push node information, and determine the first view push node from the M view push nodes based on the view push node information.

[0327] In an embodiment, the view generation apparatus is configured to send, to the service apparatus, the view information; and the service apparatus is configured to store the view information.

[0328] In some possible implementation manners, the view generation apparatus is configured to determine that a third view push node in the M view push nodes is faulty, select a fourth view push node from storage nodes other than the first view generation node and the M view push nodes in the N storage nodes to replace the third view push node, and obtain updated view push node information.

[0329] The view generation apparatus is configured to send the updated view push node information to the service apparatus.

[0330] The view generation apparatus is configured to send the view information to the fourth view push node.

[0331] In some possible implementation manners, the computing device is configured to determine that the request for the view information to the first view push node fails, determine a fifth view push node based on the view push node information, and request the view information from the fifth view push node.

[0332] In an example of the implementation manner, the computing device is configured to determine that the request for the view information fails, request the latest view push node information from the service device, determine a sixth view push node based on the latest view push node information, and request the latest view information from the sixth view push node.

[0333] For example, the computing device is configured to determine that the request for the view information fails when a number of times of the request for the view information fails is greater than or equal to a preset number of times, or when the view information is not requested within a preset time length.

[0334] In an example of the implementation manner, the distributed storage system further includes a replacement device; when the view generation device fails, the replacement device reads the view information and the view push node information from the service device. It should be noted that the replacement device and the view generation device are merely a possible naming manner, and in some possible scenarios, the view generation device can be referred to as a first view generation device, and the replacement device can be referred to as a second view generation device.

[0335] It should be noted that in the embodiments of the present application, the service device is configured to implement the steps implemented by the service node 103 in FIGS. 3 to 13, the view generation device is configured to implement the steps implemented by the first view generation node or the management node 104 in FIGS. 3 to 13, and the computing device is configured to implement the steps implemented by the computing node 101 in FIGS. 3 to 13. It should be noted that the service device is not a necessary device.

[0336] It should be noted that the view generation device, the service device, and the computing device can be implemented by software or by hardware. For example, the implementation manner of the view generation device is introduced below. Similarly, the implementation manners of the service device and the computing device can refer to the implementation manner of the view generation device.

[0337] As an example of a software functional unit, the view generation apparatus can include code running on a computing instance. The computing instance can include at least one of a physical host (computing device), a virtual machine, and a container. Further, the computing instance can be one or more. For example, the view generation apparatus can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers for running the code can be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers for running the code can be distributed in the same availability zone or in different availability zones, each availability zone including one data center or multiple data centers in a similar geographical location. Generally, one region can include multiple availability zones.

[0338] As an example of a hardware functional unit, the view generation apparatus can include at least one computing device, such as a server. Alternatively, the view generation apparatus can be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0339] It should be noted that in other embodiments, the view generation apparatus can be configured to perform any step of the view management method, the service apparatus can be configured to perform any step of the view management method, the computing apparatus can be configured to perform any step of the view management method, and the service apparatus can be configured to perform any step of the view management method. The steps performed by the view generation apparatus, the service apparatus, and the computing apparatus can be specified as needed, and the view generation apparatus, the service apparatus, and the computing apparatus can be configured to perform different steps of the view management method to implement all functions of the management platform.

[0340] The present application also provides a computing device 1800. As shown in FIG. 18, the computing device 1800 includes a bus 1802, a processor 1804, a memory 1806, and a communication interface 1808. The processor 1804, the memory 1806, and the communication interface 1808 communicate through the bus 1802. The computing device 1800 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1800.

[0341] Bus 1802 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, or the like. For ease of representation, only one bus is represented in FIG. 18, but it does not mean that there is only one bus or only one type of bus. Bus 1804 bus 1802 can include a path for transmitting information between various components (e.g., memory 1806, processor 1804, communication interface 1808) of computing device 1800.

[0342] Processor 1804 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), or the like.

[0343] Memory 1806 can include volatile memory (e.g., random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM)), flash memory, a mechanical hard disk drive (HDD), or a solid-state drive (SSD), or the like.

[0344] Executable code stored in memory 1806 is executed by processor 1804 to respectively implement the functions of the aforementioned view generation apparatus, service apparatus, and computing apparatus, thereby implementing the view management method. That is, memory 1806 has instructions for executing the view management method stored thereon.

[0345] Communication interface 1808 uses a transceiver module such as, but not limited to, a network interface card, a transceiver, or the like, to implement communication between computing device 1800 and other devices or communication networks.

[0346] The embodiments of the present application also provide a computing device cluster. The computing device cluster comprises at least one computing device. The computing device can be a server, for example, a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a notebook computer, or a smart phone.

[0347] As shown in FIG. 19, the computing device cluster comprises at least one computing device 1800. The same instructions for performing the view management method can be stored in the memory 1806 of one or more computing devices 1800 in the computing device cluster.

[0348] In some possible implementations, partial instructions for performing the view management method can also be stored in the memory 1806 of one or more computing devices 1800 in the computing device cluster, respectively. In other words, the combination of one or more computing devices 1800 can collectively execute the instructions for performing the view management method.

[0349] It should be noted that the memories 1806 in different computing devices 1800 in the computing device cluster can store different instructions, respectively, for performing partial functions of the view management system. That is, the instructions stored in the memories 1806 in different computing devices 1800 can implement the functions of one or more of the service apparatus, the view generation apparatus, and the computing apparatus.

[0350] In some possible implementations, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network or a local area network, etc.

[0351] FIG. 20 shows one possible implementation. As shown in FIG. 20, two computing devices 1800A and 1800B are connected through a network. Specifically, the computing devices are connected to the network through the communication interfaces in the computing devices. In this type of possible implementation, the memory 1806 in the computing device 1800A stores instructions for performing the functions of the service apparatus. Meanwhile, the memory 1806 in the computing device 1800B stores instructions for performing the functions of the view generation apparatus and the computing apparatus.

[0352] The connection mode between the computing device cluster shown in FIG. 20 can be that the service apparatus needs to monitor the node status of the storage node 102 according to the view management method provided by the present application, and therefore the service apparatus is separately deployed in 1800A.

[0353] It should be understood that the functions of the computing device 1800A shown in FIG. 20 can also be completed by multiple computing devices 1800. Similarly, the functions of the computing device 1800B can also be completed by multiple computing devices 1800.

[0354] The embodiment of the present application further provides a computer program product containing instructions. The computer program product can be software or program product containing instructions, which can run on a computing device or be stored in any available medium. When the computer program product runs on at least one computing device, the at least one computing device is caused to execute the view management method.

[0355] The embodiment of the present application further provides a computer readable storage medium. The computer readable storage medium can be any available medium that the computing device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk), etc. The computer readable storage medium contains instructions, which instruct the computing device to execute the view management method.

[0356] The above merely describes the specific embodiments of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0357] The embodiment of the present application further provides a computer program product containing instructions. The computer program product can be software or program product containing instructions, which can run on a computing device or be stored in any available medium. When the computer program product runs on at least one computing device, the at least one computing device is caused to execute the view management method.

[0358] The embodiment of the present application further provides a computer readable storage medium. The computer readable storage medium can be any available medium that the computing device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk), etc. The computer readable storage medium contains instructions, which instruct the computing device to execute the view management method.

[0359] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

[0360] Those skilled in the art can clearly understand the specific working process of the system, management platform or unit described above, which can refer to the corresponding process in the foregoing method embodiments, and will not be described here.

[0361] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A distributed storage system, characterized by, The distributed storage system comprises N storage nodes and a computing node, N being a positive integer, the N storage nodes comprising a first view generation node; The first view generation node is configured to determine M view push nodes from the N storage nodes, M being a positive integer less than or equal to N; The first view generation node is configured to send view information to the M view push nodes, the view information being used to indicate a rule of distributing data to the N storage nodes; The computing node is configured to determine a first view push node from the M view push nodes, request the view information from the first view push node, and perform data writing or data reading on the N storage nodes based on the requested view information.

2. The distributed storage system of claim 1, wherein The first view generation node is configured to determine an association relationship between a second view push node and H view consumption nodes, the H view consumption nodes being storage nodes other than the first view generation node and the M view push nodes in the N storage nodes, H being a positive integer less than N; The second view push node is any one of the M view push nodes; The first view generation node is configured to send the association relationship to the second view push node; The second view push node is configured to determine a view consumption node associated with the second view push node based on the association relationship, and send the view information to the associated view consumption node.

3. The distributed storage system of claim 1 or 2, wherein, The distributed storage system further comprises a service node in communication with the N storage nodes; The first view generation node is configured to determine view push node information, the view push node information being used to indicate the M view push nodes; The first view generation node is configured to send the view push node information to the service node; The computing node is configured to request the view push node information from the service node; The computing node is configured to determine a first view push node from the M view push nodes based on the view push node information.

4. The distributed storage system of claim 2, wherein The service node is configured to send node states of the N storage nodes to the first view generation node; The first view generation node is configured to determine view information based on the node states of the N storage nodes and a data distribution rule.

5. The distributed storage system of claim 3 or 4, wherein The service node is configured to send updated node states of target storage nodes in the N storage nodes to the first view generation node after determining the updated node states of the target storage nodes.

6. The distributed storage system of any one of claims 3 to 5, wherein The first view generation node is configured to send the view information to the service node; The service node is configured to store the view information.

7. The distributed storage system of any one of claims 1 to 6, wherein The first view generation node is configured to determine that a third view push node in the M view push nodes fails, select a fourth view push node from the storage nodes other than the first view generation node and the M view push nodes in the N storage nodes to replace the third view push node, and obtain updated view push node information. The first view generation node is configured to send the updated view push node information to the service node. The first view generation node is configured to send the view information to the fourth view push node.

8. The distributed storage system of any of claims 1 to 7, wherein, The distributed storage system further comprises a second view generation node. In the case where the first view generation node fails, the second view generation node is configured to read the view information and the view push node information from the service node.

9. The distributed storage system of any of claims 1 to 8, wherein The computing node is configured to determine that the request for the view information from the first view push node fails, determine a fifth view push node based on the view push node information, and request the view information from the fifth view push node.

10. The distributed storage system of claim 9, wherein, The method further comprises: The computing node is configured to determine that the request for the view information fails, request the latest view push node information from the service node, determine a sixth view push node based on the latest view push node information, and request the latest view information from the sixth view push node.

11. A view management method characterized by, The method is applied to a distributed storage system, the distributed storage system comprising N storage nodes and a computing node, the N storage nodes comprising a first view generation node, and the method comprising: The first view generation node determines M view push nodes from the N storage nodes, M being a positive integer less than or equal to N, and sends view information to the M view push nodes, the view information being used to indicate a rule of distributing data to the N storage nodes; The computing node determines a first view push node from the M view push nodes, requests the view information from the first view push node, and performs data writing or data reading on the N storage nodes based on the requested view information.

12. The method of claim 11, wherein, The method further comprises: The first view generation node determines an association relationship between a second view push node and H view consumption nodes, the H view consumption nodes being storage nodes other than the first view generation node and the M view push nodes in the N storage nodes, H being a positive integer less than N, and the second view push node being any one of the M view push nodes; The first view generation node sends the association relationship to the second view push node; The second view push node determines, based on the association relationship, a view consumption node associated with the second view push node, and sends the view information to the associated view consumption node.

13. The method according to claim 11 or 12, characterized in that, The method further comprises: The first view generation node determines view push node information, the view push node information being used to indicate the M view push nodes; The first view generation node sends the view push node information to the service node; The computing node determines a first view push node from the M view push nodes based on the view push node information. The computing node requests the view push node information from the service node; The computing node determines a first view push node from the M view push nodes based on the view push node information.

14. The method of claim 13, wherein, The method further comprises: The service node sends the node state of the N storage nodes to the first view generation node; The first view generation node determines view information based on the node state of the N storage nodes and data distribution rules.

15. The method according to claim 13 or 14, characterized in that, The method further comprises: The service node sends the updated node state of the target storage node to the first view generation node after determining the node state update of the target storage node in the N storage nodes.

16. The method according to any one of claims 13 to 15, characterized in that, The method further comprises: The first view generation node sends the view information to the service node; The service node stores the view information.

17. The method of any one of claims 11 to 16, wherein, The method further comprises: The first view generation node determines that a third view push node in the M view push nodes fails, and selects a fourth view push node from the storage nodes other than the first view generation node and the M view push nodes in the N storage nodes to replace the third view push node, to obtain updated view push node information; The first view generation node sends the updated view push node information to the service node; The first view generation node sends the view information to the fourth view push node.

18. The method of any one of claims 11 to 17, wherein, The distributed storage system further comprises a second view generation node, and the method further comprises: In the case of failure of the first view generation node, the second view generation node reads the view information and the view push node information from the service node.

19. The method of any one of claims 11 to 18, wherein, The method further comprises: The computing node determines that the request for the view information to the first view push node fails, determines a fifth view push node based on the view push node information, and requests the view information from the fifth view push node.

20. The method of claim 19, wherein, The method further comprises: The computing node determines that the request for the view information fails, requests the latest view push node information from the service node, determines a sixth view push node based on the latest view push node information, and requests the latest view information from the sixth view push node.

21. A cluster of computing devices, characterized in that, The computing device cluster comprises a plurality of computing devices, the plurality of computing devices comprising a plurality of processors and a plurality of memories, the plurality of memories storing program instructions, and the plurality of processors executing the program instructions to cause the computing device cluster to perform the method of any one of claims 11-20.

22. A computer program product comprising instructions, wherein: The instructions, when executed by the computer device cluster, cause the computer device cluster to perform the method of any one of claims 11-20.

23. A computer-readable storage medium, characterized in that, The computer program instructions, when executed by the computing device cluster, cause the computing device cluster to perform the method of any one of claims 11-20.

Images