A data processing method and related equipment

Abstract

The present application provides a data processing method and related equipment, which determines the health status of nodes in a distributed cluster by means of data calculation, and then performs corresponding operations according to the health status of the nodes to avoid the impact of cloud-edge network fluctuations on services. The method includes: the target node obtains the first initial health status of each node in the first node set; the target node obtains the second initial health status of each node in the target group; the target node determines the first target health status of each node in the target group according to the first initial health status and the second initial health status; the target node sends the first target health status of each node in the target group to the server, so that the server performs corresponding operations according to the first target health status and the second target health status of each node in the target group, and the second target health status is the health status of each node in the target group obtained by the cloud.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular to a data processing method and related equipment. Background Art

[0002] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0003] Unlike the central cloud, in edge scenarios, the first thing to face is the weak network environment at the edge of the cloud. Edge devices are often located in edge cloud computer rooms and mobile edge sites. The network environment connected to the cloud is very complex and not as reliable as the central cloud. This includes unreliable network environments at the cloud (control end) and edge end, as well as unreliable network environments between edge nodes. Even between different computer rooms in the same area, it is impossible to assume that the network quality between nodes is good.

[0004] The problem caused by weak cloud-edge network is that it affects the communication between the kubelet running on the edge node and the cloud application programming interface server (APIServer). The cloud APIServer cannot receive the heartbeat or lease renewal of the kubelet, and cannot accurately obtain the running status of the node and the pod on the node. If the duration exceeds the set threshold, the native AIServer will consider the node unavailable and take the following actions: the state of the lost node is set to NotReady or Unknown, and NoSchedule and NoExecute taints are added; the pod on the lost node is evicted and rebuilt on other nodes; the pod on the lost node is removed from the Endpoint list of the Service.

[0005] However, in the case of native Kubernetes, if Pods are frequently rebuilt due to network fluctuations, it will affect the cache effect of the service instance on the one hand, and cause the scheduling system to schedule user requests to other service instances on the other hand. Undoubtedly, these two points will have a great impact on the performance of the Content Delivery Network (CDN), and even be unacceptable.

[0006] Invention Information

[0007] The present application provides a data processing method and related equipment, which can determine the health status of nodes in a distributed cluster, and then perform corresponding operations based on the health status of the nodes, thereby avoiding the impact of cloud-edge network fluctuations on services.

[0008] The first aspect of the present application provides a data processing method, comprising:

[0009] The target node obtains a first initial health state of each node in a first node set, where the first node set is a set of nodes in a target group except the target node, the target node is any node in the target group, and the target group is any group in a distributed cluster;

[0010] The target node obtains a second initial health state of each node in the target group, the second initial health state is the health state of each node in the second node set obtained by the first target node, the second node set is a set of other nodes in the target group except the first target node, and the first target node is any node in the target group except the target node;

[0011] The target node determines a first target health state of each node in the target group according to the first initial health state and the second initial health state;

[0012] The target node sends the first target health status of each node in the target group to the server, so that the server performs corresponding operations according to the first target health status and the second target health status of each node in the target group, where the second target health status is the health status of each node in the target group obtained from the cloud.

[0013] Optionally, the target node acquiring the first initial health status of each node in the first node set includes:

[0014] The target node sends a target request to the target interface of each node in the first node set to obtain a health value returned by each node in the first node set;

[0015] The target node determines a first initial health state of each node in the first set of nodes according to a health value returned by each node in the first set of nodes.

[0016] Optionally, the target node determining, according to the health value returned by each node in the first node set, the first initial health state of each node in the first node set includes:

[0017] When the health value of the second target node is normal, the target node determines that the first initial health state of the second target node is normal, and the second target node is any one node in the first node set;

[0018] When the health value of the second target node is abnormal, the target node determines that the first initial health state of the second target node is abnormal.

[0019] A second aspect of the present application provides a data processing method, comprising:

[0020] The server receives a first target health status of each node in a target group sent by a target node, the target group being any group in a distributed cluster, the first target health status of each node in the target group being determined by the target node according to a first initial health status and a second initial health status, the first initial health status being the health status of each node in a first node set acquired by the target node, the first node set being a set of other nodes in the target group except the target node, the second initial health status being the health status of each node in a second node set received by the target node from the first target node, and the second initial health status being the health status of each node in the second node set acquired by the first target node, the second node set being a set of other nodes in the target group except the first target node, and the first target node being any node in the target group except the target node;

[0021] The server receives a second target health status of each node in the target group sent by the cloud;

[0022] The server performs corresponding operations on each node in the target group according to the first target health status and the second target health status.

[0023] Optionally, the server performing corresponding operations on each node in the target group according to the first target health status and the second target health status includes:

[0024] When the first target health state of the third target node is normal and the second target health state of the third target node is abnormal, the server stops scheduling resource objects to the third target node, and the third target node is any node in the target group;

[0025] When the first target health state of the third target node is abnormal and the second target health state of the third target node is abnormal, the server evicts the resource objects in the third target node, removes the resource objects of the third target node from the resource object list, and stops scheduling target resource objects to the third target node, wherein the resource object list stores resource objects of multiple nodes including the resource object of the third node;

[0026] When the first target health status of the third target node is normal and the second target health status of the third target node is normal, the server does not perform any operation on the third target node.

[0027] A third aspect of the present application provides a node, where the node is a target node, including:

[0028] a first acquisition unit, configured to acquire a first initial health state of each node in a first node set, wherein the first node set is a set of nodes in a target group except the target node, the target node is any node in the target group, and the target group is any group in a distributed cluster;

[0029] A second acquisition unit is used to acquire a second initial health state of each node in the target group, where the second initial health state is the health state of each node in a second node set acquired by the first target node, where the second node set is a set of other nodes in the target group except the first target node, and the first target node is any node in the target group except the target node;

[0030] a determining unit, configured to determine a first target health state of each node in the target group according to the first initial health state and the second initial health state;

[0031] A sending unit is used to send the first target health status of each node in the target group to the server, so that the server performs corresponding operations according to the first target health status and the second target health status of each node in the target group, where the second target health status is the health status of each node in the target group obtained from the cloud.

[0032] Optionally, the first acquiring unit is specifically configured to:

[0033] The target node sends a target request to the target interface of each node in the first node set to obtain the health value returned by each node in the first node set;

[0034] The target node determines a first initial health state of each node in the first set of nodes according to a health value returned by each node in the first set of nodes.

[0035] Optionally, the first acquiring unit determining the first initial health state of each node in the first node set according to the health value returned by each node in the first node set includes:

[0036] When the health value of the second target node is normal, the target node determines that the first initial health state of the second target node is normal, and the second target node is any one node in the first node set;

[0037] When the health value of the second target node is abnormal, the target node determines that the first initial health state of the second target node is abnormal.

[0038] A fourth aspect of the present application provides a server, including:

[0039] A receiving unit, used for receiving a first target health state of each node in a target group sent by a target node, the target group being any group in a distributed cluster, the first target health state of each node in the target group being determined by the target node according to a first initial health state and a second initial health state, the first initial health state being the health state of each node in a first node set acquired by the target node, the first node set being a set of other nodes in the target group except the target node, the second initial health state being the health state of each node in a second node set received by the target node from the first target node, and the second initial health state being the health state of each node in the second node set acquired by the first target node, the second node set being a set of other nodes in the target group except the first target node, and the first target node being any node in the target group except the target node;

[0040] The receiving unit is further used to receive a second target health status of each node in the target group sent by the cloud;

[0041] An execution unit is used to perform a corresponding operation on each node in the target group according to the first target health status and the second target health status.

[0042] Optionally, the execution unit is specifically used for:

[0043] When the first target health state of the third target node is normal and the second target health state of the third target node is abnormal, stop scheduling resource objects to the third target node, the third target node being any node in the target group;

[0044] When the first target health state of the third target node is abnormal and the second target health state of the third target node is abnormal, the resource objects in the third target node are evicted, the resource objects of the third target node are removed from the resource object list, and the scheduling of target resource objects to the third target node is stopped, and the resource object list stores resource objects of multiple nodes including the resource object of the third node;

[0045] When the first target health status of the third target node is normal and the second target health status of the third target node is normal, no operation is performed on the third target node.

[0046] A third aspect of the present application provides a computer device comprising at least one connected processor, memory and transceiver, wherein the memory is used to store program code, and the program code is loaded and executed by the processor to implement the steps of the data processing method described above.

[0047] A fourth aspect of the present application provides a computer-readable storage medium, which includes instructions, which, when executed on a computer, enable the computer to execute the steps of the data processing method described above.

[0048] In summary, it can be seen that in the embodiment provided by the present application, the target node obtains the first initial health state and the second initial health state of each node in the target group, and determines the first target health state of each node in the target group based on the first initial health state and the second initial health state, and sends the first target health state to the server, and the server performs corresponding operations based on the first target health state and the second target health state of each node in the target group obtained from the cloud. The health state of the nodes in the distributed cluster is determined by combining the cloud and the edge, which greatly improves the accuracy of node health state judgment in the case of weak cloud-edge network, makes the system more robust, and avoids the impact of cloud-edge network fluctuations on services. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of an application scenario of the data processing method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of another application scenario of the data processing method provided in an embodiment of the present application;

[0051] Figure 3 A technical flow diagram of a data processing method provided in an embodiment of the present application;

[0052] Figure 4 Another technical flow diagram of the data processing method provided in the embodiment of the present application;

[0053] Figure 5 Another technical flow diagram of the data processing method provided in the embodiment of the present application;

[0054] Figure 6 A schematic diagram of a virtual structure of a network node provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of a virtual structure of a server provided in an embodiment of the present application;

[0056] Figure 8 A schematic diagram of the hardware structure of a network node provided in an embodiment of the present application;

[0057] Fig. 9 A schematic diagram of the hardware structure of the server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0059] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described here can be implemented in an order other than the information illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules that appear in this application is only a logical division. There may be other division methods when implemented in actual applications, such as multiple modules can be combined or integrated into another system, or some feature vectors can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purpose of the present application.

[0060] The following are some of the terms involved in this application:

[0061] Kubernetes is a distributed cluster management system. On each node, a worker program must be run to manage the life cycle of the container. The worker program is kubelet.

[0062] The main function of kubelet is to periodically obtain the expected status of the Pod / container on the node (what container is running, the number of running replicas, how the network or storage is configured, etc.) from somewhere, and call the corresponding container platform interface to achieve this status.

[0063] The API Server provides HTTP REST interfaces such as add, delete, modify, query, and watch for various resource objects (Pod, RC, and Service, etc.), and is the data bus and data center of the entire system.

[0064] Unlike the central cloud, in edge scenarios, the first thing to face is the weak network environment at the edge of the cloud. Edge devices are often located in edge cloud data centers or mobile edge sites, and the network environment connected to the cloud is very complex and not as reliable as the central cloud. This includes unreliable network environments at the cloud (control end) and edge end, as well as unreliable network environments between edge nodes. Even between different data centers in the same area, it is impossible to assume that the network quality between nodes is good.

[0065] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of the data processing method provided in an embodiment of the present application, taking a smart factory as an example, the network between the cloud cluster 101 and the warehouse 102 and the workshop 103 is a weak network, the network between the warehouse 102 and the workshop 103 is a weak network, the edge nodes are located in the factory warehouse 102 and the workshop 103, and the control end master node is in the central computer room of Tencent Cloud.

[0066] The network between the edge devices in the warehouse 102 and the workshop 103 and the cloud cluster 101 is relatively complex, including the Internet, the fifth generation mobile communication technology (5th generation mobile networks or 5th generation wireless systems, 5th-Generation, 5G), WIFI and other forms, and the network quality is uneven and not guaranteed; at the same time, compared with the cloud network environment, since the edge devices in the warehouse 102 and the workshop 103 are connected via a local network, the network quality is definitely better than the connection between the cloud cluster and is relatively more reliable.

[0067] The problem caused by the weak cloud-edge network is that it affects the communication between the kubelet running on the edge node and the cloud APIServer. The cloud APIServer 101 cannot receive the kubelet heartbeat or lease renewal of the nodes in the warehouse 102 and the workshop 103, and cannot accurately obtain the running status of the nodes in the warehouse 102 or the workshop 103 and the Pod on the node. If the condition lasts longer than the set threshold, the cloud APIServer 102 will consider the node unavailable and take the following actions:

[0068] The status of the disconnected node is set to NotReady or Unknown, and NoSchedule and NoExecute taints are added; the Pod on the disconnected node is evicted and rebuilt on other nodes; the Pod on the disconnected node is removed from the Endpoint list of the Service.

[0069] Combine the following Figure 2The reconstruction of Pod is explained by taking the audio and video streaming scenario as an example. Figure 2 , Figure 2 Another scenario schematic diagram of the data processing method provided in the embodiment of the present application includes:

[0070] Cloud clusters 201A and 201B, edge CDN nodes 202A and 202B, edge CDN nodes 203A and 203B, and video clients 204A and 204B;

[0071] Based on user experience and cost considerations, audio and video streaming often requires improving edge cache hit rates and reducing back-to-source requests. It is common practice to schedule the same file requested by a user to the same service instance and for the service instance to cache the file.

[0072] However, in the case of native Kubernetes, if Pods are frequently rebuilt due to network fluctuations, such as Figure 2 Pod A in edge CDN node 202A is disconnected due to weak network, and reconnects after the cloud edge is disconnected. Due to the impact of the network disconnection, after the weak network is reconnected, Pod A in edge CDN node 202A will be destroyed and rebuilt, and Pod B in edge CDN node 203B will be connected to video client 204B. However, doing so will affect the cache effect of the service instance on the one hand, and will cause the scheduling system to schedule user requests to other service instances on the other hand.

[0073] In fact, if the edge node is fully operational, Pod eviction or reconstruction is completely unnecessary. In order to overcome this problem and maintain the continuous availability of services, it is not reasonable to rely solely on the connection between the edge and the cloud API Server to determine whether the edge node is normal. In order to make the system more robust, an additional judgment mechanism needs to be introduced.

[0074] In view of this, the present application provides a data processing method, which uses the health status judgment technology and mechanism of edge distributed nodes in a distributed cluster. In addition to considering the connection status between the node and the cloud API Server, it also introduces the judgment status of the edge node as an evaluation factor to make a more comprehensive status judgment on the node. In this way, the accuracy of the system in node status judgment is greatly improved in the case of weak cloud-edge network, thereby ensuring the stable operation of the service.

[0075] In addition, due to the particularity of edge networks and topology, there are often single point failures between node groups, such as the above Figure 1In the example shown in , although warehouse 102 and workshop 103 both belong to the factory area, the network connection between them may rely on a key link. Once this link is interrupted, it will cause a split between the node groups. The data processing method provided in the present application can ensure that after the two split node groups lose connection, when they determine the health status of the nodes in the group, the majority of the nodes will not be judged as abnormal, thereby avoiding the situation where the Pod can only be scheduled to a small number of nodes due to being judged as abnormal, resulting in excessive node load.

[0076] Edge devices are likely to be located in different regions and not connected to each other. The data processing method provided by this application also supports node status determination in multiple regions. It can easily group nodes according to region or other methods to implement inspection within the group; and even if regrouping, there is no need to redeploy detection components or reinitialize, which is more suitable for edge computing network conditions. After grouping, nodes will only determine the status of nodes in the same group. If nodes are not grouped, it is assumed that each node is a group by itself and will not check other nodes.

[0077] Combine the following Figure 3 The data processing method provided in the embodiment of the present application is described from the perspective of the target node.

[0078] See also Figure 3 , Figure 3 A flow chart of a data processing method provided in an embodiment of the present application includes:

[0079] 301. The target node obtains a first initial health status of each node in a first node set.

[0080] In this embodiment, the target node can obtain the first initial health status of each node in the first node set, wherein the first node set is a set of nodes in the target group except the target node, the target node is any node in the target group, and the target group is any group in the distributed cluster. That is to say, for each group in the distributed Kubernetes cluster, each node in each group periodically detects the first initial health status of other nodes in the group.

[0081] In one embodiment, the target node obtains the first initial health status of each node in the first node set, including:

[0082] The target node sends a target request to the target interface of each node in the first node set to obtain the health value returned by each node in the first node set;

[0083] The target node determines a first initial health state of each node in the first node set according to the health value returned by each node in the first node set.

[0084] In this embodiment, each node in the target group obtains the health value returned by each node by sending a request to the target interface of other nodes in the target group, and determines its corresponding health status according to the health value. When the health value of the second target node is normal, the target node determines that the first initial health state of the second target node is normal, and the second target node is any node in the first node set; when the health value of the second target node is abnormal, the target node determines that the first initial health state of the second target node is abnormal. That is to say, the target node can obtain the kubelet health status of each node in the first node set as the health status of each node by sending a request to the 10255 / healthz interface (that is, the target interface) of the kubelet of each node in the first node set; in addition, the target node can also ping the 22 port of each node in the first node set, and use the request return value as the judgment standard; it can be understood that the standard for determining the health status of the node is: if the request return value is normal, the health status of the node is considered normal, and if the request return value is abnormal, the health status of the node is considered abnormal.

[0085] 302. The target node obtains a second initial health status of each node in the target group.

[0086] In this embodiment, the target node can obtain the second initial health state of each node in the target group, and the second initial health state is the health state of each node in the second node set obtained by the first target node, and the second node set is the set of other nodes in the target group except the first target node, and the first target node is any node in the target group except the target node. That is to say, in addition to obtaining the first initial health state of other nodes by itself, each node in the target group will also obtain the second initial health state from other nodes in the target group, and the second initial health state is the health state of the nodes in the target group obtained by other nodes in the target group. For example, the target group includes nodes A, B, C and D, and node A will obtain the health state of nodes B, C and D. Similarly, node B will obtain the health state of nodes A, C and D, node C will obtain the health state of nodes A, B and D, node D will obtain the health state of nodes A, B and C, and in addition, node A will also obtain the health state of nodes in the target group obtained by nodes B, C and D, and so on, and so on, and so on. It is understandable that the target node may periodically obtain the second initial health status of each node in the target group, or may obtain it on demand, without specific limitation.

[0087] It should be noted that, through step 301, the first initial health status of each node in the first node set can be obtained, and through step 302, the second initial health status of each node in the target group can be obtained. However, there is no restriction on the execution order of these two steps. Step 301 can be executed first, or step 302 can be executed first, or they can be executed simultaneously, without specific limitation.

[0088] 303. The target node determines a first target health state of each node in the target group according to the first initial health state and the second initial health state.

[0089] In this embodiment, after the target node obtains the first initial health state of each node in the first node set and the second initial health state of each node in the target group, the target node determines the first target health state of each node in the target group based on the first initial health state and the second initial health state. That is to say, all nodes in each group in the distributed cluster periodically vote to determine the health state of the nodes in each group: through the first initial health state and the second initial health state obtained in step 301 and step 302, each node in the target group saves the health state of each node determined from its own perspective (the node's own perspective is the node's own opinion), and other nodes in the target group will also interact with each other to obtain their own health states. The first initial health status of other nodes is calculated, and the second initial health status of the nodes detected by other nodes is counted, that is, the first target health status of all nodes in the target group can be selected, for example, it can be determined by a preset value, for example, the target group includes node A, node B, node C and node D, among which, there are 2 nodes that think the health status of node A is abnormal, 3 nodes that think the health status of node B is abnormal, 0 nodes that think the health status of node C is abnormal, and 1 node that thinks the health status of node D is abnormal. If the preset value is 3, the node with the first target health status of abnormal in the target group is node B, and the nodes with the first target health status of normal in the target group are nodes A, C and D.

[0090] 304. The target node sends the first target health status of each node in the target group to the server, so that the server performs corresponding operations according to the first target health status and the second target health status of each node in the target group.

[0091] In this embodiment, after determining the first target health status of each node in the target group, the target node can send the first target health status of each node in the target group to the server, so that the service performs corresponding operations based on the first target health status of each node in the target group obtained from the target node and the second target health status of each node in the target group obtained from the cloud.

[0092] In summary, it can be seen that in the embodiment provided by the present application, the target node obtains the first initial health state and the second initial health state of each node in the target group, and determines the first target health state of each node in the target group based on the first initial health state and the second initial health state, and sends the first target health state to the server, and the server performs corresponding operations based on the first target health state and the second target health state of each node in the target group obtained from the cloud. The health state of the nodes in the distributed cluster is determined by combining the cloud and the edge, which greatly improves the accuracy of node health state judgment in the case of weak cloud-edge network, makes the system more robust, and avoids the impact of cloud-edge network fluctuations on services.

[0093] Combine the following Figure 4 The data processing method provided in the embodiment of the present application is described from the perspective of the server. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0094] See also Figure 4 , Figure 4 Another flowchart of the data processing method provided in the embodiment of the present application includes:

[0095] 401. The server receives a first target health status of each node in a target group sent by a target node.

[0096] In this embodiment, the server can receive the first target health status of each node in the target group sent by the target node, wherein the target group is any group in the distributed cluster, the first target health status of each node in the target group is determined by the target node based on the first initial health status and the second initial health status, the first initial health status is the health status of each node in the first node set obtained by the target node, the first node set is the set of other nodes in the target group except the target node, the second initial health status is the health status of each node in the second node set received by the target node from the first target node, and the second initial health status is the health status of each node in the second node set obtained by the first target node, the second node set is the set of nodes in the target group except the first target node, and the first target node is any node in the target group except the target node.

[0097] It should be noted that the above Figure 3 How the target node determines the health status of the first target has been described in detail, and will not be repeated here.

[0098] 402. The server receives a second target health status of each node in the target group sent by the cloud.

[0099] In this embodiment, the server can receive the second target health status of each node in the target group sent by the cloud. That is to say, the cloud API Server cannot receive the heartbeat or lease renewal of the node kubelet, and cannot accurately obtain the operation status of the node and the Pod on the node. If the duration exceeds the set threshold, the cloud API Server will consider the health status of the node to be abnormal, otherwise the health status of the node is normal. The server can send a request message to the cloud, and then the cloud returns the second target health status of each node in the target group obtained.

[0100] It should be noted that, through step 401, the first target health status of each node in the target group sent by the target node can be received, and through step 402, the second target health status of each node in the target group sent by the cloud can be received. However, there is no restriction on the order of execution between these two steps. Step 401 can be executed first, or step 402 can be executed first, or they can be executed at the same time, without specific limitation.

[0101] 403. The server performs corresponding operations on each node in the target group according to the first target health status and the second target health status.

[0102] In this embodiment, after receiving the first target health status and the second target health status of each node in the target group, the server can perform corresponding operations on each node in the target group according to the first target health status and the second target health status. In other words, the server can determine the final target state of each node in the target group according to the first target health status and the second target health status of each node in the target group, and then perform corresponding operations according to the final target state of each node in the target group.

[0103] In one embodiment, the server performs corresponding operations on each node in the target group according to the first target health status and the second target health status, including:

[0104] When the first target health state of the third target node is normal and the second target health state of the third target node is abnormal, the server stops scheduling resource objects to the third target node, and the third target node is any node in the target group;

[0105] When the first target health state of the third target node is abnormal, and the second target health state of the third target node is abnormal, the server evicts the resource object in the third target node, removes the resource object of the third target node from the resource object list, and stops scheduling the target resource object to the third target node, and the resource object list stores resource objects of multiple nodes including the resource object of the third node;

[0106] When the first target health status of the third target node is normal, and the second target health status of the third target node is normal, the server does not perform any operation on the third target node.

[0107] In this embodiment, the nodes in the target group conduct detection and voting to jointly determine whether each node in the target group has an abnormal state. The specific state of the node can only be determined by the consistent judgment of most nodes. In addition, although the network state between nodes is generally better than the cloud-edge network, it should be noted that the network situation between edge nodes is also very complicated, and the network between them is not 100% reliable. The state of the node cannot be determined by the node alone. It is more reliable to jointly determine the cloud-edge. Since the cloud and edge jointly determine the health status of the node, there will be three situations:

[0108] 1. The first target health status of the third target node is normal, and the second target health status of the third target node is abnormal;

[0109] 2. The first target health status of the third target node is abnormal, and the second target health status of the third target node is abnormal;

[0110] 3. When the first target health status of the third target node is normal, the second target health status of the third target node is normal.

[0111] For the first case, when the cloud determines that the node is abnormal, but other nodes believe that the node is normal, although the existing Pods will not be evicted, in order to ensure the stability of the incremental service, no new Pods will be scheduled to the node. The normal operation of the existing Pods also benefits from the edge autonomy of the edge cluster; for the second case, the server will evict the existing Pods in the third target node; and remove the Pods of the third target node from the Endpoint list, and no new Pods will be scheduled to the third target node. For the third case, the server does not perform any operation on the third target node.

[0112] It should be noted that the method in which the above-mentioned node and the cloud determine whether the node is normal can also be set according to actual conditions. For example, if the node is judged to be normal internally and the cloud determines that the node is abnormal, the final state of the node can be judged to be normal. If the node is judged to be abnormal internally and the cloud determines that the node is normal, the final health state of the node can be judged to be normal, then no operation is performed, or no new Pod is scheduled to the node. When the node is judged to be abnormal internally and the cloud also determines that the node is abnormal, the node is finally judged to be abnormal, and the existing Pods in the node are expelled; the Pods of the node are removed from the Endpoint list, and no new Pods are scheduled to the node.

[0113] Take the target group including four nodes A, B, C, and D as an example, and take the determination of the final target state of node A as an example for explanation. For example, if the first target health state of node A is normal and the second target health state of node A is abnormal, then the final health state of node A is determined to be abnormal. If the first target health state of node A is abnormal and the second target health state of node A is normal, then the final target health state of node A is determined to be abnormal. Only when the first target health state and the second target health state of node A are both normal, the final target health state of node A is determined to be normal. It should be noted that the above description of the final target health state of node A is only an example, and of course it can also be set according to actual conditions. For example, only one normal (for example, the first target health state is normal or the second target health state is normal) is needed to determine that the final target health state of the node is normal. Of course, it can also be based on the first target health state or the second target health state, and there is no specific limitation. In addition, when determining that a node is normal or abnormal, different operations can be performed according to actual conditions.

[0114] In summary, it can be seen that in the embodiment provided by the present application, the server performs corresponding operations according to the first target health status of each node in the target group sent by the node end and the second target health status of each node in the target group sent by the cloud end. The health status of the nodes in the distributed cluster is comprehensively determined by the cloud end and the edge end, which greatly improves the accuracy of node health status judgment in the case of weak cloud-edge network, makes the system more robust, and avoids the impact of cloud-edge network fluctuations on services.

[0115] See also Figure 5 , Figure 5 Another flowchart of the data processing method provided in the embodiment of the present application includes:

[0116] 501. The target node obtains a first initial health state of each node in the first node set;

[0117] 502. The target node obtains a second initial health status of each node in the target group;

[0118] 503. The target node determines a first target health state of each node in the target group according to the first initial health state and the second initial health state;

[0119] 504. The target node sends the first target health status of each node in the target group to the server;

[0120] It should be noted that steps 501 to 504 are Figure 3 Steps 301 to 304 in the above are similar. Figure 3 The details have been explained in detail in the article, so I will not repeat them here.

[0121] 505. The server receives the second target health status of each node in the target group sent by the cloud;

[0122] 506. The server performs corresponding operations according to the first target health status and the second target health status of each node in the target group.

[0123] It should be noted that steps 505 to 506 are Figure 4 Steps 402 to 403 in the above are similar. Figure 4 The details have been explained in detail in the article, so I will not repeat them here.

[0124] In summary, it can be seen that in the embodiment provided by the present application, the target node obtains the first initial health state and the second initial health state of each node in the target group, and determines the first target health state of each node in the target group based on the first initial health state and the second initial health state, and sends the first target health state to the server, and the server performs corresponding operations based on the first target health state and the second target health state of each node in the target group obtained from the cloud. The health state of the nodes in the distributed cluster is determined by combining the cloud and the edge, which greatly improves the accuracy of node health state judgment in the case of weak cloud-edge network, makes the system more robust, and avoids the impact of cloud-edge network fluctuations on services.

[0125] The above describes the present application from the perspective of a data processing method, and the following describes the present application from the perspective of a node and a server.

[0126] See also Figure 6 , Figure 6 A virtual structure diagram of a network node provided in an embodiment of the present application, the network node is a target node, including:

[0127] A first acquisition unit 601 is used to acquire a first initial health state of each node in a first node set, where the first node set is a set of nodes in a target group except the target node, the target node is any node in the target group, and the target group is any group in a distributed cluster;

[0128] A second acquisition unit 602 is used for a second initial health state of each node in the target group, where the second initial health state is the health state of each node in a second node set acquired by the first target node, where the second node set is a set of other nodes in the target group except the first target node, and the first target node is any node in the target group except the target node;

[0129] A determining unit 603, configured to determine a first target health state of each node in the target group according to the first initial health state and the second initial health state;

[0130] The sending unit 604 is used to send the first target health status of each node in the target group to the server, so that the server performs corresponding operations according to the first target health status and the second target health status of each node in the target group, and the second target health status is the health status of each node in the target group obtained from the cloud.

[0131] Optionally, the first acquiring unit 601 is specifically configured to:

[0132] The target node sends a target request to the target interface of each node in the first node set to obtain a health value returned by each node in the first node set;

[0133] The target node determines a first initial health state of each node in the first node set according to the health value returned by each node in the first node set.

[0134] Optionally, the first acquiring unit 601 determines the first initial health state of each node in the first node set according to the health value returned by each node in the first node set, including:

[0135] When the health value of the second target node is normal, the target node determines that the first initial health state of the second target node is normal, and the second target node is any one node in the first node set;

[0136] When the health value of the second target node is abnormal, the target node determines that the first initial health state of the second target node is abnormal.

[0137] See also Figure 7 , Figure 7 A virtual structure diagram of a server provided in an embodiment of the present application includes:

[0138] A receiving unit 701 is used to receive a first target health state of each node in a target group sent by a target node, the target group is any group in a distributed cluster, the first target health state of each node in the target group is determined by the target node according to a first initial health state and a second initial health state, the first initial health state is the health state of each node in a first node set acquired by the target node, the first node set is a set of other nodes in the target group except the target node, the second initial health state is the health state of each node in a second node set received by the target node from the first target node, the second initial health state is the health state of each node in a second node set acquired by the first target node, the second node set is a set of other nodes in the target group except the first target node, and the first target node is any node in the target group except the target node;

[0139] The receiving unit 701 is further configured to receive a second target health status of each node in the target group sent by the cloud;

[0140] The execution unit 702 is used to perform a corresponding operation on each node in the target group according to the first target health status and the second target health status.

[0141] Optionally, the execution unit 702 is specifically configured to:

[0142] When the first target health state of the third target node is normal and the second target health state of the third target node is abnormal, stop scheduling resource objects to the third target node, the third target node being any node in the target group;

[0143] When the first target health state of the third target node is abnormal and the second target health state of the third target node is abnormal, the resource objects in the third target node are evicted, the resource objects of the third target node are removed from the resource object list, and the scheduling of target resource objects to the third target node is stopped, and the resource object list stores resource objects of multiple nodes including the resource object of the third node;

[0144] When the first target health status of the third target node is normal and the second target health status of the third target node is normal, no operation is performed on the third target node.

[0145] See also Figure 8 In the present application, the network node includes one or more central processors 801, a memory 802 and a communication interface 803, wherein the central processor 801, the memory 802 and the communication interface 803 are interconnected via a bus.

[0146] The memory 802 may be a temporary storage or a permanent storage for storing related instructions and data, and the communication interface 803 is used to receive and send data. Furthermore, the CPU 801 may be configured to communicate with the memory 802 and execute a series of instruction operations in the memory 802 .

[0147] The target nodes in the above embodiments can all be based on the Figure 8 The structure shown.

[0148] See also Fig. 9 , Fig. 9A schematic diagram of a server structure provided for an embodiment of the present application, the server 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 922 (for example, one or more processors) and a memory 932, and one or more storage media 930 (for example, one or more mass storage devices) storing application programs 942 or data 944. Among them, the memory 932 and the storage medium 930 can be short-term storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 922 can be configured to communicate with the storage medium 930 and execute a series of instruction operations in the storage medium 930 on the server 900.

[0149] The server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and / or one or more operating systems 941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0150] The steps performed by the server in the above embodiment can be based on the Fig. 9 The server structure shown.

[0151] An embodiment of the present application also provides a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, the steps of the above-mentioned data processing method are implemented.

[0152] An embodiment of the present application further provides a processor, which is used to run a program, wherein the program executes the steps of the above-mentioned data processing method when running.

[0153] An embodiment of the present application also provides a terminal device, which includes a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program code is loaded and executed by the processor to implement the steps of the above-mentioned data processing method.

[0154] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of the above-mentioned data processing method.

[0155] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0158] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0159] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0161] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0162] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0163] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0164] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0166] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: include: The target node obtains a first initial health state of each node in a first node set, where the first node set is a set of nodes in a target group except the target node, the target node is any node in the target group, and the target group is any group in a distributed cluster; The target node obtains a second initial health state of each node in the target group, the second initial health state is the health state of each node in the second node set obtained by the first target node, the second node set is a set of other nodes in the target group except the first target node, and the first target node is any node in the target group except the target node; The target node determines a first target health state of each node in the target group according to the first initial health state and the second initial health state; The target node sends the first target health status of each node in the target group to the server, so that the server performs corresponding operations according to the first target health status and the second target health status of each node in the target group, where the second target health status is the health status of each node in the target group obtained from the cloud.

2. The method according to claim 1, characterized in that The target node acquiring the first initial health status of each node in the first node set includes: The target node sends a target request to the target interface of each node in the first node set to obtain the health value returned by each node in the first node set; The target node determines a first initial health state of each node in the first set of nodes according to a health value returned by each node in the first set of nodes.

3. The method according to claim 2, characterized in that Determining, by the target node, a first initial health state of each node in the first node set according to the health value returned by each node in the first node set includes: When the health value of the second target node is normal, the target node determines that the first initial health state of the second target node is normal, and the second target node is any one node in the first node set; When the health value of the second target node is abnormal, the target node determines that the first initial health state of the second target node is abnormal.

4. A data processing method, characterized in that: include: The server receives a first target health status of each node in a target group sent by a target node, the target group being any group in a distributed cluster, the first target health status of each node in the target group being determined by the target node according to a first initial health status and a second initial health status, the first initial health status being the health status of each node in a first node set acquired by the target node, the first node set being a set of other nodes in the target group except the target node, the second initial health status being the health status of each node in a second node set received by the target node from the first target node, and the second initial health status being the health status of each node in the second node set acquired by the first target node, the second node set being a set of other nodes in the target group except the first target node, and the first target node being any node in the target group except the target node; The server receives a second target health status of each node in the target group sent by the cloud; The server performs corresponding operations on each node in the target group according to the first target health status and the second target health status.

5. The method according to claim 4, characterized in that The server performing corresponding operations on each node in the target group according to the first target health status and the second target health status includes: When the first target health state of the third target node is normal and the second target health state of the third target node is abnormal, the server stops scheduling resource objects to the third target node, and the third target node is any node in the target group; When the first target health state of the third target node is abnormal and the second target health state of the third target node is abnormal, the server evicts the resource objects in the third target node, removes the resource objects of the third target node from the resource object list, and stops scheduling target resource objects to the third target node, wherein the resource object list stores resource objects of multiple nodes including the resource object of the third target node; When the first target health status of the third target node is normal and the second target health status of the third target node is normal, the server does not perform any operation on the third target node.

6. A network node, characterized in that: The network node is a target node, including: a first acquisition unit, configured to acquire a first initial health state of each node in a first node set, wherein the first node set is a set of nodes in a target group except the target node, the target node is any node in the target group, and the target group is any group in a distributed cluster; A second acquisition unit is used to acquire a second initial health state of each node in the target group, where the second initial health state is the health state of each node in a second node set acquired by the first target node, where the second node set is a set of other nodes in the target group except the first target node, and the first target node is any node in the target group except the target node; a determining unit, configured to determine a first target health state of each node in the target group according to the first initial health state and the second initial health state; A sending unit is used to send the first target health status of each node in the target group to the server, so that the server performs corresponding operations according to the first target health status and the second target health status of each node in the target group, where the second target health status is the health status of each node in the target group obtained from the cloud.

7. The node according to claim 6, characterized in that: The first acquisition unit is specifically used for: The target node sends a target request to the target interface of each node in the first node set to obtain the health value returned by each node in the first node set; The target node determines a first initial health state of each node in the first set of nodes according to a health value returned by each node in the first set of nodes.

8. A server, characterized in that: include: A receiving unit, used for receiving a first target health state of each node in a target group sent by a target node, the target group being any group in a distributed cluster, the first target health state of each node in the target group being determined by the target node according to a first initial health state and a second initial health state, the first initial health state being the health state of each node in a first node set acquired by the target node, the first node set being a set of other nodes in the target group except the target node, the second initial health state being the health state of each node in a second node set received by the target node from the first target node, and the second initial health state being the health state of each node in the second node set acquired by the first target node, the second node set being a set of other nodes in the target group except the first target node, and the first target node being any node in the target group except the target node; The receiving unit is further used to receive a second target health status of each node in the target group sent by the cloud; An execution unit is used to perform a corresponding operation on each node in the target group according to the first target health status and the second target health status.

9. The server according to claim 8, characterized in that: The execution unit is specifically used for: When the first target health state of the third target node is normal and the second target health state of the third target node is abnormal, stop scheduling resource objects to the third target node, the third target node being any node in the target group; When the first target health state of the third target node is abnormal and the second target health state of the third target node is abnormal, the resource objects in the third target node are evicted, the resource objects of the third target node are removed from the resource object list, and the scheduling of target resource objects to the third target node is stopped, and the resource object list stores resource objects of multiple nodes including the resource object of the third target node; When the first target health status of the third target node is normal and the second target health status of the third target node is normal, no operation is performed on the third target node.

10. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, cause the computer to execute the steps of the data processing method according to any one of claims 1 to 5.

Images