Method and apparatus for processing data

Abstract

The application relates to the technical field of Internet communication, in particular to a data processing method and device. In the data processing method, a first instruction is acquired, the first instruction is used for indicating to delete resource records of a first computing unit in a server, entity resources of the first computing unit are deleted according to the first instruction, further, a first cleaning component is controlled to delete a first port name associated with the first computing unit, and the first port name comprises a port name associated with the first computing unit and stored in a database. The data processing method provided by the application deletes the first port name associated with the first computing unit by using the first cleaning component, no record remains, repeated reconciliation is not needed, the load pressure of the server can be reduced, and the computing resources can be saved.

Description

Technical Field

[0001] The present application relates to the field of Internet communication technology, and in particular to a method and device for processing data. Background Art

[0002] With the development of Internet communication technology, load balancing technology has become widely used. Load balancing basically means distributing network traffic among multiple servers, so that no one server bears all the load by itself. When network traffic is balanced, applications can run smoothly.

[0003] Kubernetes is a widely used container cluster management system. Kubernetes, abbreviated as k8s, is an open-source platform for automating the deployment, scaling, and management of containerized applications. Kubernetes consists of a cluster master node (Master) and multiple worker nodes (Nodes), with the nodes being the executors of tasks. Worker nodes in k8s are referred to as k8s nodes.

[0004] Load balancing technology can also be used in the Kubernetes container cluster management system. For example, the Kubernetes container cluster management system uses a configuration management tool to parse the port record of the minimum load unit managed by the container cluster management system stored in a key-value store database to obtain a configuration file. The configuration file is then loaded into the load balancer. The load balancer reads this configuration file and implements mapping load balancing based on it.

[0005] For example, in the "Kubernetes" container cluster management system, the configuration management tool can be named "confd", the key-value storage database can be named "Etcd", the minimum computing unit can be called "Pod", and the load balancer can be called F5 load balancer, "haproxy" software load balancer or "nginx" software load balancer.

[0006] Typically, when a Pod is started, it registers its required port names (port keys) with slbetcd, the key-value storage database associated with load balancing. Similarly, when a Pod is deleted, the corresponding port keys in slb etcd are cleared. However, due to network issues or the Pod's own exit, port key deletion and clearing may fail, resulting in residual port key records in slb etcd, requiring further cleanup.

[0007] In current related technologies, further cleaning up residual port key records in "slb etcd" relies on the housekeeping service on k8s. Specifically, each k8s node can clean up the port keys in "slb etcd" corresponding to non-existent "Pods" through reconciliation. That is, the port record on the k8s node is compared with the port key record in "slb etcd" to see if they are consistent. If not, the remaining port key in "slb etcd" is deleted.

[0008] However, this cleanup method requires each Kubernetes node to continuously poll the slb etcd server to obtain the full slb etcd information, compare it with the port records on Kubernetes, and use this difference to remove any remaining port keys. Tens of thousands of Kubernetes nodes accessing slb etcd overload the server and cause a large amount of unnecessary recalculation, wasting computing resources. Furthermore, polling every five minutes results in a long processing interval, preventing the timely removal of any remaining pod records. This can cause the load balancer to mistakenly send network requests to non-existent pods, leading to network errors.

[0009] Therefore, how to reduce server load pressure and save computing resources has become a technical problem that needs to be solved urgently. Summary of the Invention

[0010] The data processing method and device provided in this application are used to reduce server load pressure and save computing resources.

[0011] In a first aspect, the present application provides a data processing method applied to a server, wherein the server includes a first cleaning component, and the first cleaning component is used to clean up the port names associated with the computing units in the server. The method includes: obtaining a first instruction, wherein the first instruction is used to instruct the deletion of the resource record of the first computing unit in the server; deleting the physical resources of the first computing unit according to the first instruction; and controlling the first cleaning component to delete the first port name associated with the first computing unit, wherein the first port name includes the port name associated with the first computing unit stored in a database.

[0012] In this aspect, when deleting the resource record of the first computing unit, the first cleaning component is used to delete the first port name associated with the first computing unit in the database. Compared with the prior art in which the first computing unit deletes the first port name associated with the first computing unit in the database, no port name record residue will be generated, which can save resources used to clean up the residue and reduce server load pressure.

[0013] With reference to the first aspect, in a possible implementation manner, the resource record further comprises a terminator tag, the terminator tag is used to indicate an action framework of cleaning the terminator tag which must be completed before all information in the resource record is deleted, and the method further comprises: controlling the first cleaning component to delete the terminator tag.

[0014] In the implementation manner, the deletion of the terminator tag means that the first port name record in the database has been cleaned in time before the resource record of the first computing unit is deleted, so it is not necessary to repeatedly check the database to find the residual first port name record, thereby reducing the load pressure of the server.

[0015] With reference to the first aspect, in a possible implementation manner, before the first instruction is acquired, the method further comprises: acquiring a second instruction, the second instruction is used to create a resource record of the first computing unit; creating an entity resource of the first computing unit; and registering a first port name associated with the first computing unit to the database.

[0016] In the implementation manner, the creation of the entity resource of the first computing unit and the registration of the first port name to the database facilitate the subsequent use of the load balancer.

[0017] With reference to the first aspect, in a possible implementation manner, the server further comprises a first marking component, and correspondingly, before the entity resource of the first computing unit is created, the method further comprises: controlling the first marking component to add a terminator tag to the resource record of the first computing unit.

[0018] In the implementation manner, when the first computing unit is created, the terminator tag is added to the resource record of the first computing unit, and the deletion of the terminator tag means that the first port name record in the database has been cleaned in time before the resource record of the first computing unit is deleted, so it is not necessary to repeatedly check the database to find the residual first port name record, thereby reducing the load pressure of the server.

[0019] With reference to the first aspect, in a possible implementation manner, before the first port name associated with the first computing unit is registered to the database, the method further comprises: querying first port information associated with the first computing unit, the first port information is used to indicate a first port name; and adding the first port information to the resource record of the first computing unit.

[0020] In the implementation manner, the server can obtain the first port information associated with the first computing unit, and then register the corresponding port name in the database.

[0021] With reference to the first aspect, in a possible implementation manner, the database is a key-value storage database.

[0022] In this implementation, the key-value storage database is a "key value" key-value pair storage database used to save all network configurations of the cluster and status information of all objects.

[0023] In a second aspect, the present application provides a data processing device. The processing device may include various functional modules for implementing the method of the first aspect or any possible implementation thereof. The technical effects of the processing device can refer to the technical effects of the corresponding method and are not further described here.

[0024] The processing device may include: an acquisition module for acquiring a first instruction, wherein the first instruction is used to instruct the deletion of the resource record of the first computing unit in the server; a deletion module for deleting the physical resources of the first computing unit according to the first instruction; and a control module for controlling the first cleaning component to delete the first port name associated with the first computing unit, wherein the first port name includes the port name associated with the first computing unit stored in the database.

[0025] In combination with the second aspect, in a possible implementation, the resource record also includes a terminator mark, and the terminator mark is used to indicate the action framework of clearing the terminator mark that must be completed before all information in the resource record is deleted, and the control module is also used to control the first cleaning component to delete the terminator mark.

[0026] In combination with the second aspect, in a possible implementation, the device also includes: a creation module and a registration module; the acquisition module is also used to obtain a second instruction, and the second instruction is used to create a resource record of the first computing unit; the creation module is used to create the entity resources of the first computing unit; the registration module is used to register the first port name associated with the first computing unit into the database.

[0027] In combination with the second aspect, in a possible implementation, the server also includes a first marking component; accordingly, before creating the entity resources of the first computing unit, the control module is also used to control the first marking component to add a terminator mark to the resource record of the first computing unit.

[0028] In combination with the second aspect, in a possible implementation, before registering the first port name associated with the first computing unit to the database, the device also includes: a query module for querying the first port information associated with the first computing unit, the first port information being used to indicate the first port name; and an adding module for adding the first port information to the resource record of the first computing unit.

[0029] In conjunction with the second aspect, in a possible implementation, the database is a key-value storage database.

[0030] In a third aspect, the present application provides a server comprising: a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions in the memory to execute the method described in the first aspect or any possible implementation method thereof.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect or any possible implementation method thereof.

[0032] In a fifth aspect, the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it is used to implement the method described in the first aspect or any possible implementation method thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0034] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0035] Figure 2 A schematic diagram of the architecture of the "Kubernetes" container cluster management system provided for one embodiment of the present application;

[0036] Figure 3 A flowchart of a data processing method provided in one embodiment of the present application;

[0037] Figure 4 A schematic diagram of the process of creating and registering a port name for a computing unit provided in one embodiment of the present application;

[0038] Figure 5 A schematic diagram of the process of eliminating a computing unit and clearing a port name provided in one embodiment of the present application;

[0039] Figure 6 A schematic diagram of a data processing device provided in one embodiment of the present application;

[0040] Figure 7 A schematic diagram of a device provided in another embodiment of the present application.

[0041] The specific embodiments of the application have been shown and described in the above drawings and text, and will be described in more detail in the following. These drawings and text are not intended to limit the scope of the concept of the application in any way, but to illustrate the concept of the application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0042] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same numbers are used in different drawings to represent the same or similar elements. The embodiments described in the following exemplary embodiments are not meant to represent all embodiments consistent with the application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the application as detailed in the appended claims.

[0043] The specific application scenario of the application is shown in Figure 1 The scenario includes a client 101 and a server 102. The client 101 and the server 102 can be directly or indirectly connected through wired or wireless communication, which is not limited in the application.

[0044] Optionally, in the field of Internet communication technology, the client 101 inputs a command to request the creation of a computing unit "Pod". The server 102 receives the creation command input by the client 101, and creates a corresponding entity resource to the server, and registers the port name associated with the "Pod" in the database. Alternatively, the client 101 inputs a command to request the deletion of the "Pod". Correspondingly, the server 102 receives the deletion command input by the client 101, and deletes the corresponding entity resource in the server and the port name associated with the "Pod" in the database.

[0045] With the development of Internet communication technology, load balancing technology is widely used. Load balancing basically means distributing network traffic among multiple servers, so that no server can bear all the load by itself. When the network traffic is balanced, the application program can run smoothly.

[0046] The "Kubernetes" container cluster management system is a relatively common container cluster management system at present. The "Kubernetes" container cluster management system, abbreviated as k8s, is an open source system platform for automatically deploying, expanding and managing containerized applications. The "Kubernetes" includes a cluster master (Master) and multiple worker nodes (Node), wherein the Node is the executor of tasks. The worker node in k8s can be referred to as k8s node.

[0047] Load balancing technology can also be used in the Kubernetes container cluster management system. For example, the Kubernetes container cluster management system uses a configuration management tool to parse the port record of the minimum load unit managed by the container cluster management system stored in a key-value store database to obtain a configuration file. The configuration file is then loaded into the load balancer. The load balancer reads this configuration file and implements mapping load balancing based on it.

[0048] For example, in a Kubernetes container cluster management system, the configuration management tool may be named confd, and the load balancer may be an F5 load balancer, a haproxy software load balancer, or an nginx software load balancer.

[0049] For example, Figure 2 This is a schematic diagram of the architecture of the "Kubernetes" container cluster management system provided in one embodiment of this application. Figure 2 As shown, the container cluster management system includes a cluster master node 201 , a node 202 , an interface service component 203 , a node operation agent component 204 , a computing unit 205 and a key-value storage database 206 .

[0050] Among them, the cluster master node 201 (master) is the core node of the entire "Kubernetes" container cluster management system. All execution command operations for the "Kubernetes" container cluster management system are executed by the cluster master node 201, which is responsible for the scheduling and management of the entire "Kubernetes" container cluster management system. The cluster master node 201 includes a lot of components, mainly including the interface service component 203 (Kube-apiserver). The interface service component 203 is the data bus and data center of the entire "Kubernetes" container cluster management system. It is a component deployed on the management node that exposes the "Kubernetes" container cluster management system application programming interface (API) and is the control entrance of the entire system.

[0051] Node 202 (Node): In the "Kubernetes" container cluster management system, except for the cluster master node 201, the remaining nodes are called nodes 202. Nodes 202 are all working nodes. Each node 202 will be assigned a reasonable workload by the cluster master node 201 according to the scheduling rules and is the executor of the task.

[0052] The node running agent component 204 (Kubelet) is a "Kubernetes" component deployed in the virtual machine responsible for managing the computing unit 205 (Pod) instance, running on each node 202, capable of managing the cluster itself, and capable of installing and deploying containerized applications on the cluster.

[0053] The computing unit 205 (Pod) is the smallest computing unit managed by the "Kubernetes" container cluster management system, containing one or more closely related containers, and one computing unit 205 can be regarded as a "logical host" of the application layer by a containerized environment.

[0054] The key value storage database 206 (Etcd) is a "key value" key value pair storage database, and is a very important component in the "Kubernetes" container cluster management system, used to save all network configurations and state information of all objects in the cluster.

[0055] Generally, when the computing unit 205 is started, the computing unit 205 registers the port key required by itself in the load balancing related key value storage database "slb etcd", and correspondingly, when the computing unit 205 is deleted, the computing unit 205 also cleans up the corresponding port key in the "slb etcd". However, due to some network or computing unit 205 itself exiting reasons, the deletion and cleaning of the port key may fail, resulting in residual port key records in the "slb etcd", which need to be further cleaned up.

[0056] In the related art, further cleaning up of the residual port key records in the "slb etcd" is all completed by the housekeeping service on the "Kubernetes" container cluster management system. Specifically, each "Kubernetes" container cluster management system node 202 can clean up the port key in the "slb etcd" corresponding to the non-existent computing unit 205 by accounting, that is, comparing whether the port record on the "Kubernetes" container cluster management system node 202 is consistent with the port key record in the "slb etcd", if not, the residual port key in the "slb etcd" is deleted.

[0057] Exemplarily, in the prior art, the process of creating the computing unit 205 and registering the port key to the key-value storage database 206 is as follows: the server 102 receives the request instruction for creating the computing unit 205 input by the client 101, the interface service component 203 pushes the request to create the computing unit 205 to the node operation agent component 204, the node operation agent component 204 creates the entity resources of the computing unit 205, the computing unit 205 registers the port key to the required key-value storage database 206, and the node operation agent component 204 marks that the computing unit 205 entity is created.

[0058] Optionally, the server 102 may receive the request instruction via the automation control center "kube-controller-manager" or the command line tool "kubectl", where "kube-controller-manager" is the automation control center for all resource objects in k8s, and handles routine background tasks in the cluster; "kubectl" can manage the cluster itself and install and deploy containerized applications on the cluster.

[0059] Optionally, after the computing unit 205 entity is created, the file server continuously pulls full data from the database 206 for subsequent use by the load balancer.

[0060] Furthermore, in the prior art, the process of using housekeeping reconciliation to clean up residual port key records is as follows: the server 102 receives the request instruction input by the client 101 to delete the computing unit 205, the interface service component 203 pushes the request to delete the computing unit 205 to the node operation agent component 204, the node operation agent component 204 deletes the physical resources of the computing unit 205, and the computing unit 205 deletes the port key in the key-value storage database 206. Assuming that due to some reasons such as execution timeout, crash of the computing unit 205, etc., some port key records in the key-value storage database 206 are not successfully deleted, housekeeping pulls the full amount of database data from the file server of the key-value storage database 206, compares it with the port in the computing unit 205 in the server, finds out the residual port key in the key-value storage database 206 and cleans it up.

[0061] Optionally, the server 102 receives a request instruction from the client 101 to delete the computing unit 205 , which can also be understood as the automation control center "kube-controller-manager" or the command line tool "kubectl" stamping a deletion timestamp on the computing unit 205 .

[0062] However, in this cleaning method, each k8s node 202 needs to constantly poll the file server of the key-value storage database 206 to obtain the full database data and compare it with the port record on k8s to obtain the difference set, and use this to remove the residual port key. Tens of thousands of k8s nodes access the file server of the key-value storage database 206, resulting in an overload on the file server of the key-value storage database 206, and thus causing the file server of the key-value storage database 206 to respond slowly or be unable to obtain database data records, and reconciliation may fail; polling every 5 minutes, for stable scenarios, there are a lot of unnecessary repeated calculations, wasting computing resources; and polling every 5 minutes, the processing interval is long, and the residual records of the relevant computing unit 205 cannot be removed in time, causing the load balancing to mistakenly send network requests to the computing unit 205 that no longer exists, resulting in network errors.

[0063] Therefore, how to reduce server load pressure and save computing resources has become a technical problem that needs to be solved urgently.

[0064] In view of this, the present application provides a data processing method, aiming to solve the above technical problems of the prior art.

[0065] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0066] Combine Figure 1 The application scenarios shown and Figure 2 The "Kubernetes" container cluster management system shown, Figure 3 This is a flowchart of a data processing method provided in one embodiment of the present application, which is applied to the server 102. Figure 3 As shown, the method provided in the embodiment of the present application includes S301, S302 and S303. Figure 3 The individual steps in the method shown.

[0067] S301: Obtain a first instruction, where the first instruction is used to instruct deletion of a resource record of a first computing unit in a server.

[0068] Optionally, the first instruction can be received by an automation control center "kube-controller-manager" or a command line tool "kubectl" in the server, wherein the "kube-controller-manager" is an automation control center of all resource objects in the k8s, and handles regular background tasks in the cluster; the "kubectl" can manage the cluster itself, and can install and deploy containerized applications on the cluster.

[0069] Understandably, the resource record of the first computing unit, i.e., the "Pod", includes all resources related to the first computing unit, such as entity resources of the first computing unit, a first port name associated with the first computing unit stored in a database, and a record of creating the first computing unit, etc.

[0070] The entity resources of the first computing unit include all information of the first port associated with the first computing unit and related application programs; the first port associated with the first computing unit can be understood as a port required when the first computing unit performs a computing task.

[0071] As an optional implementation, before the first instruction is obtained, the method further includes: obtaining a second instruction, the second instruction being used to create a resource record of the first computing unit, create entity resources of the first computing unit, and register a first port name associated with the first computing unit in the database.

[0072] Illustratively, the automation control center "kube-controller-manager" or the command line tool "kubectl" in the server receives the second instruction sent by the client for requesting to create the first computing unit, and then a node in the server runs a proxy component (Kubelet) to create entity resources of the first computing unit, and to query all port information of the first port associated with the first computing unit, and to directly write into the resource record of the first computing unit; further, the entity resources of the first computing unit are started, and the first port name associated with the first computing unit is registered in the corresponding database.

[0073] Optionally, the server further includes a first marking component, which is used to add a terminator mark to the resource record of the first computing unit before the entity resources of the first computing unit are created.

[0074] To put it simply, a finalizer is a framework provided by Kubernetes that specifies a cleanup action that must be performed before a data record is deleted. This action is performed by the control center corresponding to the value of the finalizer, which then clears the field. Kubernetes ensures that a data record is not deleted until it is marked as finalizer.

[0075] Correspondingly, the role of the terminator mark is that when it is deleted, it means that the first port name record in the database has been cleared in time before the resource record of the first computing unit is deleted, and there will be no port name record remaining. Compared with the prior art of repeatedly reconciling to find the remaining first port name record in the database, it can reduce computing resources and reduce server load pressure.

[0076] It should be noted that the first marking component is the load balancer key terminator marking callback program component "patch-slb-key-finalizer-webhook", which is mainly responsible for marking the terminator (finalizer) for the newly created computing unit that needs to register the port name to the database. This mark can be named "icbc.com.cn / clean-slb-etcd-key".

[0077] Optionally, the database mentioned in the above embodiment is a key-value storage database.

[0078] S302: Delete the physical resources of the first computing unit according to the first instruction.

[0079] In this step, the physical resources of the first computing unit include all information of the first port associated with the first computing unit and related applications; deleting the physical resources of the first computing unit according to the first instruction means deleting all applications related to the first computing unit.

[0080] S303: Control the first cleaning component to delete the first port name associated with the first computing unit, where the first port name includes the port name associated with the first computing unit stored in the database.

[0081] In this step, the first cleaning component is used to delete the first port name associated with the first computing unit in the database. Compared with the prior art in which the first computing unit deletes the first port name associated with the first computing unit in the database, no port name record residue is generated, which can save resources used for cleaning residues and reduce server load pressure.

[0082] Optionally, the resource record further includes a terminator tag. Before all information in the resource record of the first computing unit is deleted, the first cleaning component must clean up the terminator tag.

[0083] It should be noted that the first cleaning component is the load balancer key terminator tag cleaning program component "slb-key-finalizer-controller", which is mainly responsible for cleaning the "slb etcd key" of the first computing unit and removing the "icbc.com.cn / clean-slb-etcd-key" terminator tag after the cleaning is completed.

[0084] In the data processing method provided in the present application, when deleting the resource record of the first computing unit, the first cleaning component is used to delete the first port name associated with the first computing unit in the database. Compared with the prior art in which the first computing unit deletes the first port name associated with the first computing unit in the database, the port name in the database can be removed promptly and effectively without generating residual port name records, which can save resources used to clean up the residuals and reduce server load pressure.

[0085] As an example, based on the above embodiment, Figure 4 A schematic diagram of the process of creating and registering a port name for a computing unit provided in one embodiment of the present application. Figure 4 As shown, the cluster master node also includes a load balancer key value terminator tag callback program component and a load balancer key value terminator tag cleanup program component. Taking the creation of computing unit B as an example, the load balancer key value terminator tag cleanup program component monitors all computing unit B resources throughout the process. The processing method includes S401, S402, S403, S404, S405 and S406. The following is a detailed description Figure 4 The individual steps in the method shown.

[0086] S401: Request to create computing unit B.

[0087] In this step, taking the client on the server as an example, the request command entered by the client is received by the server's automated control center "kube-controller-manager" or the command line tool "kubectl".

[0088] S402: Mark computing unit B as a terminator.

[0089] In this step, the load balancer key value terminator mark callback program component marks the newly created computing unit B with the terminator mark of "icbc.com.cn / clean-slb-etcd-key".

[0090] S403, notify the creation of computing unit B.

[0091] In this step, the interface service component pushes the resource record of computing unit B to the node execution agent component.

[0092] S404: Create computing unit B entity resources and query port information.

[0093] In this step, the node runs the agent component to create the computing unit B entity resource and query the port information of the port required by computing unit B.

[0094] S405: Write back the port information to the resource record of computing unit B.

[0095] In this step, the node runs the agent component to update the port information of the port required by computing unit B into the resource record of computing unit B, and marks that the physical resource of computing unit B has been created.

[0096] S406 , when the computing unit B is started, the port name corresponding to the port information is registered in the database.

[0097] In this step, the physical resources of computing unit B are started and the corresponding port name is registered in the required database, for example Figure 4 The key-value storage database -xxx and the key-value storage database -yyy shown in the figure include nodes 1.1.1.1 and 1.1.1.2. In addition to the target computing unit B to be created, the computing units also include computing units A, C, and D. The key-value storage database -xxx stores information such as the port names corresponding to the computing units, such as "computing unit A-1.1.1.1-8080=1.1.1.1:8080", "computing unit B-1.1.1 .1-8090=1.1.1.1:8090” and “computing unit D-1.1.1.2-9100=1.1.1.2:9100”; the key-value storage database -yyy stores “computing unit B-1.1.1.1-8090=1.1.1.1:8090”, “computing unit C-1.1.1.2-9090=1.1.1.2:9090” and “computing unit D-1.1.1.2-9100=1.1.1.2:9100”.

[0098] As another example, based on the above embodiment, Figure 5 A schematic diagram of the process of eliminating a computing unit and clearing a port name provided in one embodiment of the present application. Figure 5As shown, the cluster master node also includes a load balancer key terminator tag callback program component and a load balancer key terminator tag cleanup program component. Taking the creation of computing unit B as an example, the processing method includes S501, S502, S503, S504, S505, S506, S507 and S508. The following is a detailed description Figure 5 The individual steps in the method shown.

[0099] S501: Add a deletion timestamp to computing unit B.

[0100] In this step, using the client on the server as an example, the client's request instruction is received by the server's automated control center "kube-controller-manager" or the command line tool "kubectl". The automated control center "kube-controller-manager" or the command line tool "kubectl" adds a deletion timestamp to computing unit B.

[0101] S502: Push the resource record of computing unit B with a deletion timestamp.

[0102] In this step, the interface service component pushes the resource record of computing unit B with a deletion timestamp to the node execution agent component and the load balancer key terminator tag cleaner component.

[0103] S503: Check and delete the port name record of computing unit B in the database.

[0104] In this step, the load balancer key terminator tag cleaner component deletes the corresponding port name record in the database according to the database information and port information required to be registered in computing unit B until success.

[0105] For example, Figure 4The databases shown in the figure include the key-value storage database -xxx and the key-value storage database -yyy. The nodes in the figure include node 1.1.1.1 and node 1.1.1.2. In addition to the target computing unit B to be created, the computing units also include computing unit A, computing unit C, and computing unit D. The key-value storage database -xxx stores information such as the port names corresponding to the computing units, such as "computing unit A-1.1.1.1-8080=1.1.1.1:8080", "computing unit B-1.1 .1.1-8090=1.1.1.1:8090" and "computing unit D-1.1.1.2-9100=1.1.1.2:9100" are stored in the key-value storage database -yyy; "computing unit B-1.1.1.1-8090=1.1.1.1:8090", "computing unit C-1.1.1.2-9090=1.1.1.2:9090" and "computing unit D-1.1.1.2-9100=1.1.1.2:9100" are stored.

[0106] S504: Delete the computing unit B entity resources.

[0107] In this step, the node runs the agent component to delete the computing unit B entity resources.

[0108] S505: The entity resource of computing unit B is marked as deleted.

[0109] In this step, the node runs the agent component to mark the computing unit B entity resource as deleted.

[0110] S506: Push the resource record of computing unit B whose computing unit entity resource is deleted.

[0111] In this step, the interface service component pushes the resource record of the computing unit B whose computing unit entity resource is deleted to the load balancer key-value terminator tag cleaner component.

[0112] S507: Check and delete the port name record of computing unit B in the database.

[0113] In this step, the load balancer key terminator tag cleaner component deletes the corresponding port name record in the database according to the database information and port information required to be registered in computing unit B until success.

[0114] S508: Erasing the terminator mark of computing unit B, thereby triggering deletion of computing unit B's resource record.

[0115] In this step, the load balancer key value terminator tag cleaner component deletes the "icbc.com.cn / clean-slb-etcd-key" terminator tag on compute unit B.

[0116] The processing method in this embodiment can accurately, timely and effectively remove the port name records in the database, reducing load balancer access errors caused by slow port name removal; and compared with the existing technology, it abolishes the database file server and housekeeper service, and abolishes meaningless full data pulling, thereby solving the processing bottleneck caused by tens of thousands of k8s nodes repeatedly accessing the file server, and greatly reducing the file server load pressure and computing resource consumption.

[0117] Optionally, the registration process of the computing unit may also be completed by an independent controller, rather than by the callback component and script of the computing unit itself.

[0118] Optionally, reporting the port information of the computing unit to the interface service component may also be completed by another independent component.

[0119] Optionally, the port resource information may not be recorded in the computing unit, for example, using a custom resource definition (CRD) of k8s for information recording.

[0120] Based on the above embodiments, Figure 6 Detailed description is given of a data processing device 600 provided in an embodiment of the present application. The device 600 includes an acquisition module 601 , a deletion module 602 , a control module 603 , a creation module 604 , a registration module 605 , a query module 606 and an addition module 607 .

[0121] Among them, the acquisition module 601 is used to obtain the first instruction, and the first instruction is used to instruct the deletion of the resource record of the first computing unit in the server; the deletion module 602 is used to delete the physical resources of the first computing unit according to the first instruction; the control module 603 is used to control the first cleaning component to delete the first port name associated with the first computing unit, and the first port name includes the port name associated with the first computing unit stored in the database.

[0122] As an example, the apparatus 600 may be used to perform Figure 3 In the method shown, for example, the acquisition module 601 is used to execute S301 , the deletion module 602 is used to execute S302 , and the control module 603 is used to execute S303 .

[0123] In one possible implementation, the resource record also includes a terminator tag, which is used to indicate an action framework for clearing the terminator tag that must be completed before all information in the resource record is deleted. The control module 603 is also used to control the first cleaning component to delete the terminator tag.

[0124] In a possible implementation, the obtaining module 601 is further configured to obtain a second instruction, the second instruction being used to create a resource record of the first computing unit; the creating module 604 is configured to create an entity resource of the first computing unit; and the registering module 605 is configured to register a first port name associated with the first computing unit to the database.

[0125] In a possible implementation, the server further includes a first marking component; and accordingly, before the entity resource of the first computing unit is created, the control module 603 is further configured to control the first marking component to add a terminator mark to the resource record of the first computing unit.

[0126] In a possible implementation, before the first port name associated with the first computing unit is registered to the database, the apparatus further includes: a querying module 606, configured to query first port information associated with the first computing unit, the first port information being used to indicate the first port name; and an adding module 607, configured to add the first port information to the resource record of the first computing unit.

[0127] In a possible implementation, the database is a key-value storage database.

[0128] Figure 7 An apparatus diagram is provided for another embodiment of the present application. Figure 7 The apparatus shown can be used to execute the method of any one of the foregoing embodiments.

[0129] As shown in Figure 7 The apparatus 700 of the embodiment includes a memory 701, a processor 702, a communication interface 703, and a bus 704. The memory 701, the processor 702, and the communication interface 703 are in communication connection with each other through the bus 704.

[0130] The memory 701 can be a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 701 can store a program, and when the program stored in the memory 701 is executed by the processor 702, the processor 702 is used to execute each step of the method shown in the foregoing embodiments.

[0131] The processor 702 can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, to execute a related program to implement each method shown in the embodiments of the present application.

[0132] The processor 702 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the method of the embodiment of the present application may be completed by an integrated logic circuit of hardware in the processor 702 or by instructions in the form of software.

[0133] The processor 702 may also be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. It may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0134] The steps of the method disclosed in conjunction with the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 701, and processor 702 reads the information in memory 701 and, in conjunction with its hardware, completes the functions required to be performed by the units included in the device of the present application.

[0135] The communication interface 703 may use, but is not limited to, a transceiver or other transceiver device to implement communication between the apparatus 700 and other devices or a communication network.

[0136] The bus 704 may include a path for transmitting information between various components of the device 700 (eg, the memory 701 , the processor 702 , and the communication interface 703 ).

[0137] It should be understood that the device 700 shown in the embodiment of the present application may be an electronic device, or may be a chip configured in the electronic device.

[0138] It should be noted that the data processing method and device in this application can be used in the field of Internet communication technology, and can also be used in any field other than the field of Internet communication technology. This application does not limit the application field of the data processing method and device.

[0139] It should be understood that the term "and / or" as used herein simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the related objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0140] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0141] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0142] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0143] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0144] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0145] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0146] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0147] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0148] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data processing method, characterized in that: Applied to a server, the server comprising a first cleaning component, the first cleaning component being used to clean up port names associated with computing units in the server, the method comprising: Obtaining a first instruction, where the first instruction is used to instruct deletion of a resource record of a first computing unit in the server; Deleting the physical resources of the first computing unit according to the first instruction; Control the first cleaning component to directly delete the first port name associated with the first computing unit, where the first port name includes the port name associated with the first computing unit stored in the database, wherein the first cleaning component runs independently of the first computing unit to ensure that there is no residual record of the port name associated with the first computing unit.

2. The method according to claim 1, characterized in that The resource record also includes a terminator tag, which is used to indicate an action framework for clearing the terminator tag that must be completed before all information in the resource record is deleted. The method further includes: Control the first cleaning component to delete the terminator mark.

3. The method according to claim 1 or 2, characterized in that Before obtaining the first instruction, the method further includes: Obtaining a second instruction, wherein the second instruction is used to create a resource record of the first computing unit; Creating a physical resource of the first computing unit; Registering a first port name associated with the first computing unit into the database.

4. The method according to claim 3, characterized in that The server further includes a first marking component; Accordingly, before creating the physical resources of the first computing unit, the method further includes: The first marking component is controlled to add a terminator marking to a resource record of the first computing unit.

5. The method according to claim 3, characterized in that Before registering the first port name associated with the first computing unit in the database, the method further includes: querying first port information associated with the first computing unit, where the first port information is used to indicate a name of the first port; Add the first port information to the resource record of the first computing unit.

6. The method according to claim 3, characterized in that The database is a key-value storage database.

7. A data processing device, characterized in that: The device comprises: An acquisition module, configured to acquire a first instruction, wherein the first instruction is used to instruct deletion of a resource record of a first computing unit in a server; a deletion module, configured to delete the physical resources of the first computing unit according to the first instruction; A control module is used to control the first cleaning component to directly delete the first port name associated with the first computing unit, where the first port name includes the port name associated with the first computing unit stored in the database, wherein the first cleaning component runs independently of the first computing unit to ensure that there is no residual record of the port name associated with the first computing unit.

8. A server, characterized in that: include: memory and processor; The memory is used to store program instructions; The processor is configured to call program instructions in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, it is used to implement the method according to any one of claims 1 to 6.

Images