ES containerized deployment method, device, computer equipment and storage medium

Abstract

The present application relates to an ES containerized deployment method, apparatus, computer equipment and storage medium, which is applied to a container system architecture, in which the nodes in the container system architecture are distributed on at least two racks, and each node in at least one rack is working normally; by creating at least one pod for carrying the ES process on each node of the container system architecture; based on node affinity and replica anti-affinity, the pods are grouped and deployed in nodes of different racks respectively; the rack perception is configured according to the distribution of the pods in the rack, so as to deploy the primary shard and at least one replica shard of the ES index in nodes of different racks respectively. Through the present application, the primary shard and at least one replica shard of the ES index are respectively deployed in nodes of different racks, so that even when an abnormality occurs in the rack, ES can rely on all normal working nodes in the rack to provide services, solving the problem of lack of reliability.

Description

Technical Field

[0001] The present application relates to the field of containerization technology, and in particular to an ES containerization deployment method, apparatus, computer equipment, and storage medium. Background Art

[0002] Elasticsearch (ES) provides a distributed multi-user full-text search engine, mainly used in cloud computing environments. It also supports multiple functions such as structured search, data analysis, complex language processing, geographic location, and object associations.

[0003] Currently, containerization technology can make more effective use of system resources. By deploying ES clusters in a containerized environment, the scheduling and orchestration mechanisms of Kubernetes (K8s) can be used to dynamically schedule and allocate resources, so as to achieve stable operation and efficient management of ES in K8s. However, when the K8s rack crashes, loses power, or loses disks, ES cannot provide services normally, so there is a problem of lack of reliability.

[0004] There is no effective solution to the problem of lack of reliability in related technologies. Summary of the invention

[0005] Based on this, it is necessary to provide an ES containerized deployment method, device, computer equipment and storage medium that can improve reliability in response to the above technical problems.

[0006] In a first aspect, an ES containerized deployment method is provided in this embodiment, which is applied to a container system architecture, wherein nodes in the container system architecture are distributed on at least two racks, and each node in at least one of the racks is working normally; the method includes:

[0007] Create at least one pod for carrying the ES process on each node of the container system architecture;

[0008] Based on node affinity and replica anti-affinity, the pods are grouped and deployed in nodes of different racks respectively;

[0009] Rack awareness is configured according to the distribution of the pod in the rack, so as to deploy the primary shard and at least one replica shard of the ES index in nodes of different racks respectively.

[0010] In some embodiments, the grouping of the pods based on the node affinity and the replica anti-affinity and deploying them in nodes of different racks respectively includes:

[0011] Setting a node label for the node;

[0012] The pods are grouped based on the node labels, so as to deploy the pods in the group in nodes of the same rack;

[0013] By utilizing the replica anti-affinity, each group of the pods is deployed in nodes of different racks.

[0014] In some of the embodiments, it also includes:

[0015] According to the number of ES nodes in the rack, the minimum number of participating nodes is configured accordingly;

[0016] When the number of available ES nodes in the rack is greater than or equal to the minimum number of participating nodes, an ES master node is determined from the available ES nodes through an election strategy.

[0017] In some of the embodiments, it also includes:

[0018] Create service resources;

[0019] The user request is obtained through the service resource, and the user request is forwarded to an available pod.

[0020] In some of the embodiments, it also includes:

[0021] The status of the pod is reported periodically, so as to determine the available pod in the service resource according to the status of the pod.

[0022] In some of the embodiments, it also includes:

[0023] When the user request is a query request, a corresponding ES node is determined according to the query request; and the query result is returned through the ES node.

[0024] In some of the embodiments, it also includes:

[0025] Configure a probe for the pod;

[0026] Through the probe, it is regularly detected whether the disk resources in the rack are abnormal, and the abnormal state of the corresponding pod is reported.

[0027] In a second aspect, an ES containerized deployment device is provided in this embodiment, which is applied to a container system architecture, wherein the nodes in the container system architecture are distributed on at least two racks, and each node in at least one of the racks works normally; the device includes:

[0028] A creation module, used to create at least one pod for carrying an ES process on each node of the container system architecture;

[0029] A deployment module, used for grouping the pods and deploying them in nodes of different racks respectively based on node affinity and replica anti-affinity;

[0030] A configuration module is used to configure rack awareness according to the distribution of the pod in the rack, so as to deploy the primary shard and at least one replica shard of the ES index in nodes of different racks respectively.

[0031] In a third aspect, a computer device is provided in this embodiment, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the ES containerized deployment method described in the first aspect is implemented.

[0032] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the ES containerized deployment method described in the first aspect is implemented.

[0033] Compared with the related art, the ES containerized deployment method, device, computer equipment and storage medium provided in this embodiment are applied to a container system architecture, in which the nodes in the container system architecture are distributed on at least two racks, and each node in at least one of the racks is working normally; by creating at least one pod for carrying the ES process on each node of the container system architecture; based on node affinity and replica anti-affinity, the pods are grouped and deployed in the nodes of different racks respectively; according to the distribution of the pods in the rack, the rack perception is configured to deploy the primary shard and at least one replica shard of the ES index in the nodes of different racks respectively. In this embodiment, each group of pods is deployed in the nodes of different racks of the container system architecture, and by configuring the rack perception, the primary shard and at least one replica shard of the ES index are deployed in the nodes of different racks respectively, so that even when an abnormal situation occurs in the rack, ES can rely on all normal working nodes in the rack to provide services normally, solving the problem of lack of reliability.

[0034] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] Figure 1is a hardware structure diagram of a terminal of an ES containerized deployment method in an embodiment;

[0037] Figure 2 is a flowchart of an ES container deployment method in an embodiment;

[0038] Figure 3 is a schematic diagram of the operation of an ES cluster in an embodiment;

[0039] Figure 4 is a schematic diagram of another operation of an ES cluster in an embodiment;

[0040] Figure 5 is a flowchart of an ES container deployment method in another embodiment;

[0041] Figure 6 The structure diagram of an ES container deployment device in an embodiment.

[0042] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, creation module; 20, deployment module; 30, configuration module. DETAILED DESCRIPTION

[0043] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0044] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0045] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure diagram of the terminal of the ES container deployment method in this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0046] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the ES containerized deployment method in the present embodiment, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0048] Elasticsearch (ES) provides a distributed multi-user full-text search engine, mainly used in cloud computing environments. It also supports multiple functions such as structured search, data analysis, complex language processing, geographic location, and object associations.

[0049] Currently, containerization technology can make more effective use of system resources. By deploying ES clusters in a containerized environment, the scheduling and orchestration mechanisms of Kubernetes (K8s) can be used to dynamically schedule and allocate resources, so as to achieve stable operation and efficient management of ES in K8s. However, when the K8s rack (Rack) experiences abnormal conditions such as downtime, power outage, or disk drop, ES cannot provide services normally, so there is a problem of lack of reliability.

[0050] In this embodiment, an ES containerized deployment method is provided, which is applied to a container system architecture. The nodes in the container system architecture are distributed on at least two racks, and each node in at least one rack operates normally. Figure 2 is a flowchart of the ES container deployment method of this embodiment. Figure 2 As shown, the method comprises the following steps:

[0051] Step S210: Create at least one pod for carrying the ES process on each node of the container system architecture.

[0052] Specifically, taking the container system architecture Kubernetes (K8s) as an example, a node is a server or device that actually stores data, such as a physical machine or server, including a master node and a worker node. Nodes are components of K8s, and all nodes contain necessary services to run pods (container groups), such as kubelet, which is responsible for the lifecycle management of pods and containers, and kube-proxy, which is responsible for network traffic routing and load balancing. Pod is the smallest unit that can be created, managed, and scheduled in K8s, and a pod can contain at least one container.

[0053] In this embodiment, the container system architecture K8s includes at least two racks, which refers to the physical framework used to install hardware such as servers, storage devices, and network devices in the data center. When deploying a K8s cluster, the nodes of the cluster are distributed on at least two racks. This deployment method helps to disperse risks and avoid the impact of single point failures or local failures on the entire cluster, so as to improve the fault tolerance and availability of the cluster.

[0054] It should be noted that each rack in this embodiment can adopt different node deployment methods, for example, the number of nodes in the K8s cluster on each rack is different, the number of pods created on each node is different, etc. Each node in the rack is working normally, that is, the rack is fully available, so that services can be provided by at least one fully available rack in the container system architecture K8s. Preferably, in order to make the cluster simpler and more controllable, the same node deployment method can be adopted in each rack.

[0055] At least one pod is created on each node in the container system architecture K8s. The pod is used to carry the ES process of the stateful application ES, that is, the ES instance. The pod encapsulates the ES process and necessary configurations and is deployed and managed as an independent unit in the cluster. Each ES process can be configured as an ES node in the ES cluster, such as a data node, master node, or coordination node. When an ES cluster is deployed in K8s, each ES node runs as a pod.

[0056] Typically, each ES node can be deployed as a separate pod to ensure resource isolation, scalability, fault tolerance, and simplicity of configuration. In this way, each pod will represent an independent ES node and cooperate with other nodes in the cluster.

[0057] Step S220: based on node affinity and replica anti-affinity, the pods are grouped and deployed in nodes of different racks.

[0058] Specifically, the master control nodes in K8s include API Server (cluster unified entrance), ControllerManager (controller management program) and Scheduler. The API Server, as the coordinator of various components, provides authentication, authorization, access control, API registration and discovery mechanisms, and provides interface services in the form of RESTful API. The addition, deletion, modification, query and monitoring of all object resources are processed by the API Server and submitted to the Etcd storage; the Controller Manager is responsible for the management of the controller. The controller and resources correspond one to one. The controller is used to maintain the status of the cluster, including fault detection, automatic expansion, rolling updates and other regular background tasks; the Scheduler is the core component in the cloud-native K8s cluster. Its role is to reasonably allocate resources to the orchestrated applications and schedule them to the appropriate computing nodes.

[0059] By using the node affinity and replica anti-affinity in the container system architecture K8s, the scheduler schedules the pods according to the specific attributes of the nodes. This can be achieved by adding custom labels to the nodes and specifying these labels as scheduling conditions in the pod scheduling policy. In this way, pods can be grouped according to node labels, and each group of pods can be deployed in nodes on different racks.

[0060] Step S230, configure rack awareness according to the distribution of pods in the racks, so as to deploy the primary shard and at least one replica shard of the ES index in nodes of different racks respectively.

[0061] Specifically, the ES index is where the associated data is stored, and is the basic unit for ES to store, search, analyze, and query data. An index can be divided into multiple shards. The primary shard is the original shard of the index data. Each index has at least one primary shard, which is responsible for receiving index requests for documents and storing the data on it. The replica shard is a copy of the primary shard, which is used for data redundancy and fault recovery. In this embodiment, the primary shard has at least one replica shard, which is used to provide redundant backup of data and enhance fault tolerance.

[0062] The ES process carried by the Pod is responsible for managing all ES indexes and their shards (including primary shards and replica shards) in the cluster. According to the deployment of the above pod grouping in the rack, the environment variables of the ES process are configured through the helm template. The name of the ES process can be obtained when the pod is started. The name indicates the distribution of the pod in the rack. The rack perception is configured according to the distribution of the pod in the rack, and the primary shard and at least one replica shard of the ES index are deployed in the nodes of different racks respectively to ensure that there are complete primary shards or replica shards on the nodes of each rack, and the primary shards and replica shards are not in the nodes of the same rack.

[0063] Through the above steps, each group of pods is deployed in the nodes of different racks of the container system architecture, and by configuring rack awareness, the primary shard and at least one replica shard of the ES index are deployed in the nodes of different racks. In this way, even if the rack crashes, power outages, disk drops, and other abnormal situations occur, ES can rely on all normal working nodes in at least one complete and available rack to provide services normally, ensuring the availability of ES services and solving the problem of lack of reliability.

[0064] In some embodiments, the above step S220, based on the node affinity and the replica anti-affinity, groups the pods and deploys them in different racks respectively, including the following steps:

[0065] Step S221, setting a node label for the node.

[0066] Specifically, in K8s, you can use the kubectl command-line tool to set node labels for nodes. Node labels can contain information about the physical location of the node (such as racks, data centers, etc.) to filter nodes using node labels.

[0067] Step S222: group the pods based on the node labels, so as to deploy the pods in the group in nodes of the same rack.

[0068] Step S223, using replica anti-affinity, deploy each group of pods in nodes of different racks.

[0069] Specifically, node affinity and replica anti-affinity are configured through stateful files. The pod scheduling strategy is written based on node labels, pods are grouped according to preset rules, and pods in the same group are deployed in the same rack. At the same time, when scheduling pods, the scheduler uses replica anti-affinity to prevent each group of pods from being deployed in the same rack as existing pods, so that each group of pods is deployed in different racks, and each rack's node has a primary shard or replica shard of the ES index.

[0070] In this embodiment, node affinity and replica anti-affinity are used to schedule pods, so that each group of pods is deployed in nodes of different racks, thereby providing a basis for subsequent configuration of rack awareness.

[0071] In some embodiments, the method further comprises:

[0072] The minimum number of participating nodes is configured according to the number of ES nodes in different racks; when the number of available ES nodes in the rack is greater than or equal to the minimum number of participating nodes, the ES master node is determined from the available ES nodes through the election strategy.

[0073] Specifically, the ES node is configured by the ES process on the pod in the rack. Since each rack can have different deployment methods, the number of ES nodes in each rack can also be different. According to the number of ES nodes in different racks, the minimum number of participating nodes is configured. Specifically, the minimum number of ES nodes in each rack can be selected as the minimum number of participating nodes. For example, if there are N ES nodes in the dual racks, there are a ES nodes in rack a, and b ES nodes in rack b, and a is less than b, then the minimum number of participating nodes is configured to a; if the same number of ES nodes are configured in the dual racks, that is, a is equal to b, then the minimum number of participating nodes is configured to N / 2. In this way, even if some ES nodes in the rack are abnormal, when the number of available ES nodes in the rack is greater than or equal to the minimum number of participating nodes, the ES master node can be determined through the election strategy to provide services to the outside world.

[0074] ES nodes include master nodes, data nodes, pre-processing nodes, and coordination nodes. An ES node can play one or more roles and is eligible to become a master node. It also stores data and can provide query services, load balancing, and data merging services. Among them, the master node controls the metadata of the entire cluster, such as the addition, deletion, and shard allocation of ES indexes. It can be determined by election strategies from the available ES nodes in all racks. Since there may be multiple candidate ES nodes in the cluster, elections are required to ensure that there is only one elected master node to avoid brain splits.

[0075] In this embodiment, by configuring the minimum number of participating nodes in the election strategy, even when some ES nodes are abnormal, as long as the number of remaining available ES nodes is greater than or equal to the minimum number of participating nodes, the ES master node can be determined through the election strategy to provide services to the outside world.

[0076] In some embodiments, the method further comprises:

[0077] Create service resources; obtain user requests through service resources and forward user requests to available pods.

[0078] Specifically, the management and scheduling of Service resources are handled by components on the K8s master node (such as kube-apiserver, kube-controller-manager, etc.). Service resources are associated with backend pods through label selectors, providing a unified access portal for pods, and forwarding received client user requests to their backend pods through the port proxy mechanism. Specifically, TCP / UDP traffic accessing service resources is forwarded to the corresponding pods, thereby achieving load balancing. Among them, it is necessary to determine the available pods based on the status of the pods in the service resources. If the pod status is abnormal, the service resources will not forward the traffic to the corresponding response pods.

[0079] The user request may be an index creation request, a query request, etc. The user request is parsed in the service resource, and all necessary information in the user request, such as the index name, field definition, and request body, is extracted to execute business logic on the corresponding pod according to the parsed user request.

[0080] For example, a container system architecture includes two racks, rack A and rack B use the same node deployment method, and each rack has the same number of n ES nodes, that is, a pod is deployed in each node of each rack, and an ES process is deployed on each pod. Figure 3 This is a schematic diagram of the operation of the ES cluster in this embodiment. Figure 3 As shown in the figure, the user sends a user request, and the service resource forwards the user request to the group a container group (pod) in rack A (including: a-1, a-2, ..., an) and the group b container group (pod) in rack B (including: b-1, b-2, ..., bn) through load balancing, and executes the corresponding business logic to return the data to the user. Figure 3 Based on the container system architecture, when an abnormality occurs in a rack, Figure 4 is a schematic diagram of another ES cluster operation in this embodiment, such as Figure 4 As shown in the figure, when an abnormal situation occurs in rack A, there is a remaining rack B in the container system architecture. Each node in the rack B is working normally, that is, it is fully available. The user sends a user request, and the service resource forwards the user request to the group b container group (pod) in rack B through load balancing. The ES cluster can still execute the corresponding business logic to return the data to the user.

[0081] In this embodiment, by creating a service resource, when a user request is received, it is forwarded to the corresponding available pod, and then the corresponding pod executes the business logic to provide external services.

[0082] In some embodiments, the method further comprises:

[0083] When the user request is a query request, the corresponding ES node is determined according to the query request; and the query result is returned through the ES node.

[0084] Specifically, the ES cluster receives query requests from the client, which contain specific query parameters such as index name, query conditions, aggregation operations, etc. The query request is parsed through the service resource in the JSON or other formats of the query DSL (Domain Specific Language, a computer language designed for a specific problem domain) to determine the index, field, filter conditions, etc. that need to be queried. The query request is distributed to the pod containing the target data, that is, the ES node. If the query involves multiple shards, the request will be distributed to the ES nodes containing these shards. Each ES node executes the query on its shard and collects matching documents. If the query request contains an aggregation operation, each node will also perform the corresponding aggregation calculation. Finally, the query results are returned to the user.

[0085] In some embodiments, the method further comprises:

[0086] Report the pod status regularly to determine the available pods in the service resources based on the pod status.

[0087] Specifically, the status of the pod is monitored through the K8s API, kubectl command or third-party monitoring tools, and the status of the pod is reported regularly. In the service resource, the available pods are determined according to the status of the pods. For pods with abnormal status, the service resource will not forward traffic to the corresponding response pods.

[0088] In this embodiment, the K8s cluster periodically reports the status of the pod, so that the available pods can be determined in the service resources.

[0089] In some embodiments, the method further comprises:

[0090] Configure probes for pods. Use probes to periodically detect whether disk resources in the rack are abnormal and report abnormal status of the corresponding pods.

[0091] Specifically, add probe configuration to the pod configuration file, and execute the script checkEslive.sh regularly to check whether the disk resource is abnormal by trying to successfully create a file on the disk. For example, the interval for script execution can be set to 10s. If an IO exception or permission exception occurs, it means that the disk resource is abnormal. The abnormal situation of the pod corresponding to the disk resource is reported to the service resource, and the pod restart is triggered.

[0092] In this embodiment, by periodically detecting whether the disk resources are abnormal, the abnormal state of the pod can be reported to the service resource, so as to determine the available pods in the service resource.

[0093] The present embodiment is described and illustrated below through preferred embodiments.

[0094] Figure 5 is a flowchart of the ES container deployment method of this embodiment. Figure 5 As shown, the method comprises the following steps:

[0095] Step S510, creating a service resource in the container system architecture; wherein the nodes in the container system architecture are distributed on at least two racks, and each node in at least one rack operates normally.

[0096] Step S520: Create at least one pod for carrying the ES process on each node of the container system architecture.

[0097] Step S530, setting a node label for the node; grouping the pods based on the node label so that the pods in the group are deployed in the nodes of the same rack; utilizing the replica anti-affinity, deploying each group of pods in the nodes of different racks.

[0098] Step S540: configure rack awareness according to the distribution of pods in the racks, so as to deploy the primary shard and at least one replica shard of the ES index in nodes of different racks respectively.

[0099] Step S550, according to the number of ES nodes in different racks, the minimum number of participating nodes is configured accordingly; when the number of available ES nodes in the rack is greater than or equal to the minimum number of participating nodes, the ES master node is determined from the available ES nodes through the election strategy.

[0100] Step S560: reporting the status of the pod regularly, so as to determine the available pods in the service resources according to the status of the pods.

[0101] Step S570: When the service resource receives a user request, it forwards the user request to an available pod to provide the service.

[0102] Through this embodiment, each group of pods is deployed in the nodes of different racks of the container system architecture, and by configuring rack awareness, the primary shard and at least one replica shard of the ES index are deployed in the nodes of different racks. In this way, even if the rack is down, power off, disk dropped, or other abnormal situations occur, ES can elect a master node from the available pods, and rely on at least one complete and available rack to provide services normally through each normally working node therein, thereby ensuring the availability of ES services and solving the problem of lack of reliability.

[0103] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0104] In this embodiment, an ES containerized deployment device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. The terms "module", "unit", "sub-unit", etc. used below can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0105] Figure 6 is a structural block diagram of the ES container deployment device of this embodiment, which is applied to a container system architecture, in which nodes are distributed on at least two racks, and each node in at least one rack is working normally. Figure 6 As shown, the device comprises:

[0106] The creation module 10 is used to create at least one pod for carrying the ES process on each node of the container system architecture.

[0107] The deployment module 20 is used to group the pods and deploy them in nodes of different racks based on node affinity and replica anti-affinity.

[0108] The configuration module 30 is used to configure the rack perception according to the distribution of the pod in the rack, so as to deploy the primary shard and at least one replica shard of the ES index in nodes of different racks respectively.

[0109] Through the device provided in this embodiment, each group of pods is deployed in the nodes of different racks of the container system architecture, and by configuring rack awareness, the primary shard and at least one replica shard of the ES index are deployed in the nodes of different racks. In this way, even if the rack crashes, power outages, disk drops, and other abnormal situations occur, ES can rely on the normal working nodes in at least one complete and available rack to provide services normally, thereby ensuring the availability of ES services and solving the problem of lack of reliability.

[0110] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0111] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0112] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0113] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0114] In addition, in combination with the ES containerized deployment method provided in the above embodiments, a storage medium may also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the ES containerized deployment methods in the above embodiments is implemented.

[0115] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0116] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0117] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0118] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0119] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A containerized deployment method for ES, characterized in that: Applied to a container system architecture, the nodes in the container system architecture are distributed on at least two racks, and each node in at least one of the racks works normally; the method includes: Create at least one pod for carrying the ES process on each node of the container system architecture; Based on node affinity and replica anti-affinity, the pods are grouped and deployed in nodes of different racks respectively; wherein the following steps are included: Setting a node label for the node; the node label includes information about the physical location of the node; The pods are grouped based on the node labels, so as to deploy the pods in the group in nodes of the same rack; Using the replica anti-affinity, each group of the pods is deployed in nodes of different racks; Rack awareness is configured according to the distribution of the pod in the rack, so as to deploy the primary shard and at least one replica shard of the ES index in nodes of different racks respectively.

2. The ES container deployment method according to claim 1, characterized in that: Also includes: According to the number of ES nodes in the rack, the minimum number of participating nodes is configured accordingly; When the number of available ES nodes in the rack is greater than or equal to the minimum number of participating nodes, an ES master node is determined from the available ES nodes through an election strategy.

3. The ES container deployment method according to claim 1, characterized in that: Also includes: Create service resources; The user request is obtained through the service resource, and the user request is forwarded to an available pod.

4. The ES container deployment method according to claim 3, characterized in that: Also includes: The status of the pod is reported periodically, so as to determine the available pod in the service resource according to the status of the pod.

5. The ES container deployment method according to claim 3, characterized in that: Also includes: When the user request is a query request, determining a corresponding ES node according to the query request; And return the query result through the ES node.

6. The ES container deployment method according to claim 1, characterized in that: Also includes: Configure a probe for the pod; Through the probe, it is regularly detected whether the disk resources in the rack are abnormal, and the abnormal state of the corresponding pod is reported.

7. An ES container deployment device, characterized in that: Applied to a container system architecture, the nodes in the container system architecture are distributed on at least two racks, and each node in at least one of the racks works normally; the device includes: A creation module, used to create at least one pod for carrying an ES process on each node of the container system architecture; A deployment module is used to group the pods and deploy them in nodes of different racks based on node affinity and replica anti-affinity; wherein the steps include: Setting a node label for the node; the node label includes information about the physical location of the node; grouping the pods based on the node label to deploy the pods in the group in nodes of the same rack; utilizing the replica anti-affinity to deploy each group of the pods in nodes of different racks; A configuration module is used to configure rack awareness according to the distribution of the pod in the rack, so as to deploy the primary shard and at least one replica shard of the ES index in nodes of different racks respectively.

8. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the ES containerized deployment method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the ES containerized deployment method according to any one of claims 1 to 6 are implemented.

Images