Automatic scaling non-intrusive gray release system

Abstract

The present invention relates to the technical field of communications. Disclosed is an automatic scaling non-intrusive gray release system, aiming to solve the technical problem of how to satisfy the requirements of a complex system for performance, high availability, customizability and non-intrusiveness, and achieving non-invasive gray release. The automatic scaling non-intrusive gray release system comprises a service instance cluster module, a resource monitoring module and an automatic scaling module; the service instance cluster module is used for using Deployment of Kubernetes or StatefulSet to create and manage service instances in a Kubernetes cluster; the resource monitoring module is used for regularly collecting resource monitoring metrics data; the automatic scaling module is used for supporting a user to configure a resource utilization rate threshold and the automatic scaling range of gray release service instances, is used for determining whether a new service instance needs to be created or whether an unhealthy service instance needs to be destroyed, and cooperates the service instance cluster module to scale a service instance cluster on the basis of a calculation result.

Description

An automatically scalable, non-intrusive grayscale release system Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to an automatically scalable and non-intrusive grayscale publishing system. Background Art

[0002] As software development becomes increasingly complex, pushing new features or versions to production without disrupting services becomes a critical issue. Grayscale releases offer a solution, allowing development teams to gradually push new versions to production, mitigating the risks of a full release.

[0003] Grayscale release involves distributing requests to new and old versions of service instances according to certain rules, so it needs to deal with issues such as request routing, service discovery, and load balancing. Traditional solutions usually integrate these functions into a centralized component, such as an API gateway or service mesh. Using Kubernetes' Ingress for grayscale release is such a traditional solution, in which the Ingress Controller (such as NGINX Ingress Controller, Traefik, etc.) plays the role of a centralized component. Ingress is an object in Kubernetes used to manage external requests to access the internal services of the cluster. It can define a set of routing rules so that external traffic can access services within the cluster. The Ingress Controller is a Pod responsible for implementing the routing rules defined by the Ingress resource.

[0004] This integrated solution has the following problems:

[0005] (1) Performance: Since all requests need to go through this centralized component, it may become a performance bottleneck and single point of failure for the system. For example, when using Ingress for grayscale release, some Ingress Controllers implemented in the Go language may experience high performance overhead when processing high concurrent requests due to factors such as Go's garbage collection.

[0006] (2) Complex expansion: To address performance issues, horizontal expansion can be used to improve processing capabilities, that is, by adding more instances to distribute traffic. However, this expansion method increases the complexity of the system, such as load balancing, instance management, and synchronization issues.

[0007] (3) Grayscale rules are less customizable: The central components of the system usually provide a set of predefined functions, such as request routing, authentication, and rate limiting. These functions are often fixed and cannot be deeply customized.

[0008] (4) Intrusiveness: The central component of the system handles all incoming and outgoing traffic and provides functions such as request routing, authentication, and rate limiting. In order to use these functions, the application often needs to be modified or configured. For example, to implement label-based routing, the service may need to add specific labels when deployed. This modification or configuration requirement for the application or service constitutes an intrusiveness.

[0009] How to meet the complex system's requirements for performance, high availability, customizability, and non-intrusiveness, and achieve non-intrusive grayscale release, is a technical problem that needs to be solved.

[0010] Summary of the Invention

[0011] The technical task of the present invention is to address the above shortcomings and provide an automatically scalable non-intrusive grayscale release system to solve the technical problem of how to meet the performance, high availability, customizability and non-intrusiveness requirements of complex systems and realize non-intrusive grayscale release.

[0012] The present invention provides an automatic scaling non-intrusive grayscale publishing system, comprising a service instance cluster module, a resource monitoring module and an automatic scaling module;

[0013] The service instance cluster module is used to create and manage service instances in a Kubernetes cluster using Kubernetes Deployment or StatefulSet. This module performs the following: exposing resource monitoring metrics, running the containerized grayscale environment, production environment, grayscale rule server, and OpenResty in the Kubernetes cluster, and implementing non-invasive grayscale release services through OpenResty. The resource monitoring metrics include basic resource monitoring metrics and customized monitoring metrics.

[0014] The resource monitoring module interacts with the service instance cluster module to periodically collect resource monitoring measurement data, wherein the resource monitoring measurement data includes basic resource monitoring measurement data reflecting system resource usage and customized monitoring measurement data;

[0015] The auto-scaling module interacts with the resource monitoring module and the service instance cluster module respectively, and provides a user interface through the API server, which supports users to configure resource utilization thresholds and auto-scaling ranges for phased release service instances through the user interface to form a configuration file. The auto-scaling module is used to determine whether to create new service instances or destroy unhealthy service instances based on resource monitoring measurement data, pre-configured resource utilization thresholds, and the auto-scaling algorithm, calculate the number of service instances that need to be adjusted, and cooperate with the service instance cluster module to scale the service instance cluster based on the calculation results.

[0016] As a preferred method, OpenResty is used to implement a non-invasive grayscale release service, including the following steps:

[0017] After the HTTP request reaches OpenResty, the Lua module in OpenResty extracts the identifier in the HTTP request and transmits it to the grayscale rule server by calling the grayscale rule server's API. The identifier includes the IP address, browser type, and login status.

[0018] The grayscale rule server uses pre-configured grayscale rules to determine whether the received logo is a grayscale logo and returns the judgment result to OpenResty;

[0019] OpenResty routes the corresponding HTTP request to the production environment or the grayscale environment based on the received judgment result.

[0020] Preferably, the service instance cluster module is used to perform the following steps to establish a non-invasive grayscale release service instance cluster:

[0021] Expose application-defined monitoring metrics: The application runs in a pod and integrates a Prometheus-supported metrics library as a client library. The application creates an HTTP endpoint through the client library and provides application-defined metrics on the HTTP endpoint. The metrics data is formatted in accordance with Prometheus's data format. Prometheus's data collection mechanism collects application-defined metrics, including request latency, query rate per second, request error rate, queue length, and business indicators.

[0022] Containerization: Perform the following four operations: Containerize the OpenResty service, the grayscale rule management service, the production environment, and the grayscale environment. This means creating a corresponding Dockerfile for each service. The Dockerfile contains all dependencies and configuration files for the corresponding service, as well as a startup script that dynamically reads configuration parameters from environment variables or configuration maps when the service starts.

[0023] Deploy the application in Kubernetes: Create a Deployment in Kubernetes. The Deployment defines the OpenResty container's operating parameters, resource limits, and the number of replicas required to create a cluster of OpenResty instances. In the YAML file that defines the Deployment, add Prometheus annotations to the metadata section of the Pod template. The Prometheus annotations are used to indicate which endpoint Prometheus should use to fetch metrics data.

[0024] Service discovery and load balancing: Create a Kubernetes Service to route traffic to the OpenResty instance. The Kubernetes Service optionally provides a stable network interface that allows other services and clients to discover and connect to the OpenResty instance.

[0025] Update grayscale rules: The OpenResty instance periodically calls the grayscale rule server API through a scheduled task to obtain the latest grayscale rules. The scheduled task uses OpenResty's ngx.timer module to periodically send HTTP requests to call the grayscale rule management module API.

[0026] Execute grayscale rules: In the OpenResty configuration file, configure the Lua module. The configured Lua module can extract the identifier in the grayscale service request and pass the grayscale service request to the grayscale rule server by calling the grayscale rule server's API. OpenResty routes the grayscale service request based on the result returned by the grayscale rule server.

[0027] Preferably, the grayscale release service instance cluster includes:

[0028] Master Node: The Master Node serves as the management node for the grayscale release service instance cluster. It manages all nodes within the cluster, sends container creation, deployment, and release commands to cluster nodes through the master server, and receives and distributes data from outside the cluster.

[0029] Worker Node, which serves as the working node of the grayscale release service instance cluster, runs in the application container and provides the Kubernetes runtime environment;

[0030] Pod, the smallest deployment unit in Kubernetes, contains at least one container. Containers share storage, networking, and runtime configuration. In a Kubernetes cluster, Pods can be scheduled and run on Worker Nodes and controlled by high-level APIs such as Deployment and StatefulSet to implement application management features including replica control and rolling upgrades.

[0031] Canary Environment container, which is used as a grayscale environment container to deploy the latest version of the service application when grayscale release services are performed;

[0032] The Production Environment container is used as a production environment container to run the current stable version of the service application. When a new version of the service application is tested in the grayscale environment and is considered stable, the new version of the service application will be deployed in the production environment container and replace the current stable version of the service application;

[0033] Canary Rules Server container, which serves as a canary rules server container and is used to run services that store and manage canary rules. Canary rules include rules that determine which HTTP requests should be routed to the canary environment and which HTTP requests should be routed to the production environment. Canary rules are based on various attributes, including the user's IP address, browser type, and login status;

[0034] An OpenResty container runs OpenResty, which is configured as a proxy server to provide the following OpenResty services: receiving all incoming traffic and routing the traffic to a grayscale environment container or a production environment container according to grayscale rules provided by a grayscale rule server.

[0035] Preferably, the Worker Node includes Kubelet components, Kube-proxy components and Container Runtime.

[0036] The Kubelet component is used to maintain the lifecycle of the Pod, perform tasks including starting and stopping containers, and communicate with the Master Node;

[0037] The Kube-proxy component implements the network rules of the Kubernetes service and forwards connections;

[0038] The Container Runtime component is responsible for running containers.

[0039] Preferably, Deployment is used to deploy stateless applications, and StatefulSet is used to deploy stateful applications.

[0040] Preferably, the resource monitoring module includes Kubelet component, Metrics Server component, Prometheus and Prometheus Adapter component.

[0041] The Kubelet component is a core component on each node in a Kubernetes cluster. As a proxy service, it communicates with containers running on the corresponding node and manages the container lifecycle and resource allocation. The Kubelet component is used to regularly collect basic resource monitoring metrics and send them to the Metrics Server component. Basic resource monitoring metrics include container CPU, memory, disk, and network bandwidth usage.

[0042] Metrics Server is a Kubernetes component that periodically collects basic resource monitoring metrics from the Kubelet component of each node and sends them to Prometheus.

[0043] Prometheus is an independent monitoring system. A scraping configuration is added to the Prometheus configuration file. Based on this scraping configuration, Prometheus can scrape custom monitoring metrics from containers in pods annotated with Prometheus. Prometheus regularly scrapes resource monitoring metrics from the Metrics Server component and pods, and stores them in its own time series database to provide decision-making data for the auto-scaling module.

[0044] Prometheus Adapter is used as a bridge between Prometheus and Kubernetes, implementing the Kubernetes Metrics API, allowing Kubernetes to query and use resource monitoring metric data stored in Prometheus.

[0045] As a preference, Prometheus provides a query interface that supports users to query historical and real-time resource monitoring measurement data. Prometheus is configured with alarm rules, which trigger alarms when resource monitoring measurement data meets the alarm conditions defined in the alarm rules.

[0046] Preferably, the automatic scaling module includes APIServer, HPA component, Scale API, Deployment and ReplicaSet.

[0047] The API Server provides a user interface that supports users in creating HPA components. Specifically, users are supported to set resource usage thresholds, such as the system's QPS (Query Per Second), CPU usage, and memory usage, through a YAML configuration file. The API Server also supports setting the automatic scaling range (i.e., the maximum and minimum number of service instances) for grayscale release service instances. The automatic scaling range for grayscale release service instances includes the maximum and minimum number of service instances.

[0048] The HPA component is the automatic scaling component in Kubernetes. It is used to regularly obtain resource monitoring metrics from the resource monitoring module, compare the obtained resource monitoring metrics with the preset resource utilization threshold, calculate the number of service instances that need to be adjusted, and implement scaling by calling the Scale API based on the calculation results.

[0049] The Scale API is a resource provided by Kubernetes for changing the number of Pods defined in Deployment and ReplicaSet.

[0050] A Deployment is a high-level controller that controls ReplicaSets by defining the desired number of Pod replicas and Pod templates, thereby implementing application operations such as scaling, updating, and rolling back.

[0051] A Deployment manages a set of ReplicaSets, which in turn manage a set of Pods through ReplicaSets. A Deployment scales the grayscale release service instance cluster through rolling updates.

[0052] Preferably, Deployment scales the grayscale release service instance cluster through rolling updates, which includes the following operations:

[0053] When a Deployment is created, the Deployment controller creates a new ReplicaSet. The ReplicaSet's template is derived from the Deployment's Pod template, and the number of replicas is derived from the Deployment's spec.replicas field. The ReplicaSet controller creates a new set of Pods based on the ReplicaSet's template and the number of replicas.

[0054] When updating a Deployment's Pod template, the Deployment controller creates a new ReplicaSet and a new set of Pods based on the new Pod template. The Deployment controller gradually reduces the number of replicas in the old ReplicaSet and increases the number of replicas in the new ReplicaSet until the number of replicas in the old ReplicaSet reaches 0 and the number of replicas in the new ReplicaSet matches the Deployment's spec.replicas field.

[0055] The automatic scalable and non-intrusive grayscale publishing system of the present invention has the following advantages:

[0056] 1. Implementing grayscale releases with OpenResty greatly improves the performance and customizability of the grayscale release system. Specifically, OpenResty's event-driven model enables excellent performance when handling high-concurrency requests, effectively avoiding performance bottlenecks. Additionally, OpenResty provides highly customizable grayscale rules through Lua scripts, enabling fine-grained control over the grayscale release process based on business needs.

[0057] 2. Use the HPA component of Kubernetes to implement automatic scaling of the grayscale release service instance cluster, which improves the high availability of the grayscale release system and reduces the waste of service resources.

[0058] 3. A non-intrusive grayscale release is achieved, which means that when implementing grayscale release, there is no need to modify the existing application code or add any specific tags or annotations. This greatly reduces the complexity of development and maintenance, allowing developers to focus more on the development of business functions rather than compatibility issues with the platform. This non-intrusive design not only improves development efficiency but also reduces the risk of introducing errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0060] The present invention will be further described below with reference to the accompanying drawings.

[0061] FIG1 is a structural block diagram of an automatically scalable, non-intrusive grayscale publishing system according to this embodiment;

[0062] FIG2 is an architectural block diagram of a non-intrusive grayscale publishing service in an automatically scalable non-intrusive grayscale publishing system according to an embodiment of the present invention;

[0063] FIG3 is an architectural block diagram of a phased release service instance cluster in an automatically scalable, non-intrusive phased release system according to an embodiment of the present invention;

[0064] FIG4 is an architectural block diagram of a resource monitoring module in an automatically scalable, non-intrusive grayscale release system according to this embodiment;

[0065] FIG5 is an architectural block diagram of an automatic scaling module in an automatic scaling non-intrusive grayscale release system according to this embodiment;

[0066] In the figure, OpenResty is a scalable web platform based on NGINX;

[0067] Kubelet is the agent component on the worker node responsible for maintaining and managing containers;

[0068] Deployment represents a resource object that describes how to deploy an application. DETAILED DESCRIPTION

[0069] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments given are not intended to limit the present invention. Unless there is a conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0070] The embodiment of the present invention provides an automatically scalable non-intrusive grayscale release system, which is used to solve the technical problem of how to meet the requirements of complex systems for performance, high availability, customizability and non-intrusiveness, and realize non-intrusive grayscale release.

[0071] Embodiment: The present invention provides an automatic scaling non-intrusive grayscale release system, including a service instance cluster module, a resource monitoring module and an automatic scaling module.

[0072] The service instance cluster module is used to create and manage service instances in a Kubernetes cluster using Kubernetes' Deployment or StatefulSet. This includes exposing resource monitoring metrics, running the containerized grayscale environment, production environment, grayscale rule server, and OpenResty in a Kubernetes cluster, and implementing non-invasive grayscale release services through OpenResty. Resource monitoring metrics include basic resource monitoring metrics and customized monitoring metrics.

[0073] Traditional centralized components, such as API (Application Programming Interface) gateways or service meshes, often require code modifications to applications or services to accommodate grayscale releases. In these cases, changes to centralized components directly impact application implementation, increasing the complexity of development and maintenance.

[0074] Using OpenResty for non-invasive grayscale release can be regarded as a highly configurable proxy server. It routes incoming traffic according to predetermined rules (for example, specific HTTP (Hypertext Transfer Protocol) header information, cookie values, etc.) and directs requests to new or old versions of service instances. In this way, the application or service itself does not need to understand or participate in the grayscale release process, thus achieving non-invasive grayscale release. Therefore, although OpenResty plays the role of a "centralized component" in this process, it does not require modification of the application or service code, making this solution more flexible and easy to maintain. In view of this, this embodiment uses OpenResty to implement a non-invasive grayscale release service.

[0075] Figure 2 shows the architecture of the non-intrusive grayscale release service. The specific operations are as follows:

[0076] (1) After the HTTP request reaches OpenResty, the Lua module in OpenResty extracts the identifier in the HTTP request and transmits the identifier to the grayscale rule server by calling the grayscale rule server's API. The identifier includes the IP address, browser type, and login status.

[0077] (2) The grayscale rule server uses the pre-configured grayscale rules to determine whether the received logo is a grayscale logo and returns the judgment result to OpenResty;

[0078] (3) OpenResty routes the corresponding HTTP request to the production environment or the grayscale environment based on the received judgment result.

[0079] After determining the implementation technology for grayscale release service instances, this example uses Kubernetes to implement a non-invasive grayscale release service instance cluster module. This module is responsible for managing and enforcing grayscale release rules and defining and exposing corresponding basic resource monitoring metrics and customized monitoring metrics based on the service-level agreement (SLA) agreement level.

[0080] As a specific implementation, the service instance cluster module is used to perform the following steps to establish a non-invasive grayscale release service instance cluster:

[0081] (1) Expose application-defined monitoring metrics: The application is an application running in a Pod (container set). A Prometheus-supported metrics library is integrated into the application as a client library. The application creates an HTTP endpoint (usually / metrics) through the client library and provides the application's customized monitoring metrics on the HTTP endpoint. The format of the monitoring metrics data conforms to the Prometheus data format. Prometheus's data collection mechanism collects application-defined monitoring metrics, including request latency, query rate per second, request error rate, queue length, and business indicators.

[0082] (2) Containerization: Perform the following four operations: Containerize the OpenResty service, the grayscale rule management service, the production environment, and the grayscale environment. This means creating a corresponding Dockerfile for each service. The Dockerfile contains all dependencies and configuration files for the corresponding service, as well as a startup script that dynamically reads configuration parameters from environment variables or configuration maps when the service starts.

[0083] (3) Deploy the application in Kubernetes: Create a Deployment in Kubernetes. The Deployment defines the OpenResty container's operating parameters, resource limits, and the number of replicas required to create a cluster of OpenResty instances. In the YAML file that defines the Deployment, add Prometheus annotations to the metadata section of the Pod template. The Prometheus annotations are used to tell Prometheus which endpoint to use to fetch resource monitoring metrics.

[0084] (4) Service discovery and load balancing: Create a Kubernetes Service to route traffic to the OpenResty instance. The Kubernetes Service optionally provides a stable network interface that allows other services and clients to discover and connect to the OpenResty instance.

[0085] (5) Update grayscale rules: The OpenResty instance periodically calls the API of the grayscale rule server through a scheduled task to obtain the latest grayscale rules. The scheduled task uses OpenResty's ngx.timer module to periodically send HTTP requests to call the API of the grayscale rule management module;

[0086] (6) Execute grayscale rules: In the OpenResty configuration file, configure the Lua module. The configured Lua module can extract the identifier in the grayscale service request and pass the grayscale service request to the grayscale rule server by calling the grayscale rule server's API. OpenResty routes the grayscale service request based on the result returned by the grayscale rule server.

[0087] Correspondingly, the architecture of the grayscale release service instance cluster built through the above operations includes the following:

[0088] (1) Master Node: The Master Node serves as the management node of the grayscale release service instance cluster. It manages all nodes in the cluster, sends functions such as creating containers, deploying containers, and publishing containers to cluster nodes through the master server, and receives and distributes data from outside the cluster.

[0089] (2) Worker Node: Worker Node serves as the working node of the grayscale release service instance cluster, runs in the application container, and provides the Kubernetes runtime environment;

[0090] (3) Pod (Container Set), Pod is the smallest deployment unit of Kubernetes. Each Pod contains at least one container. Containers share storage, network, and runtime configuration. In a Kubernetes cluster, Pod can be scheduled and run on Worker Nodes and can be controlled by high-level APIs including Deployment and StatefulSet to implement application management functions including replica control and rolling upgrades.

[0091] (4) Canary Environment container: As a grayscale environment container, the Canary Environment container is used to deploy the latest version of the service application when grayscale release is performed. The grayscale environment is usually configured to receive only a portion of the traffic (for example, based on specific user attributes or percentages) so that the new version of the service application can be tested in this environment without affecting all users.

[0092] (5) Production Environment container: The Production Environment container serves as the production environment container. This container receives most (or all if there is no grayscale release) user traffic and is used to run the current stable version of the service application. When the new version of the service application is tested in the grayscale environment and is considered stable, the new version of the service application will be deployed in the production environment container and replace the current stable version of the service application.

[0093] (6) Canary Rules Server container: The Canary Rules Server container is used to run services that store and manage canary rules. Canary rules include rules that determine which HTTP requests should be routed to the canary environment and which HTTP requests should be routed to the production environment. The canary rules are based on various attributes, including the user's IP address, browser type, and login status.

[0094] (7) OpenResty container. OpenResty runs in the OpenResty container. OpenResty is configured as a proxy server to provide the following services: receiving all incoming traffic and routing the traffic to the grayscale environment container or the production environment container according to the grayscale rules provided by the grayscale rule server.

[0095] The Worker Node consists of the Kubelet component, the Kube-proxy component, and the Container Runtime. The Kubelet component maintains the Pod lifecycle, performs tasks such as starting and stopping containers, and communicates with the Master Node. The Kube-proxy component implements the Kubernetes service's network rules and forwards connections. The Container Runtime component is responsible for running containers, such as Docker and containerd.

[0096] The resource monitoring module interacts with the service instance cluster module and is used to regularly collect resource monitoring measurement data based on resource monitoring metrics. The resource monitoring measurement data includes basic resource monitoring measurement data that reflects system resource usage and customized monitoring measurement data.

[0097] As a specific implementation, the resource monitoring module includes Kubelet components, Metrics Server (cluster-level measurement service) components, Prometheus (service monitoring system), and Prometheus Adapter components (adapters that convert monitoring data into Kubernetes API objects).

[0098] The Kubelet component is the core component of each node in the Kubernetes cluster. As a proxy service, it is used to communicate with the containers running on the corresponding node and manage the container lifecycle and resource allocation. The Kubelet component is used to regularly collect basic resource monitoring measurement data, such as the container's CPU (Central Processing Unit), memory, disk, and network bandwidth usage, and send the basic resource monitoring measurement data to the Metrics Server component.

[0099] Metrics Server is a component of Kubernetes that periodically collects basic resource monitoring metrics from the Kubelet component of each node and sends them to Prometheus.

[0100] Prometheus is a standalone monitoring system. By adding a scraping configuration to the Prometheus configuration file, Prometheus can scrape custom monitoring metrics from containers in pods annotated with Prometheus annotations. Prometheus regularly scrapes resource monitoring metrics from the Metrics Server component and pods, storing them in its own time series database to provide decision-making data for the autoscaling module.

[0101] Prometheus Adapter is used as a bridge between Prometheus and Kubernetes. It implements the Kubernetes Metrics API, allowing Kubernetes to query and use resource monitoring metrics data stored in Prometheus.

[0102] As an improvement, Prometheus provides a query interface that supports users to query historical and real-time resource monitoring measurement data. Prometheus is configured with alarm rules, which trigger alarms when resource monitoring measurement data meets the alarm conditions defined in the alarm rules.

[0103] Through the collaborative work of these components, the resource monitoring module can effectively collect, store, and use metric data to support the system's automatic scaling and phased releases. Data collection is a continuous cycle. The resource monitoring module continuously collects, processes, and provides metric data to support the automatic scaling module in making accurate decisions.

[0104] Correspondingly, in this embodiment, the resource monitoring module is configured through the following steps:

[0105] (1) Install the Kubelet component: Run the Kubelet component on each Kubernetes worker node. During the installation of the Kubelet component, configure the Kubelet component to regularly collect basic resource monitoring metrics data;

[0106] (2) Install the Metrics Server component: Install the Metrics Server component in the Kubernetes cluster. The Metrics Server component automatically collects basic resource monitoring metrics data from the Kubelet component of each node;

[0107] (3) Install Prometheus: Install Prometheus in the Kubernetes cluster and add a new scrape configuration to the Prometheus configuration file. Based on the new scrape configuration, Prometheus can scrape custom monitoring metrics for containers in pods annotated with Prometheus. Prometheus will periodically scrape resource monitoring metrics from the Metrics Server component and pods and store them in its own time series database.

[0108] (4) Install Prometheus Adapter: Install Prometheus Adapter in the Kubernetes cluster and define how to query resource monitoring metrics from Prometheus in the Prometheus Adapter configuration file, including defining the name of the resource monitoring metrics, query statements, and query cycles. Prometheus Adapter provides a Metrics API interface that allows Kubernetes to query and use resource monitoring metrics stored in Prometheus.

[0109] The auto-scaling module interacts with the resource monitoring module and the service instance cluster module respectively, and provides a user interface through the API server. This interface supports users in configuring resource utilization thresholds and the auto-scaling range of grayscale released service instances to form a configuration file. Based on resource monitoring metrics, pre-configured resource utilization thresholds, and the auto-scaling algorithm, it determines whether to create new service instances or destroy unhealthy service instances, calculates the number of service instances that need to be adjusted, and cooperates with the service instance cluster module to scale the service instance cluster based on the calculation results.

[0110] As an improvement, the auto-scaling module includes APIServer (a module that provides a user interface), HPA components, Scale API (an application programming interface for changing the size of scalable resource objects), Deployment (a resource object that describes how to deploy an application), and ReplicaSet (a replication controller).

[0111] The APIServer provides a user interface that supports users in creating HPA components. This interface allows users to set resource usage thresholds, such as the system's QPS (Query Per Second), CPU usage, and memory usage, through a YAML configuration file; and to set the automatic scaling range of service instances for grayscale releases (i.e., the maximum and minimum number of service instances).

[0112] The Horizontal Pod Autoscaler (HPA) component is an automatic scaling component in Kubernetes. It is used to periodically obtain resource monitoring metrics from the Prometheus Adapter of the resource monitoring module, compare the obtained resource monitoring metrics with the preset resource utilization threshold, calculate the number of service instances that need to be adjusted, and implement scaling by calling the Scale API based on the calculation results.

[0113] The Scale API is a resource provided by Kubernetes for changing the number of Pods defined in Deployments and ReplicaSets. For example, when using the Scale API to change the number of Pods in a Deployment, you are actually changing the spec.replicas field (which represents the desired number of Pods) in the Deployment object. Once spec.replicas changes, the Deployment controller automatically creates or deletes Pods to ensure that the current number of Pods equals spec.replicas.

[0114] A Deployment is a high-level controller that controls ReplicaSets by defining the desired number of Pod replicas and Pod templates, thereby implementing application operations such as scaling, updating, and rolling back.

[0115] A Deployment manages a set of ReplicaSets, which in turn manage a set of Pods. A Deployment scales a cluster of service instances for a grayscale release using rolling updates. The specific operations are as follows:

[0116] (1) When creating a Deployment, the Deployment controller creates a new ReplicaSet. The ReplicaSet's template comes from the Deployment's Pod template, and the number of replicas comes from the Deployment's spec.replicas field. The ReplicaSet controller creates a new set of Pods based on the ReplicaSet's template and the number of replicas.

[0117] (2) When updating a Deployment's Pod template, the Deployment controller creates a new ReplicaSet and a new set of Pods based on the new Pod template. The Deployment controller gradually reduces the number of replicas of the old ReplicaSet and increases the number of replicas of the new ReplicaSet until the number of replicas of the old ReplicaSet is 0 and the number of replicas of the new ReplicaSet is equal to the spec.replicas field of the Deployment.

[0118] This example creates an HPA using the Kubernetes API server. This requires creating a YAML file that contains reference resource metric usage thresholds, such as system QPS, CPU usage, and memory usage, as well as the maximum and minimum number of service instances. This YAML file is then applied using the kubectl command.

[0119] After the HPA is created, it periodically retrieves the specified resource monitoring metrics from the Prometheus Adapter, compares the obtained monitoring data with the user-defined resource usage thresholds, and calculates the number of service instances that need to be adjusted. The HPA adjusts the number of service instances using the Scale API. When the HPA determines that the number of service instances needs to be adjusted, it modifies the spec.replicas field in the corresponding Deployment object. The Deployment controls the number of Pods through rolling updates. This process is automatic and requires no intervention.

[0120] The automatic scalable and non-intrusive grayscale release system of this embodiment can not only ensure service performance and stability, but also effectively manage resources and implement grayscale release, providing strong support for iterative updates of services.

[0121] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the scope of protection of the present invention.

Claims

1. An automatically scalable non-intrusive grayscale publishing system, characterized in that: Includes service instance cluster module, resource monitoring module and automatic scaling module; The service instance cluster module is used to create and manage service instances in a Kubernetes cluster using Kubernetes' Deployment or StatefulSet, including the following: exposing resource monitoring metric data, running the containerized grayscale environment, production environment, grayscale rule server, and OpenResty in the Kubernetes cluster, and implementing non-intrusive grayscale release services through OpenResty, wherein the resource monitoring metric data includes basic resource monitoring metric data and customized monitoring metric data; The resource monitoring module interacts with the service instance cluster module to periodically collect resource monitoring metric data, wherein the resource monitoring metric data includes basic resource monitoring metric data reflecting system resource usage and customized monitoring metric data; The automatic scaling module interacts with the resource monitoring module and the service instance cluster module respectively, and provides a user interface through the API Server, which is used to support users to configure the resource utilization threshold and the automatic scaling range of the grayscale release service instance through the user interface to form a configuration file; based on the resource monitoring measurement data, the pre-configured resource utilization threshold and the automatic scaling algorithm, it is used to determine whether it is necessary to create a new service instance or destroy an unhealthy service instance, calculate the number of service instances that need to be adjusted, and cooperate with the service instance cluster module to scale the service instance cluster according to the calculation results.

2. According to claim 1, the automatic scalable non-intrusive grayscale publishing system is characterized in that: Implementing non-intrusive grayscale release services through OpenResty includes the following steps: After the HTTP request reaches OpenResty, the Lua module in OpenResty extracts the identifier in the HTTP request and transmits the identifier to the grayscale rule server by calling the grayscale rule server's API. The identifier includes the IP address, browser type, and login status. Grayscale rule server uses pre-configured grayscale rules to determine whether the accepted logo is grayscale Identify and return the judgment result to OpenResty; OpenResty routes the corresponding HTTP request to the production environment or the grayscale environment based on the received judgment results.

3. The automatic scalable non-intrusive grayscale publishing system according to claim 2, characterized in that: The service instance cluster module is used to perform the following steps to establish a non-intrusive grayscale release service instance cluster: Expose application-defined monitoring metric data: The application is an application running in a Pod. A metric library that supports Prometheus is integrated into the application as a client library. The application creates an HTTP endpoint through the client library and provides the application's customized metric data on the HTTP endpoint. The format of the metric data conforms to the data format of Prometheus. The data collection mechanism of Prometheus collects the application's customized metric data, including request latency, query rate per second, request error rate, queue length, and business indicators. Containerization: Perform the following four operations: Containerize the OpenResty service, grayscale rule management service, production environment, and grayscale environment. That is, create a corresponding Dockerfile for each service. The Dockerfile contains all dependencies and configuration files of the corresponding service, as well as a startup script. The startup script dynamically reads configuration parameters from environment variables or configuration maps when the service starts. Deploy the application in Kubernetes: Create a Deployment in Kubernetes. The Deployment defines the operating parameters of the OpenResty container, resource limits, and the number of replicas required to create a cluster of OpenResty instances. In the YAML file that defines the Deployment, add Prometheus annotations in the metadata section of the Pod template. Prometheus annotations are used to tell Prometheus which endpoint to fetch metrics data from. Service discovery and load balancing: Create a Kubernetes Service to route traffic to the OpenResty instance. The Kubernetes Service optionally provides a stable network interface that allows other services and clients to discover and connect to the OpenResty instance. Update grayscale rules: The OpenResty instance periodically calls the API of the grayscale rule server through a scheduled task to obtain the latest grayscale rules. The scheduled task uses the ngx.timer module of OpenResty to periodically send HTTP requests to call the API of the grayscale rule management module; Execute grayscale rules: In the OpenResty configuration file, configure the Lua module. The configured Lua module can extract the identifier in the grayscale service request and pass the grayscale service request to the grayscale rule server by calling the grayscale rule server's API. OpenResty routes the grayscale service request based on the result returned by the grayscale rule server.

4. The automatic scalable non-intrusive grayscale publishing system according to claim 3, characterized in that: The grayscale release service instance cluster includes: Master Node: The Master Node is the management node of the grayscale release service instance cluster. It is used to manage all nodes in the cluster, send functions such as creating containers, deploying containers, and publishing containers to cluster nodes through the master server, and receive and distribute data from outside the cluster. Worker Node, which serves as a working node of the grayscale release service instance cluster, runs in the application container, and provides the Kubernetes runtime environment; Pod, which is the smallest deployment unit of Kubernetes. Each Pod contains at least one container. The containers share storage, network, and running configuration. In a Kubernetes cluster, Pod can be scheduled and run on Worker Nodes and can be controlled by high-level APIs including Deployment and StatefulSet to implement application management functions including replica control and rolling upgrades. Canary Environment container, which is used as a grayscale environment container to deploy the latest version of the service application when grayscale release services are performed; Production Environment container: The Production Environment container is used as a production environment container to run the current stable version of the service application. When a new version of the service application is tested in the grayscale environment and is considered stable, the new version of the service application will be deployed in the production environment container and replace the current stable version of the service application; Canary Rules Server container, which is used as a canary rules server container to run services that store and manage canary rules. Canary rules include rules that determine which HTTP requests should be routed to the canary environment and which HTTP requests should be routed to the production environment. The canary rules are based on various attributes, including the user's IP address, browser type, and login status. An OpenResty container, in which OpenResty runs, and OpenResty is configured as a proxy server to provide the following OpenResty services: receiving all incoming traffic and routing the traffic to a grayscale environment container or a production environment container according to a grayscale rule provided by a grayscale rule server.

5. The automatic scalable non-intrusive grayscale publishing system according to claim 4, characterized in that: The Worker Node includes Kubelet components, Kube-proxy components and Container Runtime. The Kubelet component is used to maintain the life cycle of the Pod, perform tasks including starting and stopping containers, and communicate with the Master Node; The Kube-proxy component implements the network rules of the Kubernetes service and forwards connections; The Container Runtime component is responsible for running containers.

6. The automatic scalable non-intrusive grayscale publishing system according to claim 1, characterized in that: Deployment is used to deploy stateless applications, and StatefulSet is used to deploy stateful applications.

7. The automatic scalable non-intrusive grayscale publishing system according to claim 1, characterized in that: The resource monitoring module includes Kubelet component, Metrics Server component, Prometheus and Prometheus Adapter component. The Kubelet component is the core component of each node in the Kubernetes cluster. It acts as a proxy service to communicate with the container running on the corresponding node and manage the life cycle and Resource allocation: The Kubelet component is used to regularly collect basic resource monitoring metrics data and send the basic resource monitoring metrics data to the Metrics Server component. The basic resource monitoring metrics data includes the container's CPU, memory, disk, and network bandwidth usage; Metrics Server is a component of Kubernetes that collects basic resource monitoring metrics from the Kubelet component of each node and sends them to Prometheus. Prometheus is an independent monitoring system. In the Prometheus configuration file, a scrape configuration is added. Based on the scrape configuration, Prometheus can crawl custom monitoring metric data of containers with Prometheus annotations. Prometheus will regularly crawl resource monitoring metric data from the Metrics Server component and Pod, and store it in its own time series database to provide decision data for the automatic scaling module. Prometheus Adapter is used as a bridge between Prometheus and Kubernetes, implementing the Kubernetes Metrics API, allowing Kubernetes to query and use resource monitoring metrics data stored in Prometheus.

8. The automatic scalable non-intrusive grayscale publishing system according to claim 7, characterized in that: Prometheus provides a query interface that supports users to query historical and real-time resource monitoring measurement data. Prometheus is configured with alarm rules, and alarms are triggered when resource monitoring measurement data meets the alarm conditions defined in the alarm rules.

9. An automatically scalable non-intrusive grayscale publishing system according to any one of claims 1 to 8, characterized in that: The automatic scaling module includes APIServer, HPA components, Scale API, Deployment, and ReplicaSet. The APIServer provides a user interface, and supports users to create HPA components through the user interface, that is, supports users to configure resource usage thresholds and automatic scaling ranges of grayscale publishing service instances, wherein the automatic scaling range of grayscale publishing service instances includes the maximum number and minimum number of service instances; The HPA component is an automatic scaling component in Kubernetes. It is used to regularly obtain resource monitoring metric data from the resource monitoring module, compare the obtained resource monitoring metric data with the preset resource usage threshold, calculate the number of service instances that need to be adjusted, and implement scaling by calling the Scale API based on the calculation results. The Scale API is a resource provided by Kubernetes for changing the number of Pods defined in Deployment and ReplicaSet. Deployment is a high-level controller that controls ReplicaSet by defining the expected number of Pod replicas and Pod templates, thereby implementing application operations, including expansion, update, and rolling back; A Deployment manages a group of ReplicaSets, and manages a group of Pods through ReplicaSets. Deployment scales the grayscale release service instance cluster through rolling updates.

10. The automatic scalable non-intrusive grayscale publishing system according to claim 9, characterized in that: Deployment scales the grayscale release service instance cluster by rolling updates, including the following operations: When creating a Deployment, the Deployment controller creates a new ReplicaSet. The ReplicaSet template comes from the Deployment Pod template, and the number of replicas comes from the Deployment spec.replicas field. The ReplicaSet controller creates a new set of Pods based on the ReplicaSet template and the number of replicas. When updating a Deployment's Pod template, the Deployment controller creates a new ReplicaSet and a new set of Pods based on the new Pod template. The Deployment controller gradually reduces the number of replicas of the old ReplicaSet and increases the number of replicas of the new ReplicaSet until the number of replicas of the old ReplicaSet is 0 and the number of replicas of the new ReplicaSet is equal to the spec.replicas field of the Deployment.

Images