Method and system for deploying Ingress controller in host network mode
By using dummy interface and loopback interface to protect the ports of the Ingress controller in the host network mode, and safely reinforced through ingress, the problems of wide exposure and insecurity of the ports of the Ingress controller deployment mode in the existing technology are solved, and the problems of insecurity of information are improved. Performance and security.
Patent Information
- Application Number
- CN202411248553.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-05
AI Technical Summary
The existing Ingress controller deployment methods have problems such as wide port exposure, performance loss and information insecurity, especially in NodePort and LoadBalancer modes.
In host network mode, use dummy interface address and loopback interface address to protect the data face container management port and monitoring port of the Ingress controller, and expose the monitoring API to the outside through ingress, using multiple identity authentication, authentication and access restriction plug-ins for security reinforcement. Ensure that the management address is accessible only from the host where the data face container is deployed.
The security protection of the Ingress controller's data surface container management and monitoring ports is realized, avoiding the risk of information leakage, improving performance and simplifying the deployment process, and ensuring the security of cluster-sensitive information.
Smart Images

Figure CN119030867A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of computer cloud computing, and in particular to a method for deploying an Ingress controller in a host network mode, a system for deploying an Ingress controller in a host network mode, an electronic device, and a computer-readable storage medium. Background Art
[0002] In a Kubernetes cluster (hereinafter referred to as a cluster), an Ingress controller is usually deployed as the cluster's external network traffic entrance. The Nginx Ingress controller officially recommended by Kubernetes shares the data plane and control plane, which has greater security and stability risks. In addition, plug-in development is also very difficult, so there are a large number of third-party Ingress controllers to choose from.
[0003] The Kong Ingress controller uses an architecture that separates the data plane from the control plane. The data plane processes business traffic according to the ingress rules, and the control plane monitors changes in the cluster ingress rules and dynamically updates the data plane configuration. It also provides a large number of plug-and-play plug-ins such as authentication, security, current limiting, monitoring, and logging based on OpenResty. It is the first to introduce the API (Application Programming Interface) gateway into the Ingress controller, which is in line with the development direction of the Kubernetes community and is one of the ideal choices for the Ingress controller. When deploying the Kong Ingress controller (hereinafter referred to as the Ingress controller), in addition to using the officially recommended LoadBalancer for deployment, you can also use the NodePort or hostNetwork host network mode for deployment.
[0004] The Ingress controller contains two containers, the data plane and the control plane. The ports and uses of each container are as follows: 1. Data plane container monitoring: (1) Proxy port, such as 80 / 443, for access to internal services from outside the cluster, which needs to be exposed to the outside of the cluster; (2) Management port, such as 8444, used to update the ingress rule configuration, which is a sensitive port and should restrict non-local access; (3) Monitoring port, such as 8100, provides a monitoring API, and the URI (Uniform Resource Identifier) defaults to " / metrics". It can return sensitive information such as the IP and port of the container inside the cluster and should be securely reinforced. 2. Control plane container monitoring: (1) Web hook port, such as 8008, used to verify the ingress configuration, which should restrict access from outside the cluster; (2) Monitoring port: such as 10255, which provides a monitoring API, contains the Ingress controller status information, and should restrict access from outside the cluster.
[0005] However, the existing deployment method has the following disadvantages:
[0006] 1. Deployment in NodePort mode has the following problems: (1) All cluster nodes will expose the data plane container proxy port of the Ingress controller, which increases the port exposure;
[0007] (2) The NodePort mode requires the use of the cluster kube-proxy component for traffic forwarding, which results in a certain performance loss.
[0008] 2. When using the LoadBalancer mode for deployment, there are the following problems: (1) Each LoadBalancer service requires a load balancing IP address, which is usually provided by the cloud service provider. This is inconvenient for self-built and self-maintained cluster configurations; (2) The LoadBalancer mode uses NodePort at the bottom layer, which has the same problems as the NodePort mode.
[0009] 3. When using hostNetwork mode for deployment, there are the following problems: (1) The monitoring port of the data plane container is directly exposed to the host machine, which poses a risk of information leakage; (2) The data plane container management address is generally configured as the loopback interface address 127.0.0.1, which only allows local access. If the control plane container is deployed separately in Cluster mode, the control plane container cannot connect to the data plane container management address 127.0.0.1, resulting in the inability to update the ingress rule configuration. Therefore, the control plane container can only be deployed using hostNetwork mode; (3) The control plane container does not provide services outside the cluster, but all ports are exposed to the host machine in hostNetwork mode, and the web hook port needs to be accessible by each node in the cluster. It cannot be configured with the loopback interface 127.0.0.1 for security reinforcement. Summary of the invention
[0010] In order to at least solve the problems of wide port exposure, performance loss, and information insecurity existing in the deployment method of the Ingress controller in the prior art, the present disclosure provides a deployment method of the Ingress controller in a host network mode, a deployment system of the Ingress controller in a host network mode, an electronic device, and a computer-readable storage medium. In the host network mode, a dummy interface address and a loopback interface address are used to protect the data plane container management port and the monitoring port of the Ingress controller. In addition to ensuring that the management address can only be accessed from the host machine where the data plane container is deployed, the control plane container accesses the local data plane container management address; the monitoring API is exposed to the outside through ingress, and the monitoring API is securely reinforced using multiple identity authentication, authorization, access restriction and other plug-ins; the security of the data plane container monitoring API containing cluster sensitive information is ensured.
[0011] In a first aspect, the present disclosure provides a method for deploying an Ingress controller in a host network mode, the method comprising:
[0012] Deploy the control plane container and the data plane container on the same host;
[0013] In the host network mode, the local dummy interface is used to listen to the management port and provide it to the control plane container for calling. The monitoring port is listened to on the local loopback interface to provide monitoring data.
[0014] Customize a primary domain name and point it to the local data plane container monitoring address and monitoring port;
[0015] Enable the data plane container monitoring service and create ingress rules;
[0016] The monitoring API of the preset data plane container is exposed to the outside through ingress. When the monitoring server makes a monitoring interface access request through the monitoring API of the preset data plane container, the URI of the request is rewritten according to the created ingress rule, and the request is forwarded to the data plane container monitoring service. The data plane container monitoring service authenticates the request and enables specific security restrictions according to the ingress rule. After the authentication and security restriction checks are passed, the request is forwarded to the first domain name according to the service configuration, and finally the request is forwarded to the local data plane container monitoring address and monitoring port.
[0017] Furthermore, the authentication of the request and the enabling of specific security restrictions according to the ingress rules include:
[0018] Determine the plugins enabled for the service based on the ingress rules;
[0019] Requests are authenticated by account password and apikey through enabled baisc-auth and key-auth plugins;
[0020] Authenticate the permission group to which the requesting user belongs according to the enabled acl (Access Control List) plug-in;
[0021] Perform security checks on the IP address of the monitoring server based on the enabled ip-restriction plugin.
[0022] Furthermore, the method further comprises:
[0023] Call the data plane management port on the dummy interface through the control plane container to update the configuration of the ingress rule.
[0024] Furthermore, the monitoring API of the preset data plane container is obtained through the host IP where the data plane container is located and the rewritten monitoring API URI.
[0025] In a second aspect, the present disclosure provides a deployment system for an Ingress controller in a host network mode, the system comprising:
[0026] A deployment module configured to deploy the control plane container and the data plane container on the same host machine;
[0027] A setting module is configured to use the local dummy interface to listen to the management port in the host network mode, provide it to the control plane container for calling, listen to the monitoring port on the local loopback interface, and provide monitoring data;
[0028] A definition module is set to customize a first domain name and point it to the local data plane container monitoring address and monitoring port;
[0029] Create a module, set it to enable the data plane container monitoring service, and create ingress rules;
[0030] An access module is configured to expose a preset data plane container monitoring API through ingress. When a monitoring server makes a monitoring interface access request through the preset data plane container monitoring API, the URI of the request is rewritten according to the created ingress rule, the request is forwarded to the data plane container monitoring service, and the data plane container monitoring service authenticates the request and enables specific security restrictions according to the ingress rule. After the authentication and security restriction checks are passed, the request is forwarded to the first domain name according to the service configuration, and finally the request is forwarded to the local data plane container monitoring address and monitoring port.
[0031] Further, the access module includes a security reinforcement component, and the security reinforcement component is configured to:
[0032] Determine the plugins enabled for the service based on the ingress rules;
[0033] Requests are authenticated by account password and apikey through enabled baisc-auth and key-auth plugins;
[0034] Authenticate the permission group to which the requesting user belongs according to the enabled acl plug-in; and,
[0035] Perform security checks on the IP address of the monitoring server based on the enabled ip-restriction plugin.
[0036] Furthermore, the system also includes an update module:
[0037] The update module sent is set to call the data plane management port on the dummy interface through the control plane container to update the configuration of the ingress rule.
[0038] Furthermore,
[0039] The monitoring API of the preset data plane container is obtained through the host IP where the data plane container is located and the rewritten monitoring API URI.
[0040] In a third aspect, the present disclosure provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method for deploying an Ingress controller in a host network mode as described in any one of the first aspects.
[0041] In a fourth aspect, the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for deploying an Ingress controller in a host network mode described in any one of the first aspects above is implemented.
[0042] Beneficial effects:
[0043] The present invention discloses a method for deploying an Ingress controller in a host network mode, a system for deploying an Ingress controller in a host network mode, an electronic device, and a storage medium. In the host network mode, a dummy interface address and a loopback interface address are used to protect the data plane container management port and the monitoring port of the Ingress controller. In addition to ensuring that the management address can only be accessed from the host machine on which the data plane container is deployed, the control plane container accesses the local data plane container management address. The monitoring API is exposed to the outside through ingress, and its default URI is rewritten. The monitoring port is accessed through the data plane container monitoring service, and the monitoring API is securely reinforced using multiple identity authentication, authorization, access restriction and other plug-ins. The security of the data plane container monitoring API containing cluster sensitive information is ensured. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 A flowchart of a method for deploying an Ingress controller in a host network mode provided in Embodiment 1 of the present disclosure;
[0045] Figure 2 A flowchart of a prometheus monitoring server accessing a data plane container monitoring API provided in the second embodiment of the present disclosure;
[0046] Figure 3 An architectural diagram of a deployment system of an Ingress controller in a host network mode provided in Embodiment 3 of the present disclosure;
[0047] Figure 4 This is an architecture diagram of an electronic device provided in Embodiment 4 of the present disclosure. DETAILED DESCRIPTION
[0048] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the present disclosure is further described in detail below in conjunction with the drawings and embodiments. It should be understood that the specific embodiments and drawings described herein are only used to explain the present invention, rather than to limit the present invention.
[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence; and, in the absence of conflict, the embodiments in the present disclosure and the features in the embodiments can be arbitrarily combined with each other.
[0050] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The singular forms of "a", "said" and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings.
[0051] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present disclosure, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.
[0052] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned problems existing in the prior art are described in detail below with specific embodiments. It is understandable that in the embodiments of the present application, the execution subject can perform some or all of the steps in the embodiments of the present application, and these steps or operations are only examples. The embodiments of the present application can also perform other operations or deformations of various operations. In addition, each step can be performed in a different order presented in the embodiments of the present application, and it is possible not to perform all the operations in the embodiments of the present application. And, the following several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.
[0053] Figure 1 A flow chart of a method for deploying an Ingress controller in a host network mode provided in the first embodiment of the present disclosure is shown as follows: Figure 1 As shown, the method includes:
[0054] Step S101: deploy the control plane container and the data plane container on the same host machine;
[0055] Step S102: In the host network mode, the local dummy interface is used to monitor the management port, which is provided to the control plane container for calling, and the monitoring port is monitored on the local loopback interface to provide monitoring data;
[0056] Step S103: Customize a first domain name and point it to the local data plane container monitoring address and monitoring port;
[0057] Step S104: Enable the data plane container monitoring service and create ingress rules;
[0058] Step S105: The monitoring API of the preset data plane container is exposed to the outside through ingress. When the monitoring server makes a monitoring interface access request through the monitoring API of the preset data plane container, the URI of the request is rewritten according to the created ingress rule, and the request is forwarded to the data plane container monitoring service. The data plane container monitoring service authenticates the request and enables specific security restrictions according to the ingress rule. After the authentication and security restriction checks are passed, the request is forwarded to the first domain name according to the service configuration, and finally the request is forwarded to the local data plane container monitoring address and monitoring port.
[0059] For the Ingress controller, since the management port monitored by the data plane container is a sensitive port, non-local access should be restricted, and the monitoring port can return sensitive information such as the IP and port of the container inside the cluster, and access also needs to be restricted to ensure data security. However, the existing deployment method of the Ingress controller still has the problems of wide port exposure, performance loss, and information insecurity. To solve this problem, the disclosed embodiment protects the data plane container management port and monitoring port of the Ingress controller by using the dummy interface address and the loopback interface address in the host network mode, and exposes a specific monitoring API to the outside through ingress, and rewrites the access URI to forward the request to the pre-created data plane container monitoring service, so as to implement security reinforcement of the monitoring API using multiple identity authentication, authorization, security restrictions and other plug-ins, and then forward the request to the custom domain name pointing to the local data plane container monitoring address and monitoring port, so as to achieve secure and restricted access to the monitoring port. At the same time, through the management port of the dummy interface, the control plane container on the same host can verify and update the ingress configuration, etc., to control the rules for information agent forwarding, and update and synchronize to the data plane container.
[0060] About dummy interface,
[0061] The coredns component is usually installed in the cluster for cluster domain name resolution. The coredns component enables a dummy interface on each node in the cluster, and the listening address is usually 169.254.20.10. If the coredns component is not installed in the cluster, you can also enable the dummy interface yourself. For example, in the CentOS distribution, the systemd-networkd service can be used to customize the dummy interface and IP address on the host and enable it.
[0062] The dummy interface is similar to the loopback interface 127.0.0.1, which is accessible only to the local machine. However, the dummy interface can also communicate across containers and the host machine, avoiding the problem that when the host service listens to the loopback interface address 127.0.0.1 (which can only be accessed from the local machine), the container deployed on the local machine cannot access the host service.
[0063] In the embodiment of the present disclosure, according to “Table 1: Data plane container deployment configuration”, the data plane container of the Ingress controller is enabled:
[0064] 1. The data plane container uses the host network to expose proxy ports such as 80 and 443, respond to external requests, and forward them to the services in the cluster for processing;
[0065] 2. Listen to the management port on the local dummy interface and provide it to the control plane container for calling to update the ingress rule configuration;
[0066] 3. Listen to the monitoring port on the local loopback interface, provide monitoring data, and only allow local calls.
[0067] By deploying the control plane container and the data plane container on the same host, the control plane container can access the management port through the dummy interface.
[0068] Table 1: Data plane container deployment configuration
[0069]
[0070]
[0071] Then, create a new monitoring user through "Monitoring User KongConsumer Configuration", determine its authentication method, configure the account and password, configure the user's permission group and various security plug-ins, and implement powerful flow control, request forwarding, authentication and authorization functions; provide a basis for subsequent ingress rules to authenticate access requests and enable specific security restrictions.
[0072] Since the IP cannot be directly configured in ingress, define a first domain name and point it to the local data plane container monitoring address and monitoring port. For example, use the hosts plug-in of the coredns component of the cluster to customize a domain name (first domain name) for accessing the original data plane container monitoring API, such as kong.status.api, pointing to the data plane container monitoring API address in "Table 1: Data plane container deployment configuration": 127.0.0.1.
[0073] Then enable the data plane container monitoring service and create ingress rules;
[0074] According to "Table 2: Data plane container monitoring service configuration", enable the data plane container monitoring service:
[0075] 1. The service forwards the request to the external DNS, that is, forwards the ":8100 / metrics" request to "kong.status.api:8100 / metrics";
[0076] 2. This service enables various restriction plug-ins and security reinforcement plug-ins, and the service can be accessed after passing the authentication.
[0077] Table 2: Data plane container monitoring service configuration
[0078]
[0079]
[0080] And create ingress rules according to "Table 3: Ingress rules for accessing the data plane container monitoring API".
[0081]
[0082] The monitoring API of the preset data plane container is exposed externally through ingress, such as http: / / abcd / kong-admin-api / metrics. When the monitoring server makes a monitoring interface access request through the monitoring API of the preset data plane container, the access request is matched according to the created ingress rule, the URI of the request is rewritten (such as changing the / metrics of the general monitoring API to / kong-admin-api / metrics, rewriting the monitoring API, and customizing the access address of the monitoring API), and the request is forwarded to the data plane container monitoring service, and the forwarding address is: http: / / kong-status-api:8100 / metrics; and the plug-in enabled by the service of the data plane container monitoring service is used for identity authentication and specific security restrictions are enabled. After the identity authentication and security restriction checks are passed, the request is forwarded to the first domain name: kong.status.api according to the service configuration, and the request is finally forwarded to the local data plane container monitoring address and monitoring port: http: / / 127.0.0.1:8100 / metrics according to the configured domain name resolution. Thus, safe and reliable access to monitoring data is achieved.
[0083] In the host network mode, the Ingress controller does not use the cluster network, but directly uses the host network. The container port is directly exposed to the host. Instead of listening to the internal IP of the cluster, the 80 / 443 ports are directly listened to on the host instead of being accessed from the cluster. The 80 / 443 ports are directly exposed on the host, and the Ingress controller is only reserved for the currently deployed server. Only this server will expose the 80 / 443 ports of the Ingress controller, and other servers will not be exposed. In addition, since the host network is directly used, the performance will be better.
[0084] Management port, update the proxy forwarding configuration of the data plane container
[0085] The monitoring port is set to be accessible only from inside the data plane container, while the system's monitoring center, such as the prometheus monitoring server, is the monitoring port for accessing the data container from the outside. Due to the loopback interface feature, it can only be accessed from the inside. By modifying the default monitoring API and rewriting the URI, ordinary users cannot guess the correct access to the data plane container monitoring API. At the same time, the account and password of the access request are authenticated, and the apikey authentication is performed, the permission group is authenticated, and the server's IP address is security checked, etc., to ensure that although the monitoring API is exposed, it can be safely protected by the monitoring system. Achieve secure access to this monitoring API from outside the cluster.
[0086] The disclosed embodiment uses a dummy interface address and a loopback interface address to protect the data plane container management port and the monitoring port of the Ingress controller in the host network mode. In addition to ensuring that the management address can be accessed only from the host machine on which the data plane container is deployed, it also enables the control plane container to access the local data plane container management address. The monitoring API is exposed to the outside through ingress, its default URI is rewritten and forwarded to the data plane container monitoring service, and access to the monitoring port is achieved through the data plane container monitoring service. The monitoring API is securely reinforced using multiple identity authentication, authorization, access restriction and other plug-ins. The security of the data plane container monitoring API containing cluster sensitive information is ensured.
[0087] Furthermore, the authentication of the request and the enabling of specific security restrictions according to the ingress rules include:
[0088] Determine the plugins enabled for the service based on the ingress rules;
[0089] Requests are authenticated by account password and apikey through enabled baisc-auth and key-auth plugins;
[0090] Authenticate the permission group to which the requesting user belongs based on the enabled ACL plug-in;
[0091] Perform security checks on the IP address of the monitoring server based on the enabled ip-restriction plugin.
[0092] When the data plane container monitoring service authenticates the request according to the ingress rules and enables specific security restrictions, it can be achieved by setting the corresponding security detection plug-in; the implementation steps include:
[0093] Create a new monitoring user according to "Table 4: Monitoring User KongConsumer Configuration", and its authentication method is baisc-auth, key-auth and acl.
[0094] Table 4: Monitoring user KongConsumer configuration
[0095]
[0096] According to "Table 5: Monitoring User Account Password Secret Configuration", configure the Prometheus monitoring user's account and password.
[0097] Table 5: Monitoring user account password secret configuration
[0098] Secret configuration items Configuration item value Remark stringData.username Prometheus Username of the monitored user stringData.password xxxpasswordxxx Monitor user passwords stringData.kongCredType basic-auth The secret type is account password authentication
[0099] According to "Table 6: APIKey secret configuration of monitoring users", configure the APIKey value of the Prometheus monitoring user.
[0100] Table 6: APIKey secret configuration for monitoring users
[0101] Secret configuration items Configuration item value Remark stringData.key xxxkeyxxx Monitor the user's apikey value stringData.kongCredType key-auth The secret type is apikey authentication
[0102] According to "Table 7: ACL user group secret configuration of monitoring users", configure the permission group to which the Prometheus monitoring user belongs.
[0103] Table 7: ACL user group secret configuration for monitoring users
[0104] Secret configuration items Configuration item value Remark stringData.group admin Monitor the permission groups to which the user belongs stringData.kongCredType acl The secret type is acl authentication
[0105] According to "Table 8: KongPlugin configuration for account and password authentication plug-in", create a basic-auth authentication plug-in.
[0106] Table 8: KongPlugin configuration for account and password authentication plugin
[0107] KongPlugin Configuration Items Configuration item value Remark plugin basic-auth The plugin type is basic-auth authentication plugin
[0108] According to "Table 9: KongPlugin configuration of apikey authentication plug-in", create a key-auth authentication plug-in.
[0109] Table 9: KongPlugin configuration for apikey authentication plugin
[0110] KongPlugin Configuration Items Configuration item value Remark plugin key-auth The plugin type is key-auth authentication plugin config.key_names apikey The key name of the monitored user's apikey config.key_in_query true Allow passing key via request parameters
[0111] Create a new acl authentication plug-in according to "Table 10: KongPlugin configuration of acl authentication plug-in".
[0112] Table 10: KongPlugin configuration for acl authentication plugin
[0113] KongPlugin Configuration Items Configuration item value Remark plugin acl The plugin type is acl authentication plugin config.allow admin Allow users in the admin group to access
[0114] Create a new IP restriction security plugin according to "Table 11: KongPlugin configuration of IP restriction security plugin"
[0115] Table 11: KongPlugin configuration for the IP restriction security plugin
[0116] KongPlugin Configuration Items Configuration item value Remark plugin ip-restriction The plugin type is ip restriction security plugin config.allow xxxx / xx Allow access to the network segment where the prometheus server is located
[0117] After creating the corresponding security plug-in, when an access request is made, the plug-in enabled by the service added in Table 2: Data plane container monitoring service service configuration is used to perform access authentication and enable specific security restrictions, including:
[0118] The data plane container monitoring service enables the IP restriction plug-in to only allow access to the network segment where the prometheus server is located;
[0119] This service enables the basic-auth authentication plug-in, which requires authentication using an account and password;
[0120] This service also enables the key-auth authentication plug-in. In addition to the account password, you also need to use apikey for authentication;
[0121] The service also enables the acl authentication plug-in, allowing access only to users in the admin permission group;
[0122] The created prometheus monitoring user belongs to the admin authority group and is configured with an account password and apikey. The service can be accessed after passing the double authentication. Through the above process, the preset monitoring API exposed after the general URI is rewritten is implemented, and multiple identity authentication, authorization, IP restriction and other plug-ins are used for security reinforcement.
[0123] Furthermore, the method further comprises:
[0124] Call the data plane management port on the dummy interface through the control plane container to update the configuration of the ingress rule.
[0125] Enable the control plane container of the Ingress controller by following "Table 12: Control plane container deployment configuration".
[0126] Table 12: Control plane container deployment configuration
[0127]
[0128] 1. The control plane container uses the cluster network and does not expose ports to the host machine;
[0129] 2. The control plane container must be deployed on the same host as the data plane container through Pod affinity configuration;
[0130] 3. The control plane container is configured through Pod anti-affinity, and only one node can be deployed on a host node;
[0131] 4. The control plane container calls the data plane management interface through the dummy interface to update the ingress rule configuration.
[0132] The cluster network used by the control plane container does not expose any ports to the host machine and can only be accessed within the cluster network. Through some K8s-specific configurations, external access to the cluster is restricted to ensure the security of the port listened by the control plane container.
[0133] On each host, the control plane container and the data plane container are deployed on the same host. The control plane container monitors the changes in the ingress configuration of the entire cluster. When the rule changes, the ingress rules are updated to the proxy through the management port of the dummy interface's listening address 169.254.20.10. When updating the configuration, it can ensure that the configuration of each node can be updated.
[0134] Furthermore, the monitoring API of the preset data plane container is obtained through the host IP where the data plane container is located and the rewritten monitoring API URI.
[0135] The monitoring API of the preset data plane container is obtained through the host address and the rewritten monitoring API URI, so as to access the host through the host IP where the data plane container is located, and then change the general monitoring API / metrics to a specific URI: / kong-admin-api / metrics, so as to rewrite the monitoring API and customize the access address of the monitoring API so that ordinary users cannot guess it.
[0136] The disclosed embodiment uses a dummy interface address and a loopback interface address to protect the data plane container management port and the monitoring port of the Ingress controller in the host network mode. The dummy interface address can prevent external access to the host machine, and the control plane container of the same host can call the data plane management interface through the dummy interface to update the ingress rule configuration. In addition to ensuring that the management address can only be accessed from the host machine where the data plane container is deployed, the control plane container also accesses the local data plane container management address. The monitoring API is exposed to the outside through ingress, and its default URI is rewritten to prevent general users from accessing the monitoring API. The request URI is rewritten through the created ingress rule to forward the request to the data plane container monitoring service, and access to the monitoring port is achieved through the data plane container monitoring service. The monitoring API is reinforced with multiple identity authentication, authorization, access restriction and other plug-ins. The security of the data plane container monitoring API containing cluster sensitive information is ensured.
[0137] Embodiment 2 of the present disclosure further provides a method for deploying an Ingress controller in a host network mode, the purpose of which is to:
[0138] In the host network mode, use the dummy interface address and loopback interface address to protect the data plane container management port and monitoring port of the Ingress controller, expose the monitoring API through ingress, and use multiple identity authentication, authorization, IP restriction and other plug-ins to strengthen the security of the monitoring API.
[0139] 1. Use hostNetwork mode to deploy the data plane container of the Ingress controller, and use Cluster mode to deploy the control plane container of the Ingress controller;
[0140] 2. Use the dummy interface as the management address of the data plane container. In addition to ensuring that the management address can only be accessed from the host machine where the data plane container is deployed, it also enables the control plane container to access the local data plane container management address.
[0141] 3. Expose the monitoring API of the data plane container through ingress, rewrite its default URI, and use the account password + apikey to perform double identity authentication on the request of the monitoring interface. Enable IP restriction for interface access to ensure the security of the data plane container monitoring API containing cluster sensitive information.
[0142] The method comprises:
[0143] 1. Enable dummy interface
[0144] The coredns component is usually installed in the cluster for cluster domain name resolution. The coredns component enables a dummy interface on each node in the cluster, and the listening address is usually 169.254.20.10. If the coredns component is not installed in the cluster, you can also enable the dummy interface yourself. For example, in the CentOS distribution, the systemd-networkd service can be used to customize the dummy interface and IP address on the host and enable it.
[0145] The dummy interface is similar to the loopback interface 127.0.0.1 and is only accessible to the local machine. However, the dummy interface can also communicate across containers and the host machine, avoiding the problem that when the host service listens to the loopback interface address 127.0.0.1, the container deployed on the local machine cannot access the host service.
[0146] 2. Enable the data plane container of the Ingress controller according to "Table 1: Data plane container deployment configuration":
[0147] 1. The data plane container uses the host network to expose proxy ports such as 80 and 443, respond to external requests, and forward them to the services in the cluster for processing;
[0148] 2. The management port is listened on the local dummy interface and provided to the control plane container for calling to update the ingress rule configuration;
[0149] 3. Listen to the monitoring port on the local loopback interface, provide monitoring data, and only allow local calls.
[0150] Table 1: Data plane container deployment configuration
[0151]
[0152] 3. Enable the control plane container of the Ingress controller according to "Table 12: Control plane container deployment configuration":
[0153] 1. The control plane container uses the cluster network and does not expose ports to the host machine;
[0154] 2. The control plane container must be deployed on the same host as the data plane container through Pod affinity configuration;
[0155] 3. The control plane container is configured through Pod anti-affinity, and only one node can be deployed on a host node;
[0156] 4. The control plane container calls the data plane management interface through the dummy interface to update the ingress rule configuration.
[0157] Table 12: Control plane container deployment configuration
[0158]
[0159]
[0160] 4. Create a new monitoring user according to "Table 4: Monitoring User KongConsumer Configuration", and its authentication method is baisc-auth, key-auth and acl.
[0161] Table 4: Monitoring user KongConsumer configuration
[0162]
[0163] 5. According to "Table 5: Monitoring User Account Password Secret Configuration", configure the Prometheus monitoring user's account and password.
[0164] Table 5: Monitoring user account password secret configuration
[0165] Secret configuration items Configuration item value Remark stringData.username Prometheus Username of the monitored user stringData.password xxxpasswordxxx Monitor user passwords stringData.kongCredType basic-auth The secret type is account password authentication
[0166] 6. According to "Table 6: APIKey secret configuration of monitoring users", configure the APIKey value of the Prometheus monitoring user.
[0167] Table 6: APIKey secret configuration for monitoring users
[0168] Secret configuration items Configuration item value Remark stringData.key xxxkeyxxx Monitor the user's apikey value stringData.kongCredType key-auth The secret type is apikey authentication
[0169] 7. According to "Table 7: ACL user group secret configuration of monitoring users", configure the permission group to which the Prometheus monitoring user belongs.
[0170] Table 7: ACL user group secret configuration for monitoring users
[0171] Secret configuration items Configuration item value Remark stringData.group admin Monitor the permission groups to which the user belongs stringData.kongCredType acl The secret type is acl authentication
[0172] 8. According to "Table 8: Account and password authentication plug-in KongPlugin configuration", create a basic-auth authentication plug-in.
[0173] Table 8: KongPlugin configuration for account and password authentication plugin
[0174] KongPlugin Configuration Items Configuration item value Remark plugin basic-auth The plugin type is basic-auth authentication plugin
[0175] 9. According to "Table 9: KongPlugin configuration of apikey authentication plug-in", create a new key-auth authentication plug-in.
[0176] Table 9: KongPlugin configuration for apikey authentication plugin
[0177] KongPlugin Configuration Items Configuration item value Remark plugin key-auth The plugin type is key-auth authentication plugin config.key_names apikey The key name of the monitored user's apikey config.key_in_query true Allow passing key via request parameters
[0178] 10. Create a new acl authentication plug-in according to "Table 10: KongPlugin configuration of acl authentication plug-in".
[0179] Table 10: KongPlugin configuration for acl authentication plugin
[0180] KongPlugin Configuration Items Configuration item value Remark plugin acl The plugin type is acl authentication plugin config.allow admin Allow users in the admin group to access
[0181] 11. Create a new IP restriction security plugin according to "Table 11: KongPlugin configuration of IP restriction security plugin".
[0182] Table 11: KongPlugin configuration for the IP restriction security plugin
[0183] KongPlugin Configuration Items Configuration item value Remark plugin ip-restriction The plugin type is ip restriction security plugin config.allow xxxx / xx Allow access to the network segment where the prometheus server is located
[0184] 12. Customize a first domain name (customized data plane container monitoring API domain name) through the hosts plug-in of the cluster coredns component, such as kong.status.api, pointing to the data plane container monitoring API address in "Table 1: Data plane container deployment configuration": 127.0.0.1.
[0185] 13. Enable the data plane container monitoring service according to "Table 2: Data plane container monitoring service configuration":
[0186] Table 2: Data plane container monitoring service configuration
[0187]
[0188] 1. The service forwards the request to the external DNS, that is, forwards the ":8100 / metrics" request to "kong.status.api:8100 / metrics";
[0189] 2. The service enables the IP restriction plug-in, allowing access only to the network segment where the prometheus server is located;
[0190] 3. The service enables the basic-auth authentication plug-in, which requires authentication using an account and password;
[0191] 4. The service also enables the key-auth authentication plug-in. In addition to the account password, the apikey must also be used for authentication;
[0192] 5. The service also enables the acl authentication plug-in, allowing only users belonging to the admin permission group to access;
[0193] 6. The prometheus monitoring user created in the fourth step belongs to the admin permission group, and is configured with an account password and apikey. The service can be accessed after passing the double authentication.
[0194] 14. Create ingress rules according to "Table 3: Ingress rules for accessing the data plane container monitoring API".
[0195] Table 3: Ingress rules for accessing the data plane container monitoring API
[0196]
[0197] 15. The process of prometheus monitoring server accessing the data plane container monitoring API is as follows Figure 2 :
[0198] 1. The prometheus monitoring server accesses the data plane container monitoring API: http: / / abcd / kong-admin-api / metrics, and provides the account password and apikey authentication information.
[0199] 2. The data plane container matches the access request according to the ingress rule;
[0200] 3. Match the ingress rule created in step 14, rewrite the request URI, and forward the request to the data plane container monitoring service created in step 13. The forwarding address is: http: / / kong-status-api:8100 / metrics;
[0201] 4. The data plane container monitoring service performs account password and apikey authentication based on the enabled baisc-auth and key-auth plug-ins; authenticates the permission group to which the prometheus monitoring user belongs based on the enabled acl plug-in; and performs a security check on the IP address of the prometheus monitoring server based on the ip-restriction plug-in;
[0202] 5. After the authentication and IP restriction checks are passed, the request is forwarded to the external domain name: kong.status.api according to the service configuration;
[0203] 6. According to the domain name resolution configured in step 12, the request is finally forwarded to the local data plane container monitoring address and port: http: / / 127.0.0.1:8100 / metrics.
[0204] The disclosed embodiment deploys the data plane container of the Ingress controller in hostNetwork mode and deploys the control plane container of the Ingress controller in Cluster mode. Then, the dummy interface address and the loopback interface address are used to protect the data plane container management port and the monitoring port of the Ingress controller, and the rewritten monitoring API is exposed to the outside through ingress. The monitoring API is reinforced with multiple identity authentication, authorization, IP restriction and other plug-ins to ensure the security of monitoring data.
[0205] Embodiment 3 of the present disclosure also provides a deployment system of an Ingress controller in a host network mode, such as Figure 3 As shown, the system comprises:
[0206] A deployment module 11, which is configured to deploy the control plane container and the data plane container on the same host machine;
[0207] A setting module 12 is configured to use the local dummy interface to monitor the management port in the host network mode, provide it to the control plane container for calling, monitor the monitoring port on the local loopback interface, and provide monitoring data;
[0208] A definition module 13 is configured to customize a first domain name and point it to a local data plane container monitoring address and a monitoring port;
[0209] Create module 14, which is set to enable the data plane container monitoring service and create ingress rules;
[0210] The access module 15 is configured to expose the monitoring API of the preset data plane container through ingress. When the monitoring server makes a monitoring interface access request through the monitoring API of the preset data plane container, the URI of the request is rewritten according to the created ingress rule, and the request is forwarded to the data plane container monitoring service. The data plane container monitoring service authenticates the request and enables specific security restrictions according to the ingress rule. After the authentication and security restriction checks are passed, the request is forwarded to the first domain name according to the service configuration, and finally the request is forwarded to the local data plane container monitoring address and monitoring port.
[0211] Furthermore, the access module 15 includes a security reinforcement component, and the security reinforcement component is configured as follows:
[0212] Determine the plugins enabled for the service based on the ingress rules;
[0213] Requests are authenticated by account password and apikey through enabled baisc-auth and key-auth plugins;
[0214] Authenticate the permission group to which the requesting user belongs according to the enabled acl plug-in; and,
[0215] Perform security checks on the IP address of the monitoring server based on the enabled ip-restriction plugin.
[0216] Furthermore, the system further comprises an updating module 16:
[0217] The sent update module 16 is configured to call the data plane management port on the dummy interface through the control plane container to update the configuration of the ingress rule.
[0218] Furthermore,
[0219] The monitoring API of the preset data plane container is obtained through the host IP where the data plane container is located and the rewritten monitoring API URI.
[0220] The deployment system of the Ingress controller under the host network mode in the embodiment of the present disclosure is used to implement the deployment method of the Ingress controller under the host network mode in the implementation method embodiment 1 and embodiment 2, so the description is relatively simple. For details, please refer to the relevant description in the previous method embodiment, which will not be repeated here.
[0221] In addition, if Figure 4As shown, the fourth embodiment of the present disclosure further provides an electronic device, including a memory 100 and a processor 200, wherein the memory 100 stores a computer program, and when the processor 200 runs the computer program stored in the memory 100, the processor 200 executes the above-mentioned various possible methods.
[0222] The memory 100 is connected to the processor 200. The memory 100 may be a flash memory, a read-only memory or other memory, and the processor 200 may be a central processing unit or a single-chip microcomputer.
[0223] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to perform the above-mentioned various possible methods.
[0224] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), Digital Video Disc (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.
[0225] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present disclosure, but the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and substance of the present disclosure, and these modifications and improvements are also considered to be within the scope of protection of the present disclosure.
Claims
1. A method for deploying an Ingress controller in a host network mode, characterized in that: The method comprises: Deploy the control plane container and the data plane container on the same host; In the host network mode, the local dummy interface is used to listen to the management port and provide it to the control plane container for calling. The monitoring port is listened to on the local loopback interface to provide monitoring data. Customize a primary domain name and point it to the local data plane container monitoring address and monitoring port; Enable the data plane container monitoring service and create ingress rules; The monitoring application program interface API of the preset data plane container is exposed to the outside through ingress. When the monitoring server makes a monitoring interface access request through the monitoring API of the preset data plane container, the uniform resource identifier URI of the request is rewritten according to the created ingress rule, and the request is forwarded to the data plane container monitoring service. The data plane container monitoring service authenticates the request and enables specific security restrictions according to the ingress rule. After the authentication and security restriction checks are passed, the request is forwarded to the first domain name according to the service configuration, and finally the request is forwarded to the local data plane container monitoring address and monitoring port.
2. The method according to claim 1, characterized in that The request is authenticated and specific security restrictions are enabled according to the ingress rules, including: Determine the plugins enabled for the service based on the ingress rules; The request is authenticated by account password and application interface key apikey through the enabled basic authentication baisc-auth and keyword authentication key-auth plug-ins; Authenticate the permission group to which the requesting user belongs according to the enabled access control list (ACL) plug-in; Perform security checks on the IP address of the monitoring server based on the enabled ip-restriction plugin.
3. The method according to claim 1, characterized in that The method further comprises: Call the data plane management port on the dummy interface through the control plane container to update the configuration of the ingress rule.
4. The method according to claim 1, characterized in that: The monitoring API of the preset data plane container is obtained through the host IP where the data plane container is located and the rewritten monitoring API URI.
5. A deployment system for an Ingress controller in a host network mode, characterized in that: The system comprises: A deployment module configured to deploy the control plane container and the data plane container on the same host machine; A setting module is configured to use the local dummy interface to listen to the management port in the host network mode, provide it to the control plane container for calling, listen to the monitoring port on the local loopback interface, and provide monitoring data; A definition module is set to customize a first domain name and point it to the local data plane container monitoring address and monitoring port; Create a module, set it to enable the data plane container monitoring service, and create ingress rules; An access module is configured to expose a preset data plane container monitoring API through ingress. When a monitoring server makes a monitoring interface access request through the preset data plane container monitoring API, the URI of the request is rewritten according to the created ingress rule, the request is forwarded to the data plane container monitoring service, and the data plane container monitoring service authenticates the request and enables specific security restrictions according to the ingress rule. After the authentication and security restriction checks are passed, the request is forwarded to the first domain name according to the service configuration, and finally the request is forwarded to the local data plane container monitoring address and monitoring port.
6. The system according to claim 5, characterized in that The access module includes a security reinforcement component, and the security reinforcement component is configured to: Determine the plugins enabled for the service based on the ingress rules; Requests are authenticated by account password and apikey through enabled baisc-auth and key-auth plugins; Authenticate the permission group to which the requesting user belongs based on the enabled ACL plug-in; as well as, Perform security checks on the IP address of the monitoring server based on the enabled ip-restriction plugin.
7. The system according to claim 5, characterized in that The system also includes an update module: The update module sent is set to call the data plane management port on the dummy interface through the control plane container to update the configuration of the ingress rule.
8. The system according to claim 5, characterized in that The monitoring API of the preset data plane container is obtained through the host IP where the data plane container is located and the rewritten monitoring API URI.
9. An electronic device, characterized in that: It includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes the deployment method of the Ingress controller in the host network mode according to any one of claims 1 to 4.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method for deploying the Ingress controller in the host network mode according to any one of claims 1 to 4 is implemented.
Citation Information
Patent Citations
Method for automatically configuring accessing control of Docker container
CN106656980A
Container firewall system deployment method
CN107864062A
Method and system for realizing high availability of port or network, terminal and medium
CN113079207A
Network implementation method, system and equipment for running virtual machine in container and medium
CN117076051A
Method, system and device for realizing smooth migration of Ingress controller update and storage medium
CN117827256A
