An isolation and security system for use in multi-tenant container orchestration

Abstract

The invention relates to an isolation and security system running on a server for use in multi-tenant container orchestration. The system of the invention aims to improve security and efficiency by providing isolated network, storage, and control plane environments for each tenant on multi-tenant container orchestration platforms. In this context, network isolation is achieved by using tenant-specific network overlay protocols, unique VXLAN Network Identifier (VNI), distributed routing mechanisms and network policy engine. Complete storage isolation and performance continuity are ensured through separate storage pools, encryption keys, and a storage QoS manager, for each tenant. Control plane isolation is supported by tenant-specific schedulers, admission controllers, and API endpoints, thus, an independent and secure management is offered. Tenant management processes are made efficient with automatic onboarding, dynamic resource allocation, and isolation verifier, and security is guaranteed. Isolated monitoring and log management are carried out with metric aggregators, log managers and anomaly detection engines, and the data integrity and security are guaranteed throughout the system in line with the principle of zero trust architecture.

Description

[0001] AN ISOLATION AND SECURITY SYSTEM FOR USE IN MULTI-TENANT CONTAINER ORCHESTRATION

[0002] Technical Field of the Invention

[0003] The invention relates to an isolation and security system running on a server for use in multi-tenant container orchestration.

[0004] State of the Art

[0005] Container orchestration platforms are a system which automates the deployment, management, scaling, and networking of containers including software applications. These platforms are critical for the easy management of applications running with a microservice architecture in modern software development processes.

[0006] Various mechanisms have been developed to meet isolation and security requirements in multi-tenant environments, especially in systems such as container orchestration systems, which are in the state of the art; however, these mechanisms face some shortcomings and technical problems in real-world applications. For example, although namespace-based isolation provides a logical separation, it cannot guarantee a true source isolation. Therefore, a malicious pod running on the same node can gain access to the resources of other namespaces, leading to security vulnerabilities.

[0007] Role-Based Access Control (RBAC) in the state of the art regulates users' access to certain resources, but cannot provide a complete isolation at the network and storage levels. This increases data security risks by causing a high-authority user to access other tenants' data. Similarly, while network policies may be used to control traffic between pods, they are difficult to manage in complex network configurations and cannot always provide a complete isolation. An incorrectly configured network policy may lead to the unwanted network access and data leaks.

[0008] In the state of the art, the resource quotas limit the use of resources per namespace; however, it cannot offer an isolation at the node level. This may cause one tenant to consume excessive resources and to negatively impact the performance of othertenants on the same node. Additionally, while container runtime sandboxing solutions provide an additional layer of security for containers, they are not available for all applications due to the performance overhead and platform compatibility challenges.

[0009] In the state of the art, the storage isolation is another major area of vulnerability, as the existing solutions often use a shared storage infrastructure. This may cause tenants to access each other's storage or affect their performance. In addition, the shortcomings in the control plane isolation lead to problems such as the ability of one tenant's operations on the control plane to affect the other tenants. All of these shortcomings cause critical problems in ensuring security, resource isolation, and performance in a sustainable manner in multitenant environments.

[0010] Due to all these limitations in the state of the art, it has been necessary to introduce an isolation and security system for use in multi-tenant container orchestration.

[0011] Summary and Objectives of the Invention

[0012] The invention discloses an isolation and security system running on a server for use in multi-tenant container orchestration. The system of the invention aims to improve security and efficiency by providing isolated network, storage, and control plane environments for each tenant on multi-tenant container orchestration platforms. In this context, network isolation is achieved by using tenant-specific network overlay protocols, unique VXLAN Network Identifier (VNI), distributed routing mechanisms and network policy engine. Complete storage isolation and performance continuity are ensured through separate storage pools, encryption keys, and a storage QoS manager, for each tenant. Control plane isolation is supported by tenant-specific schedulers, admission controllers, and API endpoints, thus, an independent and secure management is offered. Tenant management processes are made efficient with automatic onboarding, dynamic resource allocation, and isolation verifier, and security is guaranteed. Isolated monitoring and log management are carried out with metric aggregators, log managers and anomaly detection engines, and the data integrity and security are guaranteed throughout the system in line with the principle of zero trust architecture.The object of the invention is to provide fully isolated network environments for each tenant on multi-tenant container orchestration platforms. In this context, network isolation between tenants is ensured by using special network overlay protocols and a unique tenant-specific VXLAN Network Identifier (VNI). In addition, distinct routing tables are managed for each tenant by the distributed routing mechanism, and intertenant traffic is tightly controlled thanks to the network policy engine. This increases security and prevents the unwanted interference on the network.

[0013] An object of the invention is to ensure data security and performance continuity by providing complete storage isolation between tenants. In the invention, it is aimed to prevent data sharing by creating separate storage pools for each tenant and to increase the security of data with tenant-specific encryption keys. In addition, the manager of the storage quality of service (QoS-Quality of Service) provides performance isolation for each tenant and prevents one tenant from affecting the storage performance of the other.

[0014] Another object of the invention is to provide control plane isolation in container orchestration platforms. To this end, a tenant-aware scheduler that plan tenant-specific workloads, customized admission controllers, and tenant-specific API endpoints have been introduced. These factors ensure the tenants to operate independently and securely, while at the same an independent management from other tenants is provided in the control plane.

[0015] The invention increases user experience and operational efficiency by automating the tenant management processes. The automated tenant onboarding process ensures new tenants to be quickly onboarded to the system with all resources. The resource allocation engine provides a dynamic and efficient resource allocation to tenants. In addition, the isolation status is constantly checked by means of the tenant isolation verifier, and the security is guaranteed.

[0016] The invention aims to improve system performance and security by providing an isolated monitoring and day-to-day management specifically for tenants. Thanks to the isolated metric aggregator, a separate metric aggregation and storage process is carried out for each tenant. The log manager processes and stores logs on a tenant-by-tenant basis. In addition, tenant-specific abnormal behaviors are quickly detected bymeans of the anomaly detection engine, and thus security vulnerabilities are proactively addressed.

[0017] In the invention, all transactions are considered insecure by default by adopting the principle of zero trust architecture, and thus it is aimed to increase security throughout the system. In the invention, user verification processes are strengthened by providing an integration with the external identity and access management (1AM) systems. In the invention, the security policy enforcer ensures data integrity and security by ensuring strict enforcement of security policies in all tenant interactions.

[0018] Description of the Drawings

[0019] Fig. 1. A multi-tenant container orchestration system: Basic components and a work flow diagram. Wherein: A: Start: entry of user requests into the system. 1: Advanced Network Overlay System: A system which provides network isolation between tenants and provides virtual network functions. 2: Distributed Block Storage System: A system which provides permanent, shared and reliable storage area for containers. 3: Orchestration Control Plane: A system which manages the container lifecycle, and performs functions such as scheduling, scaling, and resource allocation. 4: Tenant Management System: A system in which the customized settings such as user accounts, authorizations, resource quotas are managed for each tenant. 5: MultiTenant Monitoring and Logging: A system in which the system-wide metrics, events, and logs are collected, analyzed, and stored. 6: Security Layer: A layer which protects all layers of the system, provides functions such as authentication, authorization, and threat detection and prevention. B: End: Represents the endpoint of the work flow; it is the point at which the processed requests are returned to the user, or the results are transmitted to another system.

[0020] Fig. 2. Multi-tenant container orchestration system: isolation and security architecture -main components and interactions. Wherein; C: Virtual Extensible LAN (VXLAN) network identifiers, D: Distributed routing mechanisms, E: Network policy engine, F: Tenant-specific storage pools, G: Storage quality of service (QoS) manager I: Tenant-aware schedulers, I: Customized admission controllers, J: API server extensions K: Automated onboarding processes L:Dynamic resource allocation engine M:Continuous isolation verification N:Metric aggregation per tenant O:Tenant-specific logmanagement 0:Anomaly detection engine P:Zero trust architecture RJntegration with 1AM systems Q: security policy enforcer.

[0021] Fig. 3. A multi-tenant container orchestration system: A detailed scheme of isolation, safety and resource management mechanisms. Wherein; a: User interface, b: User, c: Application Programming Interface Gateway (API Gateway), d: Control plane, e: API server, f: Authorization, g: Authentication, h: Monitoring, I: logging, i: Data plane, j: Work loader, k: Container runtime, m: Namespace isolation, n: Role-Based Access Control (RBAC), o: Network policies, 6: Resource quotas, p: Storage isolation, r:hardware, s: Physical servers, §: Network, t: Storage.

[0022] Description of the References in the Figures

[0023] 1. Network Overlay System

[0024] 2. Distributed block storage system

[0025] 3. Orchestration control plane module

[0026] 4. Tenant management system

[0027] 5. Multi-tenant monitoring and logging module

[0028] 6. Security layer module

[0029] 1001. Optimizing the network traffic in a tenant-specific manner

[0030] 1002. Determining the resource limits specifically for the tenant

[0031] 1003. Preventing different tenants from accessing each other's data

[0032] 1004. Operating the containers securely

[0033] 1005. Isolating the storage areas between the tenants

[0034] 1006. Preventing the unauthorized access to the control plane

[0035] 1007. Managing and configuring the network policies

[0036] 1008. Defining and implementing the rules of the role-based access control (RBAC) 1009. Monitoring and managing the resource quotas

[0037] 1010. Integrating the Container Runtime Sandboxing solutions

[0038] Detailed Description of the Invention

[0039] The invention relates to an isolation and security system running on a server for use in multi-tenant container orchestration. The system of the invention operates by theintegration of more than one innovative element. The advanced Network overlay system (1) in the system of the invention provides secure communication by optimizing network traffic, thus making it possible for data flow to take place uninterruptedly and safely. The aforementioned network overlay system (1) increases the flexibility of the network by using VXLAN network identifiers. The distributed block storage system (2) in the system of the invention offers high-performance data storage and ensures the data to be accessed securely and quickly by creating tenant-specific storage pools. This structure simplifies data management while also supporting scalability. The orchestration control plane module (3) in the system of the invention increases the overall performance of the system by providing an efficient management of resources. This plane, which includes tenant-aware schedulers, optimizes resource allocation and creates a harmonious working environment between the different components of the system of the invention. The tenant management system (4) in the system of the invention automates the user management and improves the user experience by working with a dynamic resource allocation engine. The aforementioned tenant management system (4) ensures that the resources are allocated quickly and effectively depending on the needs of the users. The multi-tenant monitoring and logging module (5) in the system of the invention monitors system performance and provides reliability and sustainability by detecting anomalies. Said multi-tenant monitoring and logging module (5) assesses the performance of each tenant individually and perform an intervention when necessary by means of the per-tenant metric aggregation feature thereof. In addition, the security layer module (6) in the system of the invention increases the security of the system by applying a zero trust architecture. The aforementioned security layer module (6) continuously monitors all data and communication channels to provide protection against potential threats and thus maintains the integrity of the system.

[0040] An isolation and security system running on a server for use in the multi-tenant container orchestration according to the invention comprises:

[0041] • a network overlay system (1) which increases the flexibility of the network by using network identifiers of virtual extensible local area network (VXLAN-Virtual Extensible LAN), performs communication by optimizing the network traffic and realizes an uninterrupted data flow,• a distributed block storage system (2) which creates tenant-specific storage pools, performs data storage, facilitates data management, and supports scalability,

[0042] • an orchestration control plane module (3) which includes tenant-aware schedulers, optimizes resource allocation, manages the resources, and creates a harmonious working environment between the different components of the system,

[0043] • a tenant management system (4) which automates user management, works with a dynamic resource allocation engine, and allocates resources depending on the needs of users,

[0044] • a multi-tenant monitoring and logging module (5) which monitors system performance, evaluates the performance of each tenant individually and may intervene when necessary, detects an anomaly and aggregates metrics per tenant,

[0045] • a security layer module (6) which establishes security between tenants via the multi-tenant monitoring and logging module (5), implements a zero trust architecture, provides protection against potential threats by continuously monitoring all data and communication channels and maintains the integrity of the system.

[0046] Table 1.

[0047] Module Name Hardware Architecture Description

[0048] The server at which the user requests first arrive, wherein the server usually Start Server X86 / ARM acts as a load balancer or an API gateway. Both architectures can be supported.

[0049] Servers which provide virtual network functions and network isolation between Advanced

[0050] tenants and include the specialized network overlay Server X86 / ARM

[0051] network hardware or software-based system (1)

[0052] solutions (e.g. VXLAN). Both architectures can be supported.

[0053]

[0054] A storage cluster, typically consisting of multiple servers, which provides Distributed block persistent and shared storage and has storage system Server X86 / ARM data replication and fault tolerance (2) capabilities. Both architectures can be preferred by ARM, especially in storage- oriented systems.

[0055] Servers typically clustered for high accessibility, which manage the lifecycle of containers, and perform operations Orchestration

[0056] such as scheduling, scaling and control plane Server X86 / ARM

[0057] resource allocation. Both architectures module (3)

[0058] can be supported, but x86 may be more common in compute power intensive requirements.

[0059] An application server, typically running in conjunction with a database server, in Tenant

[0060] which the tenant-specific settings, the management Server X86 / ARM

[0061] users, and the access control policies system (4)

[0062] are managed. Both architectures can be supported.

[0063] Servers often with a high disk capacity, Multi-tenant in which the system-wide metrics, monitoring and events, and logs are collected, analyzed,

[0064] Server X86 / ARM

[0065] logging module and stored, . Both architectures can be (5) supported, and ARM architecture can be advantageous for data analysis.

[0066] Servers which provide security functions such as authentication, authorization, and threat detection and prevention, and Security layer

[0067] Server X86 / ARM may often contain specialized hardware module (6)

[0068] such as firewalls, IDS / IPS. Both architectures can be supported.

[0069]

[0070] A server, usually the same server as Start, on which the processed requests End Server X86 / ARM are returned to the user, or results are transmitted to another system. Both architectures can be supported.

[0071]

[0072] Said system may be used in systems designed for modern cloud platforms and multitenant environments.

[0073] More specifically, this network overlay system (1), which uses VXLAN (Virtual Extensible LAN) network identifiers, provides a wider isolation between virtual networks. This can be integrated with software-defined networking (SDN) solutions. VXLAN coverage requires high-performance network equipment. Therefore, it is possible to use high-speed switches and routers.

[0074] However, network interface cards (NICs) and Data Processing Units (DPUs) can be used to optimize network speed.

[0075] The distributed block storage system (2) requires high-performance storage units (e.g. NVMe (Non-Volatile Memory Express) SSDs). NVMe SSDs have been preferred, especially for multi-tenant environments that require high data access speeds.

[0076] The aforementioned orchestration control plane module (3) runs on servers and / or local processors. The orchestration control plane module (3) is a central component that manages the system resources. Therefore, high-performance multi-core processors (multi-core CPUs) and a sufficient memory (RAM) are required. It is generally integrated with the virtualized environments.

[0077] Similarly, said tenant management system (4) and the multi-tenant monitoring and logging module (5) also run on the servers.

[0078] Finally, the security layer module (6) requires hardware-based security modules (Trusted Platform Module - TPM, Hardware Security Module - HSM) to encrypt data and verify the system. Therefore, these functions may be performed with advanced firewalls and IPS / IDS devices. Firewalls, intrusion detection systems (IPS) andintrusion prevention systems (IDS) may be preferred for real-time threat detection and pre / post-attack protection.

[0079] In summary, the system of the invention operates as a whole by being supported with virtualization, software-defined networks (SDN), distributed storage solutions and cloud platforms on high-performance data center infrastructures, servers, network hardware and advanced security devices.

[0080] An operation method of an isolation and security system running on a server for use in multi-tenant container orchestration according to the invention comprises the process steps of:

[0081] i. optimizing (1001) the network traffic in a tenant-specific manner by network identifiers (C) of virtual extensible local area network (VXLAN-Virtual Extensible LAN), distributed routing mechanisms (D) and network policy engine (E) modules in the advanced network overlay system (1),

[0082] ii. determining (1002) the resource limits specifically for the tenant by tenant- aware schedulers (I) and customized admission controllers (!) in the orchestration control plane module (3), and the dynamic resource allocation engine (L) in the tenant management system,

[0083] iii. preventing (1003) different tenants from accessing each other's data by the advanced network overlay system (1), tenant-specific Storage Pools (F) in the distributed Block Storage System (2) and continuous isolation verification (M) modules in the tenant management system (4),

[0084] iv. operating (1004) the containers securely by API Server Extensions (J) in the orchestration control plane module (3) and zero trust architecture (P) and security policy enforcer (S) in the security layer module (6),

[0085] v. isolating (1005) the storage areas between the tenants by the tenant-specific storage pools (F) and Storage Quality of Service (QoS) Manager (G) modules in the distributed block storage system (2),

[0086] vi. preventing (1006) the unauthorized access to the orchestration control plane module (3) by zero trust architecture (P), integration with IAM systems (R), and security policy enforcer (S) modules in the security layer module (6), vii. managing and configuring (1007) the Network policies by the network policy engine (E) in the advanced network overlay system (1),viii. defining and implementing (1008) the rules of the role-based access control (RBAC) by the Integration with IAM Systems (R) and the security policy enforcer (S) in the security layer module (6),

[0087] ix. monitoring and managing (1009) the resource quotas by the metric aggregation per tenant (N) in the multi-tenant monitoring and logging module (5), and the dynamic resource allocation engine (L) in the tenant management system (4), x. integrating (1010) the container runtime sandboxing solutions with the third- party solutions integrated via API Server Extensions (J) in the orchestration control plane module (3) and managed by the security layer module (6).

Claims

CLAIMS1. An isolation and security system running on a server for use in multi-tenant container orchestration, characterized in that it comprises:• a network overlay system (1) which increases the flexibility of the network by using network identifiers of virtual extensible local area network (VXLAN-Virtual Extensible LAN), performs communication by optimizing the network traffic and realizes an uninterrupted data flow,• a distributed block storage system (2) which creates tenant-specific storage pools, performs data storage, facilitates data management, and supports scalability,• an orchestration control plane module (3) which includes tenant-aware schedulers, optimizes resource allocation, manages the resources, and creates a harmonious working environment between the different components of the system,• a tenant management system (4) which automates user management, works with a dynamic resource allocation engine, and allocates resources depending on the needs of users,• a multi-tenant monitoring and logging module (5) which monitors system performance, evaluates the performance of each tenant individually and may intervene when necessary, detects an anomaly and aggregates metrics per tenant,• a security layer module (6) which establishes security between tenants via the multi-tenant monitoring and logging module (5), implements a zero trust architecture, provides protection against potential threats by continuously monitoring all data and communication channels and maintains the integrity of the system.

2. An operation method of an isolation and security system running on a server for use in multi-tenant container orchestration, characterized in that it comprises the process steps of:i. optimizing (1001) the network traffic in a tenant-specific manner by network identifiers (C) of virtual extensible local area network (VXLAN- Virtual Extensible LAN), distributed routing mechanisms (D) and network policy engine (E) modules in the advanced network overlay system (1),ii. determining (1002) the resource limits specifically for the tenant by tenant-aware schedulers (I) and customized admission controllers (!) in the orchestration control plane module (3), and the dynamic resource allocation engine (L) in the tenant management system,iii. preventing (1003) different tenants from accessing each other's data by the advanced network overlay system (1), tenant-specific Storage Pools (F) in the distributed Block Storage System (2) and continuous isolation verification (M) modules in the tenant management system (4), iv. operating (1004) the containers securely by API Server Extensions (J) in the orchestration control plane module (3) and zero trust architecture (P) and security policy enforcer (S) in the security layer module (6), v. isolating (1005) the storage areas between the tenants by the tenantspecific storage pools (F) and Storage Quality of Service (QoS) Manager (G) modules in the distributed block storage system (2),vi. preventing (1006) the unauthorized access to the orchestration control plane module (3) by zero trust architecture (P), integration with IAM systems (R), and security policy enforcer (S) modules in the security layer module (6),vii. managing and configuring (1007) the Network policies by the network policy engine (E) in the advanced network overlay system (1), viii. defining and implementing (1008) the rules of the role-based access control (RBAC) by the Integration with IAM Systems (R) and the security policy enforcer (S) in the security layer module (6),ix. monitoring and managing (1009) the resource quotas by the metric aggregation per tenant (N) in the multi-tenant monitoring and logging module (5), and the dynamic resource allocation engine (L) in the tenant management system (4),x. integrating (1010) the container runtime sandboxing solutions with the third-party solutions integrated via API Server Extensions (J) in the orchestration control plane module (3) and managed by the security layer module (6).

Images