Method, device and equipment for realizing cloud service by cloud native unified control platform

Abstract

The invention relates to the technical field of computers, and discloses a method, device and equipment for a cloud native unified control platform to realize cloud services, and the method comprises the following steps: when a Kubernetes controller monitors that a new deployment file is received in an open source container arrangement platform cluster, analyzing the deployment file, obtaining at least one component included in the deployment file, and storing the at least one component in the open source container arrangement platform cluster; and according to the service type attribute of each component and the operation and maintenance characteristic information corresponding to the component, identifying the type of the component according to the service type attribute of each component, and then realizing the cloud service through the cloud platform plug-in. And the Kubernetes controller creates cloud resources corresponding to the cloud service by using the self-defined plug-in capability of the Kubernetes controller, and realizes resource configuration and deployment required by the component. Through automatic implementation and deployment of the cloud service component, the whole system efficiently runs applications, and the efficiency and flexibility of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device and equipment for implementing cloud services on a cloud-native unified control platform. Background Art

[0002] The open source container orchestration platform (Kubernetes, K8s for short) provides a declarative resource definition template. R&D personnel need to be familiar with the complex resource configuration of Kubernetes to complete basic application deployment and configuration. In the complete declaration cycle of the application, a large number of operation and maintenance features need to be configured, such as application gateway (Ingress), monitoring, dynamic expansion and contraction, traffic control, release strategy, and creation of underlying computing and storage resources. R&D personnel not only need to focus on business logic, but also need to be familiar with the functions of configuring the operation and maintenance level, and even need to understand and complete the corresponding configuration for the infrastructure.

[0003] At the infrastructure level, although applications can be run as containers on Kubernetes clusters, using Kubernetes to shield the complexity of computing resources, as business systems are deeply embedded in the cloud base, the operation and deployment of applications also rely on a large number of cloud product capabilities, such as cloud databases, operation and maintenance monitoring capabilities, cloud gateways, identity and access management (IAM authentication), cloud hard disks, etc. The configuration of these cloud products also requires additional manual settings or Terraform scripts for preliminary preparation. At the same time, in multi-cloud deployment scenarios, there are also various differences in products and configuration methods between different cloud vendors. Summary of the invention

[0004] In view of this, the present invention provides a method, device and equipment for implementing cloud services on a cloud-native unified control platform to solve the problems of separation of concerns of R&D personnel and operation and maintenance personnel, as well as differences in products and configuration methods among different cloud vendors in multi-cloud deployment scenarios.

[0005] In a first aspect, the present invention provides a method for implementing cloud services using a cloud native unified control platform, the method comprising: when a new deployment file is received in an open source container orchestration platform cluster, parsing the deployment file, obtaining at least one component included in the deployment file, and operation and maintenance characteristic information corresponding to the component, wherein the deployment file is a file uploaded in the form of an open application model definition file;

[0006] Identifying a type of the component according to a service type attribute of each component, wherein the service type attribute is one of a plurality of operation and maintenance characteristics;

[0007] When at least one component is identified as a cloud service component based on the type of each component, each cloud service component and all operation and maintenance characteristics corresponding to the cloud service component are respectively transmitted to the cloud platform plug-in corresponding to the cloud service component, so that the first cloud platform plug-in utilizes the operation and maintenance characteristics corresponding to the first cloud service component to implement the cloud service corresponding to the first cloud service component, wherein the first cloud platform plug-in is the cloud platform plug-in corresponding to the first cloud service component, and the first cloud service component is any one of the at least one cloud service components.

[0008] The method for implementing cloud services on a cloud-native unified control platform provided by the present invention has the following advantages: after receiving the deployment file, the open source container orchestration platform controller parses the deployment file. After obtaining the deployment and operation and maintenance characteristics corresponding to the cloud service components in the deployment file, the above-mentioned operation and maintenance characteristics are transmitted to the cloud platform plug-in, and the cloud platform plug-in uses the above-mentioned operation and maintenance characteristics to implement the cloud services required by the application. This solution unifies the configuration of the application layer and the operation and maintenance, infrastructure-related, and cloud product configuration models through the specifications and framework of the open application model definition file. R&D personnel focus on the logical configuration at the business level, operation and maintenance personnel configure operation and maintenance-related attributes, and the underlying IaaS provider provides corresponding cloud capability configuration to reduce the collaboration cost between different teams.

[0009] The Kubernetes controller is used to configure and deploy the resources required by Kubernetes components. The cloud resources corresponding to the cloud service are created in combination with the Kubernetes custom plug-in capabilities, thereby completing the automation of cloud service components, achieving the goals of efficiently running applications in the entire system, and improving system efficiency and flexibility.

[0010] In an optional implementation, when at least one component is identified as a cloud service component based on the type of each component, before each cloud service component and all operation and maintenance characteristic information corresponding to the cloud service component are transmitted to the cloud platform plug-in corresponding to the cloud service component, the method further includes:

[0011] Convert all operation and maintenance characteristic information corresponding to each cloud service component into a cloud service object for subsequent transmission to the cloud platform plug-in.

[0012] Specifically, all operation and maintenance feature information corresponding to each cloud service component is converted into a cloud service object. Modularizing the operation and maintenance feature information not only makes the management and maintenance of cloud services simpler and more consistent, but also greatly improves the portability of cloud service components and their corresponding operation and maintenance feature information. The subsequent cloud platform plug-in receives the cloud service object transmitted by the controller, converts the cloud service object into the parameter format required by the cloud platform, calls the corresponding application programming interface in the software development kit corresponding to the cloud platform, and implements the cloud service corresponding to the cloud service component. The cloud platform plug-in can uniformly manage and automate the configuration and deployment of cloud services, reduce the need for manual configuration, reduce the error rate, and improve efficiency. Through the cloud platform plug-in, R&D personnel do not need to have an in-depth understanding of the complexity of the cloud platform, so that they can focus more on the implementation of business logic. The cloud platform plug-in provides a consistent experience for different cloud platforms, making it easier to deploy and manage applications in a multi-cloud environment.

[0013] In an optional embodiment, when at least one component is identified as a cloud service component based on the type of each component, after each cloud service component and all operation and maintenance characteristics corresponding to the cloud service component are transmitted to the cloud platform plug-in corresponding to the cloud service component, the method further includes:

[0014] Receive access information corresponding to the cloud service fed back by at least one cloud platform plug-in.

[0015] At least one access information is converted into a target format and stored in a preset storage location in the open source container orchestration platform cluster, so as to subsequently extract the access information in the target format from the preset storage location to access the cloud service.

[0016] Specifically, after the cloud platform plug-in implements the cloud service corresponding to the cloud service component, it will return access information related to the cloud service, convert the access information into the target format and store it in the preset storage location, which can achieve unified management and maintenance of cloud service access information and simplify access rights issues. In addition, the access information is stored in the open source container orchestration platform cluster, which can easily expand storage capacity and access control policies according to actual needs to adapt to changing business needs. Extracting access information in the target format from the preset storage location to access the cloud service can ensure that this information can be accessed quickly and reliably when needed, improving the availability and reliability of cloud services.

[0017] In an optional implementation, before transmitting each cloud service component and all operation and maintenance characteristics corresponding to the cloud service component to the cloud platform plug-in corresponding to the cloud service component, the method further includes:

[0018] Extract the cloud platform plugin corresponding to the cloud service component from the pre-built plugin registry.

[0019] Specifically, different cloud platforms correspond to different cloud platform plug-ins. The open source container orchestration platform cluster implements cloud services on the cloud platform through cloud platform plug-ins. Multiple cloud platform plug-ins are stored in a pre-built plug-in registry. Before each cloud service component and all operation and maintenance features corresponding to the cloud service component are transmitted to the cloud platform plug-in corresponding to the cloud service component, the cloud platform plug-in corresponding to the cloud service component needs to be extracted from the pre-built plug-in registry. The pre-built plug-in registry can reduce resource waste because there is no need to build and maintain plug-ins separately for each cloud service component. In addition, the centralized management of the plug-in registry makes it easier to update, maintain, and monitor cloud platform plug-ins, which helps to respond to problems in a timely manner.

[0020] In an optional implementation, when at least one component is identified as an open source container orchestration platform component based on each component type attribute, the open source container orchestration platform component includes:

[0021] How to deploy and manage containers, how to self-heal, how to manage keys and configurations, how to roll back applications, application update strategies, how to collect logs, how to perform service discovery and load balancing, and how to orchestrate storage.

[0022] Specifically, by identifying the components of the container orchestration platform and their corresponding operation and maintenance feature information, it is possible to achieve automated deployment, self-repair, key and configuration management of containers, etc., greatly improving the level of operation and maintenance automation. The self-repair method and application rollback mechanism can ensure rapid recovery when problems occur in containers or services, improve the elasticity and reliability of the system, and the application update strategy can help smoothly deploy new versions of applications and reduce service interruptions during the update process. The service discovery and load balancing mechanism can improve the availability and response speed of services and ensure reasonable distribution of traffic. The storage orchestration method can optimize the use of storage resources and improve storage efficiency and data access speed. The above components realize the functions of automated deployment and expansion of applications, self-healing of faults, dynamic allocation of resources, etc., greatly improving the operation and maintenance efficiency of applications.

[0023] In an optional implementation, when at least one component is identified as an open source container orchestration platform component based on each component type attribute, the method further includes:

[0024] The open source container orchestration platform service is implemented according to the operation and maintenance characteristic information corresponding to each open source container orchestration platform component.

[0025] Specifically, the implementation of the open source container orchestration platform components is carried out according to the corresponding operation and maintenance feature information. R&D personnel develop applications, operation and maintenance personnel configure components related to application deployment and component implementation related operation and maintenance feature information, and the open source container orchestration platform implements each operation and maintenance feature information. R&D personnel focus on the logical configuration at the business level, operation and maintenance personnel configure related operation and maintenance feature information, and the open source container orchestration platform implements each operation and maintenance feature information, reducing the collaboration cost between different teams.

[0026] In an optional implementation, the deployment file is the latest deployment file extracted from the component file library and transmitted to the open source container orchestration platform controller through the open source container orchestration platform interface when the continuous deployment system monitors that there is a file update in the pre-built deployment file library.

[0027] Specifically, the continuous deployment system automatically monitors the pre-built deployment file storage warehouse. Once a new deployment file is monitored, it immediately starts the extraction and forwarding process, reducing manual intervention, speeding up the launch of new versions of applications, and improving deployment efficiency.

[0028] In a second aspect, the present invention provides a device for implementing cloud services on a cloud native unified control platform, the device comprising:

[0029] The parsing module is used to parse the deployment file when a new deployment file is received in the open source container orchestration platform cluster, and obtain at least one component included in the deployment file and operation and maintenance characteristic information corresponding to the component, wherein the deployment file is a file uploaded in the form of an open application model definition file.

[0030] The identification module is used to identify the type of the component according to the service type attribute of each component, wherein the service type attribute is one of the multiple operation and maintenance characteristics.

[0031] A transmission module is used to transmit each cloud service component and all operation and maintenance characteristic information corresponding to the cloud service component to the cloud platform plug-in corresponding to the cloud service component when at least one component is identified as a cloud service component based on the type of each component, so that the first cloud platform plug-in uses the operation and maintenance characteristic information corresponding to the first cloud service component to implement the cloud service corresponding to the first cloud service component, wherein the first cloud platform plug-in is the cloud platform plug-in corresponding to the first cloud service component, and the first cloud service component is any one of the at least one cloud service components.

[0032] The present invention provides a device for implementing cloud services on a cloud-native unified control platform, which has the following advantages:

[0033] After receiving the deployment file, the controller of the open source container orchestration platform parses the deployment file, obtains the deployment and operation and maintenance characteristics corresponding to the cloud service components in the deployment file, and transmits the above operation and maintenance characteristics to the cloud platform plug-in. The cloud platform plug-in uses the above operation and maintenance characteristics to implement the cloud services required by the application. This solution unifies the configuration and operation and maintenance, infrastructure-related, and cloud product configuration models of the application layer through the specifications and framework of the open application model definition file. R&D personnel focus on the logical configuration at the business level, operation and maintenance personnel configure operation and maintenance-related attributes, and the underlying IaaS provider provides corresponding cloud capability configuration to reduce the collaboration cost between different teams.

[0034] The Kubernetes controller is used to configure and deploy the resources required by Kubernetes components. The cloud resources corresponding to the cloud service are created in combination with the Kubernetes custom plug-in capabilities, thereby automating the implementation of cloud service components, achieving the goal of efficiently running applications in the entire system, and improving system efficiency and flexibility.

[0035] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to thereby execute the method for implementing cloud services on the cloud-native unified control platform of the first aspect or any corresponding embodiment thereof.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute a method for implementing cloud services on a cloud-native unified control platform of the first aspect or any corresponding embodiment thereof.

[0037] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the method for implementing cloud services on a cloud-native unified control platform of the above-mentioned first aspect or any corresponding embodiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 It is a flow chart of a method for implementing cloud services on a cloud-native unified control platform provided by an embodiment of the present invention;

[0040] Figure 2It is a flowchart of another method for implementing cloud services on a cloud native unified control platform provided by an embodiment of the present invention;

[0041] Figure 3 It is a flowchart of another method for implementing cloud services on a cloud-native unified control platform provided by an embodiment of the present invention;

[0042] Figure 4 It is a timing diagram of a method for implementing cloud services on a cloud-native unified control platform provided by an embodiment of the present invention;

[0043] Figure 5 This is a structural block diagram of a device for implementing cloud services on a cloud-native unified control platform provided by an embodiment of the present invention;

[0044] Figure 6 It is a schematic diagram of the hardware structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] Kubernetes provides declarative resource definition templates. R&D personnel need to be familiar with Kubernetes' complex resource configuration to complete basic application deployment and configuration. In the complete declaration cycle of an application, a large number of operation and maintenance features need to be configured, such as Ingress, monitoring, dynamic expansion and contraction, traffic control, release strategy, and creation of underlying computing and storage resources. R&D personnel cannot focus on business logic, but also need to be familiar with the functions of configuring the operation and maintenance level, and even need to understand and complete the corresponding configuration for the infrastructure.

[0047] At the infrastructure level, although applications can be run as containers on Kubernetes clusters, using Kubernetes to shield the complexity of computing resources, as business systems are deeply embedded in the cloud base, the operation and deployment of applications also rely on a large number of cloud product capabilities, such as cloud databases, operation and maintenance monitoring capabilities, cloud gateways, IAM authentication, cloud hard disks, etc. The configuration of these cloud products also requires additional manual settings or Terraform scripts for preliminary preparation. At the same time, in multi-cloud deployment scenarios, there are also various differences in products and configuration methods between different cloud vendors.

[0048] To solve the above problems, an embodiment of the present invention provides an embodiment of a cloud-native unified control platform to implement cloud services. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system (computer device) including a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0049] When developers want to deploy their applications in a Kubernetes cluster, they need to develop detailed deployment files based on business requirements. The deployment files cover all components and their corresponding operation and maintenance features required for the application to be deployed to the Kubernetes cluster to achieve the desired operating state. After developing the detailed deployment files, upload the deployment files and / or application packages to the pre-built deployment file library.

[0050] The continuous deployment system obtains deployment files from the pre-built deployment file library and transmits them to the open source container orchestration platform cluster through the open source container orchestration platform interface.

[0051] When the open source container orchestration platform controller monitors that a new deployment file is received in the open source container orchestration platform cluster, this embodiment provides a method for implementing cloud services using a cloud native unified control platform, which can be used for the above-mentioned terminal devices, such as mobile phones, tablet computers, etc. Figure 1 is a flow chart of a method for implementing cloud services on a cloud native unified control platform provided by an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0052] Step S101: parse the deployment file to obtain at least one component included in the deployment file and operation and maintenance characteristic information corresponding to the component.

[0053] Specifically, the deployment file covers all components and their corresponding operation and maintenance features required to deploy the application to the Kubernetes cluster to achieve the desired operating state, and the deployment file is formulated in the form of an open application model definition file.

[0054] Each service required for application deployment is defined as a component in the open application model definition file, and each component is configured to implement the operation and maintenance features required by the component. Among them, one component corresponds to one service, and the above components include cloud service components and / or open source container orchestration platform service components, and cloud service components include cloud service components in different cloud platforms.

[0055] Therefore, it is necessary to parse the new deployment files received in the monitored open source container orchestration platform cluster to obtain the cloud service components and their corresponding operation and maintenance feature information.

[0056] Step S102: Identify the type of the component according to the service type attribute of each component.

[0057] Specifically, one of the operation and maintenance characteristics in the operation and maintenance characteristic information corresponding to each component is a service type attribute, and this operation and maintenance characteristic indicates the type of the component. Therefore, the type of the component needs to be identified according to the service type attribute of each component.

[0058] Step S103: Transmit each cloud service component and all operation and maintenance characteristic information corresponding to the cloud service component to the cloud platform plug-in corresponding to the cloud service component.

[0059] Specifically, after identifying at least one cloud service component according to the service type attribute, each of the cloud service components and all of its corresponding operation and maintenance characteristics are transmitted to the cloud platform plug-in corresponding to the cloud service component, so that the first cloud platform plug-in uses the operation and maintenance characteristic information corresponding to the first cloud service component to implement the cloud service corresponding to the first cloud service component.

[0060] The first cloud platform plug-in is a cloud platform plug-in corresponding to the first cloud service component, and the first cloud service component is any one of the at least one cloud service component. The first cloud platform plug-in implements the creation, query, update and deletion of cloud resources in the first cloud service component according to the above operation and maintenance characteristic information.

[0061] Specifically, cloud service components include computing resources (Elastic Compute Service, ECS for short), storage resources (such as cloud databases), object storage services (Object Storage Service, OSS for short), network resources, load balancing (Server Load Balancer, SLB for short), elastic public IP (Elastic IP Address, EIP for short), virtual private cloud (Virtual Private Cloud, VPC for short), etc.

[0062] The cloud-native unified control platform provided in this embodiment implements a method for cloud services. After receiving the deployment file, the open source container orchestration platform controller parses the deployment file. After obtaining the deployment and operation and maintenance characteristics corresponding to the cloud service components in the deployment file, the above-mentioned operation and maintenance characteristics are transmitted to the cloud platform plug-in. The cloud platform plug-in uses the above-mentioned operation and maintenance characteristics to implement the cloud services required by the application. This solution unifies the configuration of the application layer and the operation and maintenance, infrastructure-related, and cloud product configuration models through the specifications and framework of the open application model definition file. R&D personnel focus on the logical configuration at the business level, operation and maintenance personnel configure operation and maintenance-related attributes, and the underlying IaaS provider provides corresponding cloud capability configuration to reduce the collaboration cost between different teams.

[0063] The Kubernetes controller is used to configure and deploy the resources required by Kubernetes components. The cloud resources corresponding to the cloud service are created in combination with the Kubernetes custom plug-in capabilities, thereby automating the implementation of cloud service components, achieving the goal of efficiently running applications in the entire system, and improving system efficiency and flexibility.

[0064] In an optional embodiment, based on the above embodiment, when at least one component is identified as a cloud service component based on the type of each component, before each cloud service component and all operation and maintenance characteristic information corresponding to the cloud service component are transmitted to the cloud platform plug-in corresponding to the cloud service component, the method further includes:

[0065] Convert all operation and maintenance characteristic information corresponding to each cloud service component into a cloud service object for subsequent transmission to the cloud platform plug-in.

[0066] Specifically, all operation and maintenance characteristic information corresponding to each cloud service component is converted into an object in a unified data structure form, and this object is transmitted to the cloud platform plug-in. The cloud platform plug-in uses this object to implement the cloud service corresponding to the cloud service component.

[0067] Converting all operation and maintenance feature information corresponding to each cloud service component into a cloud service object and modularizing the operation and maintenance feature information not only makes the management and maintenance of cloud services simpler and more consistent, but also greatly improves the portability of cloud service components and their corresponding operation and maintenance feature information.

[0068] In an optional embodiment, based on the above embodiment, the cloud platform plug-in uses this object to implement the cloud service corresponding to the cloud service component. The specific implementation method includes:

[0069] Step a1: receiving a cloud service object, and converting the operation and maintenance characteristic information in the cloud service object into a parameter format required by the cloud platform to create a cloud service instance.

[0070] Step a2, transmitting the converted object to the software development kit corresponding to the cloud platform to implement the application programming interface of the above cloud service, and realizing the cloud service corresponding to the cloud service component.

[0071] Specifically, the cloud platform plug-in acts as an intermediate station, receives the cloud service objects transmitted by the controller, converts the cloud service objects into the parameter format required by the cloud platform, calls the corresponding application programming interface in the software development kit corresponding to the cloud platform, and implements the cloud service corresponding to the cloud service component. The cloud platform plug-in can unify the management and automation of cloud service configuration and deployment, reduce the need for manual configuration, reduce the error rate, and improve efficiency. Through the cloud platform plug-in, R&D personnel do not need to have an in-depth understanding of the complexity of the cloud platform, allowing them to focus more on the implementation of business logic. The cloud platform plug-in provides a consistent experience across different cloud platforms, making it easier to deploy and manage applications in a multi-cloud environment.

[0072] In an optional embodiment, it can be used in the above-mentioned mobile terminal, such as a mobile phone, a tablet computer, etc. Figure 2 is a flow chart of a method for implementing cloud services on a cloud native unified control platform provided by an embodiment of the present invention. Based on any of the foregoing embodiments, when at least one component is identified as a cloud service component based on the type of each component, each cloud service component and all operation and maintenance characteristics corresponding to the cloud service component are transmitted to a cloud platform plug-in corresponding to the cloud service component, such as Figure 2 As shown, the process includes the following steps:

[0073] Step S201: receiving access information corresponding to a cloud service fed back by at least one cloud platform plug-in.

[0074] Specifically, after the cloud platform plug-in implements the cloud service through the above method, it will feedback access information corresponding to the cloud service. The above access information includes instance attribute information such as the cloud service instance identification information, network address, storage address, and network configuration information.

[0075] Step S202: convert at least one access information into a target format.

[0076] Specifically, after receiving the access information, the Kubernetes controller needs to convert the access information into a format that can be recognized and processed by the Kubernetes cluster so that it can be stored in the storage resource object of the Kubernetes cluster.

[0077] Step S203: storing the access information converted into the target format in a preset storage location in the open source container orchestration platform cluster.

[0078] In a specific example, after successfully creating a cloud database instance, the cloud platform plug-in returns access information of the cloud database, such as the instance ID and the intranet address, etc. The controller converts the access information into a preset target format and stores it in a resource object such as a secret or a configuration map in the Kubernetes cluster, so that the access information in the target format can be subsequently extracted from the storage location to access the cloud service.

[0079] Specifically, after the cloud platform plug-in implements the cloud service corresponding to the cloud service component, it will return access information related to the cloud service, convert the access information into the target format and store it in the preset storage location, which can achieve unified management and maintenance of cloud service access information and simplify access rights issues. In addition, the access information is stored in the open source container orchestration platform cluster, which can easily expand storage capacity and access control policies according to actual needs to adapt to changing business needs. Extracting access information in the target format from the preset storage location to access the cloud service can ensure that this information can be accessed quickly and reliably when needed, improving the availability and reliability of cloud services.

[0080] In an optional embodiment, based on any of the foregoing embodiments, before transmitting each cloud service component and all operation and maintenance characteristics corresponding to the cloud service component to the cloud platform plug-in corresponding to the cloud service component, the method further includes:

[0081] Extract the cloud platform plugin corresponding to the cloud service component from the pre-built plugin registry.

[0082] Specifically, for different cloud platforms, the open source container orchestration platform controller implements the predefined interface specifications to create the corresponding cloud platform plug-in. After the cloud platform plug-in is successfully created, it is registered in the plug-in registry of the Kubernetes cluster. When it is necessary to perform operations related to the cloud service component, the cloud platform plug-in corresponding to the cloud service component is searched and loaded from the registry according to the operation and maintenance characteristics of the service type attribute in the cloud service component.

[0083] Specifically, different cloud platforms correspond to different cloud platform plug-ins. The open source container orchestration platform cluster implements cloud services on the cloud platform through cloud platform plug-ins. Multiple cloud platform plug-ins are stored in a pre-built plug-in registry. Before each cloud service component and all operation and maintenance features corresponding to the cloud service component are transmitted to the cloud platform plug-in corresponding to the cloud service component, the cloud platform plug-in corresponding to the cloud service component needs to be extracted from the pre-built plug-in registry. The pre-built plug-in registry can reduce resource waste because there is no need to build and maintain plug-ins separately for each cloud service component. In addition, the centralized management of the plug-in registry makes it easier to update, maintain, and monitor cloud platform plug-ins, which helps to respond to problems in a timely manner.

[0084] In an optional embodiment, based on any of the foregoing embodiments, the open source container orchestration platform component includes:

[0085] How to configure network routing, how to deploy and manage containers, how to self-heal, how to store key and configuration mappings, how to roll back, how to apply update strategies, how to collect logs, how to perform service discovery and load balancing, and how to orchestrate storage.

[0086] Specifically, the Kubernetes controller can deploy, scale, and manage applications based on the open source container orchestration platform components in the deployment file.

[0087] Specifically, by identifying the components of the container orchestration platform and their corresponding operation and maintenance feature information, it is possible to achieve automated deployment, self-repair, key and configuration management of containers, etc., greatly improving the level of operation and maintenance automation. The self-repair method and application rollback mechanism can ensure rapid recovery when problems occur in containers or services, improve the elasticity and reliability of the system, and the application update strategy can help smoothly deploy new versions of applications and reduce service interruptions during the update process. The service discovery and load balancing mechanism can improve the availability and response speed of services and ensure reasonable distribution of traffic. The storage orchestration method can optimize the use of storage resources and improve storage efficiency and data access speed. The above components realize the functions of automated deployment and expansion of applications, self-healing of faults, dynamic allocation of resources, etc., greatly improving the operation and maintenance efficiency of applications.

[0088] In an optional embodiment, based on any of the foregoing embodiments, when at least one component is identified as an open source container orchestration platform component based on each component type attribute, the method further includes:

[0089] The open source container orchestration platform service is implemented according to the operation and maintenance characteristic information corresponding to each open source container orchestration platform component.

[0090] Specifically, after the Kubernetes controller identifies at least one component as an open source container orchestration platform component based on the component type attribute, it implements resource application in the open source container orchestration platform according to the operation and maintenance characteristic information corresponding to each open source container orchestration platform component.

[0091] In a specific example, for a storage orchestration component, its operation and maintenance feature information includes the specified Kubernetes interface version, storage type, storage capacity, recycling policy, access mode, etc. Among them, the storage types include persistent volume (PersistentVolume, referred to as PV), persistent volume claim (PersistentVolumeClaim, referred to as PVC) and storage class (StorageClass). The Kubernetes controller implements the storage orchestration component service according to the above operation and maintenance feature information in the deployment file.

[0092] Specifically, the implementation of the open source container orchestration platform components is carried out according to the corresponding operation and maintenance feature information. R&D personnel develop applications, operation and maintenance personnel configure components related to application deployment and component implementation related operation and maintenance feature information, and the open source container orchestration platform implements each operation and maintenance feature information. R&D personnel focus on the logical configuration at the business level, operation and maintenance personnel configure related operation and maintenance feature information, and the open source container orchestration platform implements each operation and maintenance feature information, reducing the collaboration cost between different teams.

[0093] In an optional embodiment, based on any of the foregoing embodiments, the deployment file is the latest deployment file extracted from the component file library and transmitted to the open source container orchestration platform controller through the open source container orchestration platform interface when the continuous deployment system monitors that there is a file update in the pre-built deployment file library.

[0094] Specifically, the Continuous Deployment (CD) system will monitor whether there are file updates in the pre-built deployment file library regularly or under specific trigger conditions. When there are file updates, it will obtain the application list uploaded by the developer from the pre-built deployment file library, and automatically submit the application list in the pre-built deployment file library to the Kubernetes cluster through the open source container orchestration platform interface. Among them, the application list includes application packages and / or deployment files. The open source container orchestration platform controller only monitors and parses the latest deployment files in the application list.

[0095] Specifically, the continuous deployment system automatically monitors the pre-built deployment file storage warehouse. Once a new deployment file is monitored, it immediately starts the extraction and forwarding process, reducing manual intervention, speeding up the launch of new versions of applications, and improving deployment efficiency.

[0096] In an optional embodiment, based on any of the foregoing embodiments, when a new deployment file is received in the open source container orchestration platform cluster, the deployment file is parsed to obtain the indicator monitoring task component included in the deployment file, and the operation and maintenance characteristic information corresponding to the component, and the corresponding indicator monitoring task component is implemented according to the operation and maintenance characteristic information. After the open source container orchestration platform controller uses the operation and maintenance characteristic information corresponding to the first cloud service component for the first cloud platform plug-in to implement the cloud service corresponding to the first cloud service component, the open source container orchestration platform controller performs adaptive and automatic adjustments based on the various indicator data of the cloud resources created by the cloud service collected by the indicator monitoring task component as follows:

[0097] Step c1, record the current resource configuration C(α j ), and set the weight coefficient α for each business indicator k , and resource adjustment threshold θ and resource adjustment step δ.

[0098] Step c2: Regularly obtain the application-related business indicator value M(α j， , k).

[0099] Step c3, calculating the resource demand adjustment amount according to the algorithm formula.

[0100] Step c4: updating the resource configuration of the application according to the calculated adjustment amount and the resource adjustment formula.

[0101] The resource demand adjustment formula is as follows:

[0102]

[0103] Where C(α j ) is the number of CPU cores currently allocated to the application, and the weight coefficient α of each business indicator is set k and resource adjustment threshold θ and resource adjustment step δ, M(α j， , k) is the obtained relevant business indicator value.

[0104] In a specific example, Cloud Controller obtains the CPU usage (cpu-usage) and memory usage (memory-usage) indicator values ​​of the application from the system monitoring and alarm tool (Prometheus). For example, the CPU usage of the current web-server component is 60% and the memory usage is 40%.

[0105] Calculate the resource demand adjustment amount according to the algorithm formula. For CPU resource adjustment amount calculation:

[0106]

[0107] Among them, α1 = 0.6 (cpu-usage indicator weight), M (α j， , 1) = 0.6 (the CPU usage obtained), θ = 0.5 (the set CPU utilization threshold), δ = 0.1 (the assumed resource adjustment step), C(α j ) is the number of CPU cores currently allocated to the application (assuming that the initial number of cores is 1). The CPU resource adjustment amount is calculated as:

[0108]

[0109] The new CPU core number configuration may become C(α j )+ΔC(α j )=1.072, the Cloud Controller adjusts the computing resources allocated to the web-server component through the cloud platform plug-in, which may be to increase the partial CPU quota of a computing instance.

[0110] The following is a specific example of the overall operation process of the method of the present application. Figure 3 As shown:

[0111] R&D personnel use Kubevela's CUE language configuration template to write an Open Application Model (OAM) application definition file based on business needs. The OAM application definition file includes, for example, Alibaba Cloud database components and Alibaba Cloud database operation and maintenance characteristics.

[0112] Upload the above OAM application definition file to the Git repository. CD will monitor whether there are file updates in the pre-built Git repository periodically or under specific trigger conditions. When there are file updates, it will obtain the OAM application definition file uploaded by the R&D personnel from the Git repository and automatically submit the OAM application definition file to the Kubernetes cluster through the open source container orchestration platform interface (Kubernetes API).

[0113] The open source container orchestration platform controller (Kubernetes operator) parses the above OAM application definition file. When the OAM application definition file includes Kubernetes service components, such as Ingress (network routing), Service (key storage), Deployment (deployment and management of containers), Configmap (configuration mapping), Daemonset (process monitoring), CR (custom resource) components. The Kubernetes API server in the Kubernetes operator implements the deployment of Kubernetes component services according to its corresponding operation and maintenance characteristics.

[0114] When the OAM application definition file includes cloud service components, the CloudController in the Kubernetes operator implements the creation and deployment of cloud services based on their corresponding operation and maintenance characteristics. The following uses the creation of Alibaba Cloud database services as an example to explain the implementation process of cloud service components.

[0115] The Alibaba Cloud database component includes operation and maintenance features such as the custom cloud database name, the cloud platform type to which the cloud database belongs, the database type, the storage capacity, and the security group rules. The above operation and maintenance features are encapsulated into the CloudDatabaseSpecification object.

[0116] The developer customizes a Serial Peripheral Interface (SPI) interface, such as the CloudResourceProvider interface, which includes methods such as createCloudResource, deleteCloudResource, getCloudResourceStatus, and updateCloudResourceConfiguration.

[0117] Alibaba Cloud creates the Alibaba Cloud plug-in by implementing the above interface, develops the AliyunCloudResourceProvider class to implement the CloudResourceProvider interface, and implements the createCloudResource method as follows:

[0118] Step d1, receiving a CloudDatabaseSpecification object.

[0119] Step d2: Convert the CloudDatabaseSpecification object to the parameter format required by Alibaba Cloud Relational Database Service (RDS) to create a database instance.

[0120] Step d3, use Alibaba Cloud's software development kit (Java SDK) to call RDS to create an application programming interface (API);

[0121] Step d4, passing the converted parameters to the API interface;

[0122] Step d5: Create a cloud database instance in Alibaba Cloud and generate corresponding Alibaba Cloud database resources.

[0123] After successfully creating the Alibaba Cloud database, the cloud platform plug-in returns the access information of the Alibaba Cloud database. After converting the above access information into a format that can be recognized and processed by the Kubernetes cluster, it is stored in resource objects such as secrets or configuration maps (ConfigMap) in the Kubernetes cluster.

[0124] In an optional embodiment, the timing diagram of the cloud native unified control platform implementing the cloud service method is as follows: Figure 4 As shown. The developer declares resource requirements, the Kubernetes cluster receives the resource requirements, the Kubernetes API server implements the Kubernetes component service, the Cloud Controller receives the cloud service component, the Cloud Controller calls the cloud service creation interface through the cloud platform plug-in, and the cloud platform (Cloud Provide) creates cloud resources. After the cloud platform creates the cloud resources, it returns the cloud resource access information. The Cloud Controller converts the access information into the target format and stores the access information in resource objects such as secrets or configuration maps (ConfigMap).

[0125] In this embodiment, a device for implementing cloud services on a cloud-native unified control platform is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0126] This embodiment provides a device for implementing cloud services on a cloud native unified control platform, such as Figure 5As shown, it includes: a parsing module 501, an identification module 502, and a transmission module 503.

[0127] The parsing module 501 is used to parse the deployment file when a new deployment file is received in the open source container orchestration platform cluster, and obtain at least one component included in the deployment file and operation and maintenance characteristic information corresponding to the component, wherein the deployment file is a file uploaded in the form of an open application model definition file.

[0128] Among them, the deployment file is the latest deployment file that is extracted from the deployment file library and transmitted to the open source container orchestration platform controller through the open source container orchestration platform interface when the continuous deployment system monitors that there is a file update in the pre-built deployment file library.

[0129] The identification module 502 is used to identify the type of the component according to the service type attribute of each component, wherein the service type attribute is one of the multiple operation and maintenance characteristics.

[0130] When at least one component is identified as an open source container orchestration platform component based on the service type attribute of each component, the open source container orchestration platform component includes:

[0131] How to deploy and manage containers, how to self-heal, how to manage keys and configurations, how to roll back applications, application update strategies, how to collect logs, how to perform service discovery and load balancing, and how to orchestrate storage.

[0132] The transmission module 503 is used to transmit each cloud service component and all operation and maintenance characteristic information corresponding to the cloud service component to the cloud platform plug-in corresponding to the cloud service component when at least one component is identified as a cloud service component based on the type of each component, so that the first cloud platform plug-in uses the operation and maintenance characteristic information corresponding to the first cloud service component to implement the cloud service corresponding to the first cloud service component, wherein the first cloud platform plug-in is the cloud platform plug-in corresponding to the first cloud service component, and the first cloud service component is any one of the at least one cloud service components.

[0133] In an optional embodiment, the apparatus further includes: a processing module 504;

[0134] The processing module 504 is used to convert all operation and maintenance characteristic information corresponding to each cloud service component into a cloud service object, so as to subsequently transmit the cloud service object to the cloud platform plug-in.

[0135] In an optional embodiment, the device further includes a receiving module 505;

[0136] A receiving module 505, configured to receive access information corresponding to a cloud service fed back by at least one cloud platform plug-in;

[0137] The processing module 504 is further used to convert at least one access information into a target format and store it in a preset storage location in the open source container orchestration platform cluster, so as to subsequently extract the access information in the target format from the preset storage location to access the cloud service.

[0138] In an optional embodiment, the processing module 504 is specifically configured to extract a cloud platform plug-in corresponding to the cloud service component from a pre-built plug-in registry.

[0139] In an optional embodiment, the processing module 504 is specifically configured to implement an open source container orchestration platform service according to operation and maintenance characteristic information corresponding to each open source container orchestration platform component.

[0140] The cloud-native unified control platform in this embodiment implements the cloud service in the form of a functional module, where the module refers to an application-specific integrated circuit (ASIC), a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0141] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0142] An embodiment of the present invention provides a device for implementing cloud services on a cloud-native unified control platform. After receiving the deployment file, the open source container orchestration platform controller parses the deployment file. After obtaining the deployment and operation and maintenance characteristics corresponding to the cloud service components in the deployment file, the above-mentioned operation and maintenance characteristics are transmitted to the cloud platform plug-in. The cloud platform plug-in uses the above-mentioned operation and maintenance characteristics to implement the cloud services required by the application. This solution unifies the configuration of the application layer and the operation and maintenance, infrastructure-related, and cloud product configuration models through the specifications and framework of the open application model definition file. R&D personnel focus on the logical configuration at the business level, operation and maintenance personnel configure operation and maintenance-related attributes, and the underlying IaaS provider provides corresponding cloud capability configuration to reduce the collaboration cost between different teams.

[0143] The Kubernetes controller is used to configure and deploy the resources required by Kubernetes components. The cloud resources corresponding to the cloud service are created in combination with the Kubernetes custom plug-in capabilities, thereby automating the implementation of cloud service components, achieving the goal of efficiently running applications in the entire system, and improving system efficiency and flexibility.

[0144] An embodiment of the present invention further provides a computer device, Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0145] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include an integrated circuit. The integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0146] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0147] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0148] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0149] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0150] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0151] The embodiment of the present invention also provides a computer-readable storage medium. The method provided in the above embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or is implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0152] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0153] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for implementing cloud services using a cloud native unified control platform, characterized in that: The method is executed by an open source container orchestration platform controller, and the method includes: When a new deployment file is received in the open source container orchestration platform cluster, the deployment file is parsed to obtain at least one component included in the deployment file and operation and maintenance characteristic information corresponding to the component, wherein the deployment file is a file uploaded in the form of an open application model definition file; Identifying the type of the component according to a service type attribute of each of the components, wherein the service type attribute is one of the multiple operation and maintenance characteristics; When at least one of the components is identified as a cloud service component based on the service type attribute of each of the components, each of the cloud service components and all operation and maintenance characteristic information corresponding to the cloud service component are respectively transmitted to the cloud platform plug-in corresponding to the cloud service component, so that the first cloud platform plug-in utilizes the operation and maintenance characteristic information corresponding to the first cloud service component to implement the cloud service corresponding to the first cloud service component, wherein the first cloud platform plug-in is the cloud platform plug-in corresponding to the first cloud service component, and the first cloud service component is any one of the at least one cloud service components.

2. The method according to claim 1, characterized in that When at least one of the components is identified as a cloud service component based on the type of each of the components, before transmitting each of the cloud service components and all operation and maintenance characteristic information corresponding to the cloud service component to the cloud platform plug-in corresponding to the cloud service component, the method further includes: All operation and maintenance characteristic information corresponding to each of the cloud service components is converted into a cloud service object, so as to subsequently transmit the cloud service object to the cloud platform plug-in.

3. The method according to claim 1, characterized in that: When at least one of the components is identified as a cloud service component based on the service type attribute of each of the components, after each of the cloud service components and all operation and maintenance characteristics corresponding to the cloud service components are transmitted to the cloud platform plug-in corresponding to the cloud service component, the method further includes: Receiving access information corresponding to the cloud service fed back by at least one of the cloud platform plug-ins; At least one of the access information is converted into a target format and stored in a preset storage location in the open source container orchestration platform cluster, so as to subsequently extract the access information in the target format from the preset storage location to access the cloud service.

4. The method according to any one of claims 1 to 3, characterized in that: Before respectively transmitting each of the cloud service components and all operation and maintenance characteristics corresponding to the cloud service components to the cloud platform plug-in corresponding to the cloud service component, the method further includes: A cloud platform plug-in corresponding to the cloud service component is extracted from a pre-built plug-in registry.

5. The method according to any one of claims 1 to 3, characterized in that: When at least one of the components is identified as an open source container orchestration platform component based on the service type attribute of each of the components, the open source container orchestration platform component includes: How to deploy and manage containers, how to self-heal, how to manage keys and configurations, how to roll back applications, application update strategies, how to collect logs, how to perform service discovery and load balancing, and how to orchestrate storage.

6. The method according to any one of claims 1 to 3, characterized in that: When at least one of the components is identified as an open source container orchestration platform component based on the service type attribute of each of the components, the method further includes: The open source container orchestration platform service is implemented according to the operation and maintenance characteristic information corresponding to each of the open source container orchestration platform components.

7. The method according to any one of claims 1 to 3, characterized in that: The deployment file is the latest deployment file that is extracted from the deployment file library and transmitted to the open source container orchestration platform controller through the open source container orchestration platform interface when the continuous deployment system monitors that there is a file update in the pre-built deployment file library.

8. A device for implementing cloud services using a cloud native unified control platform, characterized in that: The device comprises: A parsing module, configured to parse a new deployment file when monitoring that the open source container orchestration platform cluster receives the new deployment file, and obtain at least one component included in the deployment file and operation and maintenance characteristic information corresponding to the component, wherein the deployment file is a file uploaded in the form of an open application model definition file; an identification module, configured to identify the type of the component according to a service type attribute of each of the components, wherein the service type attribute is one of the operation and maintenance characteristics; A transmission module, for transmitting each of the cloud service components and all operation and maintenance characteristic information corresponding to the cloud service component to a cloud platform plug-in corresponding to the cloud service component, respectively, when at least one of the components is identified as a cloud service component based on the type of each of the components, so that the first cloud platform plug-in utilizes the operation and maintenance characteristic information corresponding to the first cloud service component to implement the cloud service corresponding to the first cloud service component, wherein the first cloud platform plug-in is a cloud platform plug-in corresponding to the first cloud service component, and the first cloud service component is any one of the at least one cloud service components.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for implementing cloud services on a cloud-native unified control platform as described in any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for implementing cloud services on a cloud-native unified control platform according to any one of claims 1 to 7.

Images