A method, device and electronic equipment for upgrading a container cloud multi-application

Abstract

The application relates to a container cloud multi-application upgrading method and device and electronic equipment, and is used for meeting the multi-application upgrading requirement of a container cloud in a complex scene. In the method, for a target container cloud, application information of each upgrading step indicated by each upgrading step in application group resource information is read in sequence according to a step order of the upgrading steps, at least the upgrading steps of multiple applications are contained in the application group information, and application information of a single application is indicated for a single upgrading step: if the application information of the single application contains a first upgrading type, the single application is determined to be a first application, a first flow corresponding to the first application and a first sub-workflow are obtained, and the first flow is used for referencing the first sub-workflow; if the application information of the single application contains a second upgrading type, the single application is determined to be a second application, a second flow corresponding to the second application is obtained; and based on the obtained flows, a target workflow for upgrading multiple applications in the target container cloud is generated in the order of the obtained flows.

Description

Technical Field

[0001] This application relates to the field of container cloud technology, and in particular to a method, apparatus and electronic device for upgrading multiple applications in a container cloud. Background Technology

[0002] Container cloud is a platform that uses containers as the resource partitioning and encapsulates the software runtime environment, providing users with a platform for building, deploying, and running distributed applications. Kubernetes (K8s) is the de facto standard for container cloud management and can be used to manage the lifecycle of individual workloads, that is, to manage the lifecycle of a single application running on K8s.

[0003] To address the multi-application upgrade needs arising from application systems composed of multiple applications, software package management is involved. Currently, Kubernetes primarily employs a resource management solution based on Helm Chart packages (or Chart packages for short, a collection of application resource files). The relevant technology involves defining the deployment and upgrade order of multiple application Helm Chart packages to support multi-application upgrades.

[0004] However, in certain complex scenarios, such as those requiring the execution of custom scripts that are executed independently in the order of upgrade steps, the relevant technologies cannot enable the execution of upgrade steps for other applications after the execution of the custom scripts. Therefore, the relevant technologies cannot meet the multi-application upgrade needs of container clouds in complex scenarios. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for upgrading multiple applications in a container cloud, to meet the needs of container cloud for upgrading multiple applications in complex scenarios.

[0006] Firstly, this application provides a method for upgrading multiple applications in a container cloud, the method comprising:

[0007] For the target container cloud, the application information of the application indicated by each upgrade step is read sequentially according to the step order of each upgrade step in the application group resource information; wherein, the application group resource information contains upgrade steps for at least multiple applications;

[0008] The steps for reading application information for a single upgrade step, indicating the application of a single application, are as follows:

[0009] If the application information of the single application read contains a first upgrade type, then the single application is determined to be the first application, and the first process and the first sub-workflow corresponding to the first application are obtained; wherein, the first process is used to reference the first sub-workflow, and the first sub-workflow includes at least one sub-process;

[0010] If the application information of the single application read contains a second upgrade type, then the single application is determined to be a second application, and the second process corresponding to the second application is obtained;

[0011] Based on the various processes acquired, and in the order of acquisition, a target workflow is generated for upgrading multiple applications in the target container cloud.

[0012] As one possible implementation, the application group resource information is used at least to describe the upgrade logic of the multiple applications.

[0013] As one possible implementation, obtaining the first process and the first sub-workflow corresponding to the first application includes:

[0014] Pull a custom upgrade package from the image repository, deploy the custom upgrade package, and generate a custom workflow;

[0015] Deploy the custom workflow, generate the first sub-workflow, and generate the first process that references the first sub-workflow.

[0016] As one possible implementation, the second process for obtaining the second application includes:

[0017] In response to determining that the second application is not deployed on the target container cloud, a second process for deploying the second application is generated based on the second application information.

[0018] As one possible implementation, the second process for obtaining the second application includes:

[0019] In response to determining that the second application is deployed on the target container cloud, the deployment version number of the second application is obtained;

[0020] If the deployed application version number is less than the upgrade version number in the second application information, then a second process for upgrading the second application is generated based on the second application information of the second application.

[0021] If the deployed application version number is greater than or equal to the upgrade version number in the second application information of the second application, then it is determined that no corresponding second process will be generated for the second application.

[0022] As one possible implementation, after generating the target workflow for upgrading multiple applications in the target container cloud, the method further includes:

[0023] Each process in the target workflow is executed sequentially. For a single process, the execution steps are as follows:

[0024] If the first process is executed, the first sub-workflow is activated, and each sub-process in the first sub-workflow is executed sequentially to upgrade the corresponding first application.

[0025] If the second process is executed, the corresponding second application will be upgraded based on the second process;

[0026] The multi-application upgrade of the target container is completed by executing all processes in the target workflow.

[0027] As one possible implementation, upgrading the second application based on the second process includes:

[0028] Enable the specified container according to the instructions in the second process;

[0029] Based on the specified container, pull the set of application resource files for the second process from the image repository;

[0030] The second application is upgraded by deploying the application resource file set of the second process.

[0031] Secondly, this application provides an apparatus for upgrading multiple applications in a container cloud, the apparatus comprising:

[0032] The reading unit, for the target container cloud, sequentially reads the application information of the application indicated by each upgrade step in the application group resource information, according to the step order of each upgrade step; wherein, the application group resource information contains upgrade steps for at least multiple applications;

[0033] The first generation unit, for a single upgrade step, instructs on the application information of a single application, and reads the following steps:

[0034] If the application information of the single application read contains a first upgrade type, then the single application is determined to be the first application, and the first process and the first sub-workflow corresponding to the first application are obtained; wherein, the first process is used to reference the first sub-workflow, and the first sub-workflow includes at least one sub-process;

[0035] If the application information of the single application read contains a second upgrade type, then the single application is determined to be a second application, and the second process corresponding to the second application is obtained;

[0036] The second generation unit, based on the acquired processes and in the acquisition order, generates a target workflow for upgrading multiple applications in the target container cloud. As one possible implementation, the application group resource information is used at least to describe the upgrade logic of the multiple applications.

[0037] As one possible implementation, the first generation unit is used to obtain the first process and the first sub-workflow corresponding to the first application, specifically for:

[0038] Pull a custom upgrade package from the image repository, deploy the custom upgrade package, and generate a custom workflow;

[0039] Deploy the custom workflow, generate the first sub-workflow, and generate the first process that references the first sub-workflow.

[0040] As one possible implementation, the first generation unit is used to obtain the second process corresponding to the second application, specifically for:

[0041] In response to determining that the second application is not deployed on the target container cloud, a second process for deploying the second application is generated based on the second application information of the second application.

[0042] As one possible implementation, the first generation unit is used to obtain the second process corresponding to the second application, specifically for:

[0043] In response to determining that the second application is deployed on the target container cloud, the deployment version number of the second application is obtained;

[0044] If the deployed application version number is less than the upgrade version number in the second application information of the second application, then a second process for upgrading the second application is generated based on the second application information.

[0045] If the deployed application version number is greater than or equal to the upgrade version number in the second application information, then it is determined that no corresponding second process will be generated for the second application.

[0046] As one possible implementation, the apparatus further includes: an upgrade unit; the upgrade unit is configured to, after generating the target workflow for upgrading multiple applications in the target container cloud, specifically for:

[0047] Each process in the target workflow is executed sequentially. For a single process, the execution steps are as follows:

[0048] If the first process is executed, the first sub-workflow is activated, and each sub-process in the first sub-workflow is executed sequentially to upgrade the corresponding first application.

[0049] If the second process is executed, the corresponding second application will be upgraded based on the second process;

[0050] The multi-application upgrade of the target container is completed by executing all processes in the target workflow.

[0051] As one possible implementation, the upgrade unit is used to upgrade the second application based on the second process, specifically for:

[0052] Enable the specified container according to the instructions in the second process;

[0053] Based on the specified container, pull the set of application resource files for the second process from the image repository;

[0054] The second application is upgraded by deploying the application resource file set of the second process.

[0055] Thirdly, this application provides an electronic device, the electronic device comprising:

[0056] Memory, used to store computer programs;

[0057] When the processor executes the computer program stored in the memory, it implements the above-described method steps for upgrading multiple applications in a container cloud.

[0058] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method steps for upgrading multiple applications in a container cloud.

[0059] Fifthly, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it enables the computer to implement the above-described method steps for upgrading multiple applications in a container cloud.

[0060] The beneficial effects of the embodiments of this application are briefly described as follows:

[0061] In this embodiment of the application, for the target container cloud, after sequentially reading the application information of the application indicated by each upgrade step according to the step order in the application group resource information, and then executing the corresponding reading step for the application information of a single application indicated by a single upgrade step, the reading step here is related to the type of a single application, and can be divided into the following two cases:

[0062] Scenario 1: If the application information of a single application contains the first upgrade type, then the single application is determined to be the first application, and the first process and the first sub-workflow corresponding to the first application are obtained. The first process is used to reference the aforementioned first sub-workflow, and the first sub-workflow includes at least one sub-process.

[0063] Scenario 2: If the application information of a single application contains a second upgrade type, then the single application is determined to be the second application, and the second process corresponding to the second application is obtained.

[0064] Thus, based on the first or second upgrade type contained in the application information of a single application, it can be determined whether the single application is the first or second application, and then the corresponding upgrade process plan can be obtained. Specifically:

[0065] First, by identifying a single application as the first application, we can obtain the first process corresponding to the first application, and the first sub-workflow that needs to be referenced by the first process to be executed. This first sub-workflow includes at least one sub-process. In this way, we can achieve data abstraction of the upgrade process of applications in complex scenarios. For example, in a complex upgrade scenario, a custom script that needs to be executed independently needs to be interspersed in the upgrade process of a certain application. Correspondingly, the application information of a certain application contains a first upgrade type. In order to represent the upgrade process of this complex upgrade scenario with data abstraction, we can identify it as the first application based on the first upgrade type it contains, and abstract its upgrade process into the obtained first process. The custom script that needs to be executed independently is abstracted into the first sub-workflow referenced by the first process. The specific execution process of the custom script is the sub-process in the first sub-workflow, thereby meeting the upgrade requirements of a single application in a complex upgrade scenario.

[0066] Second, by identifying a single application as the second application, the corresponding second process can be obtained. This enables the data abstraction of the application upgrade process in simple scenarios. For example, in a simple upgrade scenario, for an application with upgrade requirements, its upgrade process can be abstracted into the obtained second process, thereby meeting the upgrade requirements of a single application in a simple upgrade scenario.

[0067] As can be seen, the first process (which is also used to reference the first sub-workflow) or the second process above are based on the upgrade type contained in the application information of a single application to determine whether the single application belongs to the first or second application. In this way, it is applicable to multi-application scenarios and realizes the upgrade logic data abstraction for complex upgrade scenarios and ordinary upgrade scenarios of multiple applications.

[0068] Furthermore, by determining the upgrade type contained in the application information of each of the multiple applications, it is possible to determine whether each of the multiple applications belongs to a first application or a second application, and then obtain the first process (the first process is also used to reference the first sub-workflow) or the second process for each of the multiple applications for the upgrade. Subsequently, based on the obtained processes, a target workflow for upgrading multiple applications in the target container cloud is generated in the order of acquisition.

[0069] This application's embodiments are used to achieve automatic upgrades of multiple applications in container clouds, solving the problem that native Kubernetes can only upgrade single resources. Furthermore, it enables multi-application lifecycle management at the application level, while supporting the processing of multi-application dependency order, achieving one-time deployment and upgrade of multiple applications. Specifically, by converting application group resource information into a target workflow, a complex upgrade process is abstracted. In some complex scenarios, by introducing sub-workflows within a single process of the target workflow, multi-application upgrades in complex scenarios can be covered, thereby achieving one-click upgrades of multiple applications in container clouds, effectively improving the upgrade efficiency of multiple applications in container clouds, and reducing the upgrade complexity of multiple applications in container clouds.

[0070] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0071] Figure 1 A schematic diagram illustrating one possible application scenario provided by this application;

[0072] Figure 2 A schematic diagram illustrating a possible implementation environment for this application;

[0073] Figure 3 A flowchart of a method for upgrading multiple applications in a container cloud, provided in this application;

[0074] Figure 4 A flowchart illustrating a method for generating a target workflow provided in this application;

[0075] Figure 5 A schematic diagram illustrating the execution of the target task provided in this application;

[0076] Figure 6 A schematic diagram of a container cloud multi-application upgrade device provided in this application;

[0077] Figure 7 A schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The specific operating methods in the method embodiments can also be applied to the device embodiments or system embodiments.

[0079] It should be noted that in the description of this application, "multiple" is understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. A connected to B can represent: A and B directly connected, or A and B connected through C. Furthermore, in the description of this application, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order. For example, in an embodiment of this application, indicating application information for a single application for a single upgrade step could be the first application information of a single first application or the second application information of a single second application. "First" and "second" are used to distinguish the upgrade category within the application information of a single application.

[0080] In this embodiment of the application, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good morals.

[0081] It should be noted that this solution can be applied to container cloud multi-application upgrades in various application scenarios such as security management, intelligent monitoring, and smart cities. This solution is also suitable for tasks requiring container cloud multi-application upgrades in complex application scenarios, improving the efficiency of container cloud multi-application upgrades, and / or reducing the complexity of container cloud multi-application upgrades.

[0082] The implementation entity of this solution can be computing devices such as computing terminals and servers. By deploying on relevant computing devices and generating target workflows for multi-application upgrades in container clouds, it addresses the multi-application upgrade problem in complex scenarios of container clouds, thereby reducing the complexity of multi-application upgrades in container clouds and improving their efficiency. Of course, this is only an illustrative example of the application entity of this solution and is not specifically limited thereto.

[0083] The design concept of the container cloud multi-application upgrade method provided in the embodiments of this application will be briefly introduced below.

[0084] Kubernetes (K8s) is a portable, scalable, open-source platform for managing containerized workloads and services, enabling declarative configuration and automated deployment. The resources managed by K8s primarily include workload resources such as Deployments, StatefulSets, DaemonSets, and Jobs, as well as basic configuration resources such as ConfigMaps, Secrets, and Services. K8s focuses on container orchestration and scheduling, managing the lifecycle of individual workloads and their underlying configurations. For example, upgrading a single workload using K8s involves upgrading the application's container image file.

[0085] With the widespread adoption of Kubernetes (K8s), users are increasingly focusing on how to leverage K8s to: manage applications, manage the lifecycle of application systems composed of multiple applications, manage dependencies between multiple applications, and deploy and upgrade multiple applications.

[0086] To address the aforementioned multi-application upgrade needs, several package management solutions have been proposed. For example, in Kubernetes (K8s), a resource management solution based on Helm Chart packages is adopted as the industry standard. In this solution, each package is called a Chart, and a Chart is a directory containing a collection of application resource files. The Chart directory contains YAML files for all application-related resources, as well as a values.yaml configuration file for deploying the Chart. However, this solution primarily manages individual Charts (or their dependent sub-Charts). In multi-application upgrade scenarios, manual deployment is required. For instance, when releasing a cloud platform product version, dozens or even hundreds of Charts need to be managed and upgraded simultaneously. These Chart deployments are dependent, meaning that some Chart deployments require parameters from the previous Chart installation. In practice, managing the configuration items for each Chart (based on the values.yaml file configuring that Chart) and the orchestration order of Chart upgrades is a cumbersome process. This process requires real-time manual monitoring and manual execution according to the deployment order to complete the upgrades. It's easy to understand that manual deployment and upgrades take a long time and require real-time human monitoring, resulting in wasted time during the upgrade process. In addition, this method, which relies on human experience, will also bring some unnecessary maintenance costs.

[0087] In related technologies, automatic upgrades of multiple applications are supported by defining the deployment and upgrade order of multiple application chart packages. However, since these technologies can only define the deployment and upgrade order of chart packages, they cannot be applied to certain specific application scenarios. For example, in some complex scenarios, it is necessary to execute custom scripts to process data separately in the upgrade order of certain applications before executing the upgrade steps of other applications. Another example is that in some complex scenarios, it is necessary to first perform offline processing for a certain application, then perform data migration processing, and finally execute the upgrade steps of other applications.

[0088] It is evident that the relevant technologies cannot meet the multi-application upgrade needs of container cloud in complex scenarios.

[0089] To address the aforementioned issues, this application proposes a method for upgrading multiple applications in a container cloud. In this method, for the target container cloud, following the step order of each upgrade step in the application group resource information, the application information of the application indicated by each upgrade step is read sequentially. Then, for the application information of a single application indicated by a single upgrade step, the following reading steps are performed: if the first application information of the first application contains a first upgrade type, then the first process and first sub-workflow corresponding to the first application are obtained; if the second application information of the second application contains a second upgrade type, then the second process corresponding to the second application is obtained; then, based on the obtained processes, a target workflow for upgrading multiple applications in the target container cloud is generated according to the acquisition order.

[0090] The application group information contains upgrade steps for at least multiple applications; the first process is used to reference the first sub-workflow, which includes at least one sub-process.

[0091] This application's embodiments are used to achieve automatic upgrades of multiple applications in container clouds, solving the problem that native Kubernetes can only upgrade single resources. Furthermore, it enables multi-application lifecycle management at the application level, while supporting the processing of multi-application dependency order, achieving one-time deployment and upgrade of multiple applications. Specifically, by converting application group resource information into a target workflow, a complex upgrade process is abstracted. In some complex scenarios, by introducing sub-workflows within a single process of the target workflow, multi-application upgrades in complex scenarios can be covered, thereby achieving one-click upgrades of multiple applications in container clouds, effectively improving the upgrade efficiency of multiple applications in container clouds, and reducing the upgrade complexity of multiple applications in container clouds.

[0092] The container cloud multi-application upgrade method provided in this application embodiment can be applied to... Figure 1 The application scenario shown can include at least resource warehouse nodes, operation nodes, management nodes, computing nodes, and storage nodes.

[0093] The resource repository node includes the definition of application group resource information, which is used to obtain the application group resource information deployed by the user. The application group resource information includes application group metadata information, application group related single application list information, and application group multi-application upgrade step information. Among them, the application group multi-application upgrade step information includes at least multiple upgrade steps, which are used to describe the upgrade logic of multiple applications.

[0094] The operation node is used to interact with the user, enabling the user to deploy, configure, and manage container cloud multi-application upgrade tasks.

[0095] The management node is used to obtain application group resource information from the resource repository node, for example, see Figure 1Resource repository nodes upload images and videos to the cloud, while management nodes obtain application group resource information from the cloud. Management nodes are also used to manage compute and storage nodes in conjunction with container cloud multi-application upgrade tasks. During the management process, management nodes forward application group resource information to compute nodes.

[0096] The compute nodes are used to complete the compute tasks involved in the container cloud multi-application upgrade task based on the received application group resource information, generate the target workflow, and achieve compute acceleration.

[0097] Storage nodes are used to trace application group resource information obtained from the management storage resource repository node of the management node and the target workflow generated through the container cloud multi-application upgrade task.

[0098] It should be noted that the management node, compute node, and storage node mentioned above are different devices, or any two or three of the management node, compute node, and storage node can be integrated into the same device. The aforementioned operation nodes are not mandatory, and this solution does not impose specific limitations on them.

[0099] As one possible application scenario, in this embodiment, the application resource group nodes can be deployed on a Kubernetes cluster, and the compute nodes can be deployed on the container cloud application group backend service. After generating the target workflow, the processes in the target workflow can be executed using the workflow backend service to achieve multi-application upgrades. During the upgrade process, a set of application resource files can be pulled from the image repository, and the target client can be called to upgrade Kubernetes resources, thereby completing the container cloud multi-application upgrade. The target client can be deployed on the Kubernetes cluster, while the container cloud application group backend service, workflow backend service, and image repository can be deployed independently outside the Kubernetes cluster.

[0100] For example, such as Figure 2 The diagram shows a possible implementation environment provided by an embodiment of this application. The implementation environment includes a first sub-implementation environment for generating the target workflow and a second sub-implementation environment for executing the target workflow. The second sub-implementation environment is not mandatory and is not specifically limited here.

[0101] Specifically, users deploy and generate application group resource information on the Kubernetes cluster based on application group resource definitions. For the container cloud application group backend service responsible for monitoring application group resources, after determining that the user has deployed and generated application group resource information, the application group resource information is converted into a target workflow. For the workflow backend service responsible for monitoring the target workflow, it is used to execute the target workflow, that is, to start the corresponding containers in sequence according to the process of the target workflow to realize multi-application upgrade operations. Taking a single application upgrade as an example, it includes pulling the application resource file set of a single application from the image repository and calling the target client to execute the single application upgrade operation.

[0102] against Figure 2 It should be noted that, Figure 2 This description uses a resource management solution based on the Helm Chart package as an example to illustrate the implementation environment. Specifically, it adopts... Figure 2 The container cloud application group backend service Appgroup-controller, workflow backend service Argo-controller, and target client Helm shown are one possible implementation environment, and this application embodiment does not specifically limit them.

[0103] Using a Chart package as an example of an application resource file set, the following is a detailed description of the container cloud multi-application upgrade method provided in this application embodiment. The representation form of the application resource file set is not limited here. (Reference) Figure 3 The method includes steps 301-303, as detailed below.

[0104] Step 301: For the target container cloud, read the application information of each upgrade step in the order of the upgrade steps in the application group resource information.

[0105] The application group resource information includes upgrade steps for at least multiple applications, and the application group resource information is used to describe the upgrade logic of at least multiple applications.

[0106] Regarding application group resource information, a definition method is proposed to better describe the upgrade logic of multiple applications. It can be mainly divided into three parts: the first part is the application group metadata information, which describes the basic upgrade information of the application group; the second part is the application group related single application list information, which lists the corresponding application information for a single application; and the third part is the application group multi-application upgrade step information, which describes the upgrade logic of multiple applications.

[0107] Specifically, the application group metadata information includes: application group name, application group version number, and application group description information; the application group related single application list information may include: single application name, single application instance name, chart template name, chart version number, chart type (i.e., upgrade type), and values ​​file (i.e., the values ​​file of the implementation scheme specified in the application chart package); the application group multi-application upgrade step information includes: upgrade step definition and application group upgrade logic (used to support: sequential dependency relationship and directed graph dependency relationship).

[0108] As an example, the application group resource description can be found in Table 1, where " / / " is used as the Chinese meaning of the field in the same row.

[0109]

[0110]

[0111] Table 1

[0112] The following is a detailed explanation of some key fields in Table 1. The following form "A.a1" is used to represent the inclusion relationship between field A and field a1, where the former includes the latter, that is, A includes a1.

[0113] (1) apps: an array type, which is a list of individual applications related to the application group. It is used to list all the individual applications included in the current application group version.

[0114] (2) apps.ChartType: Chart type, which is the type of the Chart package of the sub-element in the application group's related single application list information. In this embodiment, the Chart type of a single application is used to characterize the upgrade type of a single application, including: a first upgrade type upgrade and a second upgrade type application, wherein the first upgrade type is used to define the upgrade relationship of a single application, and the second upgrade type is used to define the upgrade relationship of a single application in complex scenarios.

[0115] (3) apps.valuesFile: The values ​​file, which is the single application implementation plan, and corresponds to the file in the values ​​directory in the application group Chart package.

[0116] (4) apps.appName: Single application name, used to identify a unique application. Specifically, it can be used to determine whether different versions of the application are the same application.

[0117] (5) apps.releaseName: Single application instance name, i.e., single application deployment instance name, which is passed in by the implementation plan.

[0118] (6) apps.namespace: Specifies the namespace for a single application. If not specified, the appgroupcrd namespace can be used by default.

[0119] (7) upgrade.steps: Step information in the application group multi-application upgrade step information, used to describe the application group upgrade logic, which supports sequential dependency relationships and directed graph (e.g., DAG, Directed Acyclic Graph) dependency relationships. Among them, the definition of sequential relationship includes: "--" indicates ordered execution, and multiple "-" after "--" indicate parallel execution.

[0120] (8) upgrade.steps.indexID: Index, used to indicate the order of upgrade steps.

[0121] (8) upgrade.steps.-waitReady: An array that describes the process of waiting for the preceding application to be ready before executing this step.

[0122] (9) upgrade.steps.-supportVersion: Supports upgrading to the version range of the current application group.

[0123] It should be noted that the embodiments of this application do not specifically limit the field content, number of applications, and field naming of the above examples. Those skilled in the art can make corresponding adjustments to the aforementioned non-limiting content based on the definition of the application resource group proposed in the embodiments of this application, according to the actual situation, to obtain application group resource information.

[0124] For example, in complex multi-application upgrade scenarios, it's necessary to define custom scripts to execute in certain steps. These scripts primarily handle tasks such as disk data migration, database data modification or backup, and conversion of existing deployment parameters after service upgrades. To handle these complex upgrade scenarios, custom workflow resources can be introduced into the application group resource information. By defining sub-workflows corresponding to the custom upgrade package, some upgrade steps within the target workflow can be customized in sequence, such as the execution of custom scripts.

[0125] As a possible example, application group resource information can be found in Table 2.

[0126]

[0127]

[0128] Table 2

[0129] As shown in Table 2, a custom upgrade package with chartType "upgrade" is introduced into the application group resource information at this time. The execution order is described in the upgrade steps. After the second application upgrade, the custom upgrade workflow is executed. Since the third application upgrade requires special handling, it is not described in the application group resource information, but defined in the custom upgrade workflow.

[0130] Furthermore, as a specific implementation method, based on the current industry standard Helm Chart package encapsulation, a custom CRD can be implemented on the container cloud as an application group resource describing multiple applications. CRD (Custom Resource Definition) is a built-in Kubernetes resource type, that is, a definition of a custom resource used to describe the data structure of a user-defined resource. It is similar to native Kubernetes resources such as deployments and statefulsets. The application group resource defines the information of the Helm Chart packages corresponding to the relevant applications and describes the dependency order of each application upgrade during multi-application upgrades.

[0131] The definition of application group resources has been explained in detail above. Through these custom application group resources, the upgrade logic of multiple applications is abstracted, which can be used to describe the dependency order of multiple applications. In addition, the upgrade type of an individual application is also defined in the application group resources. Different upgrade types use different methods to generate the target workflow, thereby meeting the target requirements of multi-application upgrades in complex scenarios of container cloud.

[0132] This application embodiment targets a target container cloud and can sequentially read the application information of each upgrade step according to the step order in the application group resource information. Specifically, based on the upgrade logic in the multi-application upgrade step information of the application group, such as sequential dependencies and directed graph dependencies, the application information of each application is read sequentially based on the "indexID" in "upgrade.steps" shown in Table 1. Since the upgrade types contained in the application information of a single application may be different, and the processing methods for different upgrade types also differ, during the reading process, for each application information read, a process for composing the target workflow can be generated according to its upgrade type.

[0133] The next step will take reading application information for a single application as an example.

[0134] Step 302: For a single upgrade step, the application information for a single application is indicated. The reading steps include steps 3021 and 3022.

[0135] It should be noted that in steps 3021 and 302, the descriptions involving "first" and "second" are only used to distinguish different upgrade types for a single application.

[0136] Step 3021: If the application information of the single application read contains a first upgrade type, then the single application is determined to be the first application, and the first process and the first sub-workflow corresponding to the first application are obtained;

[0137] The first process is used to reference the first sub-workflow, which includes at least one sub-process.

[0138] In this embodiment of the application, if the first application information of the first application includes a first upgrade type, a custom upgrade package is pulled from the image repository, the custom upgrade package is deployed, a custom workflow is generated, the custom workflow is then deployed, a first sub-workflow is generated, and a first process referencing the first sub-workflow is generated.

[0139] Specifically, the first sub-process can be a custom script defined to be executed in the first process. These scripts can be used to handle some complex upgrade operations, such as disk data migration, database data modification, database data backup, and conversion of the original deployment parameters after service upgrade.

[0140] It's important to note that the first sub-process can upgrade a single application, or it can upgrade multiple applications or handle application requirements. Custom scripts can be designed according to application needs.

[0141] For example, the first application information of the first application is read. If the Chart type is determined to be the first upgrade type application, a custom workflow Helm Chart is pulled from the image repository, the custom workflow Helm Chart is deployed to create the first sub-workflow, and the first sub-workflow is referenced in the first process of the target workflow to complete the complex upgrade process.

[0142] By introducing a custom workflow, a first sub-process is generated that is referenced by the first process. The first sub-process abstracts the logic of single-application upgrades in complex scenarios, and thus becomes applicable to multi-application upgrades in complex scenarios, effectively reducing the complexity of upgrades in complex scenarios.

[0143] Step 3022: If the application information of the single application read contains a second upgrade type, then the single application is determined to be a second application, and the second process corresponding to the second application is obtained;

[0144] In this embodiment of the application, if the second application information read from the second application contains a second upgrade type, the generation of the second process can be divided into the following multiple cases.

[0145] In scenario one, in response to the determination that the second application is not deployed on the target container cloud, a second process for deploying the second application is generated based on the second application information. In other words, although there is no second application currently indicating an upgrade, the version information of the application group resource information contains relevant upgrade information for the second application. In this case, the originally indicated version upgrade process is not generated, but a second process for deploying the second application is generated.

[0146] Scenario 2: In response to determining that the second application is deployed on the target container cloud, the deployment version number of the second application is obtained. If the deployment version number is less than the upgrade version number in the second application information, a second process for upgrading the second application is generated based on the second application information. In other words, the second application currently indicating an upgrade has already been deployed and is in an existing state, and it is determined that the deployment version number of the second application does not meet the requirements. Therefore, a second process is needed to upgrade the second application from the deployment version number to the upgrade version number based on the second application information.

[0147] Scenario 3: In response to determining that the second application is deployed on the target container cloud, the deployment version number of the second application is obtained. If the deployment version number is greater than or equal to the upgrade version number in the second application information, it is determined that no corresponding second process will be generated for the second application. In other words, the second application currently indicating an upgrade has already been deployed and is in existence, and it is determined that the deployment version number of the second application has met the requirements. Therefore, no upgrade processing is needed for the second application.

[0148] It should be noted that in some special cases, such as when, in response to determining that the second application is deployed on the target container cloud and obtaining the deployment version number of the second application, the upgrade version number cannot be obtained from the second application information, no processing is performed on the second application. That is, no second process corresponding to the second application is generated. If a second process has already been generated based on the above three situations, then the second process is deleted.

[0149] By using the above method, the upgrade logic of a single application is transformed into a single process of the target workflow. Based on the same idea, this can be applied to the upgrade of multiple applications, thereby reducing the complexity of the upgrade process and improving the upgrade efficiency of multiple applications.

[0150] Step 303: Based on the acquired processes, generate a target workflow for upgrading multiple applications in the target container cloud in the order of acquisition.

[0151] Specifically, for multiple applications indicated by the application group resource information to be upgraded, the information of each application is read in sequence, and the corresponding process for each application is generated. According to the step order of the upgrade steps indicated by the application group resource information, that is, the order in which each process is obtained, the target workflow for upgrading multiple applications in the target container cloud is generated.

[0152] It is important to note that a target workflow can contain multiple referenced custom sub-workflows (e.g., the first sub-workflow). The main flow of the target workflow upgrades multiple applications step by step, and custom sub-workflows are executed at certain steps. The custom sub-workflows execute custom scripts step by step to upgrade multiple applications. In other words, a single custom sub-workflow can upgrade a single application.

[0153] As a preferred implementation method, the overall process for generating the target workflow can be found in [reference needed]. Figure 4 As shown.

[0154] like Figure 4 As shown, the process first reads the application group resource information and searches for the currently deployed application group resource with the same name in the Kubernetes cluster. If the version of the currently deployed application group does not meet the supportVersion requirement for upgrading the application group, the upgrade process ends. If the version of the currently deployed application group meets the supportVersion requirement for upgrading the application group, the generation process of the target workflow is started. Specifically, for the generation of the target workflow, after reading the single-step information upgrade.steps.indexID from the application group resource information, the detailed application information of the single application is found based on the single application name appName, and then the chart type of the single application is determined. If the single application is determined to be the first upgrade type upgrade, the custom workflow HelmChart is pulled from the image repository, the custom workflow Helm Chart is deployed to create a sub-workflow, and the sub-workflow is referenced in the target workflow to generate the process of creating the target workflow. This part is described in detail in step 3021. If the application is determined to be a second upgrade type application, the system further checks whether the currently deployed application exists. If it does not exist, the application creation process is generated. If it exists, the system compares the application's deployment version number with the upgrade version number. If the current application's deployment version number is less than the upgrade version number, the application upgrade process is generated. If the current application's deployment version number is greater than or equal to the upgrade version number, the process is skipped and no further processing is performed. See step 3022 for details. Then, based on the Steps information in the application group resource information, the system checks whether the traversal processing for indexId has been completed. If not, the indexId is incremented by 1 for the next traversal, and the traversal operation continues. If yes, the system further compares the currently deployed application list information with the apps list information in the application group resource information. If an application exists in the currently deployed application list but not in the application group apps list, the system generates a step to delete the corresponding process, thus obtaining the target workflow.

[0155] Furthermore, as a specific implementation approach, the target workflow can be implemented using Argo workflows. Argo is a cloud-native workflow framework on Kubernetes that implements each step in the workflow as a container. Argo enables users to launch multi-step pipelines using a custom DSL similar to traditional YAML. The framework provides complex loops, conditionals, and dependency management with DAGs, which helps improve flexibility in deploying application stacks and in configuring and managing dependencies. Using Argo, users can define dependencies, programmatically construct complex workflows, manage artifacts that link the output of any step as the input of subsequent steps, and monitor planned jobs in an easy-to-read UI. When users are performing multi-application upgrades, after deploying application group resources, the container cloud backend service converts the application group resources into Argo workflow step descriptions and initiates workflow execution to complete the entire upgrade process.

[0156] As a possible example, the transformed target workflow can be seen in Table 3.

[0157]

[0158]

[0159]

[0160]

[0161] Table 3

[0162] Combination Figure 2 To briefly explain the transformation principle of the target workflow in Table 3, it can be summarized as follows: the upgrade dependency order described by the application group resource information is transformed into workflow steps (processes). When Argo-controller executes the workflow, it executes the steps in sequence. Each step contains information such as the name, version number, and upgrade parameters of the Helm Chart package required for a single application upgrade. When executing a single step, Argo-controller starts a container, which is pulled using the image specified in step.template. In the container, the Helm Chart package is pulled from the image repository, and the Helm client is called to execute the single application upgrade. Through the execution of the entire workflow, the upgrade process of multiple applications in dependency order is completed.

[0163] Furthermore, in complex multi-application upgrade scenarios, it's necessary to define custom scripts to execute in certain steps. These scripts primarily handle tasks such as disk data migration, database data modification or backup, and conversion of existing deployment parameters after service upgrades. To handle these complex upgrade scenarios, custom workflow resources can be introduced into the application group resource description. Users can define a sub-workflow, that is, customize some upgrade steps in the target workflow according to their sequence, such as the execution of custom scripts.

[0164] Furthermore, after generating the target workflow, each process within the target workflow is executed sequentially to complete the multi-application upgrade process. Taking the execution of a single process as an example, if the first process is executed, the first sub-workflow is activated, and each sub-process within the first sub-workflow is executed sequentially to upgrade the first application; if the second process is executed, the specified container is activated according to the instruction image of the second process, and then, based on the specified container, the application resource file set of the second process is pulled from the image repository, and the second application is upgraded by deploying the application resource file set of the second process; this process continues until all processes in the target workflow are executed, completing the multi-application upgrade of the target container.

[0165] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish the differences in upgrade types.

[0166] For example, see Figure 5 The diagram shown illustrates the execution of the target task. Figure 5 The target workflow shown can be generated based on the application group resource information shown in Table 2. When executing the target workflow of multiple applications, the corresponding processes are executed in sequence. If a process that references a subprocess is executed, the corresponding subprocess will be executed according to the defined dependency order. At this time, the target workflow will listen to the execution status of the subprocess. Only after the subprocess is completed will the subsequent processes of the target workflow continue to be executed, thereby realizing the upgrade of multiple applications in complex scenarios.

[0167] Based on the same inventive concept, embodiments of this application also provide an apparatus for upgrading multiple applications in a container cloud. For example... Figure 6 As shown, it is a structural schematic diagram of a container cloud multi-application upgrade device according to an embodiment of this application, which may include:

[0168] The reading unit 601, for the target container cloud, sequentially reads the application information of the application indicated by each upgrade step in the application group resource information according to the step order of each upgrade step; wherein, the application group resource information contains upgrade steps for at least multiple applications;

[0169] The first generation unit 602, for a single upgrade step, instructs a single application for application information, and reads the following steps:

[0170] If the application information of the single application read contains a first upgrade type, then the single application is determined to be the first application, and the first process and the first sub-workflow corresponding to the first application are obtained; wherein, the first process is used to reference the first sub-workflow, and the first sub-workflow includes at least one sub-process.

[0171] If the application information of the single application read contains a second upgrade type, then the single application is determined to be a second application, and the second process corresponding to the second application is obtained;

[0172] The second generation unit 603 generates a target workflow for upgrading multiple applications in the target container cloud based on the acquired processes and in the order of acquisition.

[0173] As one possible implementation, the application group resource information is used at least to describe the upgrade logic of the multiple applications.

[0174] As one possible implementation, the first generation unit 602 is used to obtain the first process and the first sub-workflow corresponding to the first application, specifically for:

[0175] Pull a custom upgrade package from the image repository, deploy the custom upgrade package, and generate a custom workflow;

[0176] Deploy the custom workflow, generate the first sub-workflow, and generate the first process that references the first sub-workflow.

[0177] As one possible implementation, the first generation unit 602 is used to obtain the second process corresponding to the second application, specifically for:

[0178] In response to determining that the second application is not deployed on the target container cloud, a second process for deploying the second application is generated based on the second application information of the second application.

[0179] As one possible implementation, the first generation unit 602 is used to obtain the second process corresponding to the second application, specifically for:

[0180] In response to determining that the second application is deployed on the target container cloud, the deployment version number of the second application is obtained;

[0181] If the deployed application version number is less than the upgrade version number in the second application information of the second application, then a second process for upgrading the second application is generated based on the second application information.

[0182] If the deployed application version number is greater than or equal to the upgrade version number in the second application information, then it is determined that no corresponding second process will be generated for the second application.

[0183] As one possible implementation, the apparatus further includes: an upgrade unit 604; the upgrade unit 604 is configured, after generating the target workflow for upgrading multiple applications in the target container cloud, specifically for:

[0184] Each process in the target workflow is executed sequentially. For a single process, the execution steps are as follows:

[0185] If the first process is executed, the first sub-workflow is activated, and each sub-process in the first sub-workflow is executed sequentially to upgrade the corresponding first application.

[0186] If the second process is executed, the corresponding second application will be upgraded based on the second process;

[0187] The multi-application upgrade of the target container is completed by executing all processes in the target workflow.

[0188] As one possible implementation, the upgrade unit 604 is used to upgrade the second application based on the second process, specifically for:

[0189] Enable the specified container according to the instructions in the second process;

[0190] Based on the specified container, pull the set of application resource files for the second process from the image repository;

[0191] The second application is upgraded by deploying the application resource file set of the second process.

[0192] Based on the same inventive concept as the embodiments described above, this application also provides an electronic device that can be used for multi-application upgrades in container clouds. In one embodiment, the electronic device can be a server, a terminal device, or other electronic equipment. In this embodiment, the structure of the electronic device can be as follows: Figure 7 As shown, it includes a memory 701, a communication interface 703, and one or more processors 702.

[0193] The memory 701 is used to store computer programs executed by the processor 702. The memory 701 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0194] Memory 701 may be volatile memory, such as random-access memory (RAM); memory 701 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 701 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 701 may be a combination of the above-described memories.

[0195] Processor 702 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 702 is used to implement the above-described container cloud multi-application upgrade method when calling computer programs stored in memory 701.

[0196] Communication interface 703 is used to communicate with terminal devices and other servers.

[0197] This application embodiment does not limit the specific connection medium between the memory 701, the communication interface 703, and the processor 702 described above. This application embodiment... Figure 7 The memory 701 and the processor 702 are connected via a bus 704, and the bus 704 is in Figure 7 The connections between other components are shown in bold lines only and are not intended to be limiting. The 704 bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0198] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the container cloud multi-application upgrade methods described in the above embodiments. The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0199] According to one aspect of this application, this application also provides a computer program product that, when invoked by a computer, causes the computer to execute a container cloud multi-application upgrade method.

[0200] This application provides a method, apparatus, electronic device, and storage medium for upgrading multiple applications in a container cloud. For a target container cloud, following the step order of each upgrade step in the application group resource information, the application information of the application indicated by each upgrade step is read sequentially. For the application information of a single application indicated by a single upgrade step, the following reading steps are performed: if the first application information of the first application contains a first upgrade type, then the first process and first sub-workflow corresponding to the first application are obtained; if the second application information of the second application contains a second upgrade type, then the second process corresponding to the second application is obtained; then, based on the obtained processes, a target workflow for upgrading multiple applications in the target container cloud is generated according to the acquisition order.

[0201] The application group information contains upgrade steps for at least multiple applications; the first process is used to reference the first sub-workflow, which includes at least one sub-process.

[0202] This application embodiment is used to achieve automatic upgrades of multiple applications in container cloud, solving the problem that the current native Kubernetes can only upgrade a single resource. Furthermore, it enables multi-application lifecycle management at the application level, while supporting the processing of multi-application dependency order, achieving one-time deployment and upgrade of multiple applications. Specifically, by converting application group resource information into a target workflow, a complex upgrade process is abstracted. In some complex scenarios, by introducing sub-workflows into a single process of the target workflow, multi-application upgrades in complex scenarios can be covered, thereby achieving one-click upgrades of multiple applications in container cloud, effectively improving the upgrade efficiency of multiple applications in container cloud, and reducing the upgrade complexity of multiple applications in container cloud.

[0203] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0204] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0205] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0206] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for upgrading multiple applications in a container cloud, characterized in that, The method includes: For the target container cloud, according to the step order of each upgrade step in the application group resource information, the application information of the application indicated by each upgrade step is read sequentially; wherein, the application group resource information includes at least the upgrade steps of multiple applications, application group metadata information, and application group corresponding single application list information. The application information for a single upgrade step, indicating a single application, is read as follows: If the application information of the single application read contains a first upgrade type, then the single application is determined to be the first application, and a custom upgrade package is pulled from the image repository. The custom upgrade package is deployed to generate a custom workflow. The custom workflow is deployed to generate a first sub-workflow, and a first process referencing the first sub-workflow is generated. The first sub-workflow includes at least one sub-process. If the application information of the single application read contains a second upgrade type, then the single application is determined to be a second application, and the second process corresponding to the second application is obtained; Based on the various processes acquired, and in the order of acquisition, a target workflow is generated for upgrading multiple applications in the target container cloud. The single application mentioned here refers to the application of Helm chart.

2. The method as described in claim 1, characterized in that, The second process for obtaining the second application includes: In response to determining that the second application is not deployed on the target container cloud, a second process for deploying the second application is generated based on the second application information of the second application.

3. The method as described in claim 1, characterized in that, The second process for obtaining the second application includes: In response to determining that the second application is deployed on the target container cloud, the deployment version number of the second application is obtained; If the deployed application version number is less than the upgrade version number in the second application information of the second application, then a second process for upgrading the second application is generated based on the second application information. If the deployed application version number is greater than or equal to the upgrade version number in the second application information, then it is determined that no corresponding second process will be generated for the second application.

4. The method as described in claim 1, characterized in that, After generating the target workflow for upgrading multiple applications in the target container cloud, the process further includes: Each process in the target workflow is executed sequentially. For a single process, the execution steps are as follows: If the first process is executed, the first sub-workflow is activated, and each sub-process in the first sub-workflow is executed sequentially to upgrade the corresponding first application. If the second process is executed, the corresponding second application will be upgraded based on the second process; The multi-application upgrade of the target container is completed by executing all processes in the target workflow.

5. The method as described in claim 4, characterized in that, The upgrade of the second application based on the second process includes: Enable the specified container according to the instructions in the second process; Based on the specified container, pull the set of application resource files for the second process from the image repository; The second application is upgraded by deploying the application resource file set of the second process.

6. A device for upgrading multiple applications in a container cloud, characterized in that, The device includes: The reading unit, for the target container cloud, sequentially reads the application information of the application indicated by each upgrade step in the application group resource information according to the step order of each upgrade step; wherein, the application group resource information includes at least the upgrade steps of multiple applications, application group metadata information, and application group corresponding single application list information. The first generation unit, for a single upgrade step, instructs on the application information of a single application, and reads the following steps: If the application information of the single application read contains a first upgrade type, then the single application is determined to be the first application, and a custom upgrade package is pulled from the image repository. The custom upgrade package is deployed to generate a custom workflow. The custom workflow is deployed to generate a first sub-workflow, and a first process referencing the first sub-workflow is generated. The first sub-workflow includes at least one sub-process. If the application information of the single application read contains a second upgrade type, then the single application is determined to be a second application, and the second process corresponding to the second application is obtained; The second generation unit, based on the acquired processes and in the order of acquisition, generates a target workflow for upgrading multiple applications in the target container cloud. The single application mentioned here refers to the application of Helm chart.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a computer program stored in the memory, implements the method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-5.

9. A computer program product, characterized in that, The computer program product includes: computer program code, which, when run on a computer, causes the computer to perform the method described in any one of claims 1-5.

Images