System, method, device and medium for accessing heterogeneous virtual machine images or container cluster compilation

Abstract

The present application proposes a system, method, device and medium for accessing heterogeneous virtual machine images or container cluster compilation. The system includes: a basic component module for providing microservice discovery, microservice operation, dependency calls between microservices, data storage of microservices, and container instance operation; a network policy module for assisting calls between microservices and file distribution strategies of basic component units; a heterogeneous virtual machine builder module for managing and maintaining virtual machine builders of different architectures; a container cluster builder module for managing and monitoring the health status of container clusters and providing available healthy clusters for compilation and construction; a scheduling module for heterogeneous virtual machine images or container clusters for dynamically scheduling virtual machine images or container clusters of different architectures according to different compilation type requirements. The present application solves the configuration and compatibility issues that need to take into account the underlying environment, and can more simply manage and maintain applications, reducing the complexity caused by environmental differences.

Description

Technical Field

[0001] The present application relates to the field of software development and release, and in particular to a system, method, device and medium for accessing heterogeneous virtual machine images or container cluster compilation. Background Art

[0002] In the wave of cloud computing, heterogeneous virtual machine images and container cluster compilation technology are becoming the cornerstones of building modern IT infrastructure. These two technologies have brought revolutionary changes to software development and deployment with their unique advantages. Heterogeneous virtual machine image technology breaks through the limitations of hardware and operating systems, allowing developers to create virtual machine images that can run on different platforms. The core of this technology is that it can encapsulate applications and their dependent environments, ensuring consistent performance and behavior on different hardware architectures and operating systems, and jointly providing an isolated and consistent operating environment for applications. Container cluster compilation technology is an innovative move in the wave of containerization. Container technology has become an indispensable part of microservice architecture with its lightweight, fast startup and easy portability. Container cluster compilation technology realizes the rapid conversion from source code to container images through an automated tool chain. This process not only includes compilation and building, but also involves dependency management, security scanning and image optimization, ensuring the quality and performance of container images. Therefore, in the face of a large number of different compilation requirements of applications, access to these two technologies can meet all business scenarios. Summary of the invention

[0003] The embodiments of the present application provide a system, method, device and medium for accessing heterogeneous virtual machine images or container cluster compilation to solve the problems existing in related technologies. The technical solutions are as follows:

[0004] In a first aspect, an embodiment of the present application provides a system for accessing heterogeneous virtual machine images or container cluster compilation, including:

[0005] Basic component modules are used to provide microservice discovery, microservice operation, dependency calls between microservices, and data storage and container instance operation of microservices. The basic component modules include java, consul, k8sservice, k8sdeployment, k8spod, docker, telegraf, and mysql.

[0006] The network policy module is used to assist in the calls between microservices and the file distribution strategy of basic components. Different network strategies can be set to implement different types of scheduling or file distribution;

[0007] A heterogeneous virtual machine builder module is used to manage and maintain virtual machine builders of different architectures. Administrators can add builders of different architectures through the heterogeneous virtual machine builder module. Builder information includes architecture type, CPU, memory, disk capacity, and disk IO indicator information. Administrators can add, update, and delete builders at any time according to actual builder business needs, or form corresponding builder clusters according to actual business conditions.

[0008] The container cluster builder module is used to manage and monitor the health status of the container cluster and provide an available and healthy cluster for compilation and construction. The administrator accesses different external container clusters through the container cluster builder module and monitors the health status of each cluster, including node status, resource utilization, service availability, and cluster component status.

[0009] A module for scheduling heterogeneous virtual machine images or container clusters is used to dynamically schedule virtual machine images or container clusters of different architectures according to different compilation type requirements; the module for scheduling heterogeneous virtual machine images or container clusters provides a dynamic access to the compilation environment capability according to the actual needs of different users, and the dynamic access to the compilation environment can meet the compilation requirements of different programming languages.

[0010] In some embodiments, the basic component module includes a k8s container cluster and a virtual machine cluster, wherein:

[0011] The k8s cluster is the basis for the container cluster to perform compilation and construction. When a compilation request is received, the deployment resources transmitted based on the request are;

[0012] The deployment resource is a configuration file in yaml or json format;

[0013] The configuration file describes the application's container image, number of replicas, resource requests, and environment variable information;

[0014] The k8s cluster of the container cluster submits the deployment definition configuration to the apiServer. After the apiServer receives the deployment definition configuration, the scheduler component will be triggered. After the scheduler determines the running location of the pod, it will send instructions to the kubelet on the selected node of the cluster, instructing it to start the pod. The kubelet creates a pod on the node and starts the image instance specified in the pod to perform the compilation and construction task, thereby completing the compilation and construction task based on the container cluster;

[0015] The environment of the system for accessing heterogeneous virtual machine images or container cluster compilation is not only a container cluster, but also a virtual machine cluster mode derived from the cooperation between the java microservice dockerhost, service discovery consul, and mirror docker. When the virtual machine cluster receives a compilation request, it discovers the available dockerhost microservice node through the consul service, pulls up the docker image instance on the corresponding machine through the dockerhost microservice, executes the compilation and construction task, and completes the compilation and construction task based on the virtual machine cluster.

[0016] In some embodiments, the network policy module includes file distribution and service invocation, wherein:

[0017] In some complex network environments, accessing heterogeneous virtual machine images or container clusters to perform compilation and construction requires the corresponding network policies to be enabled in advance to prevent the basic components from being unable to perform compilation tasks due to network failures during collaborative calls.

[0018] The network strategy is divided into virtual machine cluster grouping strategy, file synchronization distribution strategy, and port calling strategy between microservices.

[0019] In some embodiments, the heterogeneous virtual machine builder module is an execution medium that provides a compilation environment for different architectures; the compilation environment includes architecture type, cpu, memory, disk capacity, and disk IO load information; the administrator can add, update, and delete build machines at any time according to the actual build machine business needs. During the execution process, when the heterogeneous virtual machine builder module receives a compilation request from the scheduling module, it combines the cpu, memory, disk capacity, and disk IO indicators, and screens out suitable build machine nodes by calculating the load balancing method of the average value over a period of time, and starts the docker image instance process on the build machine node. After the process is successfully started, a complete virtual machine compilation environment is provided for compilation and packaging until the compilation is completed, the container is destroyed, and the virtual machine builder resources gradually decrease and return to normal levels; when the virtual machines in the cluster are short of resources under concurrent conditions, machines will be dynamically added; when resources are in excess, machines will be dynamically deleted and removed from the cluster.

[0020] In some embodiments, the container cluster builder module is an execution medium that can provide k8s container clusters of different architectures. Each container cluster has a corresponding apiServer address and kubeConfig configuration. The administrator can obtain the apiServer addresses and kubeConfig credentials corresponding to these clusters through commands, and configure them to the container cluster builder module. The module can provide real-time monitoring functions, help track the health status and performance indicators of the cluster, provide backup and recovery mechanisms, and ensure that different container clusters can be quickly switched to provide scheduling for compilation and packaging in the event of a failure.

[0021] In some embodiments, the scheduling module for heterogeneous virtual machine images or container clusters is a scheduling module. When a request is sent to the scheduling module for heterogeneous virtual machine images or container clusters, the module makes a judgment based on the service discovery type of the request;

[0022] When it is determined that it is a consul type service discovery, the heterogeneous virtual machine cluster listener is triggered. Combined with the CPU, memory, disk capacity, and disk IO indicators, the load balancing method of calculating the average value over a period of time is used to select the appropriate builder node as the startup image carrier;

[0023] When it is determined that it is a k8s type service discovery, the container cluster listener is triggered, and the corresponding container cluster is selected as the carrier for starting the pod through the apiServer address configured by the listener and the kubeConfig configuration.

[0024] In a second aspect, an embodiment of the present application further provides a method for accessing a heterogeneous virtual machine image or container cluster compilation, including:

[0025] Initialize the virtual machine or container cluster node server, which includes shutting down seLinux, shutting down fireWalld, installing rsync and sysstat, adjusting the maximum number of open files, and confirming server time synchronization;

[0026] Network policies between modules are enabled, including file distribution and transmission ports, calling ports between services, password-free policies between machines, and microservice database access ports;

[0027] Synchronize basic component modules and install and start them, synchronize basic components from the source machine to the target machine, and start running components;

[0028] Build a cluster, build a container cluster according to the official k8s process, and build the corresponding number of clusters according to actual business needs;

[0029] Synchronously schedule heterogeneous virtual machine images or container clusters and install and start them, synchronously schedule services from the source machine to the target machine, and start the services;

[0030] Configure a virtual machine cluster or container cluster, enter the virtual machine IP through the heterogeneous virtual machine builder module, or set the apiServer and kubeConfig credentials of the k8s cluster in the container cluster builder module to achieve the purpose of managing the virtual machine cluster and container cluster module.

[0031] In a third aspect, an embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus, and the machine-readable instructions are executed by the processor when running, such as the steps of a method for accessing a heterogeneous virtual machine image or container cluster compilation as described above.

[0032] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for accessing a heterogeneous virtual machine image or container cluster compilation as described above are executed.

[0033] The advantages or beneficial effects of the above technical solution include at least:

[0034] Improve development efficiency: Developers can focus on code writing without having to worry about the configuration and compatibility of the underlying environment;

[0035] Reduced O&M complexity: O&M teams can more easily manage and maintain applications, reducing the complexity caused by environmental differences;

[0036] Enhanced business agility: Enterprises can respond to market changes more quickly and launch new features or updates quickly;

[0037] Optimize resource usage: Reduce operating costs through more efficient resource allocation and utilization;

[0038] Improve user experience: Rapid iteration and deployment ensure that users can experience the latest and most stable application version in a timely manner.

[0039] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for use in the embodiments will be briefly introduced below. In the accompanying drawings, unless otherwise specified, the same reference numerals throughout multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that the following drawings only illustrate certain embodiments of the present application and are therefore not to be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can also be obtained based on these drawings without paying creative work.

[0041] Figure 1 A schematic diagram of the structure of the present invention accessing heterogeneous virtual machine images or container cluster compilation;

[0042] Figure 2 A schematic diagram of the process of accessing a heterogeneous virtual machine image or container cluster compilation in the present invention;

[0043] Figure 3 This is a schematic diagram of the execution principle of the present invention accessing heterogeneous virtual machine images or container cluster compilation. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0045] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0046] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0047] As attached Figure 1 As shown, the present invention provides a method for accessing heterogeneous virtual machine images or container cluster compilation, the method includes a basic component module, a network policy module, a heterogeneous virtual machine builder module, a container cluster builder module, and a scheduling module for heterogeneous virtual machine images or container clusters. Figure 3 , a detailed introduction to each module and its execution principle:

[0048] Basic component modules are used to provide microservice discovery, microservice operation, dependency calls between microservices, and data storage and container instance operation of microservices. The basic component modules include java, consul, k8sservice, k8sdeployment, k8spod, docker, telegraf, and mysql.

[0049] The network policy module is used to assist in the calls between microservices and the file distribution strategy of basic components. Different network strategies can be set to implement different types of scheduling or file distribution;

[0050] A heterogeneous virtual machine builder module is used to manage and maintain virtual machine builders of different architectures. Administrators can add builders of different architectures through the heterogeneous virtual machine builder module. Builder information includes architecture type, CPU, memory, disk capacity, and disk IO indicator information. Administrators can add, update, and delete builders at any time according to actual builder business needs, or form corresponding builder clusters according to actual business conditions.

[0051] The container cluster builder module is used to manage and monitor the health status of the container cluster and provide an available and healthy cluster for compilation and construction. The administrator accesses different external container clusters through the container cluster builder module and monitors the health status of each cluster, including node status, resource utilization, service availability, and cluster component status.

[0052] A module for scheduling heterogeneous virtual machine images or container clusters is used to dynamically schedule virtual machine images or container clusters of different architectures according to different compilation type requirements; the module for scheduling heterogeneous virtual machine images or container clusters provides a dynamic access to the compilation environment capability according to the actual needs of different users, and the dynamic access to the compilation environment can meet the compilation requirements of different programming languages.

[0053] Execution principle: When a request is sent to schedule heterogeneous virtual machine images or container cluster services, it will be judged according to the requested service discovery type. When it is judged to be consul type service discovery, the heterogeneous virtual machine cluster listener is triggered. Combined with cpu, memory, disk capacity, and disk IO indicators, this load balancing method of calculating the average value over a period of time is used to select suitable builder nodes as the startup image carrier, pull up the docker image instance on the carrier, and perform the compilation and construction tasks. Through this series of automated steps, the compilation and construction tasks can be completed based on the virtual machine cluster. When it is judged to be k8s type service discovery, the container cluster listener is triggered. The apiServer address configured by the listener and the kubeConfig configuration select the corresponding container cluster as the carrier for starting the pod. The cluster submits the deployment definition configuration to the apiServer. After the apiServer receives the deployment definition configuration, the scheduler component will be triggered. After the scheduler determines the running location of the pod, it will send instructions to the kubelet on the selected node of the cluster, instructing it to start the pod. The kubelet creates a pod on the node and starts the image instance specified in the pod to perform the compilation and construction tasks. Through this series of automated steps, the compilation and construction tasks can be completed based on the container cluster.

[0054] As another embodiment, the present invention provides a method for accessing heterogeneous virtual machine images or container cluster compilation, as shown in the attached Figure 2 As shown, the following steps are included:

[0055] S101. Initialize the virtual machine or container cluster node server;

[0056] S102. Network strategy activation;

[0057] S103. Build a cluster;

[0058] S104. Install, start and schedule heterogeneous virtual machine images or container cluster services;

[0059] S105. Configure access to a virtual machine cluster or container cluster.

[0060] In addition, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a program, and the processor is used to run the program, and when the program runs, the method provided by the above invention is executed.

[0061] Furthermore, the present invention also provides another computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method provided by the above invention is implemented.

[0062] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A system for accessing heterogeneous virtual machine images or container cluster compilation, characterized in that: include: Basic component modules are used to provide microservice discovery, microservice operation, dependency calls between microservices, and data storage and container instance operation of microservices. The basic component modules include java, consul, k8sservice, k8sdeployment, k8spod, docker, telegraf, and mysql. The network policy module is used to assist in the calls between microservices and the file distribution strategy of basic components. Different network strategies can be set to implement different types of scheduling or file distribution; A heterogeneous virtual machine builder module is used to manage and maintain virtual machine builders of different architectures. Administrators can add builders of different architectures through the heterogeneous virtual machine builder module. Builder information includes architecture type, CPU, memory, disk capacity, and disk IO indicator information. Administrators can add, update, and delete builders at any time according to actual builder business needs, or form corresponding builder clusters according to actual business conditions. The container cluster builder module is used to manage and monitor the health status of the container cluster and provide an available and healthy cluster for compilation and construction. The administrator accesses different external container clusters through the container cluster builder module and monitors the health status of each cluster, including node status, resource utilization, service availability, and cluster component status. A module for scheduling heterogeneous virtual machine images or container clusters is used to dynamically schedule virtual machine images or container clusters of different architectures according to different compilation type requirements; the module for scheduling heterogeneous virtual machine images or container clusters provides a dynamic access to the compilation environment capability according to the actual needs of different users, and the dynamic access to the compilation environment can meet the compilation requirements of different programming languages; The heterogeneous virtual machine construction module provides a compilation environment with different architectures; The compilation environment includes architecture type, CPU, memory, disk capacity, and disk IO load information; The administrator can add, update, and delete build machines at any time according to the actual build machine business needs. During the execution process, when the heterogeneous virtual machine build machine module receives the compilation request of the scheduling module, it combines the CPU, memory, disk capacity, and disk IO indicators, and calculates the load balancing method of the average value over a period of time to screen out the appropriate build machine node, and starts the docker image instance process on the build machine node. After the process is successfully started, a complete virtual machine compilation environment is provided for compilation and packaging until the compilation is completed, the container is destroyed, and the virtual machine build machine resources gradually decrease and return to normal levels; When the cluster's virtual machines run out of resources in a concurrent situation, machines are added dynamically; when resources are excessive, machines are deleted dynamically and removed from the cluster; The container cluster builder module provides k8s container clusters of different architectures. Each container cluster has a corresponding apiServer address and kubeConfig credentials. The administrator obtains the apiServer addresses and kubeConfig credentials corresponding to these clusters through commands and configures them to the container cluster builder module. The module can provide real-time monitoring functions, help track the health status and performance indicators of the cluster, and provide backup and recovery mechanisms to ensure that different container clusters can be quickly switched to provide scheduling for compilation and packaging in the event of a failure; The scheduling module for heterogeneous virtual machine images or container clusters is a scheduling module. When a request is sent to the scheduling module for heterogeneous virtual machine images or container clusters, the module will make a judgment based on the service discovery type of the request; When it is determined that it is a consul type service discovery, the heterogeneous virtual machine cluster listener is triggered. Combined with the CPU, memory, disk capacity, and disk IO indicators, the load balancing method of calculating the average value over a period of time is used to select the appropriate builder node as the startup image carrier; When it is determined that it is a k8s type service discovery, the container cluster listener is triggered, and the corresponding container cluster is selected as the carrier for starting the pod through the apiServer address configured by the listener and the kubeConfig credentials.

2. The system for accessing heterogeneous virtual machine images or container cluster compilation according to claim 1, characterized in that: The basic component module includes: The k8s cluster is the basis for the container cluster to perform compilation and construction. When a compilation request is received, the deployment resources transmitted based on the request are; The deployment resource is a configuration file in yaml or json format; The configuration file describes the application's container image, number of replicas, resource requests, and environment variable information; The k8s cluster of the container cluster submits the deployment resources to the apiServer. After the apiServer receives the deployment resources, the scheduler component will be triggered. After the scheduler determines the running location of the pod, it will send instructions to the kubelet on the selected node of the cluster, instructing it to start the pod. The kubelet creates the pod on the node and starts the image instance specified in the pod to perform the compilation and construction tasks, thereby completing the compilation and construction tasks based on the container cluster; The environment of the system accessing heterogeneous virtual machine images or container cluster compilation is not only a container cluster, but also a virtual machine cluster mode derived from the cooperation between the Java microservice dockerhost, service discovery consul, and mirror docker. When the virtual machine cluster receives a compilation request, it discovers the available dockerhost microservice node through the consul service, pulls up the docker image instance on the corresponding machine through the dockerhost microservice, executes the compilation and construction task, and completes the compilation and construction task based on the virtual machine cluster.

3. The system for accessing heterogeneous virtual machine images or container cluster compilation according to claim 1, characterized in that: The network policy module includes: In some complex network environments, accessing heterogeneous virtual machine images or container clusters to perform compilation and construction requires the corresponding network policies to be enabled in advance to prevent the basic components from being unable to perform compilation tasks due to network failures during collaborative calls. The network strategy is divided into virtual machine cluster grouping strategy, file synchronization distribution strategy, and port calling strategy between microservices.

4. A method for accessing heterogeneous virtual machine images or container cluster compilation, characterized in that: The method for accessing heterogeneous virtual machine images or container cluster compilation is applied to the system for accessing heterogeneous virtual machine images or container cluster compilation according to any one of claims 1 to 3, and the method for accessing heterogeneous virtual machine images or container cluster compilation includes: Initialize the virtual machine or container cluster node server, which includes shutting down seLinux, shutting down fireWalld, installing rsync and sysstat, adjusting the maximum number of open files, and confirming server time synchronization; Network policies between modules are enabled, including file distribution and transmission ports, calling ports between services, password-free policies between machines, and microservice database access ports; Synchronize basic component modules and install and start them, synchronize basic components from the source machine to the target machine, and start running components; Build a cluster, build a container cluster according to the official k8s process, and build the corresponding number of clusters according to actual business needs; Synchronously schedule heterogeneous virtual machine images or container clusters and install and start them, synchronously schedule services from the source machine to the target machine, and start the services; Configure a virtual machine cluster or container cluster, enter the virtual machine IP through the heterogeneous virtual machine builder module, or set the apiServer and kubeConfig credentials of the k8s cluster in the container cluster builder module to achieve the purpose of managing the virtual machine cluster and container cluster module.

5. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor when running to perform the steps of a method for accessing a heterogeneous virtual machine image or container cluster compilation as described in claim 4.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the method for accessing a heterogeneous virtual machine image or container cluster compilation as claimed in claim 4 are executed.

Images