Rapid system setting method, medium and device

Abstract

The invention discloses a system rapid setup method, medium and device, and the system rapid setup method comprises a core integration deployment module which is used for the distribution and scheduling of automatic deployment tasks; the user interaction module interacts with the core integration deployment module and adopts a front end and rear end separated architecture, the front end is used for providing an interaction interface for a user so that the user can operate the automatic application deployment system through a browser, and the rear end is used for providing various front end requests; the application package management module interacts with the core integration deployment module and is used for managing an application package; a dependent packet management module; a mirror image management module; and a data unit module. The core integration deployment module, the user interaction module, the application package management module, the dependency package management module, the mirror image management module, the data unit module and the like which are matched with one another are adopted for matching, so that the service system can be quickly opened.

Description

Technical Field

[0001] The present invention belongs to the technical field of system architecture, and in particular relates to a method, medium and device for rapid system opening. Background Art

[0002] The current methods for rapid system deployment mainly include automated script deployment and mirroring deployment. When the business continues to develop and the data volume and concurrency reach a certain level, the database may face scalability and other problems, which will lead to migration needs such as business opening, splitting, migration, and hardware equipment maintenance.

[0003] The Chinese invention patent with publication number CN116976640A discloses an automated business generation method, device, computer equipment and storage medium. The application uses a base target big model to generate a set of business steps corresponding to the business scenario to which the actual business demand description belongs. Compared with traditional automation technology and software systems, the target big model has a better natural language understanding ability, and can generate a set of business steps that are more in line with the actual business demand description. Then, the automated arrangement of business steps based on the thinking tree simulates the process of exploring different possibilities and exploring before and after a certain possible node when making human decisions, improves the decision-making ability of automated business arrangement, and can decide an automated business generation result based on different sorting results, thereby improving the accuracy of automated business arrangement. However, the automated business generation device needs to have a sorting module, a determination module and a selection module to cooperate, resulting in a low ability to quickly deploy business and a slow application response speed. Summary of the invention

[0004] In view of the above-mentioned defects, the present invention provides a system rapid opening device.

[0005] In order to achieve the above object, the present invention provides a system rapid opening device, which comprises: A core integrated deployment module, which is used to automatically distribute and schedule deployment tasks; A user interaction module, which interacts with the core integrated deployment module and adopts a front-end and back-end separation architecture. The front-end is used to provide an interactive interface for users, so that users can operate the application automation deployment system through a browser, and the back-end is used to provide implementation of various front-end requests; An application package management module, which interacts with the core integrated deployment module and is used to manage application packages; A dependency package management module, the dependency package management module communicates with the core integrated deployment module and is used to manage dependency data; An image management module, which interacts with the core integrated deployment module and is used to provide management operations for images and image repositories; A data unit module, wherein the data unit module interacts with the user interaction module, the application package management module, the dependency package management module and the image management module respectively, and is used for continuously processing platform data.

[0006] Optimally, the request sent by the user interaction module is processed by the application package management module, which first interacts with the data unit module to persist the relevant information of the application package and the data required to initiate a cluster resource creation request, and then initiates a request to the cluster to create a persistent storage volume declaration. Storage is provided by NFS-type storage services. After the resource creation is completed, the application package data is persisted in the system cluster. When a request is used to initiate an image building task and the application package is specified to be used in the task, the corresponding data will be mounted.

[0007] Optimally, the core integrated deployment module includes a code checkout / compile module, an image building module coordinated with the code checkout / compile module, and an application deployment module coordinated with the image building module.

[0008] Optimally, the management of the dependency data includes creating dependencies, uploading dependencies, deleting dependencies, downloading dependency data, and retrieving dependencies by cluster or name. The creation of dependencies is to create information for mounting storage volumes, and the dependency upload is to mount local data to the storage volume that has been created.

[0009] Optimally, the image management includes image lifecycle management, image upload / download, image vulnerability scanning and image push, and the image repository management includes image repository backup, image repository cleanup and image repository monitoring.

[0010] Optimally, the data unit module has a system basic data interface, a mirror data management interface, a file data management interface and a cluster resource data management interface, which correspondingly use SQL to store system basic data, Harbor to manage mirror data, NFS to manage file data and Etcd to manage cluster resource data.

[0011] Optimally, the data unit module consists of three parts: a data source management module, a data unit management module, and a data packet module. The data source management module is used for data domain adaptation access management of block storage, file system, object storage, database, and message queue to form a data source used by the data unit module. The data unit management module is used for processing operations on the data unit of this node, including creation, query, deletion, import and export of data packets, etc. The data packet management module is used to package the data generated by the service instance on which the data unit is mounted, provide download and import of data packets, and realize offline migration and distribution of data.

[0012] The system rapid opening device of the present invention can realize the rapid opening of the business system by adopting the coordinated core integrated deployment module, user interaction module, application package management module, dependency package management module, image management module and data unit module.

[0013] Another object of the present invention is to provide a system rapid opening medium, which is based on the above-mentioned system rapid opening device.

[0014] Another object of the present invention is to provide a method for rapid system establishment, which is based on the above-mentioned rapid system establishment device. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of the functional modules of the device for quickly opening the system of the present invention; Figure 2 An architectural diagram of the core integrated deployment module in the device for quickly opening the system of the present invention; Figure 3 A schematic diagram of the application package management module in the device for quickly opening the system of the present invention; Figure 4 A mirror push model diagram in a device is quickly opened for the system of the present invention; Figure 5 The structure diagram of the data unit module in the device for quickly opening the system of the present invention; Figure 6 The invention discloses a processing flow of a data unit module in a device for quickly opening a system. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained under the premise of equivalent changes and modifications made by ordinary technicians in the field should belong to the scope of protection of the present invention.

[0017] like Figure 1 The system quick opening device shown mainly includes a core integrated deployment module, a user interaction module, an application package management module, a dependency package management module, an image management module and a data unit module.

[0018] Among them, the core integrated deployment module is the most critical functional module for automated deployment. This module is responsible for the distribution and scheduling of automated deployment tasks, as well as the interaction with other modules (mainly data processing). The Jenkins component can be integrated to protect the core integrated deployment module, so that the Jenkins component can complete some application integration and simple deployment tasks. The final implementation of the core integrated deployment module will call a small number of services in this area. Jenkins runs in the Pod resources of the system cluster. When a service is needed, the system will determine whether to temporarily create a slave node to provide the service. The ability of the container cloud to dynamically scale resources is fully utilized. Specifically, the core integrated deployment module includes a code checkout / compilation module, an image construction module that cooperates with the code checkout / compilation module, and an application deployment module that cooperates with the image construction module.

[0019] The core integrated deployment module implements a series of functions from source code building, code compilation, image building to application deployment. Using the pipeline concept in Jenkins, the core integrated deployment module provides a more complete and easier to use pipeline operation. This loosely coupled design method of the pipeline makes the system highly flexible and can easily adapt to the automated deployment needs of various applications. A pipeline is an application automated deployment task, which can include one or more steps and can be selected and configured according to the actual situation of the application to be deployed. This design concept also greatly facilitates the later expansion of the system. Figure 2 As shown, the connection between the system pipeline and the core integrated deployment module is revealed.

[0020] The user interaction module interacts with the core integrated deployment module. It adopts a front-end and back-end separated architecture. The front-end is used to provide an interactive interface for users to operate the application automation deployment system through a browser, and the back-end is used to provide the implementation of various front-end requests (the functions implemented by the back-end mainly include the creation, configuration, execution, and deletion of automated deployment tasks, as well as application package management, dependency management, and image management). The back-end handles the specific implementation of these functions and the interaction with the data unit module.

[0021] The application package management module interacts with the core integrated deployment module and is used to manage application packages, including uploading application packages from the local computer, retrieving application packages by type, attribute, and name, and modifying or deleting application packages. This module is designed to supplement the automated deployment of applications that cannot obtain source code from the code repository. This module is implemented using the platform's default storage service NFS; when uploading an application package, the relevant information of the application package is persisted to the database, and a storage volume is mounted using the k8s PVC resource to save the application package. When it is needed, the storage volume is remounted. Figure 3 This is the basic design idea for the application package management module.

[0022] The request sent by the user interaction module is processed by the application package management module, which first interacts with the data unit management module (i.e., the data unit module) to persist the relevant information of the application package and the data required to initiate the cluster resource creation request, and then initiates a request to create a persistent storage volume declaration (PVC) to the cluster, which is provided by the NFS type storage service. After the resource creation is completed, the application package data is persisted in the system cluster. When the system initiates a request for an image building task and specifies the use of the application package in the task, the corresponding data will be mounted.

[0023] The dependency package management module communicates with the core integrated deployment module to manage dependency data. The dependency management module is responsible for the management of dependency data, including creating dependencies, uploading dependencies, deleting dependencies, downloading dependency data, and retrieving dependencies by cluster or name. Creating dependencies is essentially creating information for mounting storage volumes. Similar to the management of application packages, it is also implemented using the platform's storage service. The difference is that dependency management can select storage services, while application package management is limited to using the default storage service, which makes the storage solution of dependency management more flexible. Dependency upload is to mount local data to the created storage volume. The dependency management module receives and processes requests for adding, deleting, and using dependencies initiated by the user interaction module. Similar to the application package management module, the request will be converted by the system into a resource application call for K8s. The difference is that when creating a persistent storage volume declaration, it supports the use of multiple storage services. All storage services currently supported by the platform can be used. In addition, unlike application package management, dependency data will only be mounted as needed when the system initiates a request for code compilation.

[0024] The image management module interacts with the core integrated deployment module to provide management operations for images and image repositories. Image management includes image lifecycle management, image upload / download, image vulnerability scanning, and image push. Image lifecycle management mainly includes the addition, deletion, modification, and query of images. The system has comprehensive permission control and can set roles for different repositories and images. Users can be assigned a role to obtain the permissions of that role, thereby obtaining read and write permissions for repositories and images. Permissions are mainly divided into three types: read-write-delete permissions, read-write permissions, and readable permissions. Image upload / download provides two methods: manual upload / download and command line upload / download; image vulnerability scanning is supported by the system-integrated general vulnerability threat scanning component Clair (Clair is an open source tool that monitors container security by analyzing vulnerabilities). Image push is the synchronization of images in two different environments, generally synchronizing the image of the test environment to the production environment. For example Figure 4 The following is the design model of the image push function. The data that needs to be synchronized for image push is mainly the data of some database tables in the system and the image data in the harbor warehouse. The system mainly completes synchronization through http requests. The management of the image warehouse includes image warehouse backup, image warehouse cleanup, and image warehouse monitoring. Image warehouse backup is to copy the image warehouse to the backup server. Rules can also be set, and then the synchronization time node can be triggered according to the rules.

[0025] The data unit module (also called the data unit management module) interacts with the user interaction module, application package management module, dependency package management module and image management module respectively to continuously process platform data. Figure 5 As shown, the data unit module has a system basic data interface, an image data management interface, a file data management interface, and a cluster resource data management interface, which correspond to using SQL to store system basic data, using Harbor to manage image data, using NFS to manage file data, and using Etcd to manage cluster resource data. The data unit management module is responsible for the persistence of platform data, and the system mainly manages four types of data: first, data closely related to platform and user interaction, which are structured; second, data related to images, which are not very structured and are stored in layers; third, file-type data such as dependency packages and application packages, which are characterized by the fact that the data itself consumes a lot of space and is not suitable for splitting; fourth, data related to cluster resources, which are characterized by strong real-time performance and high correlation with cluster resources.

[0026] The basic data of the system is stored using SQL. Most of this data is data from the interaction between the platform and the user, and it is highly structured, so it is stored in SQL for management; image data generally takes up a large space, and the image information is layered rather than structured, so the system uses Harbor to manage image data; file data also takes up a large space, and the data is not suitable for split storage. Since the data itself is large, it should be read directly when used. When the system obtains file data such as dependencies and application packages, it uses NFS to mount the storage volume and then reads the corresponding data; cluster resource data is stored by K8s's Etcd. Most of this data is highly real-time, but some data will also be associated and stored in SQL. Most of the time, the system queries through SQL and only queries Etcd when necessary.

[0027] In the rapid opening of business systems, data unit modules refer to business-related databases, caches, message queues and other data storage or transmission components. These data units play a vital role in business systems because they store and transmit business data and have a direct impact on the availability, reliability and performance of business systems.

[0028] The data unit module (referred to as data unit) plays a very important role in the rapid opening of the business system, mainly in the following aspects: 1. The data unit module is one of the basic components of the business system. It contains all the data in the business system, including data table structure, data content, data relationship, etc. When the business system is rapidly deployed, the preparation of the data unit is very critical. Only accurate, complete and clear data units can ensure the normal operation of the business system. 2. The rapid deployment of the data unit module can greatly shorten the deployment time of the business system, thereby improving the efficiency of the business system. By preparing the complete data unit in advance, problems such as data loss and inconsistent data structure can be avoided during the deployment process, thereby greatly shortening the time to go online for the business system. 3. The rapid deployment of the data unit module can ensure the data consistency and integrity of the business system. When deploying the business system, if the data unit is not fully considered and prepared, it is easy to have problems such as inconsistent data or missing data, which will have a serious impact on the normal operation of the business system and even cause the collapse of the business system. In the present invention, the main functions of the data unit acting on the rapid opening platform of the business system are data packaging, pushing, injection (injection), configuration perception, and service binding.

[0029] The data unit defined in the present invention essentially abstracts various types of data resources into individual data units, and uses the data unit as the core of unified data management, performs unified storage management on the data resources of the platform, provides a unified data access interface for upper-level services, establishes unified standards for data authority management, access control, connection, sharing and distribution, and provides a data storage support foundation for the rapid opening of business systems. It supports dynamic switching and mounting between data units and services, and has fast data access and storage capabilities. The data unit can implement data access control management and provide data migration capabilities in units of data units. Through access management of storage units such as files and directories, database tables, and message queues, storage isolation and sharing capabilities are provided. In the present invention, the data unit module consists of three parts: a data source management module, a data unit management module, and a data packet module. The data source management module is responsible for implementing data domain adaptation access management of block storage, file systems, object storage, databases, message queues, etc., and forming the data source used by the data unit.

[0030] The data unit management module is responsible for processing operations on the data unit of this node, mainly including the creation, query, deletion, import and export of data units, etc. Files, database tables or objects are packaged and encapsulated to build data units. For multi-channel data unit generation, the data flow is directed according to the load of each generation channel to ensure balanced distribution of data unit segmentation processing. The data packet module management is responsible for packaging the data generated by the service instance that mounts the data unit, providing data packet download and import, and realizing the offline migration and distribution capabilities of data. The processing flow of the data unit is as follows: Figure 6 As shown, the workflow is as follows: a) User login and authentication of user identity; b) Through the data source related interface, access to data domains such as databases, file systems, object storage, and message queues is achieved, data source access operations are completed, and the source of data storage is provided for the data unit; c) After the data source is successfully connected, the data unit is created in the data unit interface, providing the service management module with data unit mounting capabilities. After the service is installed and running, data can be continuously written. The data of the data unit can be exported as a data package, providing a basis for offline data migration; d) Offline exported data packets can be imported into a cloud-edge information system to form data units with relevant data, provide data support for services, and support management operations such as downloading and deleting data packets; e) The operation is completed and exit.

[0031] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. At the same time, the above description should be understandable and implementable for those with ordinary knowledge in the relevant technical field. Therefore, other equivalent changes or modifications that do not deviate from the concepts disclosed in the present invention should be included in the scope of protection of the present invention.

Claims

1. A system rapid opening device, characterized in that: It includes: A core integrated deployment module, which is used to automatically distribute and schedule deployment tasks; A user interaction module, which interacts with the core integrated deployment module and adopts a front-end and back-end separation architecture. The front-end is used to provide an interactive interface for users, so that users can operate the application automation deployment system through a browser, and the back-end is used to provide implementation of various front-end requests; An application package management module, which interacts with the core integrated deployment module and is used to manage application packages; A dependency package management module, the dependency package management module communicates with the core integrated deployment module and is used to manage dependency data; An image management module, which interacts with the core integrated deployment module and is used to provide management operations for images and image repositories; A data unit module, wherein the data unit module interacts with the user interaction module, the application package management module, the dependency package management module and the image management module respectively, and is used for continuously processing platform data.

2. The system rapid opening device according to claim 1, characterized in that: The request sent by the user interaction module is processed by the application package management module, which first interacts with the data unit module to persist the relevant information of the application package and the data required to initiate a cluster resource creation request, and then initiates a request to the cluster to create a persistent storage volume declaration. Storage is provided by the NFS type storage service. After the resource creation is completed, the application package data is persisted in the system cluster. When a request is used to initiate an image building task and the application package is specified to be used in the task, the corresponding data will be mounted.

3. The system rapid opening device according to claim 1, characterized in that: The core integrated deployment module includes a code checkout / compile module, an image building module coordinated with the code checkout / compile module, and an application deployment module coordinated with the image building module.

4. The system rapid opening device according to claim 1, characterized in that: The management of dependency data includes creating dependencies, uploading dependencies, deleting dependencies, downloading dependency data, and retrieving dependencies by cluster or name. Creating dependencies is to create information for mounting storage volumes, and uploading dependencies is to mount local data to the storage volumes that have been created.

5. The system rapid opening device according to claim 1, characterized in that: The image management includes image lifecycle management, image upload / download, image vulnerability scanning and image push, and the image warehouse management includes image warehouse backup, image warehouse cleanup and image warehouse monitoring.

6. The system rapid opening device according to claim 1, characterized in that: The data unit module has a system basic data interface, a mirror data management interface, a file data management interface and a cluster resource data management interface, which correspondingly adopts SQL to store system basic data, adopts Harbor to manage mirror data, adopts NFS to manage file data and adopts Etcd to manage cluster resource data.

7. The system rapid opening device according to claim 1, characterized in that: The data unit module consists of three parts: a data source management module, a data unit management module, and a data packet module. The data source management module is used for data domain adaptation access management of block storage, file system, object storage, database, and message queue to form the data source used by the data unit module. The data unit management module is used for processing operations on the data unit of this node, including creation, query, deletion, import and export of data packets, etc. The data packet management module is used to package the data generated by the service instance on which the data unit is mounted, provide download and import of data packets, and realize offline migration and distribution of data.

8. A system for quickly opening a medium, characterized in that: It is based on a system rapid opening device as described in any one of claims 1 to 7.

9. A method for rapid system opening, characterized in that: It is based on a system rapid opening device as described in any one of claims 1 to 7.

Images