Backup and restoration processing method of service system in Kubernetes cluster environment

Abstract

The invention discloses a backup and restoration processing method of a business system in a Kubernetes cluster environment, which comprises the following steps of: firstly, constructing a backup and restoration service based on java, creating a backup and restoration PVC (Poly Vinyl Chloride), and integrating the service into the backup and restoration service and a business system database class service; and the backup restoration service receives different backup parameters to realize backup of self-defined types and different database granularities, and transmits the backup parameters to the configured oss for storage. According to the technical scheme, different parameters can achieve self-defined types and backup of different database granularities, the parameters can be integrated into codes to be made into a visual operation interface, and backup and restoration operation is simplified.

Description

Technical Field

[0001] The present invention relates to the technical field of data backup, and in particular to a method for backing up and restoring a business system in a Kubernetes cluster environment. Background Art

[0002] There are currently two main methods in the Kubernetes backup and restore system: 1. Etcd backup and restore, which is usually used as the main backup method. Direct backup of Etcd is to back up all resources of the cluster. 2. Velero backup and restore. In addition to backing up the entire Kubernetes cluster, Velero can also back up or restore objects such as Type, Namespace, Label, etc. by category.

[0003] In the above two backup and restore systems, Etcd backs up the entire cluster, and Velero can perform classified backup. However, when it comes to the database, Velero can only back up and restore the entire database resources. Therefore, the granularity of the current backup and restore method needs to be further subdivided. Summary of the invention

[0004] In view of this, it is necessary to provide a backup and restore processing method for a business system in a Kubernetes cluster environment that can provide a smaller granularity data table in the database for backup and restoration.

[0005] A backup and restore processing method for a business system in a Kubernetes cluster environment is used for backup and restore of data files of a custom type of data granularity, including the following specific steps: Build a backup and restore service based on Java and integrate the service into the Kubernetes API; Create a backup and restore pvc and mount it under the backup and restore service and business system database service; The backup and restore service implements custom types and backups of different database granularities by receiving different backup parameters, and transfers the backup files to the configured OSS for storage; Restore the backup file.

[0006] Preferably, the step of integrating the service into the Kubernetes API further includes: Interact with the kubernetes cluster through the API, and store configuration files in OSS for backup files; Introduce Kubernetes dependencies into the original Java project pom file to implement the API interface for operating the kubernetes cluster, including query and modification operations on the main kubernetes components Service, Pod, Configmap, StatefulSet, Deployment, DaemonSet, Secret, and Job; Introduce minio dependency in the pom file to implement minio's operations on uploading and downloading files.

[0007] Preferably, the specific steps of creating a backup and restore pvc and mounting it under the backup and restore service and the business system database service include: In the kubernetes environment, execute the sh-5.1# kubectl create -f clb-data-pvc.yaml command to create a backup and restore pvc; The newly created database type container mounts the PVC to the database container through withVolumeMounts, and the backup service is built into the business system container.

[0008] Preferably, the backup and restore service implements the backup of custom types and different database granularities by receiving different backup parameters, and the specific steps of transferring the backup file to the configured OSS for storage include: Execute public File backup (List <string>backupItem, string path) throwsIoException command to back up the file. Among them, List <string>backupItem is the parameter passed in. The optional parameters include "APPLICATION" and "MONITOR". "APPLICATION" represents business data including postgresql database and MongoDB database data, and "MONITOR" represents monitoring data, i.e. influx database data. If only "APPLICATION" is passed in, only business data will be backed up. If both "APPLICATION" and "MONITOR" are passed in, both business data and monitoring data will be backed up. Kubernetes cluster backup includes three types: Configmap, Deployment, and StatefulSet; The Kubernetes cluster data backup is packaged and compressed into a file, and then transferred to the specific object storage through the OSS interface.

[0009] Preferably, the specific steps of restoring the backup file include: Select the backup file to be restored; The backup and restore service downloads files from OSS; Restore the backup file on the backup and restore PVC.

[0010] Preferably, the specific steps of performing the restore operation on the backup file on the backup and restore PVC include: Create a job through Kubernetes, add multiple containers to the job, and perform different restore operations in these different containers; The "shutdown-restore" container is used to stop all containers of the original business system connected to the database; The function of the "postgresql-restore" container is to restore the postgressql database; The function of the "mongo-restore" container is to restore the MongoDB database; The function of the "influx-restore" container is to restore the influx database; The function of the "restore-k8s-app" container is to restore the Configmap, Deployment, and StatefulSet of the Kubernetes cluster.

[0011] In the backup and restore processing method of the above-mentioned business system in the Kubernetes cluster environment, firstly, a backup and restore service is built based on Java, a backup and restore PVC is created, and the service is integrated into the backup and restore service and the business system database service. The backup and restore service receives different backup parameters to implement custom types and backups of different database granularities, and transmits them to the configured OSS for storage. The technical solution of the present invention can implement custom types and backups of different database granularities with different parameters, and can also be integrated into the code to make a visual operation interface to simplify the backup and restore operations. The technical solution of the present invention is simple, easy to implement, low-cost, and easy to promote. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flowchart of a method for backing up and restoring a business system in a Kubernetes cluster environment according to an embodiment of the present invention.

[0013] Figure 2 This is a schematic diagram of the operation interface of the backup and restore processing method of the business system in the Kubernetes cluster environment of the embodiment of the present invention Figure 1 .

[0014] Figure 3 This is a schematic diagram of the operation interface of the backup and restore processing method of the business system in the Kubernetes cluster environment of the embodiment of the present invention Figure 2 .

[0015] Figure 4 This is a schematic diagram of the operation interface of the backup and restore processing method of the business system in the Kubernetes cluster environment of the embodiment of the present invention Figure 3 . DETAILED DESCRIPTION

[0016] This embodiment takes the backup and restoration processing method of the business system in the Kubernetes cluster environment as an example, and the present invention will be described in detail below in conjunction with specific embodiments and drawings.

[0017] See also Figure 1 , showing a backup and restore processing method of a business system in a Kubernetes cluster environment provided by an embodiment of the present invention, the specific steps include: Step S10, build a backup and restore service based on Java, which integrates the Kubernetes API and can interact with the kubernetes cluster through the API. At the same time, the configuration file storage OSS is used to store backup files.

[0018] Step S11, interact with the kubernetes cluster through the api, and configure the configuration file to store oss for storing backup files.

[0019] Step S12: introduce the following Kubernetes dependency into the original Java project pom file.

[0020] <dependency> <groupid> io.fabric8< / groupid> <artifactid> kubernetes-client< / artifactid> < / dependency> Step S13, implement the API interface for operating the kubernetes cluster in the code, mainly including the query and modification operations of major kubernetes components such as Service, Pod, Configmap, StatefulSet, Deployment, DaemonSet, Secret, and Job.

[0021] Step S14, introduce minio dependency in the pom file.

[0022] <dependency> <groupid> io.minio< / groupid> <artifactid> minio< / artifactid> <version> ${minio.version}< / version> < / dependency> Step S15, implement minio operations such as file upload and download in the code.

[0023] / ** * @param path must be a file path (not ending with " / *) * @param objectString: file path: File object; InputStream object; * @param metadata * @param progresslistener *@throws clovdstorageException * / Void putobject(string path, object object, cloudobjectemetadatametadata, Progresslistener progresslistener) throws cloudstorageException; CloudobjectMetadata getobject(string path, file localstorage,ProgressListener listener); / ** Step S20, create a backup and restore pvc, and mount it under the backup and restore service and business system database service.

[0024] Step S21, execute the following command in the kubernetes environment to create a backup and restore pvc.

[0025] sh-5.1# kubectl create -f clb-data-pvc.yaml Step S22: The newly created database type container mounts the pvc to the database container through withVolumeMounts, and the backup service is built into the business system container.

[0026] private Container createMongocontainer(File tempDir,supportMongoProperties mongoconfig, string tempDirName) { string credential=mongoConfig.getCredentials(); int firstIndex=credential.index0f(":"); int lastIndex=credential.lastIndex0f(str:"@"); String username = credential.substring(0,firstIndex); String password = credential.substring(firstIndex + 1, lastIndex); String database= credential.substring(beginindex: lastIndex + 1); String libraryImage =getLibraryImage( service: "mongodb"); return new ContainerBuilder() .withName("mongo-backup") .withImage(libraryImage) .withCommand("bash" "-xc", mongodump mOngOdb: / / ${MONGODB_HOST}:${MONGODB_PORT}+ "--username ${MONGO USER}"+ "--password ${M0NG0 PASSWD}"+ "--authenticationDatabase ${MONGODB AUTH DATABASE}"+ "--dD ${MONGODB DATABASE}"+ "--gzip"+ ".-archive="+ tempDir.getPath()+ file.separator + "mongodb_data_$(date '+%s%3N').gz") .withEnv(envVar( name:"MONG0DB_HOST", mongoConfig.getHost()), envVar( name:"MONG0DB_PORT",String.valve0f(mongoConfig.getPort())), envVar( name:"MONG0 USER".username), envVar( name:"MONG0_PASSWD".password), envVar( name:"MONG0DB_AUTH_DATABASE". database), envVar( name:"MONG0DB_DATABASE", mongoConfig.getDatabase())) .withVolumeMounts(volumeMount(tempDir.getPath(),DATA_PVC_BACKUP_SUB_PATH)) .build(); } private container createPgcontainer(File tempDir, supportDBpropertiespgconfig, string tempDirName) { String libraryImage =getLibraryImage( service: "postgresgl"); return new ContainerBuilder() .withName("postgresql-backup") .withImage(libraryImage) .withCommand("bash","-xc, "expOrt PGPASSWORD=${PGPASSWORD}&&pg_dump " + "-h ${PGHOST}-P ${PGPORT} -U ${PGUSER} -d ${PGDATABASE}" + "--format=custom –blobs –clean --create" + "--file=" + tempDir.getPath()+ file.separator + "postgresql_data_$(date '+%s%3N').gz") .withEnv(envVar( name:"P6HosT",pgConfig.getHost()), envVar( name:"PGPORT",string.valve0f(pgconfig.getPort())), envVar( name:"PGUSER",pgConfig.getUname()), envVar( name:"PGPASSW0RD",pgConfig.getPassword()) envVar( name:"PGDATABASE",pgConfig.getDatabase())) .withVolumeMounts(volumeMount(tempDir.getPath(),DATA_PVC_BACKUP_SUB_PATH)) .build(); } private Container createInfluxcontainer(file tempDir, stringtempDirName) { String libraryImage = getLibraryImage(service:"influxdb"); return new ContainerBuilder() .withName("influx-backup") .withImage(libraryImage) .withCommand("bash"."-xc" "influx backup --host $fHosT} -b ${BUCKET} -0 $ORG} -t $fTOKEN} " +tempoir.getPath() + File.separator + "influxdb") .withEnv( envVar( name:"HoST",influxDBUrl), envVar( name:"BUCKET",influxBucketName), envVar( name:"0R6",influxDB0rg), envVar( name:"TOKEN".influxDBToken)) .withVolumeMounts(volumeMount(tempDir.getPath(),DATA_PVC_BACKUP_SUB_PATH)) .build(); } Step S30: The backup and restore service implements custom types and different database granularity backups by receiving different backup parameters, and transfers the backup files to the configured OSS for storage.

[0027] Step S31, execute public File backup (List <string>backupItem, string path)throws IoException command to back up the file, where List <string>backupItem is the parameter passed in. "APPLICATION" and "MONITOR" are optional. "APPLICATION" represents business data including postgresql database and MongoDB database data, and "MONITOR" represents monitoring data, i.e. influx database data. If only "APPLICATION" is passed in, only business data will be backed up. If both "APPLICATION" and "MONITOR" are passed in, both business data and monitoring data will be backed up.

[0028] Among them, the backup of Kubernetes cluster includes three types: Configmap, Deployment, and StatefulSet.

[0029] / / Backup configmap configuration List <configmap>cloudbility = k8sService.getConfigMapList(NAME_SPACE); File outputFile = new File(pathname: temp + " / " + "configmaps-backup.json"); try (FileOutputStream fileOutputStream = new FileOutputStream(outputFile)) { fileOutputStream.write(JsonUtil.toJson(cloudbility).getBytes(StandardCharsets.UTF_8));} catch (IOException e) { } / / Backup business deployment backupDeployment(tempDir, deploymentName: "console"); backupDeployment(tempDir, deploymentName: "executon"); backupDeployment(tempDir, deploymentName: "gateway"); backupDeployment(tempDir, deploymentName: "monitor"); backupDeployment(tempDir, deploymentName: "ocr"); backupDeployment(tempDir, deploymentName: "web"); / / Backup business statefulset backupStatefulSet(tempDir, statefulSetName: "portal"); backupStatefulSet(tempDir, statefulSetName: "scheduler"); backupStatefulSet(tempDir, statefulSetName: "nacos"); It should be noted that there are some syntax errors in the original code, such as incorrect capitalization in variable names and some missing semicolons. The translation has been made based on the corrected code as much as possible while maintaining the original meaning. / / Backup mongo, postgresql, influx data dataBackupJob(backupItem,tempDir,tempDirName); Step S32: After the above database types, i.e., Configmap, Deployment, and StatefulSet data of the Kubernetes cluster, are backed up, they are packaged and compressed into a file, and then transmitted to a specific object storage through the OSS interface.

[0030] Step S40, restoring the backup file.

[0031] Step S41, selecting the backup file to be restored.

[0032] Step S42: The backup and restore service downloads files from OSS.

[0033] / / temporary storage directory for backup files File dir = new File(getRestorePath()); / / Download historical backup files by calling OSS try { inputStream = getOssclient().readAsStream(ossBackupFileName); outputStream = new FileoutputStream(localBackupFile); IoUtil.copy(inputstream,outputstream): }catch(Exception e){ } / / Unzip the file unzip(localBackupFile,getRestorePath()); Step S43: restore the backup file on the backup and restore PVC.

[0034] Step S44, creating a job through Kubernetes, where there are multiple containers, and performing different restore operations in these different containers.

[0035] The backup file is a compressed package that contains Configmap, Deployment, StatefulSet type data of the Kubernetes cluster and postgresql, MongoDB, and influx database type data. After decompression, different restoration logics are processed according to different file types.

[0036] The most critical thing is that the service that triggers the restoration is on the original business service. The logic here is to only process the files that need to be restored and download the files to the backup restoration PVC. Then, a job is created through Kubernetes, and multiple containers are added to the job to perform different restoration operations in these different containers.

[0037] The "shutdown-restore" container is used to stop all containers of the original business system connected to the database.

[0038] / / To roll back data pg, mongo, influx, you need to stop all containers connected to the database first stopDeployment(console); stopDeployment(executor); stopDeployment(gateway); stopDeployment(monitor); stopDeployment(ocr); stopDeployment(web); Statefulset portal = getstatefulSet(NAMESPACE, name: "portal"); StatefulSet scheduler = getstatefulSet(NAMESPACE, name: "scheduler"); StatefulSet nacos = getStatefulSet(NAMESPACE, name: "nacos"); stopstatefulset(portal); stopstatefulset(scheduler); stopStatefulSet(nacos); The function of the "postgresql-restore" container is to restore the postgressql database.

[0039] The "mongo-restore" container is used to restore the MongoDB database.

[0040] The purpose of the "influx-restore" container is to restore the influx database.

[0041] The function of the "restore-k8s-app" container is to restore the Configmap, Deployment, and StatefulSet of the Kubernetes cluster.

[0042] / / Update configmap configuration / / Apply ConfiqMap to the Kubernetes cluster k8SService.updateConfigMap(namespace, configMapName, configMap); / / Update deployment restoreDeployment(files,deploymentName:"console"); restoreDeployment(files,deploymentName:"executor"); restoreDeployment(files,deploymentName:" gateway"); restoreDeployment(files,deploymentName:" monitor"); restoreDeployment(files,deploymentName:" ocr"); restoreDeployment(files,deploymentName: "web"); / / Update statefulset restorestatefulset(files statefulsettName:"portal"); restorestatefulset(files statefulsettName:"scheduler"); restorestatefulset(files.statefulsettName:"nacos"); } In this embodiment, the operation results of the visual operation interface are as follows: Figure 2 , Figure 3 and Figure 4 shown.

[0043] First, log in to the business system management console and select the following in the operation interface during backup: Select System Management -> Select Backup and Restore -> Select Create Backup -> Select the backup item -> Click Start Backup to complete the backup visualization operation.

[0044] When restoring, select in sequence on the operation interface: Select System Management -> Select Backup and Restore -> Select the backup file -> Click Restore Now -> Click Start Restore and Stop Service to complete the visual restore operation.

[0045] In the backup and restore processing method of the above-mentioned business system in the Kubernetes cluster environment, firstly, a backup and restore service is built based on Java, a backup and restore PVC is created, and the service is integrated into the backup and restore service and the business system database service. The backup and restore service receives different backup parameters to implement custom types and backups of different database granularities, and transmits them to the configured OSS for storage. The technical solution of the present invention can implement custom types and backups of different database granularities with different parameters, and can also be integrated into the code to make a visual operation interface to simplify the backup and restore operations. The technical solution of the present invention is simple, easy to implement, low-cost, and easy to promote.

[0046] It should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.< / configmap> < / string> < / string> < / string> < / string>

Claims

1. A backup and restore processing method for a business system in a Kubernetes cluster environment, used for backup and restore of data files of a custom type of data granularity, characterized in that: The specific steps include: Build a backup and restore service based on Java and integrate the service into the Kubernetes API; Create a backup and restore pvc and mount it under the backup and restore service and business system database service; The backup and restore service implements custom types and backups of different database granularities by receiving different backup parameters, and transfers the backup files to the configured OSS for storage; Restore the backup file.

2. The backup and restore processing method of the business system in a Kubernetes cluster environment according to claim 1, characterized in that: The service integration Kubernetes API step also includes: Interact with the kubernetes cluster through the API, and store configuration files in OSS for backup files; Introduce Kubernetes dependencies into the original Java project pom file to implement the API interface for operating the kubernetes cluster, including query and modification operations on the main kubernetes components Service, Pod, Configmap, StatefulSet, Deployment, DaemonSet, Secret, and Job; Introduce minio dependency in the pom file to implement minio's operations on uploading and downloading files.

3. The backup and restore processing method of the business system in the Kubernetes cluster environment as described in claim 1 is characterized in that: The specific steps of creating a backup and restore pvc and mounting it under the backup and restore service and the business system database service include: In the kubernetes environment, execute the sh-5.1# kubectl create -f clb-data-pvc.yaml command to create a backup and restore pvc; The newly created database type container mounts the PVC to the database container through withVolumeMounts, and the backup service is built into the business system container.

4. The backup and restore processing method of the business system in a Kubernetes cluster environment as claimed in claim 1, characterized in that: The backup and restore service implements custom types and different database granularity backup by receiving different backup parameters, and transfers the backup file to the configured OSS for storage. The specific steps include: Execute public File backup (List <string> backupItem, string path) throwsIoException command to back up the file.< / string> Among them, List <string> backupItem is the parameter passed in. The optional parameters include "APPLICATION" and "MONITOR". "APPLICATION" represents business data including postgresql database and MongoDB database data, and "MONITOR" represents monitoring data, i.e. influx database data. If only "APPLICATION" is passed in, only business data will be backed up. If both "APPLICATION" and "MONITOR" are passed in, both business data and monitoring data will be backed up.< / string> Kubernetes cluster backup includes three types: Configmap, Deployment, and StatefulSet; The Kubernetes cluster data backup is packaged and compressed into a file, and then transferred to the specific object storage through the OSS interface.

5. The backup and restore processing method of the business system in a Kubernetes cluster environment as described in claim 1, characterized in that: The specific steps of restoring the backup file include: Select the backup file to be restored; The backup and restore service downloads files from OSS; Restore the backup file on the backup and restore PVC.

6. The backup and restore processing method of the business system in a Kubernetes cluster environment as described in claim 5, characterized in that: The specific steps of restoring the backup file on the backup and restore PVC include: Create a job through Kubernetes, add multiple containers to the job, and perform different restore operations in these different containers; The "shutdown-restore" container is used to stop all containers of the original business system connected to the database; The function of the "postgresql-restore" container is to restore the postgressql database; The function of the "mongo-restore" container is to restore the MongoDB database; The function of the "influx-restore" container is to restore the influx database; The function of the "restore-k8s-app" container is to restore the Configmap, Deployment, and StatefulSet of the Kubernetes cluster.

Images