A Backup and Restoration Processing Method for a Business System in a Kubernetes Cluster Environment

Abstract

The present invention discloses a backup and restoration processing method for a business system in a Kubernetes cluster environment. First, a backup and restoration service is built based on Java, a backup and restoration PVC is created, and the service is integrated under the backup and restoration service and the business system database service. The backup and restoration service receives different backup parameters to implement custom type and different database granularity backups, and transmits them to the configured oss for storage. The technical solution of the present invention can implement custom type and different database granularity backups with different parameters, and can also be integrated into the code to form a visual operation interface, simplifying the backup and restoration operations.

Description

Technical Field

[0001] The present invention relates to the technical field of data backup, and particularly to a backup and restoration processing method for a business system in a Kubernetes cluster environment. Background Art

[0002] Currently, there are mainly two methods in the backup and restoration system of Kubernetes: 1. Etcd backup and restoration, which is usually used as the main backup method. Directly backing up Etcd means backing up all the resources of the cluster. 2. Velero backup and restoration. In addition to backing up the entire Kubernetes cluster, Velero can also perform classified backup or restoration on objects such as Type, Namespace, and Label.

[0003] In the above two backup and restoration systems, Etcd backs up the entire cluster, and Velero can perform classified backup. However, when it comes to databases, Velero can only back up and restore the entire database resources. Therefore, the granularity of the current backup and restoration methods needs to be further refined. Summary of the Invention

[0004] In view of this, it is necessary to provide a backup and restoration processing method for a business system in a Kubernetes cluster environment that can provide smaller-granularity data tables in the database for backup and restoration.

[0005] A backup and restoration processing method for a business system in a Kubernetes cluster environment, used for backup and restoration of data files with custom-type data granularity, includes the following specific steps:

[0006] Build a backup and restoration service based on Java and integrate the service into the Kubernetes api;

[0007] Create a backup and restoration pvc and mount it under the backup and restoration service and the business system database service;

[0008] The backup and restoration service realizes custom-type and different database granularity backups by receiving different backup parameters, and transfers the backup files to the configured oss for storage;

[0009] Perform a restoration operation on the backup files.

[0010] Preferably, the step of integrating the service into the Kubernetes api further includes:

[0011] Interact with the Kubernetes cluster through the api, and at the same time configure the file storage oss for storing backup files;

[0012] Introduce Kubernetes dependencies in the original Java project's pom file to implement the API interface for operating the Kubernetes cluster, including query and modification operations on the main Kubernetes components Service, Pod, Configmap, StatefulSet, Deployment, DaemonSet, Secret, and Job;

[0013] Introduce Minio dependencies in the pom file to implement Minio's operations for uploading and downloading files.

[0014] Preferably, the specific steps for creating a backup and restore PVC and mounting it under the backup and restore service and the business system database service include:

[0015] Execute the command sh-5.1# kubectl create -f clb-data-pvc.yaml in the Kubernetes environment to create a backup and restore PVC;

[0016] The newly created database type container mounts the PVC to the database container through withVolumeMounts, and the backup service is built into the business system container.

[0017] Preferably, the specific steps for the backup and restore service to achieve custom type and different database granularity backups by receiving different backup parameters and transfer the backup files to the configured OSS for storage include:

[0018] Execute public File backup(List <string>The backupItem, string path) throws IoException command backs up the file,

[0019] Among them, List <string>The backupItem is the passed-in parameter. The optional parameters include "APPLICATION" and "MONITOR". "APPLICATION" represents business data, including data from the postgresql database and the MongoDB database, and "MONITOR" represents monitoring data, that is, data from the influx database. If only "APPLICATION" is passed in, only business data will be backed up. If both "APPLICATION" and "MONITOR" are passed in, both business data and monitoring data will be backed up.

[0020] Backups in the Kubernetes cluster include three types: Configmap, Deployment, and StatefulSet.

[0021] The data backups in the Kubernetes cluster are packaged and compressed into a single file, and then transferred to a specific object storage through the oss interface.

[0022] Preferably, the specific steps for restoring the backup file include:

[0023] Select the backup file to be restored;

[0024] The backup restore service downloads the file from oss;

[0025] Perform a restore operation on the backup file on the backup restore PVC.

[0026] Preferably, the specific steps for performing a restore operation on the backup file on the backup restore PVC include:

[0027] Create a job through Kubernetes. There are multiple containers in the job, and different restore operations are performed in these different containers;

[0028] Among them, the role of the "shutdown-restore" container is to stop all containers in the original business system that connect to the database;

[0029] The role of the "postgresql-restore" container is to restore the postgressql database;

[0030] The role of the "mongo-restore" container is to restore the MongoDB database;

[0031] The role of the "influx-restore" container is to restore the influx database;

[0032] The function of the "restore-k8s-app" container is to restore Configmap, Deployment, and StatefulSet in the Kubernetes cluster.

[0033] In the backup and restore processing method of the above business system in the Kubernetes cluster environment, first, a backup and restore service is built based on Java, a backup and restore PVC is created, and the service is integrated under the backup and restore service and the business system database service. The backup and restore service receives different backup parameters to achieve custom type and different database granularity backups, and transmits them to the configured oss for storage. The technical solution of the present invention can achieve custom type and different database granularity backups with different parameters, and can also be integrated into the code to form a visual operation interface, simplifying the backup and restore operations. The technical solution of the present invention is simple, easy to implement, low in cost, and convenient for promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flowchart of the backup and restore processing method of the business system in the Kubernetes cluster environment according to an embodiment of the present invention.

[0035] Figure 2 is a schematic diagram of the operation interface of the backup and restore processing method of the business system in the Kubernetes cluster environment according to an embodiment of the present invention Figure 1 .

[0036] Figure 3 is a schematic diagram of the operation interface of the backup and restore processing method of the business system in the Kubernetes cluster environment according to an embodiment of the present invention Figure 2 .

[0037] Figure 4 is a schematic diagram of the operation interface of the backup and restore processing method of the business system in the Kubernetes cluster environment according to an embodiment of the present invention Figure 3 . DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] Taking the backup and restore processing method of the business system in the Kubernetes cluster environment as an example in this embodiment, the present invention will be described in detail below with reference to specific embodiments and the accompanying drawings.

[0039] Please refer to Figure 1 , which shows a backup and restore processing method of a business system in the Kubernetes cluster environment provided by an embodiment of the present invention. The specific steps include:

[0040] Step S10: Build a backup and restore service based on Java. This service integrates the Kubernetes API and can interact with the Kubernetes cluster through the API. At the same time, the configuration file is stored in OSS for storing backup files.

[0041] Step S11: Interact with the Kubernetes cluster through the API. At the same time, the configuration file is stored in OSS for storing backup files.

[0042] Step S12: Introduce the following Kubernetes dependencies in the original Java project's pom file.

[0043] <dependency>

[0044] <groupid>io.fabric8< / groupid>

[0045] <artifactid>kubernetes-client< / artifactid>

[0046] < / dependency>

[0047] Step S13: Implement the API interfaces for operating on the Kubernetes cluster in the code, mainly including query and modification operations of the main Kubernetes components such as Service, Pod, Configmap, StatefulSet, Deployment, DaemonSet, Secret, and Job.

[0048] Step S14: Introduce the Minio dependency in the pom file.

[0049] <dependency>

[0050] <groupid>io.minio< / groupid>

[0051] <artifactid>minio< / artifactid>

[0052] <version>${minio.version}< / version>

[0053] < / dependency>

[0054] Step S15: Implement operations such as file upload and download by Minio in the code.

[0055] / **

[0056] * @param path must be a file path (not ending with " / *")

[0057] * @param objectString: the relative path of the file: File object; InputStream object;

[0058] *@param metadata

[0059] *@param progresslistener

[0060] *@throws clovdstorageException

[0061] * /

[0062] Void putobject(string path, object object, cloudobjectemetadatametadata, Progresslistener progresslistener) throws cloudstorageException;

[0063] CloudobjectMetadata getobject(string path, file localstorage,ProgressListener listener);

[0064] / **

[0065] Step S20, create a backup and restore PVC and mount it under the backup and restore service and the business system database service.

[0066] Step S21, execute the following command in the kubernetes environment to create a backup and restore PVC.

[0067] sh-5.1# kubectl create -f clb-data-pvc.yaml

[0068] Step S22, the newly created database type container mounts the PVC under the database container through withVolumeMounts, and the backup service is built into the business system container.

[0069] private Container createMongocontainer(File tempDir,supportMongoProperties mongoconfig, string tempDirName) {

[0070] string credential=mongoConfig.getCredentials();

[0071] int firstIndex=credential.index0f(":");

[0072] int lastIndex=credential.lastIndex0f( str:"@");

[0073] String username = credential.substring(0,firstIndex);

[0074] String password = credential.substring(firstIndex + 1, lastIndex);

[0075] String database= credential.substring( beginindex: lastIndex + 1);

[0076] String libraryImage =getLibraryImage( service: "mongodb");

[0077] return new ContainerBuilder()

[0078] .withName("mongo-backup")

[0079] .withImage(libraryImage)

[0080] .withCommand("bash","-xc",

[0081] mongodump mOngOdb: / / ${MONGODB_HOST}:${MONGODB_PORT}+

[0082] "--username ${MONGO USER}"+

[0083] "--password ${M0NG0 PASSWD}"+

[0084] "--authenticationDatabase ${MONGODB AUTH DATABASE}"+

[0085] "--dD ${MONGODB DATABASE}"+

[0086] "--gzip"+

[0087] ".-archive="+ tempDir.getPath()+ file.separator + "mongodb_data_$(date '+%s%3N').gz")

[0088] It should be noted that there are some potential issues in the original code like incorrect syntax in the `substring` method call in line 4 and some inconsistent capitalization in the variable names and commands which might lead to compilation errors in a real programming context. The translation is done strictly according to the rules while keeping those issues intact..withEnv(envVar( name:"MONG0DB_HOST", mongoConfig.getHost()),

[0089] envVar( name:"MONG0DB_PORT",String.valve0f(mongoConfig.getPort())),

[0090] envVar( name:"MONG0 USER".username),

[0091] envVar( name:"MONG0_PASSWD".password),

[0092] envVar( name:"MONG0DB_AUTH_DATABASE". database),

[0093] envVar( name:"MONG0DB_DATABASE", mongoConfig.getDatabase()))

[0094] .withVolumeMounts(volumeMount(tempDir.getPath(),DATA_PVC_BACKUP_SUB_PATH))

[0095] .build();

[0096] }

[0097] private container createPgcontainer(File tempDir, supportDBpropertiespgconfig, string tempDirName) {

[0098] String libraryImage =getLibraryImage( service: "postgresgl");

[0099] return new ContainerBuilder()

[0100] .withName("postgresql-backup")

[0101] .withImage(libraryImage)

[0102] It should be noted that there are some potential typos in the original text (e.g., "MONG0DB", "valve0f", "MONG0 USER", "MONG0_PASSWD", "MONG0DB_AUTH_DATABASE", "postgresgl" which might affect the accurate understanding and further processing. It's advisable to double-check and correct these if possible for a more meaningful translation and interpretation in a proper context..withCommand("bash","-xc,

[0103] \"export PGPASSWORD=${PGPASSWORD}&&pg_dump \" +

[0104] \"-h ${PGHOST}-P ${PGPORT} -U ${PGUSER} -d ${PGDATABASE}\" +

[0105] \"--format=custom –blobs –clean --create\" + \"--file=\" +

[0106] tempDir.getPath()+ file.separator + \"postgresql_data_$(date '+%s%3N').gz\")

[0107] .withEnv(envVar( name:\"P6HosT\",pgConfig.getHost()),

[0108] envVar( name:\"PGPORT\",string.valueOf(pgconfig.getPort())),

[0109] envVar( name:\"PGUSER\",pgConfig.getUname()),

[0110] envVar( name:\"PGPASSW0RD\",pgConfig.getPassword())

[0111] envVar( name:\"PGDATABASE\",pgConfig.getDatabase()))

[0112] .withVolumeMounts(volumeMount(tempDir.getPath(),DATA_PVC_BACKUP_SUB_PATH))

[0113] .build();

[0114] }

[0115] It should be noted that there may be some incorrect variable names or syntax in the original text (such as "pgconfig" instead of "pgConfig" in some places), which may need to be further corrected for correct operation and understanding in a real programming or system context.private Container createInfluxcontainer(file tempDir, string tempDirName) {

[0116] String libraryImage = getLibraryImage( service:"influxdb");

[0117] return new ContainerBuilder()

[0118] .withName("influx-backup")

[0119] .withImage(libraryImage)

[0120] .withCommand("bash","-xc"

[0121] "influx backup --host $fHosT} -b ${BUCKET} -0 $ORG} -t $fTOKEN} " + tempoir.getPath() + File.separator + "influxdb")

[0122] .withEnv(

[0123] envVar( name:"HoST",influxDBUrl),

[0124] envVar( name:"BUCKET",influxBucketName),

[0125] envVar( name:"0R6",influxDB0rg),

[0126] envVar( name:"TOKEN".influxDBToken))

[0127] .withVolumeMounts(volumeMount(tempDir.getPath(),DATA_PVC_BACKUP_SUB_PATH))

[0128] .build();

[0129] }

[0130] It should be noted that there seem to be some syntax errors in the original code (e.g., incorrect use of double quotes and dollar signs in commands and variable references), which may affect the correct understanding and translation. The above translation is based on the literal content of the provided text.Step S30, the backup and restoration service realizes custom-type and different database granularity backups by receiving different backup parameters, and transfers the backup files to the configured OSS for storage.

[0131] Step S31, execute public File backup(List <string>The backupItem, string path) throws IoException command backs up the file, where

[0132] List <string>The backupItem is the passed-in parameter, "APPLICATION" and "MONITOR" are optional. "APPLICATION" represents business data including data from the PostgreSQL database and the MongoDB database, and "MONITOR" represents monitoring data, that is, data from the Influx database. If only "APPLICATION" is passed in, only business data is backed up. If both "APPLICATION" and "MONITOR" are present, both business data and monitoring data are backed up.

[0133] Among them, the backup of the Kubernetes cluster includes three types: Configmap, Deployment, and StatefulSet.

[0134] / / Backup the configmap configuration

[0135] List <configmap>cloudbility = k8sService.getConfigMapList(NAME_SPACE);

[0136] File outputFile = new File(pathname: temp + " / " + "configmaps-backup.json");

[0137] try (FileOutputStream fileOutputStream = new FileOutputStream(outputfile)) {

[0138] fileOutputstream.write(Jsonutil.toJson(cloudbility).getBytes(StandardCharsets.UTF_8));} catch (IOException e) {

[0139] }

[0140] / / Backup business deployment

[0141] backupDeployment(tempDir, deploymentName: "console");

[0142] backupDeployment(tempDir, deploymentName: "executon");

[0143] backupDeployment(tempDir, deploymentName: "gateway");

[0144] backupDeployment(tempDir, deploymentName: "monitor");

[0145] backupDeployment(tempDir, deploymentName: "ocr");

[0146] backupDeployment(tempDir, deploymentName: "web");

[0147] / / Backup business statefulset

[0148] It should be noted that there are some syntax errors in the original code, such as incorrect capitalization in variable names and some missing semicolons. The above translation has been adjusted according to the correct grammar as much as possible while maintaining the original content.backupStatefulset(tempDirstatefulsetName: "portal");

[0149] backupStatefulset(tempDirstatefulsetName: "scheduler");

[0150] backupStatefulset(tempDir.statefulsetName: "nacos");

[0151] / / Backup mongo, postgresql, and influx data

[0152] dataBackupJob(backupItem, tempDir, tempDirName);

[0153] Step S32: After backing up the data of the above database types, namely Configmap, Deployment, and StatefulSet types in the Kubernetes cluster, compress them into a file and then transfer them to a specific object storage through the oss interface.

[0154] Step S40: Perform a restore operation on the backup file.

[0155] Step S41: Select the backup file to be restored.

[0156] Step S42: The backup restore service downloads the file from oss.

[0157] / / Temporary directory for storing backup files

[0158] File dir = new File(getRestorePath());

[0159] / / Download historical backup files by calling oss

[0160] try {

[0161] inputStream = getOssclient().readAsStream(ossBackupFileName);

[0162] outputStream = new FileOutputStream(localBackupFile);

[0163] IoUtil.copy(inputstream,outputstream):

[0164] }catch(Exception e){

[0165] }

[0166] / / File decompression

[0167] unzip(localBackupFile,getRestorePath());

[0168] Step S43, perform a restore operation on the backup file on the backup restore PVC.

[0169] Step S44, create a job through Kubernetes. There are multiple containers in the job, and different restore operations are performed in these different containers.

[0170] The backup file is a compressed package that contains data of types Configmap, Deployment, and StatefulSet related to the Kubernetes cluster, as well as data of types postgresql, MongoDB, and influx databases. After decompression, different restore logics are processed according to different types of files.

[0171] The most crucial part is that the service triggering the restore is on the original business service. The logic here is to only process the selected files to be restored, download the files to the backup restore pvc, and then create a job through Kubernetes. There are multiple containers in the job, and different restore operations are performed in these different containers.

[0172] Among them, the role of the "shutdown-restore" container is to stop all containers in the original business system that connect to the database.

[0173] / / Roll back data for pg, mongo, and influx. All containers connecting to the database need to be stopped first

[0174] stopDeployment(console);

[0175] stopDeployment(executor);

[0176] stopDeployment(gateway);

[0177] stopDeployment(monitor);

[0178] stopDeployment(ocr);

[0179] stopDeployment(web);

[0180] Statefulset portal = getstatefulSet(NAMESPACE, name: "portal");

[0181] StatefulSet scheduler = getstatefulSet(NAMESPACE, name: "scheduler");

[0182] StatefulSet nacos = getStatefulSet(NAMESPACE, name: "nacos");

[0183] stopstatefulset(portal);

[0184] stopstatefulset(scheduler);

[0185] stopStatefulSet(nacos);

[0186] The "postgresql-restore" container is used to restore the postgressql database.

[0187] The "mongo-restore" container is used to restore the MongoDB database.

[0188] The "influx-restore" container is used to restore the influx database.

[0189] The "restore-k8s-app" container is used to restore Configmap, Deployment, and StatefulSet in the Kubernetes cluster.

[0190] / / Update the configmap configuration

[0191] / / Apply the ConfiqMap to the Kubernetes cluster

[0192] k8SService.updateConfigMap(namespace, configMapName, configMap);

[0193] / / Update the deployment

[0194] restoreDeployment(files, deploymentName: "console");

[0195] restoreDeployment(files, deploymentName: "executor");

[0196] restoreDeployment(files, deploymentName: "gateway");

[0197] restoreDeployment(files, deploymentName: "monitor");

[0198] restoreDeployment(files, deploymentName: "ocr");

[0199] restoreDeployment(files, deploymentName: "web");

[0200] / / Update the statefulset

[0201] restorestatefulset(files, statefulsettName: "portal");

[0202] restorestatefulset(files, statefulsettName: "scheduler");

[0203] restorestatefulset(files., statefulsettName: "nacos");

[0204] }

[0205] In this embodiment, the operation results of the visual operation interface are as Figure 2 , Figure 3 and Figure 4 shown.

[0206] First, log in to the business system management console. When backing up, make selections in sequence on the operation interface:

[0207] Select System Management -> Select Backup and Restore -> Select Create Backup -> Select Backup Items -> Click Start Backup to complete the backup visualization operation.

[0208] During the restoration operation, make selections in sequence on the operation interface:

[0209] Select System Management -> Select Backup and Restoration -> Select Backup File -> Click Restore Immediately -> Click Start Restoration and Stop Service to complete the visual restoration operation.

[0210] In the backup and restoration processing method of the above business system in the Kubernetes cluster environment, first build a backup and restoration service based on Java, create a backup and restoration PVC, and integrate this service under the backup and restoration service and the business system database service. The backup and restoration service receives different backup parameters to achieve custom type and different database granularity backups, and transmits them to the configured oss for storage. The technical solution of the present invention can achieve custom type and different database granularity backups with different parameters, and can also be integrated into the code to form a visual operation interface, simplifying the backup and restoration operations. The technical solution of the present invention is simple, easy to implement, low in cost, and convenient for promotion.

[0211] It should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.< / configmap> < / string> < / string> < / string> < / string>

Claims

1. A backup and restoration processing method for a business system in a Kubernetes cluster environment, which is used for backing up and restoring data files with a data granularity of a custom type, and is characterized in that It includes the following specific steps: Build a backup and restore service based on Java and integrate the service into the Kubernetes api; Create a backup and restore pvc and mount it under the backup and restore service and the business system database service; The backup and restore service realizes custom type and different database granularity backups by receiving different backup parameters, and transfers the backup files to the configured oss for storage; Perform a restore operation on the backup files; Among them, the specific steps for integrating the service into the Kubernetes api include: Interact with the kubernetes cluster through the api, and at the same time configure the file storage oss to store backup files; Introduce Kubernetes dependencies in the original Java project pom file to implement the api interface for operating the kubernetes cluster, including query and modification operations on the main kubernetes components Service, Pod, Configmap, StatefulSet, Deployment, DaemonSet, Secret, Job; Introduce minio dependencies in the pom file to implement the operations of minio for uploading and downloading files; Among them, the specific steps for the backup and restore service to realize custom type and different database granularity backups by receiving different backup parameters and transfer the backup files to the configured oss for storage include: Execute public File backup(List <string>Use the backupItem, string path) throws IoException command to back up the file,< / string> Among them, List <string>backupItem is the passed-in parameter, and the optional parameters include "APPLICATION" and "MONITOR". "APPLICATION" represents business data including postgresql database and MongoDB database data, and "MONITOR" represents monitoring data, that is, influx database data; if only "APPLICATION" is passed in, only business data will be backed up. If both "APPLICATION" and "MONITOR" are passed in at the same time, both business data and monitoring data will be backed up;< / string> Backups on the Kubernetes cluster include three types: Configmap, Deployment, and StatefulSet; Pack and compress the data backups on the Kubernetes cluster into a file, and then transfer it to the specific object storage through the oss interface.

2. The backup and restoration processing method of the service system according to claim 1 in a Kubernetes cluster environment, characterized in that, The specific steps for creating the backup and restore pvc and mounting it under the backup and restore service and the business system database service include: Execute the sh-5.1# kubectl create -f clb-data-pvc.yaml command in the kubernetes environment to create the backup and restore pvc; The newly created database type container mounts the pvc under the database container through withVolumeMounts, and the backup service is built into the business system container.

3. The backup and restoration processing method of the service system according to claim 1 in a Kubernetes cluster environment, characterized in that The specific steps for restoring the backup file include: Select the backup file to be restored; The backup restoration service downloads the file from OSS; Perform the restoration operation on the backup file on the backup restoration PVC.

4. The backup and restoration processing method of the service system according to claim 3 in the Kubernetes cluster environment, characterized in that, The specific steps for performing the restoration operation on the backup file on the backup restoration PVC include: Create a job through Kubernetes. The job includes multiple containers, and different restoration operations are executed in these different containers; Among them, the role of the "shutdown-restore" container is to stop all containers in the original business system that connect to the database; The role of the "postgresql-restore" container is to restore the PostgreSQL database; The role of the "mongo-restore" container is to restore the MongoDB database; The role of the "influx-restore" container is to restore the Influx database; The role of the "restore-k8s-app" container is to restore Configmap, Deployment, and StatefulSet in terms of the Kubernetes cluster.

Images