Quick backup method for TB-level database based on B / S architecture

Abstract

The invention discloses a TB-level database rapid backup method based on a B / S architecture, and relates to the technical field of data processing. Comprising the following steps: S1, designing a B / S architecture; s2, a scheduling and backup strategy; s3, optimizing data transmission; s4, data segmentation; s5, performing parallel processing; s6, managing and optimizing a backup file; and S7, performing backup verification and recovery. A B / S framework is adopted, the core function of the database migration system is deployed on the server, a user conducts migration operation through a browser, the user can achieve remote migration without installing complex software locally, the backup efficiency of the TB-level database is improved, the backup time is shortened, the backup process is simplified, the operation complexity is reduced, and the backup efficiency of the TB-level database is improved. According to the method, the user experience is improved, the security and reliability of data management are enhanced, the comprehensive coverage and rapid recovery of data are ensured, the advantages of the B / S architecture are fully utilized, and the expandability and flexibility of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method for quickly backing up a TB-level database based on a B / S architecture. Background Art

[0002] With the advent of the big data era, the backup of TB-level databases has become an important challenge in data management. Traditional backup methods usually have problems such as long backup time, large resource consumption, and complex operations. The B / S architecture, with its distributed, scalable, and easy-to-maintain characteristics, provides a new solution for backing up large databases. However, there is currently a lack of a dedicated method for quickly backing up TB-level databases based on the B / S architecture in the market.

[0003] Therefore, proposing a method for quickly backing up a TB-level database based on a B / S architecture to solve the difficulties existing in the prior art is an urgent problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the present invention provides a method for quickly backing up a TB-level database based on a B / S architecture, aiming to solve the problems such as long backup time, large resource consumption, and complex operations existing in the existing TB-level database backup methods.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for quickly backing up a TB-level database based on a B / S architecture, comprising the following steps:

[0007] S1. B / S architecture design: Adopt the B / S architecture, deploy the core functions of the database migration system on the server, and the user performs the migration operation through the browser;

[0008] S2. Scheduling and backup strategy: Adjust the backup plan according to the usage situation and change frequency of the database;

[0009] S3. Data transmission optimization: Use a compression algorithm to compress the database data, so as to optimize the database data before data transmission;

[0010] S4. Data segmentation: According to the structure and size of the database, divide the data to be backed up into multiple small blocks, and each small block contains a certain number of data records or files;

[0011] S5. Parallel processing: On the backup server, allocate an independent backup task for each data block and execute the backup tasks in parallel;

[0012] S6. Backup file management and optimization: Design the naming rules and storage paths of backup files, perform backup file search and management, and regularly clean up old backup files;

[0013] S7. Backup verification and recovery: After the backup is completed, verify the integrity of the backup data and perform recovery.

[0014] In the above method, optionally, the compression algorithms in S3 include LZ4 and Zstd.

[0015] In the above method, optionally, in S4, according to the structure and size of the database, a hashing and data partitioning method is adopted to perform a hashing operation on the data to be backed up, and the data with the same or similar hash values is divided into the same subset, so as to divide the data to be backed up into multiple small pieces, and each small piece contains a certain number of data records or files.

[0016] In the above method, optionally, in S5, the backup tasks are executed in parallel through a multi-threaded or multi-process manner, and the execution status of each task is monitored to ensure load balancing and efficient utilization of resources among tasks.

[0017] In the above method, optionally, a task parallel model is adopted to execute the backup tasks. The tasks are assigned to multiple processors, and each processor independently executes the tasks. The task parallel model is expressed as:

[0018]

[0019] where g i [[ID=2,2]] is a function of the task processed by processor i, X i,m is the input of task m assigned to processor i, is a check indicating whether the processor has completed the assignment, and n is the number of processors.

[0020] In the above method, optionally, the verification process in S7 includes hash verification and data comparison.

[0021] In the above method, optionally, the recovery process in S7 includes partial recovery and full recovery.

[0022] From the above technical solutions, it can be seen that compared with the prior art, the present invention provides a method for quickly backing up a TB-level database based on the B / S architecture, and has the following beneficial effects:

[0023] (1) Adopting the B / S architecture, the core functions of the database migration system are deployed on the server, and users can perform migration operations through a browser. Users can achieve remote migration without installing complex software locally, which improves the flexibility and usability of the system;

[0024] (2) Combining the incremental backup technology and the parallel processing technology, only the data that has changed since the last backup is backed up, and the parallel processing of backup tasks is realized by using a multi-core processor and a distributed storage system, which greatly shortens the backup time and reduces resource consumption;

[0025] (3) The present invention improves the efficiency of TB-level database backup, shortens the backup time, reduces resource consumption, simplifies the backup process, reduces the operation complexity, improves the user experience, enhances the security and reliability of data management, ensures the comprehensive coverage and rapid recovery of data, makes full use of the advantages of the B / S architecture, and improves the scalability and flexibility of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to the provided drawings.

[0027] Figure 1 It is a flowchart of a method for quickly backing up a TB-level database based on the B / S architecture provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0029] Refer to Figure 1 As shown, the present invention discloses a method for quickly backing up a TB-level database based on the B / S architecture, including the following steps:

[0030] S1. B / S architecture design: Adopt the B / S architecture, deploy the core functions of the database migration system on the server, and the user performs the migration operation through the browser;

[0031] S2. Scheduling and backup strategy: Adjust the backup plan according to the usage situation and change frequency of the database;

[0032] S3. Data transmission optimization: Use a compression algorithm to compress the database data, so as to optimize the database data before data transmission;

[0033] S4. Data segmentation: According to the structure and size of the database, divide the data to be backed up into multiple small pieces, and each small piece contains a certain number of data records or files;

[0034] S5. Parallel Processing: On the backup server, assign an independent backup task to each data block and execute the backup tasks in parallel;

[0035] S6. Backup File Management and Optimization: Design backup file naming rules and storage paths, perform backup file search and management, and regularly clean up old backup files;

[0036] S7. Backup Verification and Recovery: After the backup is completed, perform integrity verification and recovery on the backup data.

[0037] Furthermore, the compression algorithms in S3 include LZ4 and Zstd.

[0038] Furthermore, in S4, according to the structure and size of the database, a hashing and data splitting method is adopted. Hash operations are performed on the data to be backed up, and data with the same or similar hash values are divided into the same subset, thereby splitting the data to be backed up into multiple small blocks, and each small block contains a certain number of data records or files.

[0039] Furthermore, in S5, the backup tasks are executed in parallel through multi-threading or multi-processing, and the execution status of each task is monitored to ensure load balancing and efficient utilization of resources among tasks.

[0040] Furthermore, a task parallel model is adopted to execute the backup tasks. The tasks are assigned to multiple processors, and each processor independently executes the tasks. The task parallel model is expressed as:

[0041]

[0042] where g i is the function of the task processed by processor i, X i,m is the input of task m assigned to processor i, is the check indicating whether the processor has completed the assignment, and n is the number of processors.

[0043] Furthermore, the verification process in S7 includes hash verification and data comparison.

[0044] Furthermore, the recovery process in S7 includes partial recovery and full recovery.

[0045] In a specific embodiment, it includes the following content

[0046] S1. B / S Architecture Design: Adopt the B / S architecture, deploy the core functions of the database migration system on the server, and users perform migration operations through the browser. In this way, users can achieve remote migration without installing complex software locally, improving the flexibility and usability of the system.

[0047] S2. Scheduling and Backup Strategies: Design a scheduling control module to dynamically adjust the backup plan according to the usage and change frequency of the database. Meanwhile, formulate a reasonable backup strategy, such as combining regular full backups with daily incremental backups, to ensure comprehensive data coverage and rapid recovery.

[0048] S3. Data Transmission Optimization: Before data transmission, efficiently compress the database data to reduce the network transmission volume and lower the transmission latency. Adopt compression algorithms suitable for large amounts of data, such as LZ4, Zstd, etc., to ensure a balance between compression speed and compression ratio.

[0049] S4. Data Segmentation: According to the structure and size of the database, divide the data to be backed up into multiple small chunks, each of which contains a certain number of data records or files. The segmentation strategy can be adjusted according to actual needs to balance the efficiency of parallel processing and resource consumption.

[0050] S5. Parallel Processing: On the backup server, assign an independent backup task to each data block and execute these tasks in parallel through multi-threading or multi-processing. Meanwhile, monitor the execution status of each task to ensure load balancing and effective utilization of resources among tasks.

[0051] S6. Backup File Management and Optimization: Design a reasonable naming rule and storage path for backup files to facilitate the search and management of backup files. Meanwhile, regularly clean up old backup files to free up storage space.

[0052] S7. Backup Verification and Recovery: After the backup is completed, verify the integrity of the backup data to ensure the accuracy and availability of the backup data. The verification process can include various methods such as hash verification and data comparison. Provide an efficient recovery mechanism that can quickly recover data in case of data loss or system failure. The recovery process can support partial recovery or full recovery to meet the requirements in different scenarios.

[0053] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and reference can be made to the description in the method part for related parts.

[0054] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A TB-level database fast backup method based on the B / S architecture, characterized in that, It includes the following steps: S1. B / S architecture design: Adopt the B / S architecture, deploy the core functions of the database migration system on the server, and users perform migration operations through the browser; S2. Scheduling and backup strategy: Adjust the backup plan according to the usage and change frequency of the database; S3. Data transmission optimization: Use compression algorithms to compress the database data, so as to optimize the database data before data transmission; S4. Data segmentation: According to the structure and size of the database, divide the data to be backed up into multiple small pieces, and each small piece contains a certain number of data records or files; S5. Parallel processing: On the backup server, assign an independent backup task to each data block and execute the backup tasks in parallel; S6. Backup file management and optimization: Design backup file naming rules and storage paths, search for and manage backup files, and regularly clean up old backup files; S7. Backup verification and recovery: After the backup is completed, perform integrity verification and recovery on the backup data.

2. A TB-level database fast backup method based on the B / S architecture according to claim 1, characterized in that the compression algorithms in S3 include LZ4 and Zstd.

3. A TB-level database fast backup method based on the B / S architecture according to claim 1, characterized in that in S4, according to the structure and size of the database, adopt a hashing and data segmentation method, perform a hashing operation on the data to be backed up, and divide the data with the same or similar hash values into the same subset, so as to divide the data to be backed up into multiple small pieces, and each small piece contains a certain number of data records or files.

4. A TB-level database fast backup method based on the B / S architecture according to claim 1, characterized in that in S5, execute the backup tasks in parallel through multi-threading or multi-processing, and monitor the execution status of each task to ensure load balancing and effective utilization of resources between tasks.

5. A TB-level database fast backup method based on the B / S architecture according to claim 4, characterized in that adopt a task parallel model to execute the backup tasks, the tasks are assigned to multiple processors, and each processor independently executes the tasks. The task parallel model is expressed as: Among them, g i is a function of the task processed by processor i, X i,m is the input of task m assigned to processor i, is a check indicating whether the processor has completed the assignment, and n is the number of processors.

6. A TB-level database fast backup method based on the B / S architecture according to claim 1, characterized in that the verification process in S7 includes hash verification and data comparison.

7. A TB-level database fast backup method based on the B / S architecture according to claim 1, characterized in that the recovery process in S7 includes partial recovery and full recovery.

Images