Database gray release method and system based on proxy jump table

Abstract

The invention relates to the field of database management and software release, and discloses a database gray release method and system based on a proxy jump table. Based on a k8s side vehicle proxy mode and by using DBproxy software which is universal in the industry as database proxy software which is used for being compatible with new and old version access requests, the invention provides a new database gray release method. According to the invention, unified release task management is carried out by adopting an independent database grayscale release task management module. And in the gray release task, a management module is used to create and manage the side vehicle container. And completing initialization preparation work of the new-version database by utilizing the side vehicle container, copying persistent data resources of the old-version database instance, agency an access request of the new-version application and the old-version application, and redirecting the access request of the old-version application to the old-version database instance according to the redirection table. And finally, the database gray release management module issues an old database instance removal and destruction instruction to complete updating of a new version database instance.

Description

Technical Field

[0001] The present invention relates to the field of database management and software publishing, and in particular to a method and system for grayscale publishing of a database based on a proxy jump table. Background Art

[0002] k8s (Kubernetes) is an open source container cluster management system for automated deployment, expansion and management of containerized applications. It provides automated container orchestration, self-healing capabilities, horizontal expansion, service discovery and load balancing, which can help developers better manage containerized applications. Deploying applications in k8s makes the applications virtualized applications that can be dynamically started and destroyed. It no longer relies on physical hardware devices and can be scaled up and down in a timely manner according to business needs.

[0003] Grayscale release is a software version release strategy that allows a product or service to be open to a subset of users before it is officially released to all users, in order to test the stability and effectiveness of new features or services. This release method achieves a transition from quantitative change to qualitative change by gradually expanding the audience, starting with a small number of users, to a larger proportion of users, and finally covering all users. The advantage of grayscale release is that the new version can coexist with the old version, while observing the performance of the new version, so as to find problems and make adjustments in a timely manner.

[0004] Deploying database clusters in a K8s environment and performing grayscale releases of applications have become common solutions in the industry.

[0005] Usually, as a persistent data storage resource in the software architecture, the database instance is deployed separately to ensure data security, integrity, and recoverability, and the business accesses it uniformly through the database gateway. During the grayscale release of the business software version, there will be access requests for the old and new versions of the database at the same time. In the process of grayscale release, as the proportion of old version application copies decreases, the proportion of new version application copies increases, resulting in a smooth transition upgrade phenomenon.

[0006] In the traditional cluster deployment mode, operation and maintenance personnel use traffic management tools such as software and hardware L4 / L7 gateway agents to issue traffic forwarding policies to application gateways in advance after predicting the access IP or access region or other tags during the gray release of applications, and complete the cleanup and destruction of old version database resources after the gray release is completed. In addition, data migration and synchronization operations need to be completed in advance to ensure that the business data queried by the new and old applications during the gray release of applications are consistent. Figure 1 The database deployment method for the grayscale release of traditional application clusters.

[0007] In common practice, not only does it require the coordination of software and hardware, but it also requires the simultaneous deployment of two sets of software versions, which takes up double the computing resources. At the same time, operation and maintenance personnel manually configure business access policies, increasing application deployment costs.

[0008] As the company's business grows, it automatically completes the grayscale release adaptation of the database while maintaining a relatively loose coupling between the database and the application. It completes smooth grayscale upgrades and database data migration without business perception, and the database table structure upgrade has become a new requirement for product operation and maintenance deployment.

[0009] K8s sidecar mode is a service governance deployment mode in k8s. It separates the main application from the auxiliary tasks by decomposing the core functions into multiple containers, creates multiple containers in a k8s pod, and shares resources in the namespace such as network disk volumes. The sidecar mode is an important design pattern in cloud computing. The sidecar mode is similar to the gateway mode in design, but its granularity is finer. It equips each service with a "sidecar", which can be understood as an agent. All communications of this service are completed through this agent. This agent is created and destroyed together with the service. Functions such as service registration, service discovery, monitoring, traffic control, logging, service current limiting and service service fuse can be made into standardized components and modules. There is no need to realize their functions separately to consume the energy and time of business development to develop and debug these functions, so that truly high cohesion and low coupling software can be developed. Summary of the invention

[0010] The k8s sidecar mode can be used to complete the preliminary initialization preparation of container resources and to proxy the main application services. The sidecar mode proxy main application responds to access requests of multiple application versions and redirects to the old version database instance according to the intercepted access requests, becoming an ideal proxy access solution in the process of database grayscale release, and plays a vital role in adapting to multi-version application access requests.

[0011] In the process of selecting the database proxy program, the present invention selects the industry-wide DBproxy software as the database proxy software for compatibility with the access requests of the old and new versions. And the forwarding of the access requests of the new and old databases is completed by issuing a redirection table. At the same time, the progress statistics management of the release task can be completed until the entire grayscale release task is completed.

[0012] Therefore, the present invention proposes a new database grayscale release method based on the k8s sidecar proxy mode. This method uses an independent database grayscale release task management module to perform unified release task management. In the grayscale release task, the management module is used to create and manage the sidecar container. The sidecar container is used to complete the initialization preparation of the new version of the database, copy the persistent data resources of the old version of the database instance, proxy the access requests of the new and old versions of the application, and redirect the access requests of the old version of the application to the old version of the database instance according to the redirection table. Finally, the database grayscale release management module issues the removal and destruction instructions of the old database instance to complete the update of the new version of the database instance.

[0013] In a first aspect, the present invention provides a database upgrade method based on grayscale release, comprising the following steps: Step 1: Create a grayscale release task; Step 2: Import the old database instance migration SQL command and DBproxy proxy jump table required for this grayscale release task, and start the new database instance; Step 3: Migrate the old instance data to the new instance, including the persistent volume copy of the old instance and the migration SQL execution; Step 4: DBproxy parses the version information of the SQL request according to the proxy jump table and redirects it to the corresponding database instance; Step 5: Gradually remove old instances as the task progresses until the new instances are replaced; Step 6: When there are no old instances in the DBproxy proxy resource pool, the grayscale release task is completed.

[0014] Furthermore, the DBproxy proxy jump table imported in step 1 is a global data structure managed by the database grayscale release task, which is used to record the mapping relationship between the new and old SQL version information requested by the user and the new and old database instances.

[0015] Furthermore, the DBproxy proxy resource pool is a collection of old instance resources hosted by the database grayscale release task, which is used to store and dynamically update the list of database instances that can currently be used to process old SQL requests.

[0016] Furthermore, in step 3, the old instance data is migrated in the following manner: 1) Copy the persistent volume of the old instance to the new instance; 2) Execute the migration SQL command of the old instance in the new instance to complete the conversion of data structure and content.

[0017] Furthermore, the SQL request redirection method of DBproxy in step 4 includes: 1) Query the DBproxy proxy jump table to obtain the version information of the SQL request; 2) Find the corresponding database instance from the DBproxy proxy resource pool based on the version information; 3) Forward the SQL request to the found database instance to complete the access.

[0018] Furthermore, removing the old instance in step 5 includes: (1) Remove the old instance from the DBproxy proxy resource pool; (2) Delete the container or VM resources corresponding to the old instance to release computing and storage resources.

[0019] Furthermore, the persistent volume copy in step 3 is implemented through a container orchestration tool, preferably a Persistent Volume mechanism of Kubernetes, for seamlessly migrating old instance data to the new instance.

[0020] In a second aspect, the present invention provides a database grayscale publishing system based on a proxy jump table, comprising: (1) Database grayscale release task management module, which is used to uniformly manage the grayscale release process, including database instance creation, initialization, data migration, proxy resource pool maintenance, destruction of old database instances, and scheduling and status management of other related modules; (2) a DBproxy proxy module, which is scheduled by the database grayscale release task management module and is used to parse access requests according to the jump table and forward them to the corresponding new and old database instances; (3) a database instance management module, which is controlled by the database grayscale release task management module and is used to manage the dynamic allocation of database instances, resource optimization, and migration of persistent data; (4) A log recording and status monitoring module, which is called by the database grayscale release task management module and is used to record the log of the grayscale release process and monitor the execution status of the task in real time.

[0021] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the method as described above.

[0022] In a fourth aspect, the present invention provides an electronic device comprising one or more processors, one or more memories and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method as described above.

[0023] Advantages of the invention: (1) The database grayscale publishing system provided by the present invention can achieve: Compatible with different versions of database access requests; Decouple the application upgrade process from the database upgrade process; Introducing database proxy components to reduce the system's dependence on specific network hardware resources; The present invention can effectively avoid the resource overhead caused by the simultaneous deployment of two database systems by introducing the k8s sidecar proxy mode, and introduces a database grayscale release task management module to automatically manage and collect the grayscale release progress of database instances. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is the traditional cluster grayscale release database deployment method of the present invention.

[0025] Figure 2 It is a module diagram of the database grayscale release task management system of the present invention.

[0026] Figure 3 It is a flowchart of the database grayscale publishing method of the present invention.

[0027] Figure 4 This is the redirection flow chart of the database grayscale release system. DETAILED DESCRIPTION

[0028] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0029] The purpose of the present invention is to provide a method and system for grayscale release of a database based on a proxy jump table in a k8s environment. The release system includes the following components: Database grayscale release task management module SQL for migrating data from old database instances DBproxy proxy matching engine DBproxy proxy jump table DBproxy proxy resource pool Database grayscale release task management module: completes DBproxy proxy jump table initialization, DBproxy proxy resource pool initialization, DBproxy proxy matching engine deployment, database old instance data migration SQL deployment, database new instance creation, old instance removal, database old instance access migration proxy resource pool maintenance and other peripheral management tasks.

[0030] SQL statement for migrating old database instance data: This statement is used to complete the migration of historical data of old database instance. This statement is managed by the database grayscale release task management module and sent to the created new database instance for execution.

[0031] DBproxy proxy matching engine: completes the parsing of database access requests, queries the DBproxy proxy jump table, and queries the DBproxy proxy resource pool.

[0032] DBproxy proxy jump table: A global data structure initialized by the database grayscale release task management module, used to match SQL access requests of different versions.

[0033] DBproxy proxy resource pool: The global data structure is initialized by the database grayscale release task management module to record instance version information. When the instance is updated, the task management module updates the table entry.

[0034] The method for implementing the present invention comprises the following steps: Create a new database instance; The persistent volume copy of the old database instance; Execute SQL to migrate data from old database instances; Migrate access to old database instances; DBproxy accesses the proxy jump table to forward SQL; Remove the old database instance; The DBproxy access redirection method in step 5 is specifically as follows: Get DBproxy redirection policy; DBproxy checks the SQL statement content version according to the redirection strategy: ver1 or ver2; Query the DBproxy redirection table and select the candidate database instance number; DBproxy forwards database access SQL to the candidate database instance; The DBproxy redirection strategy obtained in step 1 is a global strategy managed by the database grayscale release task management module.

[0035] In step 2, DBproxy parses the current SQL access statement and matches the DBproxy redirection strategy to obtain the redirection result.

[0036] The candidate database instances in step 3 are selected in a round-robin manner.

[0037] The DBproxy in step 4 obtains the candidate database instance IP address through the candidate database instance ID and forwards the request.

[0038] Example 1 Figure 2 , this embodiment includes a k8s database multi-instance cluster and a database grayscale release system.

[0039] The k8s database multi-instance cluster uses the kubelet v1.28.2 system, and the database multi-instance image uses the docker.io / library / mysql:latest version image. The DBproxy module in the database grayscale release system uses the modified v0.20 version. The initialization process of the database grayscale release system is as follows Figure 4 First, import the database change SQL and the DBproxy redirection policy. After the configuration import is completed, the database grayscale release system creates a new database instance by calling kubectl.

[0040] The database grayscale release system provides DBproxyredirect policy table query and DBproxy redirect table query services to the DBproxy module by listening to the system TCP port 11089. When a new database query request enters a new database instance, the DBproxy module parses the SQL statement and executes the redirection strategy to check the SQL statement content version: ver1 or ver2.

[0041] The DBproxy module queries the first candidate database instance of ver1 or ver2 in the DBproxy redirection table in a round-robin manner and uses it as the target instance for subsequent SQL redirection.

[0042] Based on the same technical solution, the present invention also discloses a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, enable the computing device to execute the above-mentioned database grayscale publishing method and system based on the proxy jump table.

[0043] Based on the same technical solution, the present invention also discloses a computing device, including one or more processors, one or more memories and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include methods and systems for executing the above-mentioned database grayscale release method based on the proxy jump table.

[0044] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0045] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0046] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0047] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0048] Finally, it should be noted that the above implementation modes are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned implementation modes, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned implementation modes, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various implementation modes of the present application.

Claims

1. A database upgrade method based on grayscale release, characterized in that: The following steps are involved: (1.1) Create a grayscale release task; (1.2) Import the old database instance migration SQL commands and DBproxy proxy jump table required for the grayscale release task, and start the new database instance; (1.3) Migrate the old instance data to the new instance, including copying the old instance's persistent volume and executing the migration SQL; (1.4) DBproxy parses the version information of the SQL request according to the proxy jump table and redirects it to the corresponding database instance; (1.5) Gradually remove old instances as the task progresses until the replacement with new instances is completed; (1.6) When there are no old instances in the DBproxy proxy resource pool, the grayscale release task ends.

2. The method according to claim 1, characterized in that The DBproxy proxy jump table imported in the step (1.1) is a global data structure managed by the grayscale release task, which is used to record the mapping relationship between the new and old SQL version information requested by the user and the new and old database instances.

3. The method according to claim 1, characterized in that The DBproxy proxy resource pool is a collection of old instance resources hosted by the grayscale release task. It is used to store and dynamically update the list of database instances that can currently be used to process old SQL requests.

4. The method according to claim 1, characterized in that In step (1.3), the old instance data is migrated in the following way: (4.1) Copy the persistent volume of the old instance to the new instance; (4.2) Execute the migration SQL command of the old instance in the new instance to complete the conversion of data structure and content.

5. The method according to claim 1, characterized in that The SQL request redirection method of DBproxy in step (1.4) includes: (5.1) Query the DBproxy proxy jump table to obtain the version information of the SQL request; (5.2) Find the corresponding database instance from the DBproxy proxy resource pool based on the version information; (5.3) Forward the SQL request to the found database instance to complete the access.

6. The method according to claim 1, characterized in that Removing the old instance in the step (1.5) includes: (6.1) Remove the old instance from the DBproxy proxy resource pool; (6.2) Delete the container or virtual machine resources corresponding to the old instance to release computing and storage resources.

7. The method according to claim 1, characterized in that The persistent volume copy in step (1.3) is implemented through a container orchestration tool to seamlessly migrate the old instance data to the new instance.

8. A database grayscale publishing system based on a proxy jump table, characterized in that: include: The database grayscale release task management module is used to uniformly manage the grayscale release process, including database instance creation, initialization, data migration, proxy resource pool maintenance, destruction of old database instances, and scheduling and status management of other system modules; The DBproxy proxy module is scheduled by the database grayscale release task management module and is used to parse the access request according to the proxy jump table and forward it to the corresponding new and old database instances; A database instance management module, which is controlled by the database grayscale release task management module and is used to manage the dynamic allocation of database instances, resource optimization, and migration of persistent data; The log recording and status monitoring module is called by the database grayscale release task management module to record the log of the grayscale release process and monitor the execution status of the task in real time.

9. A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, characterized in that: When the instructions are executed by a computing device, the computing device is caused to perform the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The method comprises one or more processors, one or more memories and one or more programs, wherein the one or more programs are stored in the one or more memories and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method as claimed in any one of claims 1 to 7.

Images