A Database Proxy Implementation Method Based on a Responsive Thread Pool

Abstract

The present invention provides a method for implementing a database proxy based on a reactive thread pool, which relates to the technical field of database proxies and includes: starting the proxy software to obtain a list of databases to be proxied; adding the databases in the database list to the proxy software to obtain a database distribution list; the proxy software dynamically creates a reactive thread pool for each database in the database distribution list respectively; the proxy software receives and parses the instructions of the client through the first reactive thread pool to obtain the corresponding database in the instruction content; the proxy software forwards the instructions to the database through the second reactive thread pool corresponding to the database. The present invention directly passes the right of data routing from the proxy software to the client application itself. When the proxy software is unavailable, the application can directly connect to the database to complete the service, realizing a high-performance and lightweight data proxy; and realizes the hot update of the proxy database during the low-traffic period by means of reactive programming.

Description

Technical Field

[0001] The present invention relates to the technical field of database proxy, and in particular to a method for implementing a database proxy based on a reactive thread pool. Background Art

[0002] When an enterprise conducts daily production and operation activities, a series of application programs will be developed to provide services for users; the basic enterprise software architecture is that multiple server programs initiate database connections to perform data addition, deletion, modification, and query; as the business of the enterprise server develops, the number of programs to be deployed gradually increases; the read and write pressure on the database becomes larger. For common databases, capacity expansion or modification of the deployment plan is carried out, changing to a master-slave mode or a read-write separation mode. After the database deployment plan is modified, a large number of modifications need to be made on the application software side to adapt to the new database deployment plan. Therefore, a variety of database proxy software has emerged.

[0003] Most of the existing proxy software implements a specific version of the mysql protocol. When the mysql versions are inconsistent, proxying cannot be carried out.

[0004] In addition, the implementation of mysql high availability by the existing database proxy software has a certain conflict with the high availability solution of the database itself. Once the proxy software becomes unavailable, the application program cannot perform degradation processing, thus causing more problems. Summary of the Invention

[0005] In view of the above problems, the present invention provides a method for implementing a database proxy based on a reactive thread pool, enabling the proxy software to focus on data proxy, directly passing the right of data routing from the proxy software to the application program itself, realizing high-performance and lightweight data proxy. When the proxy software is unavailable, the application program can directly connect to the database to complete the service.

[0006] To achieve the above object, the present invention provides a method for implementing a database proxy based on a reactive thread pool, including:

[0007] The proxy software starts and obtains the list of databases to be proxied;

[0008] Add the databases in the database list to the proxy software to obtain a database distribution list;

[0009] The proxy software dynamically creates a reactive thread pool for each database in the database distribution list respectively;

[0010] The proxy software receives and parses the client's instructions through the first reactive thread pool in a multi-threaded manner, and obtains the corresponding database in the instruction content;

[0011] The proxy software forwards the instruction to the database through the second reactive thread pool corresponding to the database.

[0012] As a further improvement of the present invention, parsing the instruction of the parsing client includes:

[0013] Performing frame parsing on the instruction to determine whether the instruction statement includes various operations of a transaction;

[0014] If so, splitting the transaction operations, and for each operation, obtaining the database corresponding to the instruction according to the node name declared in the instruction statement.

[0015] As a further improvement of the present invention, the proxy software is provided with a database list acquisition interface, and the database list to be proxied is acquired through the database list acquisition interface to obtain a JSON-formatted database list.

[0016] As a further improvement of the present invention, the proxy software is provided with a database addition interface, and each database in the database list is added to the proxy software through the database addition interface;

[0017] The parameters passed by each database to the proxy software include: the node name, routing, and read / write attributes of the database.

[0018] As a further improvement of the present invention, the second thread pool forwards the instruction to the corresponding database according to the routing of the corresponding database.

[0019] As a further improvement of the present invention, the proxy software deletes the added database through the database deletion interface.

[0020] As a further improvement, the node name of the database to be deleted is passed to the database deletion interface;

[0021] The proxy software removes the node name from the database dispatch list, and waits until the task queue in the second reactive thread pool corresponding to the node name is empty and all execution threads are in the sleep state, and then performs a null value processing on the reactive thread pool.

[0022] As a further improvement of the present invention, the proxy software has a database modification interface;

[0023] The parameters to be modified, including the node name, routing, and read / write attributes of the database, are passed to the database modification interface. The proxy software compares the passed parameters to be modified with the original parameters of the corresponding database. If they are the same, no modification is made.

[0024] As a further improvement of the present invention, the proxy software compares the incoming parameter to be modified with the original parameter in the corresponding database. If they are different, the proxy software pauses the task queue dispatching in the second reactive thread pool, waits until all execution threads are in the sleep state, and then destroys all the execution threads;

[0025] Modify the parameters of the corresponding database according to the parameter to be modified, recreate the execution threads and redispatch the task queue.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0027] The present invention directly passes the right of data routing from the proxy software to the client application itself, realizes high-performance lightweight data proxy by means of reactive programming, and when the proxy software is unavailable, the application can directly connect to the database to complete the service.

[0028] In view of the fact that the current open-source proxy software does not implement a many-to-many database connection pool and the online dynamic configuration of the proxy software affects the use of the proxy software when configuring database information, the present invention makes use of the flexible design model of reactive programming to realize the hot update of the proxy database during the low-traffic period, including the addition, deletion, and modification of the database. After the online configuration is completed, it will take effect immediately.

[0029] The application program connection database proxy software of the present invention is multi-threaded, and the proxy software uses a reactive thread pool for docking; the connection from the proxy software to the database is also multi-threaded, and a reactive thread pool will be dynamically created for each proxy database, realizing the high availability of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flowchart of a method for implementing a database proxy based on a reactive thread pool disclosed in an embodiment of the present invention;

[0031] Figure 2 It is a schematic diagram of the processing process of a database proxy software based on a reactive thread pool disclosed in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. The steps S1, S2, etc. in the embodiments are only a feasible solution and do not limit the specific steps. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0033] The following further describes the present invention in detail with reference to the accompanying drawings:

[0034] As Figure 1 shown, a method for implementing a database proxy based on a reactive thread pool provided by the present invention includes:

[0035] S1. The proxy software starts and obtains the list of databases to be proxied;

[0036] Among them,

[0037] After the proxy software starts, it will default to start the http service and open the corresponding api (interface) for dynamically managing the proxy databases, including:

[0038] (1) Database list acquisition interface ( http: / / {ip}:30010 / databaselist ), obtain the list of databases to be proxied through the database list acquisition interface, and obtain a JSON-formatted database list.

[0039] (2) Database addition interface ( http: / / {ip}:30010 / adddatabase ), see step S2 for details;

[0040] (3) Database deletion interface ( http: / / {ip}:30010 / delDatabase ), pass the node name of the database to be deleted to the database deletion interface;

[0041] The proxy software removes the node name from the database dispatch list, and waits until the task queue in the second reactive thread pool corresponding to the node name is empty and all execution threads are in the sleep state, and then performs a null value processing on the reactive thread pool.

[0042] (4) Database modification interface ( http: / / {ip}:30010 / modDatabase ); Pass the parameters to be modified to the database modification interface, including the node name, routing, and read / write attributes of the database. The proxy software compares the parameters to be modified passed in with the original parameters of the corresponding database. If they are the same, no modification is made.

[0043] If they are different, the proxy software pauses the dispatch of the task queue in the second reactive thread pool, waits until all execution threads are in the sleep state, and then destroys all execution threads;

[0044] Modify the parameters of the corresponding database according to the parameters to be modified, recreate the execution threads and redispatch the task queue.

[0045] Furthermore,

[0046] The database addition interface, the database deletion interface, and the database modification interface can be called in real time during the running process after the proxy software starts, dynamically add, delete, and modify the database information to be proxied by the proxy software, realize online configuration, and the configuration takes effect immediately after completion.

[0047] S2. Add the databases in the database list to the proxy software to obtain a database distribution list;

[0048] Among them,

[0049] Add each database in the database list to the proxy software through a database addition interface to obtain a database distribution list;

[0050] The parameters passed by each database to the proxy software include: the node name, routing, and read / write attributes of the database, in the format of:

[0051] {

[0052] Dbname:main

[0053] Mysqlurl:

[0054] "jdbc:mysql: / / 127.0.0.1:3306 / seckill?useUnicode=true&characterEncoding=utf8&autoReconnect=true&rewriteBatchedStatements=TRUE"

[0055] Role:W / R / WR

[0056] }

[0057] Dbname is the basic name of the SQL routing, which is globally unique. If the dbnames of two databases are the same, they cannot be added;

[0058] Mysqlurl is the url for database connection, and the proxy software connects to the proxy database through this string.

[0059] Role is the role in the database, specifically the read / write attribute, where W represents "write only", R represents "read only", and WR represents "both read and write".

[0060] S3. The proxy software dynamically creates a reactive thread pool for each database in the database distribution list respectively;

[0061] Among them,

[0062] The connection from the proxy software to each database is multi-threaded, and a reactive thread pool will be dynamically created for each proxy database.

[0063] S4. The proxy software receives and parses the client's instructions through the first reactive thread pool in multiple threads, and obtains the corresponding database in the instruction content;

[0064] Among them,

[0065] The client application connection proxy software is multi-threaded, and the proxy software docks through the first reactive thread pool.

[0066] Furthermore,

[0067] Perform frame parsing on the instruction to determine whether the instruction statement includes various operations of the transaction;

[0068] If so, split the transaction operations, and for each operation, obtain the corresponding database of the instruction according to the node name declared in the instruction statement.

[0069] Specifically,

[0070] Based on the TCP protocol, perform "frame" parsing on the SQL instruction to analyze whether the parsing statement includes operations such as the start, commit, and rollback of the transaction.

[0071] SQL forwarding, forward to the corresponding database according to the declaration in the SQL statement, and if there is no declaration, forward to the default database. The form of this declaration is to fill in the corresponding node name in the comment of the SQL statement.

[0072] Based on the technical solution of TCP frame parsing, there is no need to participate in the version matching process between the client and the server. It only requires that the client version of the application can match the server.

[0073] S5. The proxy software forwards the instruction to the database through the second reactive thread pool corresponding to the database.

[0074] Among them,

[0075] The database receives and executes the client's instruction.

[0076] Advantages of the present invention:

[0077] (1) Transmit the right of data routing directly from the proxy software to the client application itself, achieve high-performance lightweight data proxy with reactive programming, and when the proxy software is unavailable, the application can directly connect to the database to complete the service.

[0078] (2) Aiming at the current open-source proxy software that does not implement a many-to-many database connection pool and the situation that the use of the proxy software is affected when configuring database information for the proxy software, with the flexible design model of reactive programming, realize the hot update of the proxy database during the low-traffic period, including the addition, deletion, and modification of the database. After the online configuration is completed, it will take effect immediately.

[0079] (3) The application program connection database proxy software of the present invention is multi-threaded, and the proxy software uses a reactive thread pool for docking; the connection from the proxy software to the database is also multi-threaded, and a reactive thread pool will be dynamically created for each proxy database, achieving high availability of the database.

[0080] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for implementing a database proxy based on a reactive thread pool, characterized in that Including: The proxy software starts and obtains a list of databases to be proxied; Add the databases in the database list to the proxy software to obtain a database distribution list; The proxy software dynamically creates a reactive thread pool for each database in the database distribution list; The proxy software receives and parses the client's instructions through the first reactive thread pool to obtain the corresponding database in the instruction content; Perform frame parsing on the instruction to determine whether the instruction statement includes multiple operations of a transaction; if so, split the transaction operations, and for each operation, obtain the database corresponding to the instruction according to the node name declared in the instruction statement; Based on the TCP protocol, perform frame parsing on the SQL instruction to parse whether the statement contains operations for starting, committing, and rolling back a transaction; SQL forwarding, forward to the corresponding database according to the declaration in the SQL statement, and if there is no declaration, forward to the default database. The form of this declaration is to fill in the corresponding node name of the node name in the comment of the SQL statement; The proxy software forwards the instruction to the corresponding database through the second reactive thread pool; The proxy software deletes the added database through the database deletion interface; Pass the node name of the database to be deleted to the database deletion interface; The proxy software removes the node name from the database distribution list, and waits until the task queue in the second reactive thread pool corresponding to the node name is empty and all execution threads are in the sleep state, and then performs nullification processing on the reactive thread pool; Pass the parameters to be modified to the database modification interface, including the node name, routing, and read / write attributes of the database. The proxy software compares the parameters to be modified passed in with the original parameters of the corresponding database; If they are the same, no modification is made; If they are different, the proxy software suspends the dispatching of the task queue in the second reactive thread pool, waits until all execution threads are in the sleep state, destroys all the execution threads; modifies the parameters of the corresponding database according to the parameters to be modified, re-creates the execution threads and re-dispatches the task queue; The proxy software is provided with a database list acquisition interface, and obtains a list of databases to be proxied through the database list acquisition interface to obtain a database list in JSON format.

2. The database proxy implementation method according to claim 1, wherein: The proxy software is provided with a database addition interface, and through the database addition interface, adds each database in the database list to the proxy software; Each database passes parameters to the proxy software, including: the node name, routing, and read / write attributes of the database.

3. The database proxy implementation method according to claim 2, wherein: The second reactive thread pool forwards the instruction to the corresponding database according to the routing of the corresponding database.

Images