Database switching method and device based on data middle layer

Abstract

The present invention provides a database switching method and device based on a data middle layer, and relates to the field of computer technology. The present invention is applied to a data middle layer adopting an ORM model, wherein the data middle layer communicates with a business system through a unified API, and manages connections with multiple databases through a connection pool; the data middle layer connects to the business system through a unified API, and connects to multiple databases through a connection pool, so that by constructing the data middle layer, the decoupling of the business and the database is achieved, and the flexibility and maintainability of database adaptation are improved; at the same time, the data middle layer realizes the expansion of a target database for executing a query operation through a target special function, and realizes the switching of the target database by reconstructing a target query statement, which avoids the failure of a single database query to a certain extent and improves the query efficiency.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a database switching method and device based on a data middle layer. Background Art

[0002] Different types of databases have different query syntaxes. When switching to a new database in a project developed by a business system, backend developers need to learn the new database syntax and adjust and maintain the backend code. This process increases the difficulty of project development and maintenance costs. Moreover, as the project progresses, if a new database needs to be added, it will become difficult to maintain the adaptation of multiple different types of databases. Summary of the invention

[0003] The purpose of the present invention is to provide a database switching method and device based on a data middle layer to achieve decoupling of business and database and improve the flexibility and maintainability of database adaptation.

[0004] In a first aspect, an embodiment of the present invention provides a database switching method based on a data middle layer, which is applied to a data middle layer using an object-relational mapping (ORM) model, wherein the data middle layer communicates with a business system through a unified API, and the data middle layer manages connections with multiple databases through a connection pool; the method comprises:

[0005] Receiving an original query request sent by a target business system through a unified API, the original query request including an original query statement and a target database identifier;

[0006] Parse the original query statement through the ORM model to obtain the original query information, which includes the original query content and the target special function;

[0007] Determine a target database from at least one candidate database; wherein the at least one candidate database includes a first database corresponding to the target database identifier and a second database adapted to the target special function;

[0008] Based on the original query content and the target special function, a target query statement corresponding to the target database is constructed;

[0009] Perform data query of the target query statement in the target database through the connection pool to obtain the query result;

[0010] When the query result indicates that the query fails, re-execute the step of determining a target database from at least one candidate database to switch the target database;

[0011] When the query result indicates that the query is successful, the query result is returned to the target business system.

[0012] Further, determining a target database from at least one candidate database includes:

[0013] Determine at least one candidate database based on a first database corresponding to the target database identifier and a second database adapted to the target special function;

[0014] Prioritize each candidate database to obtain a ranking result; wherein, when the first database is one of the second databases, the priority of the first database is higher than other databases in the second database;

[0015] Based on the ranking results, a target database is selected from each candidate database.

[0016] Further, based on the first database corresponding to the target database identifier and the second database adapted to the target special function, at least one candidate database is determined, including:

[0017] determining whether the first database is one of the second databases;

[0018] If not, the second database is determined as a candidate database.

[0019] Furthermore, the original query content includes query keywords, query fields and query conditions; based on the original query content and the target special function, a target query statement corresponding to the target database is constructed, including:

[0020] Based on the keyword library corresponding to the target database, keyword conversion is performed on the query keyword to obtain the target keyword;

[0021] Based on the target keywords, query fields, query conditions, target special functions and grammatical rules corresponding to the target database, a target query statement is constructed.

[0022] Further, when the query result indicates that the query fails, the step of determining a target database from at least one candidate database is re-executed, including:

[0023] When the query result indicates that the query fails, marking the target database corresponding to the query result as having executed the query operation, and determining whether there is an optional database in at least one candidate database that has not executed the query operation;

[0024] If there is an optional database, at least one candidate database is updated as an optional database, and the step of determining the target database from the at least one candidate database is re-executed.

[0025] Furthermore, the above method also includes:

[0026] If no optional database exists, a query failure is returned to the target business system.

[0027] Furthermore, the multiple databases include Greenplum, MySQL, PostgreSQL, Doris, Elasticsearch, Oracle, ClickHouse, DAMO, MongoDB, Hive, Hbase, SQL Server and neo4j.

[0028] In a second aspect, an embodiment of the present invention further provides a database switching device based on a data middle layer, which is applied to a data middle layer using an ORM model, wherein the data middle layer communicates with a business system through a unified API, and the data middle layer manages connections with multiple databases through a connection pool; the device includes:

[0029] A receiving module, used to receive an original query request sent by a target business system through a unified API, wherein the original query request includes an original query statement and a target database identifier;

[0030] The parsing module is used to parse the original query statement through the ORM model to obtain the original query information, which includes the original query content and the target special function;

[0031] A determination module, used to determine a target database from at least one candidate database; wherein the at least one candidate database includes a first database corresponding to the target database identifier and a second database adapted to the target special function;

[0032] A construction module, used to construct a target query statement corresponding to a target database based on the original query content and the target special function;

[0033] A query module, used to perform data query of a target query statement in a target database through a connection pool to obtain query results;

[0034] The determination module is further configured to, when the query result indicates that the query fails, re-execute the step of determining a target database from at least one candidate database to switch the target database;

[0035] The return module is used to return the query result to the target business system when the query result indicates that the query is successful.

[0036] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the database switching method based on the data intermediate layer of the first aspect is implemented.

[0037] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the database switching method based on the data intermediate layer of the first aspect is executed.

[0038] The database switching method and device based on the data middle layer provided in the embodiment of the present invention are applied to the data middle layer adopting the ORM model, the data middle layer communicates with the business system through a unified API, and the data middle layer manages the connection with multiple databases through a connection pool; the method comprises: receiving the original query request sent by the target business system through the unified API, the original query request comprises the original query statement and the target database identifier; parsing the original query statement through the ORM model to obtain the original query information, the original query information comprises the original query content and the target special function; determining the target database from at least one candidate database; wherein at least one candidate database comprises a first database corresponding to the target database identifier and a second database adapted to the target special function; constructing a target query statement corresponding to the target database based on the original query content and the target special function; performing data query of the target query statement in the target database through the connection pool to obtain the query result; when the query result indicates that the query fails, re-executing the step of determining the target database from at least one candidate database to switch the target database; when the query result indicates that the query is successful, returning the query result to the target business system. The data middle layer connects to the business system through a unified API and connects to multiple databases through a connection pool. By building this data middle layer, the decoupling of business and database is achieved, and the flexibility and maintainability of database adaptation are improved. At the same time, the data middle layer realizes the expansion of the target database for executing query operations through target special functions, and realizes the switching of target databases by reconstructing target query statements, which avoids the failure of single database queries to a certain extent and improves query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A schematic diagram of the system architecture of a database management system provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of a database management system provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of a flow chart of a database switching method based on a data middle layer provided in an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of the structure of a database switching device based on a data middle layer provided by an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] With the storage of multi-dimensional data, the types of databases in database management systems are increasing. However, the business and the database are coupled, and the switching of the database depends on the maintenance of the back-end developers, resulting in poor flexibility and maintainability of the project. Based on this, an embodiment of the present invention provides a database switching method and device based on a data middle layer. By building a unified data middle layer, the flexibility and maintainability of the project can be improved, and the development can be more focused on the implementation of business logic, rather than paying too much attention to the differences in the underlying database. The method can be applied to database management systems.

[0047] like Figure 1 As shown, a database management system provided by an embodiment of the present invention adopts a three-layer structure model consisting of a service layer, a data middle layer and a data source layer, wherein the service layer is provided with business code for processing the business logic of the application; the data middle layer adopts the ORM (Object Relational Mapping) model, and manages the database connection with multiple databases in the data source layer through a connection pool, that is, the data middle layer manages the connection with multiple databases in the data source layer through a connection pool; the data source layer can include multiple types of databases, such as Greenplum, MySQL, PostgreSQL, Doris, Elasticsearch, Oracle, ClickHouse, DAMO, MongoDB, Hive, Hbase, SQLServer and neo4j, etc. Figure 2 As shown, the data middle layer communicates with the business system of the application layer through a unified API, and the data middle layer communicates with multiple databases through corresponding APIs.

[0048] The above database management system can realize data query through the interface of outer layer encapsulation data (i.e. unified API) and decoupling of business and database. The project and bottom layer (i.e. data source layer) data can be managed separately. The bottom layer data remains unchanged. When the project business changes, the interface with the database remains unchanged relative to the project business; the data middle layer remains unchanged, and the upper and lower layers (i.e. service layer and data source layer) change separately without affecting each other. The above database management system can realize data parsing and query statement rewriting, that is, converting data query statements into query statements of different database standards. The standard MySQL statements can be converted into executable statements of various databases through SQL parser and SQL executor. The execution subject (i.e. target database) and query conditions can be extracted, and data analysis can be performed based on this, and then the query statements can be converted. A set of persistence layers compatible with multiple databases can be written to solve the differences in SQL language and SQL standards of various databases, thereby reducing the learning cost of back-end developers, code adjustment and maintenance costs.

[0049] The framework of the data middle layer can use the ORM model, such as Hibernate or MyBatis, to simplify the operation of multiple databases. These frameworks provide an abstraction layer that can abstract the differences between different databases and provide a unified API to the upper-level business system, so that developers can focus on business logic instead of dealing directly with multiple databases. The ORM model provides object-relational mapping, in which the classes in the ORM correspond to the data tables in the database, the objects in the ORM correspond to the data rows in the database, and the attributes in the ORM correspond to the fields in the database. The ORM model has the following advantages:

[0050] 1. Simplify operations: The ORM model provides an abstraction layer that allows developers to operate the database in an object-oriented manner without having to write cumbersome SQL statements. This greatly simplifies database operations and improves development efficiency;

[0051] 2. Improve efficiency: Due to the automation and abstraction of the ORM model, developers can quickly build applications and reduce the workload of writing repetitive code. At the same time, the ORM framework also provides rich query and operation functions, allowing developers to write business logic more efficiently;

[0052] 3. Reduced Dependency: The ORM model allows developers to focus more on the implementation of business logic rather than the specific operations of the database. This helps reduce developers' dependence on specific databases and makes applications more portable and scalable.

[0053] You can choose Hibernate as the ORM model. Hibernate has good scalability, comes with a log system, and can report errors and prompts when dirty data appears. Hibernate's mapping mechanism is to associate Java objects with databases. You no longer need to pay attention to SQL generation and result mapping, so you can focus more on business and better meet project requirements.

[0054] The database connection is managed through the connection pool of the data middle layer: when the system is initialized, the database connection is stored as an object in the memory. When the user needs to access the database, an established idle connection object is taken from the connection pool; after use, the connection is returned to the connection pool for the next request to access. Optionally, the data middle layer can organize the connections generated by DriverManager into a connection pool, and then provide connections from the connection pool; DriverManager is a class used to manage database drivers.

[0055] In this way, a consistent data query platform is established (i.e., a unified API for the data middle layer is built to handle the differences between different databases), and the database and business are decoupled. When the underlying database changes, the upper-level business system does not need to modify the query logic; the database relationship is hidden from the outside, making the transition from SQL query to object query more convenient; it helps to simplify the development process, reduce the pressure on developers, and improve the independence of business logic and data layer, bringing greater flexibility and maintainability to the project, making it smoother to adapt to new databases and expand business. The above database management system can be used for, but not limited to, the collection and analysis of social data.

[0056] To facilitate understanding of this embodiment, a database switching method based on a data middle layer disclosed in an embodiment of the present invention is described in detail below.

[0057] The embodiment of the present invention provides a database switching method based on a data middle layer. The method is applied to a data middle layer using an ORM model. The data middle layer communicates with a business system through a unified API. The data middle layer manages connections with multiple databases through a connection pool. The multiple databases may include Greenplum, MySQL, PostgreSQL, Doris, Elasticsearch, Oracle, ClickHouse, DAMO, MongoDB, Hive, Hbase, SQL Server, and neo4j. Figure 3 The flowchart of a database switching method based on the data middle layer is shown, and the method mainly includes the following steps S310 to S370:

[0058] Step S310: receiving an original query request sent by a target business system through a unified API, where the original query request includes an original query statement and a target database identifier.

[0059] The target business system can send an original query request to the data middle layer through a unified API. The original query statement in the original query request can be a preset general query statement, such as a MySQL statement. It should be noted that the general query statement can be set according to actual needs, and this embodiment does not limit the general query statement. The target database identifier in the original query request is used to uniquely identify the database to be queried.

[0060] Step S320, parsing the original query statement through the ORM model to obtain original query information, the original query information includes the original query content and the target special function.

[0061] The original query content may include query keywords, query fields, query conditions, etc. The target special function refers to the special function in the original query statement. The special function is a method customized by the database. Different types of databases may have different special functions. For example, MySQL's special functions may include concat() function, substring() function, ifnull() function, group_concat() function, and date_format() function, etc. Oracle's special functions may include to_char() function, to_number() function, and to_date() function, etc.

[0062] For example, an example of an original query statement is as follows:

[0063] select task_id as text_id,

[0064] get_json_string(message,'$.id') as post_id,

[0065] get_json_string(message,'$.pk') as post_pk,

[0066] import_time as created_time,

[0067] now() as import_time,

[0068] from_unixtime(get_json_string(message,'$.task_info.timeStamp') / 1000),

[0069] from_unixtime(get_json_string(message,'$.taken_at')) as message_time,

[0070] from_unixtime(get_json_string(message,'$.created_at')) as fetching_time,

[0071] get_json_string(message,'$.local_videos') as local_video_list

[0072] from ods_ins_test

[0073] where type='topics' and to_date(import_time) = to_date(date_sub(now(),interval 1 day))

[0074] The target special functions in the above original query statement include from_unixtime() function, to_date() function and get_json_string() function.

[0075] Step S330, determining a target database from at least one candidate database; wherein the at least one candidate database includes a first database corresponding to the target database identifier and a second database adapted to the target special function.

[0076] Considering that special functions are only applicable to some databases, such as the get_json_string() function is only applicable to Doris, Hive, and MySQL, but not to Hbase and Greenplum, the target special function may not be adapted to the first database corresponding to the target database identifier, which may be caused by the user selecting the wrong target database identifier. Based on this, this embodiment uses the target special function to expand the target database, and realizes the query of multiple databases by switching the target database, which avoids the failure of a single database query to a certain extent and improves the query efficiency.

[0077] In some possible embodiments, the above step S330 can be implemented through the following process: determine at least one candidate database based on the first database corresponding to the target database identifier and the second database adapted to the target special function; prioritize each candidate database to obtain a ranking result; wherein, when the first database is one of the second databases, the priority of the first database is higher than that of other databases in the second database; based on the ranking result, select the target database from each candidate database.

[0078] In one possible implementation, both the first database and the second database may be used as candidate databases. In another possible implementation, considering that when the target special function does not match the first database corresponding to the target database identifier, the first database cannot perform the data query operation corresponding to the target special function, the candidate database may be determined in the following manner: determining whether the first database is one of the second databases; if not, determining the second database as the candidate database.

[0079] The default priorities of different types of databases can be determined first. When sorting the priorities, if the first database is one of the second databases, the priority of the first database is set to the highest, and the priorities of other candidate databases use the default priorities. The databases can be sorted in order of high to low priority or in order of low to high priority. The target database can select the candidate database with the highest priority in the sorting result. For example, when sorting in order of high to low priority, the candidate database with the highest ranking is selected as the target database.

[0080] In some possible embodiments, the default priority of the above-mentioned candidate databases can be determined in the following manner: for each candidate database, historical query data of the candidate database is obtained, and the historical query data includes indicator data of multiple indicators within a preset time range, and the multiple indicators include query response time, query throughput, query success rate, query complexity and query frequency, etc.; based on the historical query data of the candidate database, the indicator value of each indicator corresponding to the candidate database is calculated; based on preset weight data, the indicator values ​​of each indicator corresponding to the candidate database are weighted and summed to obtain a comprehensive score of the candidate database; the comprehensive scores of each candidate database are sorted, and based on the sorting results, the default priority of each candidate database is determined; among them, the candidate database with the highest comprehensive score has the highest default priority.

[0081] Specifically, the above preset time range can be set according to actual needs and is not limited here. For example, the preset time range is the most recent month. Among the above multiple indicators, query response time (Query Response Time, QRT) refers to the average query time required, query throughput (Query Throughput, QT) refers to the number of queries that can be completed per unit time, query success rate (Query Success Rate, QSR) refers to the proportion of successfully completed queries, query complexity (Query Complexity, QC) refers to the complexity of the query statement, and query complexity can be measured by the type of SQL statement or execution plan. Query frequency (Query Frequency, QF) refers to the number of queries that appear per unit time.

[0082] In a possible implementation, the indicator value of each indicator can be obtained as follows:

[0083] (1) Query response time indicator value

[0084]

[0085] in, It is i The response time of the query, N is the total number of queries;

[0086] (2) Query the throughput index value QT

[0087]

[0088] in, T is the total time;

[0089] (3) Query success rate indicator value QSR

[0090]

[0091] in, is the number of queries that completed successfully;

[0092] (4) Query complexity index value QC , can be assigned corresponding scores based on pre-defined levels;

[0093] (5) Query frequency index value QF , directly obtained by statistics.

[0094] The above weight data includes the weight of each indicator. The sum of the weights of each indicator is equal to 1. The comprehensive score of the candidate databaseS It can be calculated by the following formula:

[0095]

[0096] in, w QRT is the weight of query response time, w QT is the weight of query throughput, w QSR is the weight of the query success rate, w QC is the weight of query complexity, w QF is the weight of the query frequency.

[0097] For example, the indicator values ​​and weights of each indicator corresponding to database A and database B are shown in Table 1 below:

[0098] Table 1

[0099]

[0100] Then, the comprehensive score of database A is:

[0101]

[0102] The comprehensive score of database B is:

[0103]

[0104] Therefore, the default priority of database A is higher than the default priority of database B.

[0105] Step S340: construct a target query statement corresponding to the target database based on the original query content and the target special function.

[0106] The above-mentioned original query content includes query keywords, query fields and query conditions. Taking into account that the keywords in different databases are different, based on this, the above-mentioned step S340 can be implemented through the following process: based on the keyword library corresponding to the target database, the query keywords are converted into keywords to obtain the target keywords; based on the target keywords, query fields, query conditions, target special functions and the grammatical rules corresponding to the target database, the target query statement is constructed.

[0107] The keyword libraries and grammar rules corresponding to different databases are all pre-set. When constructing a target query statement, you can first obtain the keyword library and grammar rules corresponding to the target database, and then perform keyword conversion and query statement reconstruction.

[0108] Step S350, performing a data query of the target query statement in the target database through the connection pool to obtain a query result.

[0109] The data middle layer selects an idle connection object corresponding to an established target database from the connection pool, sends the target query statement to the target database, the target database performs a data query for the target query statement, and returns the query result to the data middle layer, and then puts the connection back into the connection pool.

[0110] Step S360: When the query result indicates that the query fails, step S330 is re-executed to switch the target database.

[0111] In some possible embodiments, in order to avoid repeated queries, the above step S360 may include: when the query result indicates that the query fails, the target database corresponding to the query result may be marked as having executed the query operation, and it is determined whether there is an optional database in at least one candidate database that has not executed the query operation; if there is an optional database, at least one candidate database is updated to an optional database, and the step of determining the target database from at least one candidate database is re-executed.

[0112] Furthermore, in the case of query failure, an error may be reported to the target business system. Based on this, the above method further includes: if there is no selectable database, returning query failure to the target business system.

[0113] Step S370: When the query result indicates that the query is successful, the query result is returned to the target business system.

[0114] The data middle layer can format the query results and return the formatted query results to the target business system.

[0115] In the embodiment of the present invention, the data middle layer is connected to the business system through a unified API and to multiple databases through a connection pool. In this way, by constructing the data middle layer, the decoupling of the business and the database is achieved, and the flexibility and maintainability of database adaptation are improved. At the same time, the data middle layer realizes the expansion of the target database for executing query operations through target special functions, and realizes the switching of the target database by reconstructing the target query statement, which avoids the failure of a single database query to a certain extent and improves the query efficiency.

[0116] Corresponding to the above-mentioned database switching method based on the data middle layer, the embodiment of the present invention also provides a database switching device based on the data middle layer, which is applied to the data middle layer using the ORM model. The data middle layer communicates with the business system through a unified API, and the data middle layer manages connections with multiple databases through a connection pool. Figure 4The schematic diagram of the structure of a database switching device based on a data intermediate layer is shown, and the device includes:

[0117] The receiving module 401 is used to receive an original query request sent by a target business system through a unified API, where the original query request includes an original query statement and a target database identifier;

[0118] A parsing module 402 is used to parse the original query statement through the ORM model to obtain original query information, where the original query information includes the original query content and the target special function;

[0119] The determination module 403 is used to determine a target database from at least one candidate database; wherein the at least one candidate database includes a first database corresponding to the target database identifier and a second database adapted to the target special function;

[0120] A construction module 404 is used to construct a target query statement corresponding to the target database based on the original query content and the target special function;

[0121] Query module 405, used to perform data query of target query statement in target database through connection pool to obtain query result;

[0122] The determination module 403 is further configured to, when the query result indicates that the query fails, re-execute the step of determining a target database from at least one candidate database to switch the target database;

[0123] The return module 406 is used to return the query result to the target business system when the query result indicates that the query is successful.

[0124] In the embodiment of the present invention, the data middle layer is connected to the business system through a unified API and to multiple databases through a connection pool. In this way, by constructing the data middle layer, the decoupling of the business and the database is achieved, and the flexibility and maintainability of database adaptation are improved. At the same time, the data middle layer realizes the expansion of the target database for executing query operations through target special functions, and realizes the switching of the target database by reconstructing the target query statement, which avoids the failure of a single database query to a certain extent and improves the query efficiency.

[0125] Furthermore, the above-mentioned determination module 403 is specifically used to: determine at least one candidate database based on the first database corresponding to the target database identifier and the second database adapted to the target special function; prioritize each candidate database to obtain a ranking result; wherein, when the first database is one of the second databases, the priority of the first database is higher than that of other databases in the second database; based on the ranking result, select the target database from each candidate database.

[0126] Furthermore, the determination module 403 is further configured to: determine whether the first database is one of the second databases; if not, determine the second database as a candidate database.

[0127] Furthermore, the above original query content includes query keywords, query fields and query conditions; the above construction module 404 is specifically used to: based on the keyword library corresponding to the target database, perform keyword conversion on the query keywords to obtain target keywords; based on the target keywords, query fields, query conditions, target special functions and grammatical rules corresponding to the target database, construct a target query statement.

[0128] Furthermore, the above-mentioned determination module 403 is also used to: when the query result indicates that the query fails, mark the target database corresponding to the query result as having executed the query operation, and determine whether there is an optional database in at least one candidate database that has not executed the query operation; if there is an optional database, update at least one candidate database to an optional database, and re-execute the step of determining the target database from at least one candidate database.

[0129] Furthermore, the returning module 406 is also used to return a query failure to the target business system if there is no optional database.

[0130] Furthermore, the above-mentioned multiple databases include multiple ones of Greenplum, MySQL, PostgreSQL, Doris, Elasticsearch, Oracle, ClickHouse, DAMO, MongoDB, Hive, Hbase, SQL Server and neo4j.

[0131] The database switching device based on the data intermediate layer provided in this embodiment has the same implementation principle and technical effects as those of the aforementioned database switching method embodiment based on the data intermediate layer. For the sake of brief description, for matters not mentioned in the embodiment of the database switching device based on the data intermediate layer, reference may be made to the corresponding contents in the aforementioned embodiment of the database switching method based on the data intermediate layer.

[0132] like Figure 5 As shown, an electronic device 500 provided by an embodiment of the present invention includes: a processor 501, a memory 502 and a bus, the memory 502 stores a computer program that can be run on the processor 501, when the electronic device 500 is running, the processor 501 and the memory 502 communicate through the bus, and the processor 501 executes the computer program to implement the above-mentioned database switching method based on the data middle layer.

[0133] Specifically, the memory 502 and the processor 501 can be general-purpose memories and processors, which are not specifically limited here.

[0134] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the database switching method based on the data intermediate layer in the previous method embodiment is executed. The computer-readable storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0135] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0136] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0137] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and a part of a module, a program segment or a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A database switching method based on a data middle layer, characterized in that: Applied to a data middle layer using an object-relational mapping (ORM) model, the data middle layer communicates with a business system through a unified API, and the data middle layer manages connections with multiple databases through a connection pool; the method includes: Receiving an original query request sent by a target business system through the unified API, wherein the original query request includes an original query statement and a target database identifier; Parsing the original query statement through the ORM model to obtain original query information, wherein the original query information includes original query content and target special function; Determine a target database from the candidate databases; wherein the candidate databases include a first database corresponding to the target database identifier and a second database adapted to the target special function; Based on the original query content and the target special function, construct a target query statement corresponding to the target database; Performing a data query of the target query statement in the target database through the connection pool to obtain a query result; When the query result indicates that the query fails, re-execute the step of determining the target database from the candidate databases to switch the target database; When the query result indicates that the query is successful, the query result is returned to the target business system.

2. The method according to claim 1, characterized in that The step of determining a target database from the candidate databases includes: Determine a candidate database based on a first database corresponding to the target database identifier and a second database adapted to the target special function; Prioritizing each of the candidate databases to obtain a ranking result; wherein, when the first database is one of the second databases, the priority of the first database is higher than that of other databases in the second database; Based on the ranking result, a target database is selected from each of the candidate databases.

3. The method according to claim 2, characterized in that The determining of the candidate database based on the first database corresponding to the target database identifier and the second database adapted to the target special function includes: determining whether the first database is one of the second databases; If not, the second database is determined as the candidate database.

4. The method according to claim 1, characterized in that: The original query content includes query keywords, query fields and query conditions; the target query statement corresponding to the target database is constructed based on the original query content and the target special function, including: Based on the keyword library corresponding to the target database, the query keyword is converted into a target keyword; A target query statement is constructed based on the target keyword, the query field, the query condition, the target special function and the grammatical rules corresponding to the target database.

5. The method according to claim 1, characterized in that When the query result indicates that the query fails, re-executing the step of determining the target database from the candidate databases includes: When the query result indicates that the query fails, marking the target database corresponding to the query result as having executed the query operation, and determining whether there is an optional database in the candidate database that has not executed the query operation; If the optional database exists, the candidate database is updated to the optional database, and the step of determining the target database from the candidate database is re-executed.

6. The method according to claim 5, characterized in that The method further comprises: If the optional database does not exist, a query failure is returned to the target business system.

7. The method according to any one of claims 1 to 6, characterized in that The multiple databases include Greenplum, MySQL, PostgreSQL, Doris, Elasticsearch, Oracle, ClickHouse, DAMO, MongoDB, Hive, Hbase, SQL Server and neo4j.

8. A database switching device based on a data middle layer, characterized in that: Applied to a data middle layer adopting an ORM model, the data middle layer communicates with a business system through a unified API, and the data middle layer manages connections with multiple databases through a connection pool; the device includes: A receiving module, configured to receive an original query request sent by a target business system through the unified API, wherein the original query request includes an original query statement and a target database identifier; A parsing module, used to parse the original query statement through the ORM model to obtain original query information, wherein the original query information includes original query content and target special function; A determination module, used to determine a target database from a candidate database; wherein the candidate database includes a first database corresponding to the target database identifier and a second database adapted to the target special function; A construction module, used to construct a target query statement corresponding to the target database based on the original query content and the target special function; A query module, used to perform data query of the target query statement in the target database through the connection pool to obtain a query result; The determining module is further configured to, when the query result indicates that the query fails, re-execute the step of determining the target database from the candidate databases to switch to the target database; The return module is used to return the query result to the target business system when the query result indicates that the query is successful.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the database switching method based on the data intermediate layer according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the database switching method based on a data intermediate layer according to any one of claims 1 to 7 is executed.

Images