Multi-database driving method and device based on HTTP (Hyper Text Transport Protocol)

Abstract

The invention provides a multi-database driving method and device based on an HTTP protocol, and the method comprises the steps: calling a JDBC method in response to a report tool after a client and a server establish a connection for authority verification, intercepting the calling of the JDBC method by the client, serializing the JDBC method into an HTTP request, and sending the HTTP request to the server through the HTTP protocol, the server side converts the HTTP request into a native JDBC method and forwards a corresponding SQL statement to a local database to execute query, the server side obtains a result set returned in a batch streaming mode after the database responds to the SQL statement, and the result set is serialized and then returned to the client side in a batch streaming mode; a client side pulls a plurality of batches of result sets in batches in advance and carries out local caching, the result sets are returned to a report tool, the report tool preferentially obtains data from the local cache, and when the local cache is used up, a next batch of requests are automatically triggered.

Description

Technical Field

[0001] This application relates to the field of database access technology, and more specifically, to a multi-database driver method and apparatus based on the HTTP protocol. Background Technology

[0002] The JDBC driver system is the core technology foundation for Java's ability to connect to and manipulate databases. It uses a standardized interface to shield the underlying differences between various databases, enabling Java applications to interact with various relational databases in a unified manner.

[0003] Currently, existing JDBC direct connection solutions include traditional JDBC direct connection solutions, data warehouse ETL solutions, and HTTP API encapsulation solutions. In the traditional JDBC direct connection solution, the reporting tool directly configures the database connection string, username, and password for each business system and connects to the database through the native JDBC Driver. In the data warehouse ETL solution, the business system data is periodically synchronized to the data warehouse, and the reporting tool accesses the data warehouse instead of the source database.

[0004] However, JDBC direct connection solutions pose a risk of exposing database connection information, increasing security risks; data warehouse ETL solutions have high data latency, making it difficult to access the latest data in business systems in real time, and there is a problem of data redundancy; and data warehouse ETL solutions generally have low transmission efficiency for large result sets, and the development and adaptation costs are high. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a multi-database driver method and apparatus based on the HTTP protocol. Under the HTTP protocol, it utilizes a separate JDBC driver architecture for permission verification, performs batch serialization processing between the client and server, and forwards the data to the database. Based on protocol optimization, it streams and consumes the result set in batches, hiding database connection information. Through complex permission verification, it reduces security risks and achieves real-time data access, avoiding data redundancy. Simultaneously, through streaming batch processing and protocol optimization, it reduces the number of network interactions and improves the transmission efficiency of large result sets. Furthermore, it provides a fully compatible driver that can be applied to various databases, reducing development, adaptation, and maintenance costs.

[0006] In a first aspect, embodiments of this application provide a multi-database driver method based on the HTTP protocol, applied to a split JDBC driver architecture, wherein the split JDBC driver architecture includes a client and a server; the method includes: After the client and the server establish a connection and successfully verify permissions, in response to the reporting tool calling a JDBC method, the client deployed on the reporting tool intercepts the JDBC method call, serializes the JDBC method into an HTTP request, and sends the HTTP request to the server via the HTTP protocol; wherein, serialization is performed differently for different types of data; The server receives and parses the HTTP request, converts the HTTP request into a native JDBC method, and forwards the corresponding SQL statement to the local database for query execution. The multi-database adapter deployed on the server handles the syntax differences of different databases for the JDBC method based on a preset processing method. The server obtains the result set returned in batches after receiving the SQL statement response from the database, serializes the result set, and returns it to the client in batches. The client pre-fetches multiple batches of the result set and caches them locally, and then returns the result set to the reporting tool. The reporting tool prioritizes retrieving data from the local cache and automatically triggers the next batch of requests when the local cache is exhausted.

[0007] In one possible implementation, the protocol layers of the client and the server are implemented through the following steps: The protocol layer of the client and the server is based on the HTTP protocol, and the transport layer supports multiple data transmission methods; among them, the multiple data transmission methods include at least long connection, Gzip compression, and batch processing; The transport layer is implemented based on long-lived multiplexed connections and Gzip compressed response bodies; wherein, the optimization of the protocol layer is achieved by long-lived multiplexed TCP connections, Gzip compression for data compression, and binary transmission encoding followed by chunked transmission.

[0008] In one possible implementation, the permission verification includes: In response to the client's request for table group permissions, the server defines the mapping relationship between table groups and database tables, and defines the access scope; wherein, the access scope represents the range of authorized table groups that the client can only access; In response to the client specifying the target table group corresponding to the authorization in the URL parameters, the server verifies whether the target table group exists within the access scope; If the target table group does not exist within the access scope, the original SQL statement is dynamically rewritten based on the database type so that the rewritten SQL statement filters out table groups that are not authorized for access; wherein, the database type is determined by parsing the SQL statement.

[0009] In one possible implementation, the permission verification further includes: The client stores the application interface key in the HTTP header of the HTTP protocol and transmits the key through the URL parameters and the HTTP header; The server verifies the validity of the key through a pre-configured keystore and associates it with the table group access permissions of the access scope to verify whether the key has the table group access permissions of the access scope.

[0010] In one possible implementation, the permission verification includes: The table group is grouped according to a preset grouping standard to obtain the grouped table groups, and the fields of the table groups are determined. Define and assign access permissions for the fields of the table group, assign the access permissions of the fields to the corresponding roles, and assign the roles to the corresponding users; wherein, the access permissions of the fields correspond to different users or roles; Identify the user executing the reporting tool, verify the user's access permissions, and execute the corresponding table group content based on the user's access permissions after successful verification.

[0011] In one possible implementation, the method further includes: To address the differences in methods used with different databases, the server dynamically masks methods based on the database type. The metadata is uniformly trimmed, and the corresponding metadata is returned according to the scope of the table group.

[0012] In one possible implementation, serialization is achieved through the following steps: If the data to be serialized is numeric, then the data is converted into a target array; If the data to be serialized is a string, the data is converted into the first target encoding, and special characters are escaped when they exist. If the data to be serialized is a date, the data is converted to the target ISO format, and special characters are escaped if present. If the data to be serialized is binary, the data is converted to a second target encoding and escaped if special characters are present.

[0013] Secondly, embodiments of this application also provide a multi-database driver device based on the HTTP protocol, applied to a split JDBC driver architecture, the split JDBC driver architecture including a client and a server; the device includes: The sending module is used to intercept the JDBC method call deployed on the reporting tool after the client and the server establish a connection for authorization verification and the verification is successful. The client serializes the JDBC method call into an HTTP request and sends the HTTP request to the server via the HTTP protocol. The serialization is performed differently for different types of data. The query module is used by the server to receive and parse the HTTP request, convert the HTTP request into a native JDBC method, and forward the corresponding SQL statement to the local database for query execution; wherein, the multi-database adapter deployed on the server uniformly handles the syntax differences of different databases for the JDBC method based on a preset processing method; The acquisition module is used for the server to acquire the result set returned in batches after the database responds with the SQL statement, serialize the result set and return it to the client in batches, and the client to fetch multiple batches of the result set in advance and cache them locally, and to return the result set to the reporting tool. The triggering module is used by the reporting tool to prioritize retrieving data from the local cache and automatically trigger the next batch of requests when the local cache is exhausted.

[0014] In one possible implementation, the protocol layers of the client and the server are implemented through the following steps: The protocol layer of the client and the server is based on the HTTP protocol, and the transport layer supports multiple data transmission methods; among them, the multiple data transmission methods include at least long connection, Gzip compression, and batch processing; The transport layer is implemented based on long-lived multiplexed connections and Gzip compressed response bodies; wherein, the optimization of the protocol layer is achieved by long-lived multiplexed TCP connections, Gzip compression for data compression, and binary transmission encoding followed by chunked transmission.

[0015] In one possible implementation, the sending module is specifically used for: In response to the client's request for table group permissions, the server defines the mapping relationship between table groups and database tables, and defines the access scope; wherein, the access scope represents the range of authorized table groups that the client can only access; In response to the client specifying the target table group corresponding to the authorization in the URL parameters, the server verifies whether the target table group exists within the access scope; If the target table group does not exist within the access scope, the original SQL statement is dynamically rewritten based on the database type so that the rewritten SQL statement filters out table groups that are not authorized for access; wherein, the database type is determined by parsing the SQL statement.

[0016] In one possible implementation, the sending module is specifically used for: The client stores the application interface key in the HTTP header of the HTTP protocol and transmits the key through the URL parameters and the HTTP header; The server verifies the validity of the key through a pre-configured keystore and associates it with the table group access permissions of the access scope to verify whether the key has the table group access permissions of the access scope.

[0017] In one possible implementation, the sending module is specifically used for: The table group is grouped according to a preset grouping standard to obtain the grouped table groups, and the fields of the table groups are determined. Define and assign access permissions for the fields of the table group, assign the access permissions of the fields to the corresponding roles, and assign the roles to the corresponding users; wherein, the access permissions of the fields correspond to different users or roles; Identify the user executing the reporting tool, verify the user's access permissions, and execute the corresponding table group content based on the user's access permissions after successful verification.

[0018] In one possible implementation, the device further includes: The blocking module is used to dynamically block different databases based on their methodological differences and through the server. The trimming module is used to uniformly trim metadata and return the corresponding metadata according to the scope of the table group.

[0019] In one possible implementation, serialization is achieved through the following steps: If the data to be serialized is numeric, then the data is converted into a target array; If the data to be serialized is a string, the data is converted into the first target encoding, and special characters are escaped when they exist. If the data to be serialized is a date, the data is converted to the target ISO format, and special characters are escaped if present. If the data to be serialized is binary, the data is converted to a second target encoding and escaped if special characters are present.

[0020] Thirdly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the multi-database driver method based on the HTTP protocol as described in any of the first aspects.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the multi-database driver method based on the HTTP protocol as described in any one of the first aspects.

[0022] This application provides a multi-database driver method and apparatus based on the HTTP protocol. After establishing a connection between the client and server, performing authorization verification, and successfully verifying the authorization, in response to a reporting tool calling a JDBC method, the client deployed on the reporting tool intercepts the JDBC method call, serializes the JDBC method into an HTTP request, and sends the HTTP request to the server via the HTTP protocol. The server receives and parses the HTTP request, converts the HTTP request into a native JDBC method, and forwards the corresponding SQL statement to the local database for query execution. The server obtains the result set returned in batches after receiving the SQL statement from the database, serializes the result set, and returns it to the client in batches. The client pre-fetches multiple batches of result sets in batches and caches them locally, and then returns the result sets to the reporting tool. The reporting tool prioritizes retrieving data from the local cache and automatically triggers the next batch of requests when the local cache is exhausted. This application employs a separate JDBC driver architecture for permission verification under the HTTP protocol. Batch serialization processing is performed between the client and server before forwarding to the database. Result sets are then streamed and consumed in batches based on protocol optimization, hiding database connection information. Complex permission verification reduces security risks and enables real-time data access, avoiding data redundancy. Furthermore, streaming batch processing and protocol optimization reduce the number of network interactions, improving the transmission efficiency of large result sets. In addition, a fully compatible driver is provided, applicable to various databases, reducing development, adaptation, and maintenance costs.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a multi-database driver method based on the HTTP protocol provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a multi-database driver device based on the HTTP protocol according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0027] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0028] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0029] Considering that the JDBC driver system is the core technology foundation for Java's ability to connect to and manipulate databases, it shields the underlying differences between different databases through a standardized interface, enabling Java applications to interact with various relational databases in a unified manner.

[0030] Currently, existing JDBC direct connection solutions include traditional JDBC direct connection solutions, data warehouse ETL solutions, and HTTP API encapsulation solutions. In the traditional JDBC direct connection solution, the reporting tool directly configures the database connection string, username, and password for each business system and connects to the database through the native JDBC Driver. In the data warehouse ETL solution, the business system data is periodically synchronized to the data warehouse, and the reporting tool accesses the data warehouse instead of the source database.

[0031] However, JDBC direct connection solutions pose a risk of exposing database connection information, increasing security risks; data warehouse ETL solutions have high data latency, making it difficult to access the latest data in business systems in real time, and there is a problem of data redundancy; and data warehouse ETL solutions generally have low transmission efficiency for large result sets, and the development and adaptation costs are high.

[0032] To address this issue, this application provides a multi-database driver method and apparatus based on the HTTP protocol. Under the HTTP protocol, it employs a separate JDBC driver architecture for permission verification. Batch serialization processing is performed between the client and server, and the data is forwarded to the database. Based on protocol optimization, the result set is streamed and batch-transmitted for consumption, hiding database connection information. Complex permission verification reduces security risks and enables real-time data access, avoiding data redundancy. Simultaneously, streaming batch processing and protocol optimization reduce the number of network interactions, improving the transmission efficiency of large result sets. Furthermore, a fully compatible driver is provided, applicable to various databases, reducing development, adaptation, and maintenance costs.

[0033] Figure 1 This is a flowchart illustrating a multi-database driver method based on the HTTP protocol provided in an embodiment of this application. The multi-database driver method based on the HTTP protocol of this application is applied to a split JDBC driver architecture; the split JDBC driver architecture includes a client (Stub client) and a server (Skeleton server); the client is deployed on the reporting tool to implement the standard JDBC interface; the server is deployed on each business system. For example... Figure 1 As shown in the embodiments of this application, the multi-database driver method based on the HTTP protocol may specifically include: S101. After establishing a connection between the client and server, performing authorization verification, and successfully verifying the authorization, in response to the reporting tool calling a JDBC method, the client deployed on the reporting tool intercepts the JDBC method call, serializes the JDBC method into an HTTP request, and sends the HTTP request to the server via the HTTP protocol.

[0034] S102. The server receives and parses the HTTP request, converts the HTTP request into a native JDBC method, and forwards the corresponding SQL statement to the local database for query execution.

[0035] S103. The server obtains the result set returned in batches after receiving the SQL statement response from the database, serializes the result set, and returns it to the client in batches. The client pre-fetches multiple batches of result sets and caches them locally, and returns the result sets to the reporting tool.

[0036] S104. The reporting tool prioritizes retrieving data from the local cache and automatically triggers the next batch of requests when the local cache is exhausted.

[0037] The aforementioned multi-database driver method based on the HTTP protocol employs a separate JDBC driver architecture for permission verification under the HTTP protocol. Batch serialization processing is performed between the client and server, and the data is forwarded to the database. Result sets are then streamed and consumed in batches according to protocol optimizations, hiding database connection information. Complex permission verification reduces security risks and enables real-time data access, avoiding data redundancy. Simultaneously, streaming batch processing and protocol optimization reduce the number of network interactions, improving the transmission efficiency of large result sets. Furthermore, a fully compatible driver is provided, applicable to various databases, reducing development, adaptation, and maintenance costs.

[0038] The exemplary steps described above in the embodiments of this application are illustrated below with specific examples: S101: After establishing a connection between the client and server, performing authorization verification, and successfully verifying the authorization, in response to the reporting tool calling a JDBC method, the client deployed on the reporting tool intercepts the JDBC method call, serializes the JDBC method into an HTTP request, and sends the HTTP request to the server via the HTTP protocol.

[0039] It can be added here that the multi-database driver method based on the HTTP protocol in this application requires the construction of a corresponding environment. Specifically, it involves constructing a client-side and server-side environment in a split JDBC driver architecture. The client deployed on the reporting tool forwards JDBC operations to the server deployed on each business system via the HTTP protocol. The protocol layer of the client and server is implemented through the following steps: the client and server protocol layers are based on the HTTP protocol (e.g., HTTP / 1.1+), and the transport layer supports multiple data transmission methods; these methods include at least keep-alive connections, Gzip compression, and batch processing; the transport layer is implemented based on multiplexed connections using keep-alive and Gzip compressed response bodies; protocol layer optimization is achieved through multiplexed TCP connections using keep-alive, data compression using Gzip compression, and chunked transmission after binary transmission encoding; data return is a batch streaming return, meaning that the target number of rows of data are compressed into data packets using Gzip compression before being returned.

[0040] Specifically, this application constructs a high-performance transmission protocol with a corresponding performance optimization mechanism. At the transport layer, it uses HTTP Keep-alive to reuse connections and Gzip to compress the response body. HTTP Keep-alive reuses TCP connections, reducing handshake overhead, while Gzip compression compresses text data, reducing network latency.

[0041] In this embodiment, the JDBC method is a JDBC operation, and serialization is used to process different types of data. After the connection is established between the client and the server, permission verification is performed. If the verification is successful, when the reporting tool calls the JDBC method, the client deployed on the reporting tool intercepts the JDBC method call, serializes the JDBC method into an HTTP request, and sends the HTTP request to the server via the HTTP protocol for subsequent processing.

[0042] Optionally, during permission verification, in response to a client requesting table group permissions, the server defines the mapping relationship between table groups and database tables, as well as the access scope. In response to the client specifying the target table group corresponding to the authorization in the URL parameters, the server verifies whether the target table group exists within the access scope. If the target table group does not exist within the access scope, the original SQL statement is dynamically rewritten based on the database type, so that the rewritten SQL statement filters out unauthorized table groups. Here, the access scope refers to the range of table groups that can be accessed; that is, the access scope represents the range of authorized table groups that the client can only access. The database type is determined by parsing the SQL statement.

[0043] For example, the mapping relationship between table groups and database tables can be defined in queryTableGroup.properties, such as userMenuGroup = T_User, T_Menu, User_View; the server-side configuration of queryTableGroup.properties defines the scope of accessible table groups (such as userMenuGroup), and the client can only access authorized table groups.

[0044] Continuing, the client specifies the target table group corresponding to the authorization in the URL, for example, specifying queryTableGroup=userMenuGroup. The server checks whether the table group exists and dynamically rewrites the SQL to restrict access to only the authorized table.

[0045] Optionally, the client stores the API (Application Programming Interface) key in the HTTP header and transmits the key via URL parameters and the HTTP header. The server verifies the key's validity using a pre-configured keystore and associates it with access permissions for the specified table groups to verify whether the key has the necessary access permissions. Here, the API key is the API key, and the HTTP header is the HTTP header.

[0046] For example, the client puts the APIKey in the HTTP Header (X-API-Key: JKLKJJKLJK-767698), the server verifies the validity of the APIKey through the configured keystore and associates it with table group access permissions, and the client passes the APIKey for authentication through the URL parameter queryTableGroup and the HTTP Header, without needing a database account.

[0047] Optionally, the table groups are grouped based on preset grouping criteria to obtain the grouped table groups, and the fields of the table groups are determined. Access permissions for the fields of the table groups are defined and assigned, with field access permissions assigned to corresponding roles, and roles assigned to corresponding users. The user executing the reporting tool is determined, and the user's access permissions are verified. After successful verification, the corresponding table group content is executed based on the user's access permissions. Specifically, the access permissions for each field correspond to different users or roles.

[0048] Therefore, by defining and assigning access permissions to the fields of a table group, field-level permission verification can be achieved.

[0049] In summary, for a split JDBC driver architecture, the stub client intercepts JDBC calls, serializes them into HTTP requests, supports nextBatch batch retrieval of ResultSets (e.g., retrieval in batches of 1000 rows by default), and performs local caching. Additionally, the stub client is deployed on the reporting tool (such as an ABI server), implementing standard JDBC interfaces (Driver / Connection / ResultSet, etc.) to serialize JDBC operations into HTTP requests.

[0050] S102, the server receives and parses the HTTP request, converts the HTTP request into a native JDBC method, and forwards the corresponding SQL statement to the local database for query execution.

[0051] In this embodiment, the multiple database adapters deployed on the server uniformly handle the syntax differences of different databases for JDBC methods based on a preset processing method; the server receives and parses the HTTP request sent by the client, converts the HTTP request into the native JDBC method, and forwards the corresponding SQL statement to the local database for query execution and subsequent processing.

[0052] S103: The server retrieves the result set returned in batches after receiving the SQL statement response from the database, serializes the result set, and returns it to the client in batches. The client pre-fetches multiple batches of result sets and caches them locally, and then returns the result sets to the reporting tool.

[0053] In this embodiment, the server forwards SQL statements to the local database for query execution, obtains the result sets returned by the database in batches, serializes the result sets, and returns them to the client in batches. The client pre-fetches multiple batches of result sets and caches them in a local queue, and then returns the result sets to the reporting tool. Data transmission can be performed using a JDBC batch streaming protocol based on the HTTP protocol.

[0054] Here, the Skeleton server parses HTTP requests, converts them into native JDBC operations, and handles syntax differences across multiple database adapters (such as compatibility of Oracle / PG metadata methods), resolving compatibility issues between different drivers. Additionally, deployed on various business systems, it receives HTTP requests, forwards them to the local database, and serializes and returns the results.

[0055] S104, the reporting tool prioritizes retrieving data from the local cache, and automatically triggers the next batch of requests when the local cache is exhausted.

[0056] In this embodiment of the application, the reporting tool prioritizes retrieving data from the local cache when obtaining data, and automatically triggers the next batch of requests when the local cache is exhausted or no data exists in the local cache.

[0057] The multi-database driver method based on the HTTP protocol provided in this application embodiment establishes a connection between the client and the server, performs authorization verification, and after successful verification, responds to the reporting tool's call to a JDBC method. The client deployed on the reporting tool intercepts the JDBC method call, serializes the JDBC method into an HTTP request, and sends the HTTP request to the server via the HTTP protocol. The server receives and parses the HTTP request, converts it into a native JDBC method, and forwards the corresponding SQL statement to the local database for query execution. The server obtains the database's response SQL statement and returns the result set in batches, serializes the result set, and returns it to the client in batches. The client pre-fetches multiple batches of result sets in batches and caches them locally, and then returns the result sets to the reporting tool. The reporting tool prioritizes retrieving data from the local cache and automatically triggers the next batch of requests when the local cache is exhausted. This application presents a multi-database driver method based on the HTTP protocol. Under the HTTP protocol, it employs a split JDBC driver architecture for permission verification. Batch serialization processing is performed between the client and server before forwarding to the database. Based on protocol optimization, the result set is streamed and batch-transmitted for consumption, hiding database connection information. Complex permission verification reduces security risks and enables real-time data access, avoiding data redundancy. Simultaneously, streaming batch processing and protocol optimization reduce the number of network interactions, improving the transmission efficiency of large result sets. Furthermore, it provides a fully compatible driver that can be applied to various databases, reducing development, adaptation, and maintenance costs.

[0058] Furthermore, to address the differences in methods across different databases, dynamic masking is performed on the server side based on the database type; metadata is uniformly trimmed, and the corresponding metadata is returned according to the scope of the table group.

[0059] Specifically, this application employs a unified metadata adaptation mechanism. For example, it dynamically masks differences in DatabaseMetaData methods across different databases (e.g., Oracle does not support getSuperTypes()). It also features secure metadata trimming, dynamically filtering DatabaseMetaData methods (e.g., blocking sensitive or database-unsupported methods like getSuperTables), and returning metadata by table group scope to avoid exposing the full database structure.

[0060] Furthermore, the server creates an independent connection pool for each secret key and automatically releases connections when the table group exceeds a preset threshold. After the server captures a database exception, it returns a corresponding standardized error code to the client so that the client can convert the error code into a corresponding exception class.

[0061] Specifically, the server creates an independent connection pool (such as HikariCP) for each APIKey and configures timeout at the table group level (for example, userMenuGroup.timeout = 300s), and the connections are automatically released when the timeout occurs. In addition, after the server captures a database exception, it returns a standardized error code, and the client converts the error code into a SQLException while retaining the original stack trace.

[0062] In this way, the connection pool, timeout control, and exception transmission mechanism are implemented.

[0063] Furthermore, for the client and the server, end-to-end time monitoring is performed to obtain the execution times of the client and the server. The execution times are written into a preset timer table and analyzed based on the timer table.

[0064] For example, end-to-end performance monitoring (client / server / DB execution times) is performed, and the data can be written into the timer_stats table for analyzing bottlenecks.

[0065] It can be supplemented that the serialization of this application is implemented through the following steps: in response to the data to be serialized being numeric, the data is converted into a target array; in response to the data to be serialized being a string, the data is converted into a first target encoding and escaped when there are special characters; in response to the data to be serialized being a date type, the data is converted into a target ISO format and escaped when there are special characters; in response to the data to be serialized being binary, the data is converted into a second target encoding and escaped when there are special characters. Here, the target array is an array in a target format.

[0066] Specifically, for example, the serialization rules of this application are as follows: for numeric types, directly convert to JSON numeric types; for strings, perform UTF-8 encoding and escape special characters (such as " to \"); for date types, ISO 8601 format (yyyy-MM-dd'T'HH:mm:ss.SSSZ); for binary, Base64 encoding (such as BLOB fields).

[0067] In addition, the HTTP protocol format of this application is as shown in Table 1 below: Table 1

[0068] In summary, this application applies to a split JDBC driver architecture, where the stub client forwards JDBC operations to the Skeleton server via the HTTP protocol; it features a table group isolation mechanism, with the server configuring table groups to define access scopes and the client specifying authorized table groups via URL parameters; it supports batch streaming transmission, with the client pulling data packets and caching them locally; it includes multiple database adapters to uniformly handle differences in metadata methods and SQL syntax between Oracle / PG / MySQL; and it features dynamic metadata filtering, automatically trimming DatabaseMetaData sensitive methods based on database type.

[0069] Therefore, this application achieves complete hiding of database connection information, controls access permissions only through APIKey + table group, thereby improving security; implements field-level permission control, restricting the range of data visible to the client; reduces the number of network interactions, improving the throughput of large result set queries; supports heterogeneous databases without modifying client code, reducing maintenance costs; avoids exposing the internal structure of the database, and solves compatibility issues between different drivers.

[0070] Figure 2 This is a schematic diagram of the structure of a multi-database driver device based on the HTTP protocol provided in the embodiments of this application; as shown below. Figure 2 As shown, the HTTP-based multi-database driver device 200 of this application embodiment is applied to a split JDBC driver architecture, which includes a client and a server; specifically, it may include: The sending module 201 is used to establish a connection between the client and the server for authorization verification. After successful verification, in response to the reporting tool calling a JDBC method, the client deployed on the reporting tool intercepts the JDBC method call, serializes the JDBC method into an HTTP request, and sends the HTTP request to the server via the HTTP protocol. The serialization process is performed differently for different types of data. The query module 202 is used to receive and parse HTTP requests on the server side, convert the HTTP requests into native JDBC methods, and forward the corresponding SQL statements to the local database for query execution. Among them, the multi-database adapter deployed on the server side handles the syntax differences of different databases for JDBC methods based on a preset processing method. The acquisition module 203 is used to obtain the result set returned in batches after the database responds with the SQL statement, serialize the result set and return it to the client in batches. The client pre-fetches multiple batches of result sets and caches them locally, and returns the result set to the reporting tool. Trigger module 204 is used by the reporting tool to prioritize retrieving data from the local cache and automatically trigger the next batch of requests when the local cache is exhausted.

[0071] In one possible implementation, the protocol layer of the client and server is achieved through the following steps: The client and server protocol layers are based on the HTTP protocol, and the transport layer supports multiple data transmission methods; among these, at least persistent connections, Gzip compression, and batch processing are supported. The transport layer is implemented based on long-lived multiplexed connections and Gzip-compressed response bodies; the optimization of the protocol layer is achieved through long-lived multiplexed TCP connections, Gzip compression for data compression, and chunked transmission after binary transmission encoding.

[0072] In one possible implementation, the sending module is specifically used for: In response to a client's request for table group permissions, the server defines the mapping relationship between table groups and database tables, as well as the access scope; where the access scope represents the range of table groups that the client can only access. In response to the client specifying the target table group corresponding to the authorization in the URL parameters, the server verifies whether the target table group exists within the access scope; If the target table group is not within the access scope, the original SQL statement is dynamically rewritten based on the database type so that the rewritten SQL statement filters out table groups that are not authorized for access; the database type is determined by parsing the SQL statement.

[0073] In one possible implementation, the sending module is specifically used for: The client stores the application interface key in the HTTP header of the HTTP protocol and transmits the key through URL parameters and HTTP headers; The server verifies the validity of the key through a pre-configured keystore and associates it with the access permissions of the table group within the access scope to verify whether the key has the access permissions for the table group within the access scope.

[0074] In one possible implementation, the sending module is specifically used for: The table groups are grouped based on a preset grouping standard to obtain the grouped table groups, and the fields of the table groups are determined. Define and assign access permissions for fields in a table group, assign field access permissions to corresponding roles, and assign roles to corresponding users; where field access permissions correspond to different users or roles. Identify the user who will execute the reporting tool, verify the user's access permissions, and execute the corresponding table group content based on the user's access permissions after successful verification.

[0075] In one possible implementation, the apparatus further includes: The masking module is used to dynamically mask methods for different databases based on the database type via the server. The trimming module is used to uniformly trim metadata and return the corresponding metadata according to the scope of the table group.

[0076] In one possible implementation, serialization is achieved through the following steps: If the data to be serialized is numeric, then the data is converted into a target array; If the data to be serialized is a string, the data is converted to the first target encoding, and special characters are escaped if present. If the data to be serialized is a date, the data is converted to the target ISO format and escaped if special characters are present. If the data to be serialized is binary, the data is converted to a second target encoding and escaped if special characters are present.

[0077] The multi-database driver device based on the HTTP protocol provided in this application embodiment establishes a connection between the client and the server, performs authorization verification, and after successful verification, responds to the JDBC method call by the reporting tool. The client deployed on the reporting tool intercepts the JDBC method call, serializes the JDBC method into an HTTP request, and sends the HTTP request to the server via the HTTP protocol. The server receives and parses the HTTP request, converts the HTTP request into a native JDBC method, and forwards the corresponding SQL statement to the local database for query execution. The server obtains the result set returned in batches after receiving the SQL statement from the database, serializes the result set, and returns it to the client in batches. The client pre-fetches multiple batches of result sets in batches and caches them locally, and then returns the result sets to the reporting tool. The reporting tool prioritizes retrieving data from the local cache and automatically triggers the next batch of requests when the local cache is exhausted. This application presents a multi-database driver device based on the HTTP protocol. Under the HTTP protocol, it employs a separate JDBC driver architecture for permission verification. Batch serialization processing is performed between the client and server, and the data is forwarded to the database. Based on protocol optimization, the result set is streamed and batch-transmitted for consumption. This hides database connection information, reduces security risks through complex permission verification, and enables real-time data access while avoiding data redundancy. Furthermore, streaming batch processing and protocol optimization reduce the number of network interactions, improving the transmission efficiency of large result sets. In addition, it provides a fully compatible driver that can be applied to various databases, reducing development, adaptation, and maintenance costs.

[0078] like Figure 3As shown in the embodiment of this application, an electronic device 300 includes a processor 301, a memory 302, and a bus. The memory 302 stores machine-readable instructions that can be executed by the processor 301. When the electronic device is running, the processor 301 communicates with the memory 302 via the bus. The processor 301 executes the machine-readable instructions to perform the steps of the multi-database driver method based on the HTTP protocol described above.

[0079] Specifically, the memory 302 and processor 301 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 301 runs the computer program stored in the memory 302, it can execute the above-mentioned multi-database driver method based on the HTTP protocol.

[0080] Corresponding to the above-described multi-database driver method based on the HTTP protocol, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described multi-database driver method based on the HTTP protocol.

[0081] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0082] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0084] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the deployment methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0085] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-database driving method based on HTTP protocol, characterized in that, The method is applied to a split JDBC driver architecture, and the split JDBC driver architecture comprises a client and a server. After the client and the server establish a connection and pass the permission verification, in response to a report tool calling a JDBC method, the client deployed on the report tool side intercepts the JDBC method call, serializes the JDBC method into an HTTP request, and sends the HTTP request to the server through the HTTP protocol; wherein serialization is different for different types of data. The server receives and parses the HTTP request, converts the HTTP request into an original JDBC method, and forwards the corresponding SQL statement to a local database for query; wherein a multi-database adapter deployed on the server uniformly processes the syntax differences of different databases for the JDBC method based on a preset processing method; After the server obtains the result set returned by the database in batches and in a streaming manner in response to the SQL statement, the server serializes the result set and returns it to the client in batches and in a streaming manner; the client pre-pulls multiple batches of the result set and locally caches them, and returns the result set to the report tool; The report tool preferentially obtains data from the local cache, and when the local cache is exhausted, automatically triggers the next batch request.

2. The method of claim 1, wherein, The protocol layer of the client and the server is implemented by the following steps: The protocol layer of the client and the server is based on the HTTP protocol, and the transmission layer supports multiple data transmission modes; wherein the multiple data transmission modes at least include long connection, Gzip compression, and batch processing; The transmission layer is implemented based on the multiplexing connection of long connection and the response body of Gzip compression; wherein the optimization of the protocol layer is achieved by multiplexing TCP connection, data compression through Gzip compression, and binary transmission encoding and block transmission.

3. The method of claim 2, wherein, The permission verification comprises: In response to the client requesting table group permissions, the server defines the mapping relationship between table groups and database tables, and defines the access range; wherein the access range represents that the client can only access the authorized table group range; In response to the client specifying the target table group corresponding to the authorization in the URL parameter, the server checks whether the target table group exists in the access range; When the target table group does not exist in the access range, the original SQL statement is dynamically rewritten based on the type of the database, so that the rewritten SQL statement filters the table groups that are not authorized to access; wherein the type of the database is determined by parsing the SQL statement.

4. The method of claim 3, wherein, The permission verification further comprises: The client stores the secret key of the application programming interface into the HTTP header of the HTTP protocol, and transmits the secret key through the URL parameter and the HTTP header; The server checks the validity of the secret key through a pre-configured key library, and associates the table group access permission of the access range to check whether the secret key has the table group access permission of the access range.

5. The method of claim 4, wherein, The permission verification comprises: grouping the table groups based on preset grouping criteria to obtain grouped table groups, and determining fields of the table groups; defining and assigning access permissions of the fields for the fields of the table groups, assigning the access permissions of the fields to corresponding roles, and assigning the roles to corresponding users; wherein the access permissions of the fields correspond to different users or roles; determining a user executing the report tool, verifying access permissions of the user, and executing corresponding table group content based on the access permissions of the user after the verification is passed.

6. The method of claim 1, wherein, The method further comprises: dynamically shielding based on types of databases through the server for method differences of different databases; unifying and clipping metadata, and returning corresponding metadata according to ranges of the table groups.

7. The method of claim 1, wherein, Serialization is achieved by the following steps: in response to the data to be serialized being numerical, converting the data into a target array; in response to the data to be serialized being a string, converting the data into a first target encoding, and escaping if there are special characters; in response to the data to be serialized being date type, converting the data into a target ISO format, and escaping if there are special characters; in response to the data to be serialized being binary, converting the data into a second target encoding, and escaping if there are special characters.

8. A multi-database driving apparatus based on an HTTP protocol, characterized by comprising: Applied to a split JDBC driver architecture, the split JDBC driver architecture comprises a client and a server; the device comprises: a sending module, configured to, after establishing a connection between the client and the server for permission verification and successful verification, in response to a report tool calling a JDBC method, intercepting, by the client deployed on the report tool side, the JDBC method call, serializing the JDBC method into an HTTP request, and sending the HTTP request to the server through an HTTP protocol; wherein serialization is different for different types of data; a query module, configured to receive and parse, by the server, the HTTP request, convert the HTTP request into an original JDBC method, and forward a corresponding SQL statement to a local database to execute a query; wherein a multi-database adapter deployed on the server uniformly processes syntax differences of different databases for the JDBC method based on a preset processing method; an acquisition module, configured to acquire, by the server, a result set returned in batches and in a streaming manner after the database responds to the SQL statement, serialize the result set, and return the result set to the client in batches and in a streaming manner; the client pre-pulls multiple batches of the result set and locally caches the result set, and returns the result set to the report tool; a triggering module, configured to, the report tool preferentially acquires data from the local cache, and when the local cache is exhausted, automatically triggers a next batch of requests.

9. An electronic device, comprising: The method comprises: A processor, a memory, and a bus, the memory storing machine readable instructions executable by the processor, the processor in communication with the memory via the bus when the electronic device is running, the machine readable instructions, when executed by the processor, performing the steps of the HTTP protocol based multi-database driving method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer readable storage medium storing a computer program, the computer program, when executed by a processor, performing the steps of the HTTP protocol based multi-database driving method of any one of claims 1 to 7.

Images