Multi-data-source online interface service system

Abstract

The invention provides a multi-data-source online interface service system, which belongs to the technical field of data services and comprises a data source management module, an interface configuration module, a dynamic parameter processing module, a data caching and synchronizing module, a visual integration module and a security management module. A user can connect various data sources through custom configuration, support data extraction in a full scale acquisition or dynamic SQL query mode, and support parameterized requests to realize data dynamic screening. And after the interface service is modified, the returned data can be synchronized to the large visual screen in real time, so that the instantaneity of data display is ensured. Through the method and the device, the problems of single data source, rigid configuration, update delay and the like in the traditional interface service are solved, the flexibility and the response efficiency of the data service are remarkably improved, and efficient technical support is provided for a big data visualization scene.

Description

Technical Field

[0001] The present invention relates to the technical field of data services, and particularly to an online interface service system for multiple data sources. Background Art

[0002] In data-driven business scenarios, data visualization dashboards rely on backend interface services to obtain multi-source heterogeneous data in real time. Traditional interface service solutions have the following problems:

[0003] Single data source: Most systems only support a single database connection and cannot flexibly switch or integrate multiple types of data sources (such as MySQL, Oracle, API interfaces, etc.);

[0004] Lack of interface flexibility: The data acquisition method is fixed (such as only supporting full table reading), lacking the ability of dynamic parameterized query, and it is difficult to meet the personalized needs of complex business scenarios;

[0005] Update delay: After the interface configuration is modified, the service needs to be restarted or manually refreshed, resulting in delayed data display and affecting the timeliness of decision-making;

[0006] Security risks: Dynamic SQL queries lack a parameter verification mechanism, which is prone to security hazards such as SQL injection.

[0007] In the prior art, although there are individual solutions that support multi-data source access, they generally lack unified online configuration management capabilities, and the integration degree of dynamic query and real-time synchronization functions is relatively low. Therefore, there is an urgent need for a solution that supports online switching of multiple data sources, flexible interface configuration, and real-time data synchronization to improve the efficiency and security of data services. Summary of the Invention

[0008] In order to solve the above technical problems, the present invention provides an online interface service system for multiple data sources, aiming to provide flexible and efficient data access and dynamic update capabilities for data visualization dashboards.

[0009] The technical solution of the present invention is:

[0010] An online interface service system for multiple data sources, comprising:

[0011] A data source management module for online configuration, connection testing, and dynamic switching of multiple types of data sources;

[0012] An interface configuration module that supports two modes of full table data acquisition and dynamic SQL query, and generates corresponding API endpoints;

[0013] A dynamic parameter processing module that verifies, filters, and binds incoming parameters to pre-compiled SQL;

[0014] Data caching and synchronization module, which realizes query result caching and real-time data update of the visualization dashboard;

[0015] Visualization integration module, which provides standard data format and real-time push function;

[0016] Security management module, which conducts identity authentication, permission control and SQL injection prevention.

[0017] Furthermore,

[0018] The dynamic SQL mode supports users to customize parameterized query statements and automatically parses the parameter types to generate API documents.

[0019] Furthermore,

[0020] Full table mode: After the user selects the data source and table name, the system automatically generates a full data query interface;

[0021] Dynamic SQL mode: Users can write parameterized SQL statements, define dynamic parameters, and the system parses the SQL and generates corresponding API endpoints.

[0022] Furthermore,

[0023] The dynamic parameter processing module receives the dynamic parameters passed in by the front end, conducts type verification and security filtering; binds the parameters to the pre-compiled SQL statement to generate the final query request.

[0024] Furthermore,

[0025] The data caching and synchronization module is configured to set the caching policy according to business requirements and automatically refresh the cache after the interface is modified.

[0026] Furthermore,

[0027] The security management module restricts dynamic SQL to only allow SELECT operations and intercepts high-risk instructions.

[0028] Furthermore,

[0029] The data source management module supports the configuration of the data source for the API interface, including the definition of the request header, authentication method and response format.

[0030] The beneficial effects of the present invention are

[0031] Multi-source compatibility: Supports online switching of heterogeneous data sources to meet the data integration requirements of complex business scenarios;

[0032] Flexibility and scalability: The full table and dynamic SQL dual modes adapt to different business requirements, and the parameterized design improves the interface reuse rate;

[0033] Real-time performance: Modifications to interface configurations and data updates take effect immediately, ensuring the timeliness of decision-making on the visual dashboard;

[0034] High performance: Optimize query efficiency through caching mechanisms and pre-compiled SQL to reduce the database load;

[0035] Security: Multiple protection mechanisms effectively prevent data leakage and malicious attacks;

[0036] Ease of use: The online configuration interface is user-friendly. Interfaces can be quickly generated without coding, lowering the usage threshold. Description of the Drawings

[0037] Figure 1 It is a schematic diagram of the workflow of the present invention. Detailed Embodiments

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0039] The present invention provides an online interface service solution for multiple data sources, with the following specific objectives:

[0040] 1. Dynamic access to multiple data sources: Support custom configuration of multiple types of data sources (such as relational databases, NoSQL,

[0041] API interfaces), enabling quick switching and unified management;

[0042] 2. Flexible interface service modes: Provide two data acquisition modes - full table reading and dynamic SQL query. The latter supports parameterized requests to achieve dynamic data filtering;

[0043] 3. Online configuration and real-time effectiveness: Allow users to modify interface configurations (such as data sources, SQL statements) online, and the modifications will take effect immediately without restarting the service;

[0044] 4. Real-time data synchronization: The data returned by the interface can be immediately pushed to the visual dashboard to ensure strong consistency between data display and backend services;

[0045] 5. Security guarantee: Built-in parameter verification and SQL injection protection mechanisms to ensure the security of dynamic queries;

[0046] 6. Efficient caching and performance optimization: Reduce the database load through data caching strategies and improve the response speed in high-frequency query scenarios.

[0047] Specifically include:

[0048] 1. Data source management module

[0049] 1) Support online configuration of multiple data source types (such as MySQL, PostgreSQL, etc.), and provide connection testing and status monitoring functions;

[0050] 2) Encrypt and store data source information, and support dynamic switching of multi-environment configurations (development, testing, production).

[0051] 2. Interface configuration module

[0052] 1) Full table mode: After the user selects the data source and table name, the system automatically generates a full data query interface;

[0053] 2) Dynamic SQL mode: The user can write parameterized SQL statements, define dynamic parameters (such as time range, filtering conditions), and the system parses the SQL and generates the corresponding API endpoints;

[0054] 3. Dynamic parameter processing module

[0055] 1) Receive dynamic parameters passed in from the front end, and perform type verification and security filtering (such as preventing SQL injection);

[0056] 2) Bind the parameters to the pre-compiled SQL statement to generate the final query request.

[0057] 4. Data caching and synchronization module

[0058] 1) Cache the results of high-frequency queries, and set the cache expiration time to balance real-time performance;

[0059] 2) After the interface configuration is modified, automatically refresh the cache and trigger the data update of the visualization dashboard.

[0060] 5. Visualization integration module

[0061] 1) Provide a standard API response format (such as JSON);

[0062] 6. Security management module

[0063] 1) Interface access requires identity authentication and permission verification, and controls the data access scope according to roles;

[0064] 2) Perform syntax analysis and dangerous operation interception (such as DROP, DELETE) before executing dynamic SQL.

[0065] Users can connect to multiple data sources through custom configurations, support data extraction through full-scale tables or dynamic SQL queries, and support parameterized requests to achieve dynamic data filtering. After the interface service is modified, the returned data can be synchronized to the visualization dashboard in real time to ensure the immediacy of data display. Through this solution, problems such as single data source, rigid configuration, and delayed updates in traditional interface services are solved.

[0066] The above are only the preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An online interface service system for multiple data sources, characterized in that: include: Data source management module, used for online configuration, connection testing and dynamic switching of multiple types of data sources; The interface configuration module supports two modes: full table data acquisition and dynamic SQL query, and generates corresponding API endpoints; Dynamic parameter processing module, which verifies, filters and binds incoming parameters to precompiled SQL; Data caching and synchronization module, to achieve query result caching and real-time data update of the visualization screen; Visual integration module, providing standard data format and real-time push function; The security management module performs identity authentication, permission control and SQL injection protection.

2. The system according to claim 1, characterized in that The dynamic SQL mode supports user-defined parameterized query statements and automatically parses parameter types to generate API documents.

3. The system according to claim 1 or 2, characterized in that: Full table mode: After the user selects the data source and table name, the system automatically generates a full data query interface; Dynamic SQL mode: Users can write parameterized SQL statements, define dynamic parameters, parse SQL and generate corresponding API endpoints.

4. The system according to claim 1 or 2, characterized in that: The dynamic parameter processing module receives the dynamic parameters passed in by the front end, performs type verification and security filtering; binds the parameters to the precompiled SQL statement to generate the final query request.

5. The system according to claim 1 or 2, characterized in that: The data cache and synchronization module is configured to set cache policies according to business needs and automatically refresh the cache after the interface is modified.

6. The system according to claim 1, characterized in that: The security management module restricts dynamic SQL to only allow SELECT operations to be executed, intercepting high-risk instructions.

7. The system according to claim 6, characterized in that Interface access requires identity authentication and permission verification, and data access scope is controlled by role; Perform syntax analysis and intercept dangerous operations before dynamic SQL is executed.

8. The system according to claim 1, characterized in that The data source management module supports the data source configuration of the API interface, including the request header, authentication method and response format definition.

Images