Multi-dimensional data index management system and method based on Spring Boot

Abstract

The invention discloses a Spring Boot-based multi-dimensional data index management system and method, and the system employs a Spring Boot micro-service architecture, and comprises a data source management module, an index definition module, a task scheduling module, and a data display and export module. The method comprises the following steps: in response to a data source added by a user through a management interface of a data source management module, performing a connectivity test on a corresponding database; the data source passing the test is initialized; in response to data source information selected by a user through a visual interface of an index definition module and a correspondingly written SQL query statement, creating an index definition; in response to a task plan preset by a user through the task scheduling module, generating a scheduling strategy, and recording a task state; executing and aggregating data; and displaying and exporting the data. According to the method, dynamic SQL configuration and a visual index definition method are supported, statistical logic can be adjusted without coding, business personnel can conveniently participate in index definition, and a cross-data-source joint query and task scheduling mechanism is realized.

Description

Technical Field

[0001] This application belongs to the field of medical data management technology, and relates to a multidimensional data indicator management system and method based on Spring Boot. Background Technology

[0002] Currently, hospitals have multiple independent information systems, such as HIS, LIS, PACS, and EMR systems. The data generated by these systems is scattered across different databases, with varying formats and structures. Given this data environment, traditional hospital data statistics methods suffer from the following problems: 1. Data sources are scattered, requiring manual extraction of data from multiple systems, which is inefficient; 2. Statistical indicators are fixed and cannot be flexibly adjusted according to business needs; 3. The query statement is hard-coded in the program, and modifying it requires redeploying the system; 4. Lack of a unified data indicator management platform, resulting in inconsistent indicator definitions; 5. Poor real-time performance, unable to meet dynamic monitoring requirements.

[0003] Furthermore, most existing data statistics tools are designed for a single data source, lack the ability to manage multiple data sources in a unified manner, and do not support dynamic SQL configuration, thus failing to meet the complex and ever-changing data statistics needs of hospitals. It is necessary to improve them. Summary of the Invention

[0004] To address the shortcomings of the aforementioned existing technologies, this application provides a multidimensional data indicator management system and method based on Spring Boot, which supports dynamic SQL configuration and visual indicator definition methods. It allows for adjustment of statistical logic without coding, facilitates business personnel participation in indicator definition, and implements cross-data source joint queries and task scheduling mechanisms.

[0005] To achieve the above objectives, the present invention employs the following techniques: The multidimensional data metrics management system based on Spring Boot adopts a Spring Boot microservice architecture, including: The data source management module is used for data source configuration, managing connection configurations with various types of databases, and providing data source connectivity testing and connection pool management functions. The indicator definition module provides a visual interface for users to define data indicators and write SQL query statements to complete the indicator definition creation. The task scheduling module provides task planning functionality, implements scheduled task plans based on Quartz or Spring Scheduler, aggregates data from multiple databases to a unified central database according to the user's preset task plan, and calls the query logic in the pre-created indicator definition to execute cross-data source query calculation tasks in the central database and stores the calculated indicator results in a specified location. The data display and export module is used to read the calculated indicator results and display them in the form of visualization components, and supports exporting the results to files in various formats.

[0006] Furthermore, it also includes a permission management module, which is used to implement user role and data permission control based on Spring Security.

[0007] This paper describes a multidimensional data metric management method based on Spring Boot, implemented using a Spring Boot-based multidimensional data metric management system. The steps include: S100, Data Source Connection Configuration: In response to a data source added by a user through the management interface of the data source management module, the added data source includes the data source name and its corresponding database connection information, including JDBC URL, user ID, password and driver class name; Perform connectivity tests to the corresponding database; For data sources that pass the test, the data source management module creates a connection object for the actual connection established between the data source and the corresponding database, generates a data source ID, and saves the data source name, data source ID, database name, and connection object as data source information to the connection pool. S200, Data Indicator Definition: In response to the user's selection of data source information and corresponding SQL query statement through the visualization interface of the indicator definition module, the SQL query statement is validated, and an indicator definition is created based on the selected data source information and the corresponding SQL query statement when the validation is successful. The created indicator definition includes indicator ID, indicator description, and SQL query calculation formula. The SQL query calculation formula contains location information associated with the database name in the selected data source information. S300, Task Scheduling Configuration: In response to the task plan preset by the user through the task scheduling module, a scheduling strategy is generated and the task status is recorded; the task plan includes the task name, data source, execution frequency, mapping table configuration, and mapping field configuration, and the data source includes the database name and data table name; S400, Data Execution and Aggregation: When the task scheduling module triggers execution according to the scheduling policy, it reads the data source in the task plan, obtains the connection object from the connection pool according to the database name in the data source, establishes a connection with the corresponding database through the obtained connection object, and obtains the original data from the database with the established connection according to the data table name in the data source. The task scheduling module uses mapping table configuration and mapping field configuration to synchronize the table names and fields of the original data it obtains, and then stores the synchronized data in a unified central database. The mapping table configuration and mapping field configuration are used to determine the mapping relationship between the synchronized data and its original source database. Based on the metric calculation request issued by the user, the system calls the corresponding metric definition, performs the query calculation from the central database according to the SQL query calculation formula in the called metric definition, stores the calculated metric result in Redis or a specified database, and updates the task status. S500, Data Display and Export: The indicator results are read through the data display and export module, and the results are displayed in charts according to the time range selected by the user through the data display and export module using the visualization component. The data display and export module responds to the export operation performed by the user through the data display and export module, generating and exporting files in Excel and / or PDF formats.

[0008] The beneficial effects of this invention are as follows: 1. Achieve cross-system data integration and unified management: Through the data source management module, the system can connect to multiple heterogeneous databases such as HIS, LIS, PACS, and EMR scattered within the hospital, as well as different types of databases, to establish a unified connection pool and central database. This effectively solves the problem of scattered data sources and the need for manual extraction in the traditional approach, and achieves unified access and management across data sources. 2. Supports flexible and configurable indicator definition and dynamic query. The system provides a visual interface that allows users to customize data indicators by writing and validating SQL statements and preset task plans. Statistical logic and query rules can be adjusted without modifying program code or redeploying the system, significantly improving the flexibility of indicator management and the speed of response to business needs. Through task scheduling, data is stored in the central database and aggregated and mapped, providing a unified data model foundation for subsequent more complex cross-indicator related queries and aggregation queries. 3. Enhance the automation and real-time performance of data processing: Based on the task scheduling module, the system can automatically execute cross-data source query and calculation tasks according to preset plans and store the results in a specified location. It supports result visualization and export in multiple formats, effectively improving the shortcomings of traditional statistical methods such as poor real-time performance and inability to dynamically monitor, and realizing the automation of data processing and real-time monitoring. Attached Figure Description

[0009] Figure 1 This is a system structure block diagram of an embodiment of this application.

[0010] Figure 2 This is a flowchart of the method steps in an embodiment of this application.

[0011] Figure 3 This is an example of a chart display in the data display and export module of this application embodiment.

[0012] Figure 4 This is a second example of a chart display in the data display and export module of this application embodiment.

[0013] Figure 5 This is Example 3 of the chart display of the data display and export module in this application embodiment.

[0014] Figure 6 This is an example of an Excel file exported by the data display and export module in this application embodiment. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the implementation methods of the present invention will be described in detail below with reference to the accompanying drawings. However, the embodiments described in this invention are only some embodiments of the present invention, and not all embodiments.

[0016] This application provides a multi-dimensional data indicator management system based on Spring Boot, employing a Spring Boot microservice architecture, such as... Figure 1 As shown, it includes modules such as data source management, indicator definition, task scheduling, and data display and export.

[0017] Specifically, the data source management module is used for data source configuration, managing connection configurations with various types of databases, and providing data source connectivity testing and connection pool management functions. Among them, the various types of databases can be multiple heterogeneous databases distributed within the hospital, such as HIS, LIS, PACS, EMR, etc., including various specific types of MySQL, Oracle, SQL Server, Hive, etc.

[0018] The indicator definition module provides a visual interface for users to define data indicators and write SQL queries to complete indicator definition creation; it supports dynamic SQL configuration and parameterized queries. Indicator definitions can be associated with multiple data sources and support cross-database joins.

[0019] The task scheduling module provides task planning functionality, implementing scheduled task plans based on Quartz or Spring Scheduler; it supports real-time querying and offline statistical tasks. The task scheduling module aggregates data from multiple databases into a unified central database according to user-preset task plans; it also calls the query logic in pre-created metric definitions to execute cross-data source query calculation tasks in the central database and stores the calculated metric results in a specified location.

[0020] The data display and export module is used to read the calculated indicator results and display them in the form of a visualization component, which integrates ECharts; and supports exporting the results to files in various formats.

[0021] Preferably, the system further includes a permission management module for implementing user role and data permission control based on Spring Security.

[0022] This application also provides a multidimensional data indicator management method based on Spring Boot, which is implemented using the multidimensional data indicator management system based on Spring Boot as described in the previous embodiments. Figure 2 As shown, the steps include: S100, Data Source Connection Configuration: In response to data sources added by users through the management interface of the data source management module, the added data source includes the data source name and its corresponding database connection information; the connection information consists of configuration parameters required to create a connection, including JDBC URL, user ID and password, driver class name, etc. Example 1: Data source name: Inpatient EMR system_Oracle Connection information: JDBC URL: jdbc:oracle:thin:@ / / 192.168.1.101:1521 / emrdb_svc User ID: emr_reader Password: encrypted_password_here Driver class name: oracle.jdbc.OracleDriver; Example 2: Data source name: Test LIS system_SQL Server Connection information: JDBC URL: jdbc:sqlserver: / / 192.168.1.102:1433;databaseName=lis_prod User ID: lis_reader Password: encrypted_password_here Driver class name: com.microsoft.sqlserver.jdbc.SQLServerDriver; After adding, perform a connectivity test to the corresponding database; Once the data source passes the test, initialization is complete. For data sources that pass the test, the data source management module creates a connection object from the actual connection established between the data source and the corresponding database, generates a data source ID, and saves the data source name, data source ID, database name, and connection object as data source information to the connection pool for use in subsequent steps. If the test fails, the data source addition fails, and the failed data source is deleted.

[0023] S200, Data Indicator Definition: In response to the user's selection of data source information and corresponding SQL query statement through the visualization interface of the indicator definition module, the SQL query statement is validated, and an indicator definition is created based on the selected data source information and the corresponding SQL query statement when the validation is successful. The created indicator definition includes indicator ID, indicator description, and SQL query calculation formula. The SQL query calculation formula contains location information associated with the database name in the selected data source information.

[0024] Indicator definition example: Indicator ID: 1987813229677535234; Indicator Description: Workload statistics for medical staff; SQL query calculation formula (SQL query statement): select COUNT(1) from datacenter..workwhere user_id = #{userId}.

[0025] When creating an indicator definition, multiple data sources can be selected simultaneously to create an indicator definition that is associated with multiple data sources. The SQL query calculation formula contains location information that is associated with the database names of the selected multiple data sources, thereby enabling "cross-data source" queries when performing subsequent query calculations.

[0026] S300, Task Scheduling Configuration: In response to the task plan preset by the user through the task scheduling module, a scheduling strategy is generated and the task status is recorded. The task plan includes the task name, data source (including database name and table name), execution frequency (using corn expressions, implemented through Quartz or Spring Scheduler), mapping table configuration, and mapping field configuration.

[0027] Task planning example: Task Name: Healthcare Workload Aggregator Data source: ["emrdb_svc..work", "lis_prod..work"] Execution frequency: 0 0 0-1 * * ? * Mapping table configuration: { “datacenter”:[“emrdb_svc..work”,“lis_prod..work”]} Mapping field configuration: { “emrdb_svc..work”:{ "emrdb_svc_name":"data_center_name", "lis_prod_id":"data_center_id" }, “lis_prod..work”:{ "lis_prod_name":"data_center_name", "lis_prod_id":"data_center_id" } }

[0028] S400, Data Execution and Aggregation: When the task scheduling module triggers execution according to the scheduling policy, it reads the data source in the task plan, obtains the connection object from the connection pool according to the database name in the data source, establishes a connection with the corresponding database through the obtained connection object, and obtains the original data from the database with the established connection according to the data table name in the data source.

[0029] The task scheduling module synchronizes the table names and fields of the original data it acquires through mapping table configuration and mapping field configuration. After synchronization, the data is stored in a unified central database. The mapping table configuration and mapping field configuration are used to determine the mapping relationship between the synchronized data and its original source database, thus providing a unified data model foundation for subsequent more complex cross-index related queries and aggregation queries.

[0030] Based on the metric calculation request issued by the user, the system calls the corresponding metric definition, performs the query calculation from the central database (which has already aggregated all the original data) according to the SQL query calculation formula in the called metric definition, stores the calculated metric result in Redis or a specified database, and updates the task status.

[0031] When the data sources in the preset task plan include multiple database names and multiple table names, and each database name corresponds to at least one table name, the task scheduling module executes in parallel based on each database name: obtaining connection objects from the corresponding data source information from the connection pool, establishing a connection with the corresponding database through the obtained connection objects, and retrieving the original data from the database with the established connection based on the table name corresponding to the database name; in the parallel execution branch, when a database name corresponds to multiple table names, the original data is retrieved from the database with the established connection sequentially based on the table name in that parallel execution branch.

[0032] The data aggregation method is preset by the user through the task scheduling module. The data aggregation methods include one or more of the following: basic aggregation, group aggregation, window function, and custom aggregation. Basic aggregation includes one or more of the following: SUM, COUNT, AVG, MAX, and MIN. Group aggregation includes multi-dimensional grouping by GROUP BY. Window functions include window functions that support the OVER clause. Custom aggregation includes aggregation logic defined by the user through the task scheduling module.

[0033] S500, Data Display and Export: The data display and export module reads the indicator results, and the visualization component displays the results in charts according to the time range selected by the user through the data display and export module, such as... Figures 3-5 As shown.

[0034] The data visualization and export module responds to the export operation performed by the user through this module, generating and exporting files in Excel and / or PDF formats, such as... Figure 6 As shown.

[0035] Preferably, the task scheduling module records the task status to the monitoring log starting from S300, updates the monitoring log according to the task execution process, records the task execution results in the monitoring log, records task execution error information in the monitoring log, and provides the monitoring log to the data display and export module for users to retrieve, view and / or export.

[0036] In the above steps, the permission management module can be used to control the permissions of different users for different operations.

[0037] The above description is only a preferred embodiment of this application and is not intended to limit this application. Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application.

Claims

1. A multi-dimensional data indicator management system based on Spring Boot, characterized in that, The multi-dimensional data index management system based on Spring Boot comprises the following modules: a data source management module for data source configuration, management of connection configuration of multiple types of databases, and provision of data source connectivity test and connection pool management functions; an index definition module for providing a visual interface for users to define data indexes and write SQL query statements, and to complete index definition creation; a task scheduling module for providing task plan setting functions, implementing a timing task plan based on Quartz or Spring Scheduler, aggregating data from multiple databases to a unified central database according to a user-preset task plan, and calling the query logic in the pre-created index definition to perform cross-data source query calculation tasks in the central database and store the calculated index results in a specified location; a data display and export module for reading the calculated index results and displaying them in the form of visual components, and supporting the export of the results into files in multiple formats.

2. The Spring Boot based multi-dimensional data indicator management system as claimed in claim 1, wherein, The multiple types of databases include multiple types of databases in MySQL, Oracle, SQL Server, and Hive. 3.The Spring Boot based multi-dimensional data indicator management system according to claim 1, wherein, The visual components are integrated with ECharts.

4. The Spring Boot based multi-dimensional data indicator management system as claimed in claim 1, wherein, The system further comprises a permission management module for implementing user role and data permission control based on Spring Security.

5. A method for managing multi-dimensional data indicators based on Spring Boot, characterized by, The multi-dimensional data index management system based on Spring Boot is implemented by the following steps: S100, data source connection configuration: in response to a data source added by a user through a management interface of the data source management module, wherein the added data source comprises a data source name and corresponding database connection information, and the connection information comprises a JDBC URL, a user ID, a password, and a driver class name; performing a connectivity test on the corresponding database; for a data source that passes the test, the data source management module creates an actual connection established with the corresponding database as a connection object, generates a data source ID, and saves the data source name, data source ID, database name, and connection object as data source information to a connection pool; S200, data index definition: in response to data source information selected by a user through a visual interface of the index definition module and a corresponding written SQL query statement, verifying the SQL query statement, and creating an index definition based on the selected data source information and the corresponding written SQL query statement when the verification is passed; wherein the created index definition comprises an index ID, an index description, and a SQL query calculation formula; the SQL query calculation formula contains positioning information associated with the database name in the selected data source information; S300, task scheduling configuration: in response to a task plan preset by a user through the task scheduling module, generating a scheduling strategy, and recording a task state; the task plan comprises a task name, a data source, an execution frequency, a mapping table configuration, and a mapping field configuration, and the data source comprises a database name and a data table name; S400, data execution and aggregation: When the task scheduling module is triggered to execute according to the scheduling strategy, the data source in the task plan is read, the connection object in the corresponding data source information is obtained from the connection pool according to the database name in the data source, the connection with the corresponding database is established through the obtained connection object, and the original data is obtained from the database with which the connection is established according to the data table name in the data source; The task scheduling module synchronizes the table name and the field of the obtained original data through the mapping table configuration and the mapping field configuration, and stores the synchronized data in the unified central database; wherein the mapping table configuration and the mapping field configuration are used to determine the mapping relationship between the synchronized data and the database of the original source thereof; According to the index calculation demand issued by the user, the index definition corresponding to the index calculation demand is called, the SQL query calculation formula in the called index definition is used to query and calculate from the central database, and the calculated index result is stored in Redis or a specified database, and the task state is updated; S500, data display and export: The index result is read through the data display and export module, and the chart display of the index result is performed through the visual component according to the time range selected by the user through the data display and export module; The data display and export module generates and exports an Excel and / or PDF format file in response to the export operation performed by the user through the data display and export module.

6. The Spring Boot based multi-dimensional data indicator management method of claim 5, wherein, When the data source in the preset task plan includes multiple database names and multiple data table names, and each database name corresponds to at least one data table name, the task scheduling module performs the following operations in parallel based on each database name: obtaining the connection object in the corresponding data source information from the connection pool, establishing the connection with the corresponding database through the obtained connection object, and obtaining the original data from the database with which the connection is established according to the data table name corresponding to the database name; in the parallel execution branch, when one database name corresponds to multiple data table names, the original data is obtained from the database with which the connection is established according to the data table names in sequence in the parallel execution branch.

7. The Spring Boot based multi-dimensional data indicator management method of claim 5, wherein, The task scheduling module stores the obtained original data in the unified central database and performs data aggregation and mapping table configuration and mapping field configuration; The data aggregation mode is preset by the user through the task scheduling module, and the data aggregation mode includes one or more of basic aggregation, grouping aggregation, window function, and custom aggregation; the basic aggregation includes one or more of SUM, COUNT, AVG, MAX, and MIN; the grouping aggregation includes GROUP BY multi-dimensional grouping; the window function includes a window function supporting an OVER clause; the custom aggregation includes the aggregation logic customized by the user through the task scheduling module. 8.The Spring Boot-based multidimensional data indicator management method of claim 5, wherein, Multiple data source information can be selected simultaneously when creating the index definition, so as to create the index definition associated with multiple data source information, and the SQL query calculation formula in the index definition contains positioning information associated with each database name in the selected multiple data source information. 9.The Spring Boot-based multidimensional data indicator management method of claim 5, wherein, The execution frequency adopts a corn expression, and is implemented through Quartz or Spring Scheduler.

10. The Spring Boot based multi-dimensional data indicator management method of claim 5, wherein, The task scheduling module records the task state from S300 to the monitoring log, updates the monitoring log according to the execution process of the task, and records the task execution result in the monitoring log; and records the task execution error information in the monitoring log. The monitoring log is provided to the data display and export module for the user to call, view and / or export.

Images