An active data integration method, system, medium, device and terminal

Abstract

The present application belongs to the technical field of data integration, data fusion and software architecture, and discloses an active data integration method, system, medium, equipment and terminal. A data source monitoring module is used to monitor a database accessed by a business system, and an active data integration service is required to integrate data modules accessed by the business system into an extensible integrated database. Preliminary integration, fusion and sharing of data are realized according to changes in user request data monitored. The active data integration system comprises an active data integration framework, an integration method based on a micro-service strategy and a data interface for providing agile data integration, extensible local database support, active data integration, micro-service access for composite data access, lightweight data relationship mapping and reverse data structure relationship support, and can maximize the guarantee of the premise of free development of business systems by each company, realize data integration and sharing of each business system and accelerate the development speed of the business system.

Description

Technical Field

[0001] The present invention belongs to the technical field of data integration, data fusion and software architecture, specifically to a business system data integration, fusion and sharing system in a specific field, and especially to an active data integration method, system, medium, equipment and terminal. Background Art

[0002] Currently, with the development of information, digital, and intelligent technologies, various business systems in the pharmaceutical sector are being widely deployed and applied, placing new demands on data integration and sharing, as well as agile system development. On the one hand, data integration and sharing between business systems deployed at different companies and at different times are required; on the other hand, agile and rapid data integration and shared access support is required for new business systems to accelerate business system deployment and application.

[0003] There are two main approaches to existing data integration technology: data warehouses and virtual integration systems. Data warehouses extract, transform, and load data from different data sources and store it in a data warehouse. The data structures of the data sources and the data warehouse, as well as the mapping relationships between them, are analyzed and stored to ensure correct mapping between data sources and data in the data warehouse, eliminating semantic conflicts and data redundancy. Business systems access data through the data warehouse, providing each business system with a global view of the data. Virtual integration systems rewrite business system queries when they are accessed, converting global data access into operations on different data sources. Wrappers are used to forward different query requests to the relevant data sources and the results returned by the data sources to upper-level applications. Data access is performed through virtual warehouses, allowing business systems to access different data sources through wrappers. Clearly, these two approaches primarily aim to provide data analysis and mining services through data integration, rather than supporting shared data access across different businesses. These approaches are unable to meet the current deployment and development needs of specific business systems, particularly in the pharmaceutical sector. This is reflected in the following aspects:

[0004] First, it fails to effectively support data sharing between business systems. Business systems within specific pharmaceutical sectors increasingly require the development of data interfaces between different systems to facilitate data sharing. This not only provides data support for other business systems, but also avoids duplicate development of the same business. Traditional approaches to integrating two types of data, derived from the integration of data from different data sources, provide data access primarily for analytical queries and lack effective support for data access across different business systems. In short, existing technologies struggle to understand data sources and formats across diverse business systems developed over time and space, making traditional data integration and sharing approaches ineffective.

[0005] Secondly, it cannot effectively support optimizing data access services based on business access patterns. The data access performance of business systems is often load-sensitive, and different access patterns can have significant differences in access performance for different data loads. However, data warehouse-based and virtual integration systems do not optimize data access based on the access patterns of business systems, making it difficult to effectively support load-sensitive data access services. In short, existing traditional business data integration technologies cannot effectively support data sharing across different business systems, but instead provide support for specific data analysis or mining services. Different business systems still directly access different local data sources, while data integration is specifically designed for those who need to conduct global data analysis and mining.

[0006] Third, it fails to provide data architecture support for agile development of business systems. Integrated query and access to existing data is a critical requirement in specific business areas, but access to business data varies from company to company, database to database, and from different time periods, making it difficult to provide agile data access architecture support for newly developed business systems. In short, existing technologies, such as traditional data sharing and access methods provided through services or interfaces, are unable to effectively meet the changing needs of business over time.

[0007] Through the above analysis, the problems and defects of the existing technology are as follows:

[0008] (1) Existing data integration technologies cannot effectively meet the needs of data sharing between business systems, and are difficult to meet the deployment and development of business systems in specific fields, especially in the medical field.

[0009] (2) Existing data integration technologies cannot effectively support the optimization of data access services based on business access patterns, and are unable to effectively support load-sensitive data access services. Traditional integration methods focus more on providing global data for query and analysis of business data in specific areas, rather than providing optimized data access strategies for different business systems. In other words, it is difficult for different business systems to obtain more optimized data access strategies from data integration, and for load-sensitive business systems, data access optimization cannot be performed based on data access patterns.

[0010] (3) There are barriers to accessing business data from different companies, different databases, and different periods. Existing data integration technologies fail to provide data architecture support for the agile development of business systems. This is mainly manifested in the following ways: first, traditional data integration methods have poor automation levels and are difficult to support the agile development and deployment of business systems with localized data views; second, traditional data integration methods have their own advantages and disadvantages and are difficult to effectively meet the rapidly changing needs of businesses in specific areas. Data warehouses require data relationship mapping in advance, and virtual integration systems need to understand the access strategies of different data sources in advance. These are both good support for the integration of established business data, but difficult to provide good support for rapidly changing businesses. Summary of the Invention

[0011] In response to the problems existing in the prior art, the present invention provides an active data integration method, system, medium, device and terminal, and in particular relates to an active data integration method, system, medium, device and terminal based on the field of hospital informatization.

[0012] The present invention is implemented as follows: an active data integration method, the active data integration method comprising:

[0013] For existing business systems, the data source monitoring module is used to monitor the database accessed by the business system, and the active data integration service is used to integrate the data modules accessed by the business system into the scalable integrated database, and the initial integration, fusion and sharing of data are achieved based on the monitored user request data changes.

[0014] Furthermore, the active data integration method further includes:

[0015] Scalable local database support, active data integration architecture, microservice access for composite data access, lightweight data relationship mapping, and reverse data structure relationship support.

[0016] Furthermore, the active data integration method comprises the following steps:

[0017] Step 1: Use the data integration framework for business systems under development or in pre-development, and utilize the active data integration service and business access performance service to record the data request commands and request processes of the business systems.

[0018] Step 2: Active data integration services generate multiple microservices based on user requests using a microservices architecture approach, and implement data integration between business system data and a globally scalable integrated database, as well as local temporary access to data views.

[0019] Step 3: Based on user access patterns and business access performance, optimize the data method and provide high-performance data access services by generating microservices.

[0020] Step 4: Provide integrated database metadata and relationship services to business systems through the extended access framework, and provide rapid data integration services to business systems under development or in pre-development.

[0021] Step five: The databases of different business systems in specific fields are integrated and merged with the integrated data database at the bottom layer to form a flat structure, providing data access and data sharing services for upper-level applications.

[0022] Furthermore, the active data integration method further includes:

[0023] (1) Based on the data connection of the business system, the active data integration system generates a service to connect to the local database of the business system and completes the connection to the local database of the business system;

[0024] (2) Using the service generated in step (1), the system actively completes the database replication corresponding to the business system, and integrates the databases of different business systems into a global database;

[0025] (3) Record the data access process and content of the business system.

[0026] Furthermore, the active data integration method further includes:

[0027] (1) Generate data access microservices based on the recorded data access process and content;

[0028] (2) Convert access to different databases or different tables into local access that depends on step (1), and convert cross-table and cross-database access into generating and accessing temporary local tables through microservices;

[0029] (3) The integrated system uses microservice strategies to achieve access to the local database of the business system and the global integrated database.

[0030] Furthermore, the active data integration method further includes:

[0031] (1) The system generates a data relationship diagram by forcing developers to specify the data access process and access content, and the database connection method;

[0032] (2) The new business system developer applies for connection and access to existing data through the data relationship diagram generated in step (1), and / or generates a new data access microservice through the data relationship diagram;

[0033] (3) The entire system optimizes services based on user access patterns; the entire system and method are applied to the medical field, but are not limited to the medical field.

[0034] Another object of the present invention is to provide an active data integration system that applies the active data integration method. The active data integration system includes an extended access framework, active data integration services, business access performance services, a data source monitoring module, metadata and relationship services, an extensible integrated database, and a data interface.

[0035] The extended access framework is used to provide integrated database metadata and relationship services to the business system, and record the business system's connection, access and result information to the database; the business system includes the research and development business system and the pre-research business system;

[0036] The active data integration service is used to integrate data modules accessed by business systems into an extensible integrated database, including MySQL, Oracle, Sql Server and XML;

[0037] The business access performance service is used to optimize data methods and provide high-performance data access services by generating microservices;

[0038] The data source monitoring module is used to monitor the database accessed by the business system;

[0039] The data interface is used to provide agile data integration.

[0040] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0041] For existing business systems, the data source monitoring module is used to monitor the database accessed by the business system, and the active data integration service is used to integrate the data modules accessed by the business system into the extensible integrated database, and the initial integration, fusion and sharing of data are achieved based on the data changes requested by the monitored user.

[0042] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0043] For existing business systems, the data source monitoring module is used to monitor the database accessed by the business system, and the active data integration service is used to integrate the data modules accessed by the business system into the extensible integrated database, and the initial integration, fusion and sharing of data are achieved based on the data changes requested by the monitored user.

[0044] Another object of the present invention is to provide an information data processing terminal, which is used to implement the active data integration system.

[0045] In combination with the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solutions to be protected by the present invention from the following aspects:

[0046] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty of solving these problems, this paper closely combines the technical solutions to be protected by the present invention and the results and data during the research and development process, and analyzes in detail and in depth how the technical solutions of the present invention solve the technical problems and some creative technical effects brought about by solving the problems. The specific description is as follows:

[0047] The active data integration system provided by the present invention includes an active data integration framework, an integration method based on microservices strategies, and a data interface that provides agile data integration. The system primarily features scalable local database support, active data integration, microservice access for composite data access, lightweight data relationship mapping, and support for reverse data structure relationships. Furthermore, the active data integration system provided by the present invention is based on the pharmaceutical field and includes an active data access framework, an active data integration shared access method supporting local and global data sources, and a data access interface that supports agile development.

[0048] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by the present invention are described in detail as follows:

[0049] The active data integration system provided by the present invention can realize data integration and sharing of various business systems under the premise of ensuring that each company can freely develop its business system to the greatest extent, and provide simple and effective data access support for the subsequent development of new business systems, so as to speed up the development of business systems and data integration and sharing.

[0050] Third, as auxiliary evidence for the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0051] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are:

[0052] After the technical solution of the present invention is transformed, on the one hand, it can reduce the data integration and development costs for business development companies; on the other hand, it can provide rapid and continuous data integration development for industry business systems, reducing business development and management costs. By utilizing the results of this invention, it is expected that the cost of business system development companies and business entrusting units can be reduced by about 20% respectively (according to project management: module development such as data management and development accounts for more than 50% of the total development cost), and it will increase as the current data volume increases. In addition, after the technical solution of the present invention is transformed, it also brings great convenience to data integration and sharing within specific industry fields, is conducive to data integration and sharing and secondary development within specific industry fields, provides great functional and performance convenience for mining and analyzing data, and improves the data reuse value within the industry field.

[0053] (2) The technical solution of the present invention solves the technical problems that people have been eager to solve but have never been able to solve successfully:

[0054] The technical solution of the present invention provides a new idea for data integration and sharing of business systems within the industry field, and solves the problem of data sharing among different business systems. Whether it is the data integration and sharing previously provided through data warehouses, data virtual integration systems, etc., or the data access services for different business systems provided by providing standard interfaces, data services, and secondary development services in some practical applications, there are problems such as these interfaces and services are static and difficult to dynamically respond to changing business needs. The technical solution of the present invention fully takes into account the dynamic changes and scalability of real-world application needs, realizes a dynamic registration mechanism for data access among different business systems, and realizes the scalability and dynamic adaptability of data integration services. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0056] Figure 1 is a flow chart of the active data integration method provided by an embodiment of the present invention;

[0057] Figure 2 This is a structural block diagram of the active data integration system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] In view of the problems existing in the prior art, the present invention provides an active data integration method, system, medium, device and terminal. The present invention is described in detail below with reference to the accompanying drawings.

[0060] 1. Explanatory Examples In order to enable those skilled in the art to fully understand how to implement the present invention, this section provides an illustrative example that expands upon the technical solutions of the claims.

[0061] like Figure 1 As shown, the active data integration method provided by the embodiment of the present invention includes the following steps:

[0062] S101: Use the data integration framework for the business system under development or in pre-development, and use the active data integration service and business access performance service to record the data request command and request process of the business system respectively;

[0063] S102, the active data integration service generates multiple microservices based on user requests using a microservice architecture approach, and implements data integration between business system data and a globally scalable integrated database, as well as local temporary access to data views;

[0064] S103, optimizing data access performance based on user access patterns and business access performance, and providing high-performance data access services by generating microservices;

[0065] S104: Provide integrated database metadata and relationship services to business systems through an extended access framework, and provide rapid data integration services to business systems under development or in pre-development.

[0066] S105, the databases of different business systems in specific fields are integrated and merged with the integrated data database at the bottom layer to form a flat structure, providing data access and data sharing services for upper-level applications.

[0067] like Figure 2 As shown, the active data integration system provided by the present invention includes an active data integration framework, an integration method based on microservices strategies, and a data interface that provides agile data integration. Its key features include scalable local database support, active data integration, microservice access for composite data access, lightweight data relationship mapping, and support for reverse data structure relationships.

[0068] Among them, the active data integration system includes an active data access framework, an active data integration shared access method supported by local and global data sources, and a data access interface supported by agile development.

[0069] The data extension access framework provided by the embodiment of the present invention can record information such as the connection, access and results of the business system to the database;

[0070] (a) Based on the data connection of the business system, the active data integration system generates a service for connecting to the local database of the business system and completes the connection to the local database of the business system;

[0071] (b) Using the services generated in (a), the system actively completes the database replication corresponding to the business system and integrates the databases of different business systems into a global database;

[0072] (c) Record the data access process and content of the business system.

[0073] The active data integration system provided by an embodiment of the present invention further includes:

[0074] (a) Generate data access microservices based on the recorded data access process and content;

[0075] (b) Convert access to different databases or different tables into local access that depends on (a). Specifically, convert cross-table and cross-database access into temporary local tables generated and accessed through microservices.

[0076] (c) The integration system uses microservice strategies to achieve access to the local database of the business system and the global integration database.

[0077] The active data integration system provided by an embodiment of the present invention further includes:

[0078] (a) The system can generate a data relationship diagram by forcing developers to specify the data access process and access content, the database connection method, etc.

[0079] (b) New business system developers can apply for connection and access to existing data through the data relationship diagram generated in (a), or generate new data access microservices through the data relationship diagram;

[0080] (c) The entire system can optimize services based on user access patterns;

[0081] (d) The entire system and method are applicable to the medical field, but are not limited to the medical field.

[0082] 2. Application Examples: In order to demonstrate the creativity and technical value of the technical solution of the present invention, this section provides application examples of the claimed technical solution on specific products or related technologies.

[0083] The present invention is mainly aimed at data integration and sharing in specific fields and industries, and has good applications in business systems under research and pre-research, and also provides solutions for business systems that have been developed.

[0084] (1) For the business system that has been developed, the data source monitoring module in the attached figure is used to monitor the data source of the business system and obtain the operation records of the data source. Currently, three methods are used: one is to directly monitor the logs and other contents of the data source, such as Sql Server, Mysql, etc.; the other is to use an intermediate proxy server, using network monitoring and proxy switching technology to achieve monitoring of the data access process. At the same time, the data integration between the data source and the integrated data database is completed, and the data relationship mapping between different data sources is completed through the monitoring content. The data access service is registered in the active data integration service in the attached figure, and the metadata and relationship services in the attached figure are notified to register.

[0085] (2) For business systems under development or in pre-development, the extended access framework in the appendix is ​​mandatory for data access. The access framework is used to record the access process, generate microservices based on the access process, and complete metadata and relationship service registration. This module mainly uses microservice architecture technology to dynamically support the innovation of microservices based on the data access method of the business system.

[0086] The invention solution provides better support for data integration of business systems under development and pre-development. Through this method, the data access of business systems can be uniformly registered in the system of the invention. It can provide services for the following scenarios:

[0087] First, when a business system under research or pre-research hopes to obtain data support from other business systems, there is no need for other business systems to provide interfaces or data.

[0088] Second, when existing data content is needed during the development of a business system under research or pre-research, it is only necessary to use the microservice interface to obtain the data and perform local processing.

[0089] Third, when the third line needs to mine and analyze data within the entire industry field, it only needs to process the integrated data. It can even provide data mining and analysis services by combining the use of previously registered microservice data access interfaces and independently added microservice data access interfaces.

[0090] Fourth, by monitoring the data access patterns of the business system, access performance can be improved by adding temporary tables and other means on the local data source side.

[0091] 3. Evidence of the effects of the embodiments: The embodiments of the present invention have achieved some positive effects during the development or use process, and indeed have great advantages over the existing technology. The following content describes them with reference to the data, charts, etc. of the experimental process.

[0092] Functionally, this invention offers breakthrough improvements over data warehouses, integrated virtual systems, and traditional interface or service approaches. First, it autonomously generates data access interfaces based on business systems, providing data access support for other business systems. Second, it can transfer access to local business data sources to the solution provided by this invention, optimizing the data access performance of existing business systems based on access patterns. Third, it can provide minimal-cost data integration and sharing support for business systems of different companies at different times. These capabilities are difficult to achieve using traditional integration methods.

[0093] In terms of cost-effectiveness, an analysis of three typical existing business systems reveals that each requires data sharing with other business systems, accounting for 5%, 15%, and 25% of costs, respectively. The later a business system is developed, the greater its reliance on data from earlier systems, resulting in a higher cost. Preliminary estimates suggest that data integration and sharing costs for new business systems under development or pre-development in mature industries will account for approximately 20%. Business units within these sectors are also experiencing increasing demand for data integration and sharing. This is driven by a growing need to aggregate internal data for reporting to industry superiors and a growing demand to leverage integrated data for analysis and new business opportunities.

[0094] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0095] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. An active data integration method, characterized in that: The active data integration method includes: For existing business systems, the data source monitoring module is used to monitor the database accessed by the business system. Active data integration services are used to integrate the data modules accessed by the business system into the scalable integrated database. The initial integration, fusion, and sharing of data are achieved based on the monitored user request data changes. The active data integration method further includes: Scalable local database support, active data integration, microservice access for composite data access, lightweight data relationship mapping, and reverse data structure relationship support; The active data integration method comprises the following steps: Step 1: Use the data integration architecture for business systems under development or in pre-development, and use the active data integration service and business access performance service to record the data request commands and request processes of the business systems respectively; Step 2: Active data integration services generate multiple microservices based on user requests using a microservices architecture approach, and implement data integration between business system data and a globally scalable integrated database, as well as local temporary access to data views. Step 3: Based on user access patterns and business access performance, optimize the data method and provide high-performance data access services by generating microservices. Step 4: Provide integrated database metadata and relationship services to business systems through the extended access framework, and provide rapid data integration services to business systems under development or in pre-development. Step five: The databases of different business systems in specific fields are integrated and merged with the integrated data database at the bottom layer to form a flat structure, providing data access and data sharing services for upper-level applications.

2. The active data integration method according to claim 1, wherein: The active data integration method further includes: (1) Based on the data connection of the business system, the active data integration system generates a service to connect to the local database of the business system and completes the connection to the local database of the business system; (2) Using the service generated in step (1), the system actively completes the database replication corresponding to the business system, and integrates the databases of different business systems into a global database; (3) Record the data access process and content of the business system.

3. The active data integration method according to claim 1, wherein: The active data integration method further includes: (1) Generate data access microservices based on the recorded data access process and content; (2) Convert access to different databases or different tables into local access that depends on step (1), and convert cross-table and cross-database access into generating and accessing temporary local tables through microservices; (3) The integrated system uses microservice strategies to achieve access to the local database of the business system and the global integrated database.

4. The active data integration method according to claim 1, wherein: The active data integration method further includes: (1) The system generates a data relationship diagram by forcing developers to specify the data access process and access content, and the database connection method; (2) The new business system developer applies for connection and access to existing data through the data relationship diagram generated in step (1), and / or generates a new data access microservice through the data relationship diagram; (3) The entire system optimizes services based on user access patterns; the entire system and method are applied to the medical field, but are not limited to the medical field.

5. An active data integration system using the active data integration method according to any one of claims 1 to 4, characterized in that: The active data integration system includes an extended access framework, active data integration services, business access performance services, a data source monitoring module, metadata and relationship services, an extensible integrated database, and a data interface; The extended access framework is used to provide integrated database metadata and relationship services to the business system, and record the business system's connection, access and result information to the database; the business system includes the research and development business system and the pre-research business system; The active data integration service is used to integrate data modules accessed by business systems into an extensible integrated database, including MySQL, Oracle, Sql Server and XML; The business access performance service is used to optimize data methods and provide high-performance data access services by generating microservices; The data source monitoring module is used to monitor the database accessed by the business system; The data interface is used to provide agile data integration.

6. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the active data integration method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the active data integration method according to any one of claims 1 to 4.

8. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the active data integration system as claimed in claim 5.

Images