A data source extension method, device, equipment and medium

Abstract

The application provides a data source extension method and device, equipment and medium. The method obtains data source configuration information, generates a uniform resource locator based on a connection template string, inputs the uniform resource locator and connection pool parameter information into a connection pool library, establishes a connection between a target data source and an application integration platform, which is recorded as a first connection, and if the data source configuration information changes, the first connection is updated according to the changed data source configuration information to realize the extension of the data source. According to the data source configuration information, the application can establish a connection between the target data source and the application integration platform, and when the data source configuration information changes, the first connection is updated according to the changed data source configuration information, thereby improving the adaptability, scalability and management efficiency between the application integration platform and the data source.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and medium for extending a data source. Background Technology

[0002] As the central nervous system of the Industrial Internet platform, the application integration platform facilitates interaction, data storage, and data management between the Industrial Internet platform and numerous internal and external systems. With the widespread use of application integration platforms, they need to support the dynamic access of various heterogeneous data sources. However, different data source types differ significantly in their drivers, connection management, query optimization, and data access methods, making it difficult for traditional integration methods to meet the needs of rapid adaptation, dynamic expansion, and efficient management.

[0003] Therefore, there is an urgent need for an efficient and flexible method to extend data sources in order to improve the system's data adaptability, operational stability, and maintainability. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, this application provides a data source extension method, apparatus, device and medium to solve the technical problems of low data source adaptability, poor scalability and low management efficiency.

[0005] According to one aspect of the embodiments of this application, a data source extension method is provided. The method includes: obtaining data source configuration information; the data source configuration information includes: a target data source type, a connection template string, and connection pool parameter information; the connection template string consists of the server address, server account, and server password of an application integration platform; generating a Uniform Resource Locator (URL) based on the connection template string; inputting the URL and the connection pool parameter information into a connection pool library to establish a connection between the target data source and the application integration platform, denoted as a first connection; if the data source configuration information changes, updating the first connection according to the changed data source configuration information to achieve data source extension.

[0006] In one embodiment of this application, after obtaining the first connection, the method further includes: acquiring connection status information of the first connection and caching the connection status information; the connection status information includes: the number of connections and the connection creation time; if the change in the number of connections is greater than a preset change threshold, then the connection is determined to be abnormal and a connection abnormality alarm is triggered; if the connection creation time is greater than a preset duration threshold, then the connection is determined to be abnormal and a connection abnormality alarm is triggered.

[0007] In one embodiment of this application, if the connection status information further includes a connection identifier, and the data source configuration information further includes a target data source demand tenant identifier and a correspondence between the demand tenant identifier and the connection identifier, then after caching the connection status information, the method further includes: if the connection identifier and the demand tenant identifier have a correspondence, then calling the connection with the connection identifier is allowed; if the connection identifier and the demand tenant identifier do not have a correspondence, then calling the connection with the connection identifier is prohibited.

[0008] In one embodiment of this application, if the data source configuration information further includes: a driver class path of the target data source and data source operation configuration information, then after establishing the first connection, the method further includes: obtaining a structured query language operation implementation class for the target data source; the structured query language operation implementation class is generated based on the driver class path and the data source operation configuration information; obtaining a data source operation type from the data source operation configuration information, performing structured query language conversion on the target data source according to the data source operation type; using the converted target data source as input to the structured query language operation implementation class, and running the structured query language operation implementation class to obtain the operation result of the target data source.

[0009] In one embodiment of this application, the process of generating the Structured Query Language (SML) operation implementation class based on the driver class path and the data source operation configuration information includes: generating a data access object (DME) implementation class based on the driver class path and the data source operation configuration information; registering the DME implementation class to bind the target data source with the data source operation, thereby obtaining the SML operation implementation class of the target data source; and representing the data source operation through the data source operation configuration information.

[0010] In one embodiment of this application, the process of running the Structured Query Language (SCL) operation implementation class further includes: obtaining the running status information of the SCL operation implementation class, the running status information including: execution time, error rate, and slow query ratio; if the execution time exceeds a preset time threshold, then a running abnormality is determined and a running abnormality alarm is triggered; if the error rate exceeds a preset error rate threshold, then a running abnormality is determined and a running abnormality alarm is triggered; if the slow query ratio exceeds a preset query ratio, then a running abnormality is determined and a running abnormality alarm is triggered.

[0011] In one embodiment of this application, the data source operation includes at least one of: querying data, pagination preview, table structure query, adding data, deleting data, and modifying data.

[0012] According to one aspect of the embodiments of this application, a data source extension device is provided, comprising: an information acquisition module, configured to acquire data source configuration information; the data source configuration information includes: a target data source type, a connection template string, and connection pool parameter information; the connection template string consists of a server address, server account, and server password of an application integration platform; a connection establishment module, configured to generate a Uniform Resource Locator (URL) based on the connection template string; input the URL and the connection pool parameter information into a connection pool library to establish a connection between the target data source and the application integration platform, denoted as a first connection; and a connection update module, configured to update the first connection according to the changed data source configuration information if the data source configuration information changes, thereby realizing the extension of the data source.

[0013] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the data source extension method as described above.

[0014] According to one aspect of the embodiments of this application, a readable storage medium is provided that stores computer-readable instructions thereon, which, when executed by a computer's processor, cause the computer to perform the data source extension method as described above.

[0015] The beneficial effects of this application are as follows: This application obtains data source configuration information, generates a Uniform Resource Locator (URL) based on the connection template string, inputs the URL and connection pool parameter information into the connection pool library, and establishes a connection between the target data source and the application integration platform, denoted as the first connection. If the data source configuration information changes, the first connection is updated according to the changed data source configuration information to achieve data source expansion. The above process establishes a connection between the target data source and the application integration platform based on the data source configuration information, and updates the first connection according to the changed data source configuration information when the data source configuration information changes, thereby improving the adaptability, scalability, and management efficiency between the application integration platform and the data source.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0018] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application;

[0019] Figure 2 This is a flowchart illustrating a data source expansion method in an exemplary embodiment of this application;

[0020] Figure 3 This is a functional block diagram illustrating dynamic expansion of data sources based on an application integration platform, as shown in an exemplary embodiment of this application.

[0021] Figure 4 This is a technical architecture diagram based on an application integration platform, illustrated in an exemplary embodiment of this application;

[0022] Figure 5 This is a schematic diagram illustrating the data source visualization effect based on an application integration platform, as shown in an exemplary embodiment of this application.

[0023] Figure 6 This is a block diagram illustrating a data source extension device in an exemplary embodiment of this application;

[0024] Figure 7 This is a schematic diagram of the structure of a computer system for an electronic device, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0025] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0026] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0027] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0028] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0029] Reference Figure 1 As shown, the system architecture may include a storage device 101 and an electronic device 102. The electronic device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, or a neural network computer. Those skilled in the art can use the electronic device 102 to obtain data source configuration information, generate a Uniform Resource Locator (URL) based on a connection template string, input the URL and connection pool parameter information into a connection pool library, and establish a connection between the target data source and the application integration platform, designated as the first connection. If the data source configuration information changes, the first connection is updated according to the changed data source configuration information. The storage device 101 is used for data source configuration information. In this embodiment, the storage device 101 uses Random Access Memory (RAM) or Read-Only Memory (ROM) for storage and provides the information to the electronic device 102 for processing.

[0030] Indicatively, after obtaining the data source configuration information stored in the storage device 101, the electronic device 102 generates a Uniform Resource Locator (URL) based on the connection template string, inputs the URL and connection pool parameter information into the connection pool library, and establishes a connection between the target data source and the application integration platform, denoted as the first connection. If the data source configuration information changes, the first connection is updated according to the changed data source configuration information. The above process establishes a connection between the target data source and the application integration platform based on the data source configuration information, and updates the first connection according to the changed data source configuration information when the data source configuration information changes, thereby improving the adaptability, scalability, and management efficiency between the application integration platform and the data source.

[0031] It should be noted that the data source extension method provided in this application embodiment is generally executed by electronic device 102, and correspondingly, the data source extension device is generally disposed in electronic device 102.

[0032] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0033] Figure 2 This is a flowchart illustrating a data source expansion method in an exemplary embodiment of this application. This data source expansion method can be executed by a computing processing device, which may be... Figure 1 The electronic device 102 shown is illustrated. (Refer to...) Figure 2 As shown, this data source extension method includes at least steps S210 to S230, which are described in detail below:

[0034] In step S210, data source configuration information is obtained. In one embodiment of this application, the data source configuration information includes: target data source type, connection template string, and connection pool parameter information; the connection template string consists of the server address, server account, and server password of the application integration platform. Target data source types include: MySQL (MyStructured Query Language) database, Oracle database, SQL Server database, PostgreSQL (Postgres Structured Query Language) database, TiDB (Ti Database), etc. Connection pool parameter information includes maximum number of connections, minimum number of connections, maximum connection time limit, etc. The data source configuration information also includes: the name of the target data source, the version number of the target data source, etc. The target data source type is composed of 0-127 bits of encoding, possessing global uniqueness and immutability.

[0035] In one embodiment of this application, before obtaining the data source configuration information, it is necessary to configure, manage and maintain the data source configuration information in the configuration center. The configuration center includes Nacos (Naming and Configuration Service), Apollo (Apollo Configuration Center) or other custom configuration components.

[0036] In step S220, a Uniform Resource Locator (URL) is generated based on the connection template string; the URL and connection pool parameter information are input into the connection pool library to establish a connection between the target data source and the application integration platform, denoted as the first connection. In one embodiment of this application, the URL includes: protocol, hostname, port, path, etc., and the connection pool library can be the HikariCP connection pool library, or the Druid connection pool library, etc.

[0037] In step S230, if the data source configuration information changes, the first connection is updated according to the changed data source configuration information to extend the data source. In one embodiment of this application, the changes in data source configuration information include changes in at least one of the target data source type, connection template string, and connection pool parameter information. If the target data source type changes, a connection between the changed target data source and the application integration platform is established by inputting the Uniform Resource Locator (URL) and connection pool parameter information into the connection pool library. If the connection template string changes, the URL changes, and a connection between the target data source and the application integration platform is established by inputting the changed URL and connection pool parameter information into the connection pool library. If the connection pool parameter information changes, a connection between the target data source and the application integration platform is established by inputting the URL and the changed connection pool parameter information into the connection pool library. Other cases where the data source configuration information changes are not listed here.

[0038] In one embodiment of this application, the listener component of the configuration center can monitor the data source configuration information in real time. When the data source configuration information changes, it triggers the hot refresh logic of the data source connection instance, dynamically loads the updated data source connection instance, and ensures the consistency and real-time performance of the data source configuration information during the application integration platform operation.

[0039] In one embodiment of this application, a connection between the target data source and the application integration platform is established based on the data source configuration information, and when the data source configuration information changes, the first connection is updated based on the changed data source configuration information, thereby improving the adaptability, scalability and management efficiency between the application integration platform and the data source.

[0040] In one embodiment of this application, after obtaining the first connection, the data source expansion method further includes:

[0041] The connection status information of the first connection is obtained and cached. In one embodiment of this application, connection status information is collected in real time using real-time monitoring and alerting tools such as the Prometheus component. After collection, the connection status information of the first connection is cached using a caching framework (e.g., Java Caffeine). Subsequent requests directly schedule connection resources (e.g., connection identifier) ​​through the cache, thereby reducing database connection overhead. The connection status information includes: number of connections, connection creation time, connection identifier, number of active threads, number of idle connections, and acquisition time, etc.

[0042] If the change in the number of connections exceeds a preset threshold, a connection anomaly is determined, and a connection anomaly alarm is triggered. In one embodiment of this application, the preset threshold is set according to actual conditions, and the alarm information is distributed to multiple channels such as industrial internet platforms and emails through Alertmanager to ensure rapid fault response. Alertmanager is an open-source alarm management and notification routing system, typically used in conjunction with monitoring tools (e.g., Prometheus components) for centralized processing, deduplication, grouping, and sending of alarm notifications.

[0043] If the connection creation time exceeds a preset time threshold, the connection is deemed abnormal, and a connection abnormality alarm is triggered. In one embodiment of this application, the preset time threshold is set according to actual conditions, for example, 3 seconds. The alarm information will be distributed to multiple channels such as industrial internet platforms and emails through Alertmanager to ensure rapid fault response. Alertmanager is an open-source alarm management and notification routing system, typically used in conjunction with monitoring tools (e.g., Prometheus components) for centralized processing, deduplication, grouping, and sending of alarm notifications.

[0044] In one embodiment of this application, if the connection state information further includes a connection identifier, and the data source configuration information further includes a target data source demand tenant identifier and a correspondence between the demand tenant identifier and the connection identifier, then after caching the connection state information, the data source extension method further includes:

[0045] If there is a correspondence between the connection identifier and the demand tenant identifier, then invoking the connection with the connection identifier is permitted. In one embodiment of this application, when there is a correspondence between the connection identifier and the demand tenant identifier, it indicates that the demand tenant has access rights to the connection identifier. If the connection identifier can establish a correspondence with multiple demand tenant identifiers simultaneously, it indicates that multiple demand tenants have access rights to the same connection identifier. The access rights include: selection, insertion, update, deletion, creation, etc.

[0046] If there is no correspondence between the connection identifier and the demand tenant identifier, then calling the connection with the connection identifier is prohibited. In one embodiment of this application, when there is a correspondence between the connection identifier and the demand tenant identifier, it indicates that the demand tenant does not have access to the connection identifier. Through the correspondence between the connection identifier and the demand tenant identifier, multi-tenant scenarios are supported, and configuration management and access control functions for tenant isolation are provided.

[0047] In one embodiment of this application, if the data source configuration information further includes: the driver classpath of the target data source and data source operation configuration information, then after establishing the first connection, the data source extension method further includes:

[0048] The structured query language (SCL) operation implementation class of the target data source is obtained. In one embodiment of this application, the SCL operation implementation class is generated based on the driver classpath and data source operation configuration information. The process of generating the SCL operation implementation class based on the driver classpath and data source operation configuration information includes: generating a data access object (DAO) implementation class based on the driver classpath and data source operation configuration information; registering the DAO implementation class to bind the target data source with the data source operation, thereby obtaining the SCL operation implementation class of the target data source. The driver classpath is used to indicate the download address or acquisition path of the database driver, and the download address or acquisition path supports remotely pulling the database driver and dynamically loading it.

[0049] Furthermore, the data source operation type is obtained from the data source operation configuration information, and a structured query language (SCL) conversion is performed on the target data source based on the data source operation type. In one embodiment of this application, the data source operation includes at least one of: querying data, pagination preview, table structure query, adding data, deleting data, and modifying data. The process of performing SCL conversion on the target data source based on the data source operation type includes: determining the adapter type based on the data source operation type and the target data source type; and performing SCL conversion on the target data source using the corresponding type of adapter. Adapter types include: MySQL adapter, Oracle adapter, DM adapter, etc. The process of performing SCL conversion on the target data source using the corresponding type of adapter includes: dynamically loading the corresponding type of adapter using dynamic proxy technology, and injecting the adapter into the runtime environment of the SCL operation implementation class through a plug-in injection mechanism. The corresponding type of adapter loads the database driver according to the driver class path registered in the configuration center, obtains the target data source through the database driver, and implements the SCL conversion operation for the target data source.

[0050] The transformed target data source is used as input to a Structured Query Language (SCL) operation implementation class, which is then run to obtain the operation results from the target data source. In one embodiment of this application, the operation results from the target data source include: querying the target data source to obtain query data; performing a paginated preview of the target data source to obtain paginated preview results; performing a table structure query on the target data source to obtain table structure query results; adding data to the target data source to obtain results after adding data; deleting data from the target data source to obtain results after deleting data; and modifying data in the target data source to obtain modified results, etc.

[0051] In one embodiment of this application, the process of generating a Structured Query Language (SCL) operation implementation class based on the driver classpath and data source operation configuration information includes:

[0052] Based on the driver classpath and data source operation configuration information, a data access object implementation class is generated. In one embodiment of this application, the driver classpath and data source operation configuration information are used to generate the data access object implementation class in bytecode format using dynamic proxy technology. The bytecode format can employ the ByteBuddy bytecode generation library or other bytecode manipulation techniques.

[0053] Registering the implementation class of the data access object binds the target data source to the data source operations, resulting in the structured query language operation implementation class of the target data source; the data source operations are represented by the data source operation configuration information. In one embodiment of this application, the process of registering the data access object implementation class includes: using the data access object implementation class as a Mapper class, and registering the generated Mapper class to the Configuration object of the MyBatis persistence layer framework using a preset plugin mechanism; and achieving automatic binding between the target data source and the data source operations by registering through the MapperRegistry component or dynamically constructing the MappedStatement class, thereby achieving flexible expansion of the data access layer. This preset plugin mechanism includes three core modules: dynamic proxy construction, container registration, and binding module, which can automatically complete the generation, registration, and data source routing functions of the data access object implementation class based on the interface definition.

[0054] In one embodiment of this application, the process of running the Structured Query Language operation implementation class further includes:

[0055] The runtime status information of the Structured Query Language (SCL) operation implementation class is obtained, including execution time, error rate, and slow query ratio. In one embodiment of this application, the runtime status information is collected in real time using real-time monitoring and alerting tools such as Prometheus, and anomaly detection is performed on the collected runtime status information.

[0056] If the execution time exceeds a preset time threshold, an operational anomaly is determined, and an operational anomaly alarm is triggered. In one embodiment of this application, the preset time threshold is set according to actual conditions, and the alarm information is distributed to multiple channels such as industrial internet platforms and emails through Alertmanager to ensure rapid fault response. Alertmanager is an open-source alarm management and notification routing system, typically used in conjunction with monitoring tools (e.g., Prometheus) for centralized processing, deduplication, grouping, and sending of alarm notifications.

[0057] If the error rate exceeds a preset error rate threshold, an operational anomaly is determined, and an operational anomaly alarm is triggered. In one embodiment of this application, the preset error rate threshold is set according to actual conditions, and alarm information is distributed to multiple channels such as industrial internet platforms and emails through Alertmanager to ensure rapid fault response. Alertmanager is an open-source alarm management and notification routing system, typically used in conjunction with monitoring tools (e.g., Prometheus) for centralized processing, deduplication, grouping, and sending of alarm notifications.

[0058] If the slow query ratio exceeds the preset query ratio, an operational anomaly is determined, and an operational anomaly alarm is triggered. In one embodiment of this application, the preset query ratio is determined based on actual conditions, and alarm information is distributed to multiple channels such as industrial internet platforms and emails via Alertmanager to ensure rapid fault response. Alertmanager is an open-source alarm management and notification routing system, typically used in conjunction with monitoring tools (e.g., Prometheus) for centralized processing, deduplication, grouping, and sending alarm notifications.

[0059] In one embodiment of this application, the data source operation includes at least one of: querying data, pagination preview, table structure query, adding data, deleting data, and modifying data. In another embodiment of this application, after obtaining the query results, pagination preview results, table structure query results, results after adding data, results after deleting data, and modification results, the query results, pagination preview results, table structure query results, results after adding data, results after deleting data, and modification results are displayed in real-time using Prometheus real-time monitoring and alerting tools and graphical components (Echarts or AntV) for user convenience.

[0060] Figure 3 This is a functional block diagram illustrating dynamic expansion of data sources based on an application integration platform, as shown in an exemplary embodiment of this application. Figure 3 As shown in the diagram, the functional module includes: physical data source, visual configuration management interface, configuration center, data source dynamic expansion module, connection management module, data source adaptation module, and data source monitoring module.

[0061] Physical data sources include: MySQL database, Oracle database, SQL Server database, DM adapter, Doris adapter, etc.

[0062] The visual configuration management interface includes: a configuration operation module, a data preview module, a monitoring and query module, and an application integration module. The configuration operation module is used to configure the data source configuration information; the data preview module is used to preview the operation results of the target data source; the monitoring and query module is used to monitor and query the connection status information between the target data source and the application integration platform, as well as the running status information of the structured query language operation implementation class; the application integration module integrates the configuration center, the data source dynamic expansion module, the connection management module, the data source adaptation module, the data source monitoring module, the configuration operation module, the data preview module, and the monitoring and query module, enabling them to work collaboratively.

[0063] The configuration center includes: a configuration management module, a dynamic storage module, a configuration persistence module, and an event-driven module. The configuration management module is used to manage the configuration information of the data source (e.g., update and modify); the dynamic storage module is used to dynamically store the configuration information of the data source; the configuration persistence module is used to persistently save the configuration information of the data source; and the event-driven module is used to trigger the interaction of different components through the configuration information of the data source.

[0064] The data source dynamic extension module is used to generate a data access object implementation class by using dynamic proxy technology (e.g., Bytebuddy proxy technology) to generate the driver classpath and data source operation configuration information; and to register the data access object implementation class by using dynamic injection technology, so that the target data source is bound to the data source operation, and the structured query language operation implementation class of the target data source is obtained.

[0065] The connection management module is used to input Uniform Resource Locator (URL) and connection pool parameter information into a high-performance connection pool library (e.g., HikariCP) to establish a connection between the target data source and the application integration platform.

[0066] The data source adaptation module is used to obtain the data source operation type from the data source operation configuration information, determine the adapter type based on the data source operation type and the target data source type, perform structured query language conversion on the target data source through the adapter of the corresponding type, use the converted target data source as the input of the structured query language operation implementation class, run the structured query language operation implementation class, and obtain the operation result of the target data source.

[0067] The data source monitoring module includes a monitoring indicator collection module, a visualization display module, and a monitoring and early warning module. The monitoring indicator collection module is used to collect connection status information and operation status information; the visualization display module is used to visualize the operation results of the target data source; the monitoring and early warning module is used to determine whether the connection status information is abnormal and to issue an alarm when an abnormality occurs; and the monitoring and early warning module is used to determine whether the operation status information is abnormal and to issue an alarm when an abnormality occurs.

[0068] Figure 4 This is an exemplary embodiment of the technical architecture diagram based on an application integration platform, in which... Figure 4 The technical architecture diagram includes: physical data source, visual management and application integration, access layer, core business layer, dynamic configuration center and monitoring center.

[0069] Physical data sources include: MySQL, Oracle, SQL Server, DM adapter, Doris adapter, etc.

[0070] Visual management and application integration includes: a configuration operation module, a data preview module, a monitoring and query module, and an application integration module. The configuration operation module is used to configure the data source configuration information; the data preview module is used to preview the operation results of the target data source; the monitoring and query module is used to monitor and query the connection status information between the target data source and the application integration platform, as well as the running status information of the structured query language operation implementation class; the application integration module integrates the configuration center, the data source dynamic expansion module, the connection management module, the data source adaptation module, the data source monitoring module, the configuration operation module, the data preview module, and the monitoring and query module, enabling them to work collaboratively.

[0071] The access layer is used to access the core business layer.

[0072] The core business layer includes: a dynamic proxy module, a dynamic injection module, a high-performance connection management module, a data source adaptation module, a configuration hot update module, and a driver management module. The dynamic proxy module uses dynamic proxy technology (e.g., Bytebuddy proxy technology) to generate data access object implementation classes from driver classpaths and data source operation configuration information. The dynamic injection module registers the data access object implementation classes using dynamic injection technology, binding the target data source to the data source operation to obtain the target data source's Structured Query Language (SCL) operation implementation class. The high-performance connection management module inputs Uniform Resource Locator (URI) and connection pool parameter information into a high-performance connection pool library (e.g., HikariCP) to establish a connection between the target data source and the application integration platform. The data source adaptation module obtains the data source operation type from the data source operation configuration information, determines the adapter type based on the data source operation type and the target data source type, performs SCL conversion on the target data source using the corresponding adapter, uses the converted target data source as input to the SCL operation implementation class, runs the SCL operation implementation class, and obtains the operation result from the target data source. The configuration hot update module monitors for changes in the target data source. The driver management module triggers and manages the interaction of different components based on the data source configuration information.

[0073] The dynamic configuration center includes Nacos components, Apollo components, or preset configuration components; the configuration persistence module is used to persistently save the data source configuration information.

[0074] The monitoring center includes a data acquisition / preview module and a monitoring and alarm module. The data acquisition / preview module is used to collect and preview connection status information and operation status information; the monitoring and alarm module is used to determine whether there are any abnormalities in the connection status information and to issue an alarm when an abnormality occurs; the monitoring and alarm module is used to determine whether there are any abnormalities in the operation status information and to issue an alarm when an abnormality occurs.

[0075] Figure 5 This is a schematic diagram illustrating the data source visualization effect based on an application integration platform, as shown in an exemplary embodiment of this application. Figure 5 As shown, the visualized information of the data source includes: data source name, group, connection type, Uniform Resource Locator (URL), creation time, data permissions, etc.

[0076] In one embodiment of this application, by displaying the visual information of the data source, it is not only convenient for users to view, but also convenient for users to operate.

[0077] This application obtains data source configuration information, generates a Uniform Resource Locator (URL) based on the connection template string, inputs the URL and connection pool parameter information into the connection pool library, and establishes a connection between the target data source and the application integration platform, denoted as the first connection. If the data source configuration information changes, the first connection is updated according to the changed data source configuration information. The above process establishes a connection between the target data source and the application integration platform based on the data source configuration information, and updates the first connection according to the changed data source configuration information when the data source configuration information changes, thereby improving the adaptability, scalability, and management efficiency between the application integration platform and the data source.

[0078] The following describes an apparatus embodiment of this application, which can be used to execute the data source expansion method in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the data source expansion method described above.

[0079] Figure 6 This is a block diagram illustrating a data source extension device according to an exemplary embodiment of this application. The device can be applied to… Figure 1 The implementation environment shown is specifically configured in electronic device 102. This device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0080] like Figure 6 As shown, the exemplary data source extension device 600 includes:

[0081] The information acquisition module 601 is used to obtain data source configuration information.

[0082] The connection establishment module 602 is used to generate a Uniform Resource Locator (URL) based on the connection template string; input the URL and connection pool parameter information into the connection pool library to establish a connection between the target data source and the application integration platform, which is denoted as the first connection.

[0083] The connection update module 603 is used to update the first connection according to the changed data source configuration information if the data source configuration information changes, so as to realize the expansion of the data source.

[0084] In one embodiment of this application, the data source configuration information includes: the target data source type, the connection template string, and connection pool parameter information; the connection template string consists of the server address, server account, and server password of the application integration platform. The target data source type includes: MySQL (MySQL Structured Query Language) database, Oracle database, SQL Server database, PostgreSQL (Postgres Structured Query Language) database, TiDB (Ti Database), etc. The connection pool parameter information includes the maximum number of connections, the minimum number of connections, and the maximum connection timeout. The data source configuration information also includes: the name of the target data source, the version number of the target data source, etc. The target data source type is composed of a 0-127 bit encoding, possessing global uniqueness and immutability.

[0085] In one embodiment of this application, before obtaining the data source configuration information, it is necessary to configure, manage and maintain the data source configuration information in the configuration center. The configuration center includes Nacos (Naming and Configuration Service), Apollo (Apollo Configuration Center) or other custom configuration components.

[0086] In one embodiment of this application, the Uniform Resource Locator includes: protocol, hostname, port, path, etc., and the connection pool library can be HikariCP connection pool library, or Druid connection pool library, etc.

[0087] In one embodiment of this application, changes in data source configuration information include: changes in at least one of the target data source type, connection template string, and connection pool parameter information. If the target data source type changes, a connection is established between the changed target data source and the application integration platform by inputting the Uniform Resource Locator (URL) and connection pool parameter information into the connection pool library. If the connection template string changes, the URL changes, and a connection is established between the target data source and the application integration platform by inputting the changed URL and connection pool parameter information into the connection pool library. If the connection pool parameter information changes, a connection is established between the target data source and the application integration platform by inputting the URL and the changed connection pool parameter information into the connection pool library. Other scenarios where data source configuration information changes are not listed here.

[0088] In one embodiment of this application, the listener component of the configuration center can monitor the data source configuration information in real time. When the data source configuration information changes, it triggers the hot refresh logic of the data source connection instance, dynamically loads the updated data source connection instance, and ensures the consistency and real-time performance of the data source configuration information during the application integration platform operation.

[0089] In one embodiment of this application, a connection between the target data source and the application integration platform is established based on the data source configuration information, and when the data source configuration information changes, the first connection is updated based on the changed data source configuration information, thereby improving the adaptability, scalability and management efficiency between the application integration platform and the data source.

[0090] It should be noted that the data source expansion device and the data source expansion method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the data source expansion device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation here.

[0091] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the data source extension method provided in the above embodiments.

[0092] Figure 7 This is a schematic diagram illustrating the structure of a computer system for an electronic device, as shown in an exemplary embodiment of this application. It should be noted that... Figure 7The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0093] like Figure 7 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 702 or programs loaded from storage portion 708 into Random Access Memory (RAM) 703. The RAM 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An Input / Output (I / O) interface 705 is also connected to the bus 704.

[0094] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0095] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs various functions defined in the system of this application.

[0096] Another aspect of this application provides a computer-readable storage medium storing computer-readable instructions that, when executed by a computer's processor, cause the computer to perform the data source extension methods provided in the various embodiments described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0097] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0098] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for extending a data source, characterized in that, The method includes: Obtain data source configuration information; the data source configuration information includes: target data source type, connection template string, and connection pool parameter information; the connection template string consists of the application integration platform's server address, server account, and server password; Based on the connection template string, a Uniform Resource Locator (URL) is generated; the URL and the connection pool parameter information are input into the connection pool library to establish a connection between the target data source and the application integration platform, which is denoted as the first connection. If the data source configuration information changes, the first connection is updated according to the changed data source configuration information to expand the data source. The changes in data source configuration information include changes in at least one of the following: target data source type, connection template string, and connection pool parameter information. If the target data source type changes, a connection is established between the changed target data source and the application integration platform by inputting the Uniform Resource Locator (URL) and connection pool parameter information into the connection pool library. If the connection template string changes, resulting in a change in the URL, a connection is established between the target data source and the application integration platform by inputting the changed URL and connection pool parameter information into the connection pool library. If the connection pool parameter information changes, a connection is established between the target data source and the application integration platform by inputting the URL and the changed connection pool parameter information into the connection pool library.

2. The data source extension method according to claim 1, characterized in that, After obtaining the first connection, the method further includes: Obtain the connection status information of the first connection and cache the connection status information; the connection status information includes: the number of connections and the connection creation time; If the change in the number of connections exceeds a preset change threshold, a connection anomaly is determined and a connection anomaly alarm is triggered. If the connection creation time exceeds a preset time threshold, the connection is determined to be abnormal, and a connection abnormality alarm is triggered.

3. The data source extension method according to claim 2, characterized in that, If the connection status information further includes a connection identifier, and the data source configuration information further includes a target data source demand tenant identifier and a correspondence between the demand tenant identifier and the connection identifier, then after caching the connection status information, the method further includes: If the connection identifier corresponds to the demand tenant identifier, then invoking the connection with the connection identifier is permitted; If the connection identifier does not correspond to the tenant identifier, then calling the connection with the connection identifier is prohibited.

4. The data source extension method according to claim 2, characterized in that, If the data source configuration information further includes: the driver classpath of the target data source and data source operation configuration information, then after establishing the first connection, the method further includes: Obtain the Structured Query Language (SCL) operation implementation class of the target data source; the SCL operation implementation class is generated based on the driver class path and the data source operation configuration information; In addition, the data source operation type is obtained from the data source operation configuration information, and the target data source is converted to structured query language according to the data source operation type; The transformed target data source is used as input to the structured query language operation implementation class, and the structured query language operation implementation class is run to obtain the operation result of the target data source.

5. The data source extension method according to claim 4, characterized in that, The process of generating the structured query language operation implementation class based on the driver class path and the data source operation configuration information includes: Based on the driver classpath and the data source operation configuration information, a data access object implementation class is generated; The data access object implementation class is registered to bind the target data source with the data source operation, thereby obtaining the structured query language operation implementation class of the target data source; the data source operation is characterized by the data source operation configuration information.

6. The data source extension method according to claim 4, characterized in that, The process of running the structured query language operation implementation class also includes: Obtain the runtime status information of the structured query language operation implementation class, including execution time, error rate, and slow query ratio; If the execution time exceeds a preset time threshold, an operational abnormality is determined, and an operational abnormality alarm is triggered. If the error rate exceeds the preset error rate threshold, the operation is determined to be abnormal, and an operation abnormality alarm is triggered. If the slow query ratio exceeds the preset query ratio, the system is deemed to be in operation abnormal and an operation abnormality alarm is triggered.

7. The data source extension method according to claim 4, characterized in that, The data source operations include at least one of the following: querying data, pagination preview, table structure query, adding data, deleting data, and modifying data.

8. A data source extension device, characterized in that, include: The information acquisition module is used to obtain data source configuration information; The data source configuration information includes: target data source type, connection template string, and connection pool parameter information; the connection template string consists of the application integration platform's server address, server account, and server password; The connection establishment module is used to generate a Uniform Resource Locator (URL) based on the connection template string; input the URL and the connection pool parameter information into the connection pool library to establish a connection between the target data source and the application integration platform, which is denoted as the first connection; The connection update module is used to update the first connection according to the changed data source configuration information if the data source configuration information changes, thereby expanding the data source. The changes in data source configuration information include changes in at least one of the following: target data source type, connection template string, and connection pool parameter information. If the target data source type changes, a connection is established between the changed target data source and the application integration platform by inputting the Uniform Resource Locator (URL) and connection pool parameter information into the connection pool library. If the connection template string changes, resulting in a change in the URL, a connection is established between the target data source and the application integration platform by inputting the changed URL and connection pool parameter information into the connection pool library. If the connection pool parameter information changes, a connection is established between the target data source and the application integration platform by inputting the URL and the changed connection pool parameter information into the connection pool library.

9. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the data source extension method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the computer's processor, cause the computer to perform the data source extension method as described in any one of claims 1 to 7.

Images