Method, device and equipment for data source management, medium and program product

Abstract

The invention relates to the technical field of big data, and discloses a data source management method, device and equipment, a medium and a program product, and the method comprises the steps: obtaining a business demand, and calling at least one of the following modules based on the business demand: a distribution module, a cache module, a connection pool module and an execution module; if the distribution module is called, the service demand is sent to at least one of the following modules: a cache module, a connection pool module and an execution module; if the cache module is called, a cache demand is determined based on a service demand, and the cache demand is sent to at least one of the following modules: a connection pool module and an execution module; if the connection pool module is called, determining a connection demand based on a service demand and / or a cache demand, and sending the connection demand to an execution module; and if the execution module is called, obtaining and determining an execution demand based on at least one of a business demand, a cache demand and a connection demand, and executing related operation. And the response speed to the business demand is improved.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a method, device, equipment, medium and program product for data source management. Background Art

[0002] In the field of big data technology, data source management is based on data management and crawling of various heterogeneous data sources. Data source management has frequent data interactions with other modules in the big data module, and it is necessary to ensure the robustness and high performance of data source management.

[0003] The multi-data source management in the related art has a low response speed to business needs. Summary of the invention

[0004] In view of this, the present invention provides a method, apparatus, device, medium and program product for data source management to solve the problem of low response speed of multi-data source management to business needs.

[0005] In a first aspect, the present invention provides a method for data source management, the method comprising: obtaining business requirements, and calling at least one of the following modules based on the business requirements: a distribution module, a cache module, a connection pool module, and an execution module; if the distribution module is called, the business requirements are sent to at least one of the following modules: the cache module, the connection pool module, and the execution module; if the cache module is called, the cache requirements are determined based on the business requirements, and the cache requirements are sent to at least one of the following modules: the connection pool module and the execution module; if the connection pool module is called, the connection requirements are determined based on the business requirements and / or the cache requirements, and the connection requirements are sent to the execution module; if the execution module is called, the execution requirements are obtained and determined based on at least one of the following: the business requirements, the cache requirements, and the connection requirements, and related operations are performed.

[0006] In an optional embodiment, the method further includes, based on the business needs, converting the initial data source into a target data source, including: determining an intermediate data source; establishing a mapping relationship between the intermediate data source data type and other data source data types, and obtaining a mapping table in which the target intermediate data source data type corresponds to at least one other data source data type; and converting the initial data source into the target data source via the mapping table.

[0007] In an optional implementation, converting the initial data source into the target data source via the mapping table includes: if the target other data source data type in the mapping table includes multiple initial data types, determining the target data type with the highest priority among the initial data types.

[0008] In an optional implementation, the distribution module, the cache module, the connection pool module and the execution module include a first attribute; the method further includes: determining the number of executions of the distribution module, the cache module, the connection pool module and the execution module based on the first attribute.

[0009] In an optional embodiment, the calling of at least one of the following modules: a distribution module, a cache module, a connection pool module and an execution module includes: based on annotation or triggering, calling at least one of the following modules: the distribution module, the cache module, the connection pool module and the execution module.

[0010] In an optional embodiment, the method also includes: based on the business needs, determining whether to call the cache module; if the cache module is called, determining whether there is cache data, and if the cache data exists, executing the business needs; if the cache module is not called or the cache data does not exist, determining whether to call the connection pool module; if the connection pool module is called, determining whether there is an available connection object, and if the available connection object exists, obtaining the connection object and executing the business needs; if the connection pool module is not called or the available connection object does not exist, creating a connection object, obtaining the connection object, and executing the business needs.

[0011] In a second aspect, the present invention provides a device for data source management, the device comprising: a distribution module, a cache module, a connection pool module and an execution module; the distribution module is used to obtain business requirements and send the business requirements to at least one of the following modules: the cache module, the connection pool module and the execution module; the cache module is used to obtain business requirements from the distribution module, determine cache requirements based on the business requirements, and send the cache requirements to at least one of the following modules: the connection pool module and the execution module; the connection pool module is used to obtain and determine connection requirements based on the business requirements from the distribution module and / or the cache requirements of the cache module, and send the connection requirements to the execution module; the execution module is used to obtain and determine execution requirements based on at least one of the following: the business requirements, the cache requirements and the connection requirements, and perform related operations.

[0012] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for data source management of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for data source management of the above-mentioned first aspect or any corresponding embodiment thereof.

[0014] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method for data source management of the first aspect or any corresponding embodiment thereof.

[0015] The method for data source management provided by the embodiment of the present invention freely assembles distribution capabilities, caching capabilities, connection pool capabilities and execution capabilities based on business needs. According to the assembly effect, cache, connection pool and business demand distribution logic can be added to different functions, which can be quickly expanded and maintained, increasing the efficiency of concurrent execution, improving the response speed to business needs, and reducing the development cost of data source capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of a process flow of a method for data source management provided by an embodiment of the present invention is shown;

[0018] Figure 2 A schematic flow chart of another method for data source management provided by an embodiment of the present invention is shown;

[0019] Figure 3 A schematic diagram showing the dependency relationships of each module;

[0020] Figure 4 A schematic diagram of a business level call is shown;

[0021] Figure 5 A schematic diagram of the structure of a device for data source management is shown;

[0022] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0024] The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B may be singular or plural. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0025] According to an embodiment of the present invention, a method embodiment for data source management is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] In this embodiment, a method for data source management is provided, which can be used in a terminal, such as a mobile phone, a tablet computer, a desktop computer, or a server. Figure 1 A schematic diagram of a process for managing a data source provided by an embodiment of the present invention is shown. Figure 1 As shown, the process includes the following steps:

[0027] Step S101, obtaining business requirements, and calling at least one of the following modules based on the business requirements: a distribution module, a cache module, a connection pool module, and an execution module.

[0028] In this step, the data source is the source of data, which is a device or original media that provides some required data. All information for establishing a database connection is stored in the data source. Just as you can find a file in the file system by specifying the file name, you can find the corresponding database connection by providing the correct data source name. Depending on the source of the data, the data source includes: file data source, database, big data engine data source and third-party interface data source. Among them, the database includes relational database and non-relational database.

[0029] In data source management, business requirements involve multiple aspects, aiming to ensure the validity, security, availability and performance of data. Business requirements include: data connection and access, data query and reporting, data caching and performance optimization, etc.

[0030] Step S102: if the distribution module is called, the business demand is sent to at least one of the following modules: a cache module, a connection pool module and an execution module.

[0031] The distributed structure is a complete system that is divided into independent subsystems according to business functions. In the distributed structure, each subsystem is called a "service". These subsystems can run independently in the Web container and communicate with each other through RPC.

[0032] Different business modules are deployed on different servers or the same business module is split into multiple sub-businesses and deployed on different servers, which can solve the problem of high concurrency and provide scalability and high availability. The main scenarios for using distributed services are distributed storage and distributed computing. In distributed storage, data can be sharded to multiple nodes, which can not only improve performance (scalability), but also use multiple nodes to back up the same data.

[0033] Applying this step to a distributed structure can increase concurrent execution efficiency and improve response speed to business needs.

[0034] Step S103: if the cache module is called, the cache requirement is determined based on the business requirement, and the cache requirement is sent to at least one of the following modules: a connection pool module and an execution module.

[0035] In this step, if the business needs involve frequent access to the same data or data queries that require a quick response, the business needs can be sent to the cache module. The cache module will check whether the required data already exists in the cache. If it does, the data in the cache is directly returned; if it does not exist, the data may be obtained from the data source, cached, and then returned to the caller.

[0036] The cache module temporarily stores data in memory or other high-speed storage media to quickly access and read data. Data cache is generally used for data that needs to be accessed frequently, such as table data in a database or page data in a Web application.

[0037] Data caching can improve the speed and performance of data access and avoid repeated queries to databases or other data sources, thereby reducing system load and response time. Data caching can be implemented using various technologies, such as memory caching, distributed caching, client caching, etc.

[0038] Step S104: if the connection pool module is called, the connection requirement is determined based on the business requirement and / or cache requirement, and the connection requirement is sent to the execution module.

[0039] In this step, when the business demand requires establishing a connection with a data source to perform data query, update or other operations, the business demand can be sent to the connection pool module.

[0040] The database connection pool is responsible for allocating, managing, and releasing database connections. It allows applications to reuse an existing database connection instead of re-establishing one. It releases database connections whose idle time exceeds the maximum idle time to avoid missing database connections due to failure to release database connections. This can improve the performance of database operations.

[0041] The basic idea of ​​the connection pool is to store the database connection as an object in the memory when the system is initialized. When the user needs to access the database, instead of establishing a new connection, an established idle connection object is taken out from the connection pool. After use, the user does not close the connection, but puts the connection back into the connection pool for the next request to access. The establishment and disconnection of the connection are managed by the connection pool itself. At the same time, the initial number of connections in the connection pool, the upper and lower limits of the connection, the maximum number of uses of each connection, the maximum idle time, etc. can also be controlled by setting the parameters of the connection pool. The number and usage of database connections can also be monitored through its own management mechanism.

[0042] Step S105, if the execution module is called, the execution requirement is obtained and determined based on at least one of the following: business requirement, cache requirement and connection requirement, and related operations are performed.

[0043] In this step, the execution module can be implemented through a database driver. The database driver acts as a bridge between the application and the database. The database driver is a software component that provides interfaces and functions for communicating with a specific database system. It is mainly used to connect to the database, perform database operations, data conversion and mapping, error handling and exception handling, and performance optimization.

[0044] The database driver acts as a communication bridge between the application and the database, allowing the application to easily interact with the database, perform database operations, and handle related errors and exceptions. It simplifies the database programming process and provides access to and control over specific database systems.

[0045] The method for data source management provided in this embodiment freely assembles distribution capabilities, caching capabilities, connection pool capabilities, and execution capabilities based on business needs. According to the assembly effect, caching, connection pools, and business demand distribution logic can be added to different functions. It can be quickly expanded and maintained, which increases the efficiency of concurrent execution, improves the response speed to business needs, and reduces the development cost of data source capabilities.

[0046] The data types defined by different data sources are similar, but there are also many custom data types, such as the LONG RAW data type in the Oracle database. The LONG RAW data type does not exist in MySQL and SQLServer. Before synchronizing data from the Oracle database to the MySQL database, it is necessary to map and convert the LONG RAW data type in the Oracle database. However, each type of data requires mapping of the corresponding data type in other data sources, which greatly increases program development and maintenance costs.

[0047] In some optional embodiments, the aforementioned method for data source management further includes: based on business needs, converting the initial data source into a target data source, including: determining an intermediate data source; establishing a mapping relationship between the intermediate data source data type and other data source data types, and obtaining a mapping table in which the target intermediate data source data type corresponds to at least one other data source data type; and converting the initial data source into the target data source via the mapping table.

[0048] In this embodiment, an intermediate type conversion standard is selected, each data type in each data source type is divided, and the default data type corresponding to each intermediate type is determined, and a data type mapping relationship network is established to facilitate rapid acquisition of the target conversion data type.

[0049] This implementation is described by way of example, using java.sql.Types as an intermediate data source, matching and mapping all data types under each data source type with the values ​​of java.sql.Types, and establishing a mapping table between each data source and java.sql.Types. For example, the MySQL driver package contains the com.mysql.cj.MysqlType type, which implements the conversion between MySQLType and JDBCType, and JDBCType and Types already have a direct mapping relationship, so this type can be used directly for type conversion.

[0050] For Oracle, the Oracle driver package contains the oracle.jdbc.OracleType type, but lacks direct type conversion with java.sql.Types. Therefore, you can add mappings with java.sql.Types one by one according to the OracleType enumeration.

[0051] Determine the mapping table between the initial data source and java.sql.Types, and find the specified mapping relationship from this mapping table. For example, for the int type of MySQL, first find the Types.INTEGER corresponding to the int type from the mapping table. Determine the mapping table between the target data source and java.sql.Types, and find the specified mapping relationship from this mapping table.

[0052] Through the mapping of multiple data sources, the original one-to-one mapping relationship is converted into a pair of java.sql.Types mapping, and the original development cost is reduced from N^2 to N, which greatly reduces the development complexity. In addition, there is no need to worry about how to convert each type with other types, and it can be more friendly and compatible with various heterogeneous data source types.

[0053] In some optional implementations, converting the initial data source into a target data source via a mapping table includes: if the target other data source data type in the mapping table includes multiple initial data types, determining the target data type with the highest priority among the initial data types.

[0054] In this implementation, since java.sql.Types is a relatively unified type, there may be multiple mapping chains, so when establishing a mapping table, you can specify the type with the highest default priority. For example, if the target type is Oracle, find the corresponding int or integer from the mapping table according to Types.INTEGER. At this time, you can set integer as the type with the highest default priority in the mapping table, and the final result after conversion is integer.

[0055] At the same time, when there are multiple mapping chains, in addition to using the default specified type to handle, you can also select the optimal target data type by combining the length or number of digits set for the field type during conversion. On this basis, in addition to implementing the conversion of table types, we also implement the conversion of each data type and java.sql.Types under javaType, that is, when MySQL's int is converted to Oracle's integer, it is first converted to java.lang.Integer according to the conversion logic of MySQL and Types.INTEGER, and then converted to the corresponding Integer object of Oracle according to the conversion logic of Oracle and Types.INTEGER.

[0056] In this way, setting the priority of the target data type can further improve business processing efficiency.

[0057] In some optional implementations, the distribution module, cache module, connection pool module and execution module include a first attribute; the aforementioned method for data source management also includes: determining the number of executions of the distribution module, cache module, connection pool module and execution module based on the first attribute.

[0058] In this implementation, the public interfaces Provider and Factory can be specified, with Factory as the calling entry and Provider as the entry for each module extension. Factory uses @Autowire to inject all implemented Providers in the form of a Map, which is convenient for subsequent specified Provider calls. TransmittableThreadLocal can be used to save the modules specified by the current business needs. The saved data is the corresponding module parameter object, which contains the first attribute (scope) and the provider attribute. The scope corresponds to two enumeration values: once (ONCE) and multiple times (MULTI). Provider indicates which implementation provider is specified.

[0059] In each module, different Provider interfaces are implemented, and the module can be called through the Serial Peripheral Interface (SPI) to facilitate better function expansion.

[0060] In this way, by setting the first attribute, the overhead of frequently creating and closing database connections can be reduced, the system performance can be improved, and the efficiency of processing business needs can be further improved.

[0061] In some optional implementations, calling at least one of the following modules: a distribution module, a cache module, a connection pool module, and an execution module includes: calling at least one of the following modules: a distribution module, a cache module, a connection pool module, and an execution module based on annotations or triggers.

[0062] In this implementation, custom annotations can be added according to each module, corresponding to the distribution module @Distributed, the cache module @Cache, the connection pool module @Pool, and the data source execution module @Plugin. Each annotation has a defaultProvider parameter, which can specify the Provider to call and cache to TransmittableThreadLocal. In the specified code module, the custom annotation can be assigned to the corresponding method, which means that this method will be processed using the corresponding module.

[0063] For example, the method for querying the database list can be annotated with @Cache, @Pool, and @Plugin, indicating that when querying the database list, the connection pool module will be used to create a jdbc connection, and then the specified data source execution module will be used to call it, and finally the data will be cached for use.

[0064] In each module, Aspect Oriented Programming (AOP) can be used to process the respective custom annotations, using @Before to cut in, saving the module parameter object with scope=MULTI in TransmittableThreadLocal, and using @After to remove this module parameter object.

[0065] In this embodiment, a manually triggered method may be added to each module, such as a cache module, using startDistributed() and endDistributed() as dividing boundaries. Code within the boundaries can use the distribution module, startXX means opening the module, and endXX means closing the module.

[0066] You can also use useDistributed() to specify the effect on the next execution call without affecting other code usage. After using the manual trigger method, the module parameter object will be saved in TransmittableThreadLocal, start saves scope = MULTI, use saves scope = ONCE, and end removes this module parameter object. The manual trigger method can also specify the provider.

[0067] You can use spring.factories to create custom starters for each module and specify the default Provider implementation through dependencies. Implement a Factory that implements global unified operations. The project to which this Factory belongs depends on the starter of each module in the form of a provider. At the same time, implement the call chain of the execution module from cache, connection pool, distribution, and data source.

[0068] Factory uses Class.forName to determine whether it actually depends on the starter of the corresponding module, and uses ApplicationContext to obtain the dependency injection of the capability module. The actual business engineering module can use Factory to trigger the data source capability and depend on the corresponding starter according to the actual business needs. It also depends on more providers to add more implementation methods for each capability module, and makes a specific call through the provider in the module parameter object saved by TransmittableThreadLocal.

[0069] In this way, for each new data source type or each new connection pool, dynamic expansion can be achieved through the overall architectural level of this solution, which can reduce the coupling of the code and reduce development costs and learning costs.

[0070] In some optional embodiments, Figure 2 A schematic diagram of another method for data source management provided by an embodiment of the present invention is shown. Figure 2 The aforementioned method for data source management also includes: step S201: based on business needs, determine whether to call the cache module; step S202: if the cache module is called, determine whether there is cache data, if there is cache data, go to step S207; step S203: if the cache module is not called or there is no cache data, determine whether to call the connection pool module; step S204: if the connection pool module is called, determine whether there is an available connection object, step S205: if there is an available connection object, obtain the connection object, go to step S207; step S206: if the connection pool module is not called or there is no available connection object, create a connection object, go to step S205; step S207: execute business needs.

[0071] In this way, various modules can be dynamically expanded and assembled to meet business processing in more scenarios, while improving concurrency capabilities and response speed.

[0072] In some optional implementations, more different connection pool modules and cache modules can be introduced to increase compatibility with different data source types based on the database connection pool and data cache layer, thereby increasing the scalability and free adaptability of different data source types. Depending on different business scenarios, you can freely configure whether to use module capabilities such as connection pool, cache, and request distribution. For example, when obtaining metadata such as MySQL libraries and tables, you can configure HikaciCP as a connection pool, and specify Redis as a cache. When obtaining Csv data, you can only configure Redis as a cache, etc. At the same time, if there are a large number of queries, such as the need to count the data information of each field of a table, the statistical operations of each field can be distributed to each node for processing, and then summarized and returned.

[0073] Figure 3 A schematic diagram showing the dependency relationship of each module is shown. Figure 3 As shown, the distribution module Provider301 can send business requirements to the distributed coordination service (ZooKeeper) 302, the cluster 303, the HikariC connection pool 304, the GenericObjectPool connection pool 306 or the memory 309; the cache module 306 can call the Redis cache 308 or the memory 309 through the cache Provider307; the cache module 306 sends the cache requirements to the connection pool module 310, and the connection pool module 310 can call the HikariC connection pool 304, the RedisPooling connection pool 305 or the GenericObjectPool connection pool 306 through the connection pool Provider311; the connection pool module 310 sends the connection requirements to the execution module 317, and the execution module 317 creates a connection object 315 based on the API operation 314 or the connection requirement, and performs a connection test verification 316 based on the created connection object.

[0074] In some optional implementations, by distributing the modules and replicating and forwarding the required module requests, creation loss of subsequent requests from other data source management system nodes can be reduced.

[0075] Figure 4 Figure 1 shows a schematic diagram of a business level call. Figure 4As shown, the business module 401 includes: a database list 402, a data table list 403 and table information 404; the cache module 405 includes: a Hive cache 406, a Mysql cache 407 and an Oracle cache 408; the connection pool module 409 includes: a Hive connection pool 410, a Mysql connection pool 411 and an Oracle connection pool 412; the data source 413 includes: Hive414, Mysql415 and Oracle416; the business module 401 sends the cache acquisition demand to the cache module 405, sends the connection pool acquisition demand to the connection pool module 409, and sends the API acquisition demand to the data source 413; the cache module 405 obtains the cache demand, sends the connection pool request to the connection pool module 409, or sends the cache demand to the data source 413; the connection pool module 409 obtains the connection pool request and makes an API call according to the pooled object; the data source module 413 is used to perform database connection and API operations.

[0076] In this way, by adding support for dynamic expansion loading of cache modules, connection pool modules, and distributed modules based on the ability to dynamically load data sources, and allowing free assembly of various modules, the concurrency and response speed improvements in different business scenarios can be met.

[0077] In this embodiment, a device for data source management is also provided, which is used to implement the above embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0078] This embodiment provides a device for data source management. Figure 5 A schematic diagram of the structure of a device for data source management is shown, Figure 5 As shown, the device for data source management includes: a distribution module 501, a cache module 502, a connection pool module 503 and an execution module 504;

[0079] The distribution module 501 is used to obtain business requirements and send the business requirements to at least one of the following modules: a cache module, a connection pool module and an execution module.

[0080] The cache module 502 is used to obtain the business requirements from the distribution module, determine the cache requirements based on the business requirements, and send the cache requirements to at least one of the following modules: the connection pool module and the execution module.

[0081] The connection pool module 503 is used to obtain and determine the connection requirements based on the business requirements from the distribution module and / or the cache requirements of the cache module, and send the connection requirements to the execution module.

[0082] The execution module 504 is used to obtain and determine the execution requirement based on at least one of the following: business requirement, cache requirement and connection requirement, and perform related operations.

[0083] In some optional implementations, the aforementioned device for data source management includes:

[0084] The conversion module is used to convert the initial data source into the target data source based on business needs, including: determining the intermediate data source; establishing a mapping relationship between the intermediate data source data type and other data source data types, and obtaining a mapping table in which the target intermediate data source data type corresponds to at least one other data source data type; and converting the initial data source into the target data source via the mapping table.

[0085] In some optional implementations, the conversion module includes:

[0086] The conversion unit is used to determine the target data type with the highest priority among the initial data types if the target other data source data type in the mapping table includes multiple initial data types.

[0087] In some optional implementations, the distribution module, cache module, connection pool module and execution module include a first attribute; based on the first attribute, the execution times of the distribution module, cache module, connection pool module and execution module are determined.

[0088] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0089] The device for data source management in this embodiment is presented in the form of a functional unit, where the unit refers to an application specific integrated circuit (ASIC) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0090] The embodiment of the present invention also provides a computer device having the above Figure 5 An apparatus for data source management is shown.

[0091] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of a graphical user interface on an external input / output device (such as a display device coupled to an interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0092] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0093] The aforementioned memory 20 stores instructions executable by at least one processor 10, so that the aforementioned at least one processor 10 executes the method shown in the above embodiment.

[0094] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0095] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0096] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0097] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (such as a light emitting diode) and a tactile feedback device (such as a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0098] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0099] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0100] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for data source management, characterized in that The method comprises: Obtaining business requirements, and calling at least one of the following modules based on the business requirements: a distribution module, a cache module, a connection pool module, and an execution module; If the distribution module is called, the business demand is sent to at least one of the following modules: the cache module, the connection pool module and the execution module; If the cache module is called, a cache requirement is determined based on the business requirement, and the cache requirement is sent to at least one of the following modules: the connection pool module and the execution module; If the connection pool module is called, a connection requirement is determined based on the business requirement and / or the cache requirement, and the connection requirement is sent to the execution module; If the execution module is called, the execution requirement is obtained and determined based on at least one of the following: the business requirement, the cache requirement and the connection requirement, and the related operation is performed.

2. The method according to claim 1, characterized in that The method further includes, based on the business requirement, converting the initial data source into a target data source, including: Identify intermediate data sources; Establishing a mapping relationship between the intermediate data source data type and other data source data types, and obtaining a mapping table in which the target intermediate data source data type corresponds to at least one other data source data type; The initial data source is converted into the target data source via the mapping table.

3. The method according to claim 2, characterized in that The converting the initial data source into the target data source via the mapping table comprises: If the target other data source data type in the mapping table includes multiple initial data types, the target data type with the highest priority among the initial data types is determined.

4. The method according to claim 1, characterized in that: The distribution module, the cache module, the connection pool module and the execution module include a first attribute; the method further includes: Based on the first attribute, the execution times of the distribution module, the cache module, the connection pool module and the execution module are determined.

5. The method according to claim 1, characterized in that The calling of at least one of the following modules: a distribution module, a cache module, a connection pool module and an execution module includes: Based on annotation or triggering, at least one of the following modules is called: the distribution module, the cache module, the connection pool module and the execution module.

6. The method according to claim 1, characterized in that The method further comprises: Based on the business requirements, determine whether to call the cache module; If the cache module is called, it is determined whether there is cache data, and if the cache data exists, the business requirement is executed; If the cache module is not called or the cache data does not exist, determine whether to call the connection pool module; If the connection pool module is called, it is determined whether there is an available connection object. If there is an available connection object, the connection object is obtained to execute the business requirement; If the connection pool module is not called or the available connection object does not exist, a connection object is created, the connection object is obtained, and the business requirement is executed.

7. A device for data source management, characterized in that: The device comprises: a distribution module, a cache module, a connection pool module and an execution module; The distribution module is used to obtain business requirements and send the business requirements to at least one of the following modules: the cache module, the connection pool module and the execution module; The cache module is used to obtain the business requirements from the distribution module, determine the cache requirements based on the business requirements, and send the cache requirements to at least one of the following modules: the connection pool module and the execution module; The connection pool module is used to obtain and determine the connection requirements based on the business requirements from the distribution module and / or the cache requirements of the cache module, and send the connection requirements to the execution module; The execution module is used to obtain and determine the execution requirement based on at least one of the following: the business requirement, the cache requirement and the connection requirement, and perform related operations.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for data source management according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for data source management according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method for data source management according to any one of claims 1 to 6.

Images