Data access method and device of data source and processing equipment

Abstract

The invention provides a data access method and device of a data source and processing equipment, and aims to provide a novel multi-data source integration access scheme and solve the operation cost problem that data source connection still needs to be established in real time under some conditions in an existing scheme. According to the method, seamless integration of multiple data sources can be stably and efficiently realized, and the method has high compatibility, data quality, processing efficiency, data security, flexibility and expansibility, so that the data access requirements of high-quality multiple data sources can be met.

Description

Technical Field

[0001] This application relates to the field of data access, and specifically to a data access method, device, and processing device for a data source. Background Art

[0002] For the operation of an enterprise, there are now a large number of data sources in the enterprise's network architecture, providing the data support of the source data required for the development of related business services. In this case, relevant technical personnel are required to build the corresponding data source connections one by one, which obviously has the problems of low processing efficiency and high labor costs.

[0003] In this case, a data source connection pool mechanism can be introduced. First, build the data source connections in advance and add them to the data source connection pool. In this way, when it is necessary to read and write data from the corresponding data source, the pre-built data source connections in the data source connection pool can be directly called for use, effectively reducing the operation cost of the original need to build data source connections in real time.

[0004] However, the inventors of this application noticed that in actual situations, in a large and complex network architecture, it is inevitable to involve a large number of different types of data sources. The pre-built data source connections in the data source connection pool may have problems of not being compatible with the data source types involved in the data source, so it is still necessary to build the adapted data source connections in real time, which still has the problem of operation cost to a certain extent. Summary of the Invention

[0005] This application provides a data access method, device, and processing device for a data source, which provides a novel multi-data source integration access solution. Without the operation cost problem of still needing to build data source connections in real time in some cases like the existing solutions, it can stably and efficiently achieve the seamless integration of multi-data sources, with high compatibility, data quality, processing efficiency, data security, flexibility, and scalability. Therefore, it can meet the high-quality data access requirements of multi-data sources.

[0006] In a first aspect, this application provides a data access method for a data source, and the method includes:

[0007] Determine the first data source to be accessed in the network architecture;

[0008] Collect the configuration information of the first data source;

[0009] In the business system, through the data source management service, create a corresponding second data source and a data source connection based on the configuration information of the first data source, and put the data source connection into the data source connection pool;

[0010] When there is a target business service related to the first data source, based on the second data source and the data source connections created in the data source connection pool, process the target business service.

[0011] Combined with the first aspect of this application, in the first possible implementation manner of the first aspect of this application, the specific data source types of the first data source include relational database types, data warehouse types, time series database types, in-memory database types, and file storage system types. The relational database types specifically involve oracle, mysql, db2, and sqlServer. The data warehouse types specifically involve hive and Olap. The time series database types specifically involve influxdb. The in-memory database types specifically involve Redis and Memcached. The file storage system types specifically involve fastDFS and OSS.

[0012] Combined with the first aspect of this application, in the second possible implementation manner of the first aspect of this application, determining the first data source to be accessed in the network architecture includes:

[0013] From the candidate data source set, based on the application requirements, analyze and evaluate in combination with data structure, format, access method, and security requirements to determine the first data source to be accessed in the network architecture.

[0014] Combined with the first aspect of this application, in the third possible implementation manner of the first aspect of this application, the configuration information specifically includes database connection address, username, password, connection information, authentication credentials, and access permissions.

[0015] Combined with the first aspect of this application, in the fourth possible implementation manner of the first aspect of this application, the method further includes:

[0016] Assign a unique identifier to the second data source;

[0017] When there is a target business service related to the first data source, based on the second data source and the data source connections created in the data source connection pool, process the target business service, including:

[0018] When there is a target business service related to the first data source, directly call the second data source and the data source connections created in the data source connection pool with the assigned unique identifier to process the target business service.

[0019] Combined with the first aspect of this application, in the fifth possible implementation manner of the first aspect of this application, the method further includes:

[0020] Verify the data source connections in the second data source and the data source connection pool by simulating the processing results of connection and data return to verify whether they are running properly.

[0021] In combination with the first aspect of the present application, in the sixth possible implementation manner of the first aspect of the present application, the second data source is encapsulated in the database connection pool in an independent encapsulation manner.

[0022] In a second aspect, the present application provides a data access device for a data source. The device includes:

[0023] A determination unit, configured to determine a first data source to be accessed in the network architecture where it is located;

[0024] An acquisition unit, configured to acquire the configuration information of the first data source;

[0025] A creation unit, configured to create a corresponding second data source and a data source connection in the service system through a data source management service based on the configuration information of the first data source, and put the data source connection into the data source connection pool;

[0026] A processing unit, configured to process a target service based on the second data source and the data source connection created in the data source connection pool when there is a target service related to the first data source.

[0027] In combination with the second aspect of the present application, in the first possible implementation manner of the second aspect of the present application, the specific data source types of the first data source include relational database types, data warehouse types, time series database types, in-memory database types, and file storage system types. The relational database type specifically involves oracle, mysql, db2, and sqlServer. The data warehouse type specifically involves hive and Olap. The time series database type specifically involves influxdb. The in-memory database type specifically involves Redis and Memcached. The file storage system type specifically involves fastDFS and OSS.

[0028] In combination with the second aspect of the present application, in the second possible implementation manner of the second aspect of the present application, the determination unit is specifically configured to:

[0029] Analyze and evaluate from the candidate data source set based on application requirements in combination with data structure, format, access method, and security requirements to determine the first data source to be accessed in the network architecture where it is located.

[0030] In combination with the second aspect of the present application, in the third possible implementation manner of the second aspect of the present application, the configuration information specifically includes a database connection address, a username, a password, connection information, authentication credentials, and access permissions.

[0031] In combination with the second aspect of the present application, in the fourth possible implementation manner of the second aspect of the present application, the device further includes:

[0032] Allocation unit, used to allocate a unique identifier for the second data source;

[0033] Processing unit, specifically used for:

[0034] When there is a target business service related to the first data source, directly call the second data source and the data source connection created in the data source connection pool with the allocated unique identifier to process the target business service.

[0035] Combined with the second aspect of the present application, in the fifth possible implementation manner of the second aspect of the present application, the device further includes:

[0036] Verification unit, used to verify the second data source and the data source connection in the data source connection pool by simulating the processing results of connection and data return, so as to verify whether it is running normally.

[0037] Combined with the second aspect of the present application, in the sixth possible implementation manner of the second aspect of the present application, the second data source is encapsulated in the database connection pool in an independent encapsulation manner.

[0038] In a third aspect, the present application provides a processing device, including a processor and a memory. A computer program is stored in the memory. When the processor calls the computer program in the memory, it executes the method provided in the first aspect of the present application or any one of the possible implementation manners of the first aspect of the present application.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores multiple instructions, and the instructions are suitable for being loaded by a processor to execute the method provided in the first aspect of the present application or any one of the possible implementation manners of the first aspect of the present application.

[0040] From the above content, it can be concluded that the present application has the following beneficial effects:

[0041] Aiming at the data access target of multiple data sources, the present application provides a novel multi-data source integration access scheme. Without the operation cost problem of having to build a data source connection in real time in some cases as in the existing scheme, it can stably and efficiently achieve seamless integration of multiple data sources, and has high compatibility, data quality, processing efficiency, data security, flexibility and scalability. Therefore, it can meet the high-quality data access requirements of multiple data sources. Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0043] Figure 1 A schematic flowchart of a data access method for the data source of this application;

[0044] Figure 2 A schematic structural diagram of a data access device for the data source of this application;

[0045] Figure 3 A schematic structural diagram of a processing device of this application. Detailed implementation manners

[0046] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of this application.

[0047] Terms such as "first" and "second" in the specification, claims and above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described here can be implemented in an order different from that shown or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or modules does not necessarily have to be limited to those steps or modules clearly listed, but may include other steps or modules not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of steps that appear in this application does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The named or numbered process steps can be changed in the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0048] The division of modules in this application is a logical division. In actual implementation, there may be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection between modules can be in an electrical or other similar form, which is not limited in this application. And the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed to multiple circuit modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application.

[0049] Before introducing the data access method for the data source provided by this application, the background content involved in this application will be introduced first.

[0050] The data access method, device, and computer-readable storage medium for the data source provided by this application can be applied to a processing device, and are used to provide a novel multi-data source integration access solution. Without the operation cost problem of having to set up a data source connection in real time in some cases as in the existing solutions, it can stably and efficiently achieve seamless integration of multiple data sources, with high compatibility, data quality, processing efficiency, data security, flexibility, and scalability. Therefore, it can meet the data access requirements for high-quality multi-data sources.

[0051] For the data access method of the data source mentioned in this application, the execution subject can be a data access device for the data source, or different types of processing devices such as a server, physical host, or user equipment (UE) that integrates the data access device for the data source. Among them, the data access device for the data source can be implemented in hardware or software. The UE can specifically be a terminal device such as a smart phone, tablet computer, notebook computer, desktop computer, or personal digital assistant (PDA). The processing device can be set up in the form of a device cluster.

[0052] It can be understood that this application is targeted at the data access scenario of multiple data sources within the enterprise / company network architecture. Therefore, the processing device that executes the data access method for the data source of this application, or the processing device that carries the application service corresponding to the data access method for the data source of this application, is usually a device node responsible for the network work of data access in the network architecture in actual situations. It can be either a dedicated network node or a network node that additionally configures the network service of data access on the basis of the original network service.

[0053] As an example, the processing device can specifically be a server device in the network architecture. Of course, in actual situations, with the flexible adjustment of application requirements, the processing device can also be a physical host or other devices in the network architecture. Even more, the processing device can be outside the network architecture and provide corresponding data access services for the enterprise / company network architecture in the form of a third-party service, which may also occur in some special situations.

[0054] Next, the data access method for the data source provided by this application will be introduced.

[0055] First, refer to Figure 1 , Figure 1A schematic flowchart of a data access method for the data source of the present application is shown. The data access method for the data source provided by the present application may specifically include the following steps S101 to S104:

[0056] Step S101, determine the first data source to be accessed in the network architecture.

[0057] It can be understood that when facing the data access requirements of relevant data sources in the network architecture, it may involve the determination process of the specific data source to be accessed or waiting to be accessed. For the convenience of description, the data source waiting to be accessed determined here is collectively referred to as the first data source, and the subsequent data sources are referred to as the second data sources.

[0058] In specific operations, the first data source can either be a data source manually specified by a staff member or a data source automatically identified by the system with an access requirement.

[0059] For example, when a new data source is added to the network architecture, the system can default it as the new first data source and add it to the set of the first data sources; for another example, when there is a data source in the network structure that switches from the offline state to the online state, the system can default it as the new first data source and add it to the set of the first data sources; for another example, when the system determines that relevant service requirements need to configure relevant data sources, the system can use the corresponding data source as the new first data source and add it to the set of the first data sources; for another example, when it is determined through a corresponding update event that the network architecture has been updated, the system can identify all the data sources that can be accessed in the network architecture as the first data sources and directly obtain the set of the first data sources.

[0060] Among them, for the data sources that can be accessed in the solution of the present application, the definition of the data source may also be involved behind it. That is to say, before determining whether to access the data source, it may also involve the judgment of whether it is the data source concerned and targeted by the solution of the present application, rather than directly including all the theoretically relevant data sources in the scope of the data sources involved in the solution of the present application. Correspondingly, the solution of the present application is considered from the enterprise / company level, and the data sources to be accessed are themselves considered based on the operation requirements of the enterprise / company.

[0061] As an exemplary embodiment, determining the first data source to be accessed in the network architecture here may specifically include:

[0062] From the set of candidate data sources, based on the application requirements, analyze and evaluate in combination with data structure, format, access method, and security requirements to determine the first data source to be accessed in the network architecture.

[0063] It can be understood that in practical applications, for the data sources to be accessed through the data source access service involved in the solution of the present application, in-depth analysis and evaluation processing are involved to take into account considerations in dimensions / aspects such as specific application requirements, data structures, formats, access methods, and security requirements, so as to determine a first data source that better conforms to the actual situation and in-depth requirements.

[0064] The candidate data source set can be initially determined by the above-mentioned first data source determination method, and then through the analysis and evaluation processing involved in the embodiments herein, the truly required first data source can be obtained.

[0065] In addition, it can be understood that the solution of the present application is described from the perspective of processing a single data source. In specific applications, the solution can be processed either based on a single data source or based on multiple data sources, or the access work of multiple data sources can be configured simultaneously.

[0066] As an exemplary embodiment herein, the specific data source types of the first data source can include relational database types, data warehouse types, time series database types, in-memory database types, and file storage system types, etc. Among them, there are:

[0067] 1. The relational database type can specifically involve oracle, mysql, db2, and sqlServer, etc.;

[0068] 2. The data warehouse type specifically involves hive and Olap, etc.;

[0069] 3. The time series database type specifically involves influxdb, etc.;

[0070] 4. The in-memory database type specifically involves Redis and Memcached, etc.;

[0071] 5. The file storage system type specifically involves fastDFS and OSS, etc.

[0072] Among them, it can be understood that the above-mentioned series of specific data sources of different types can be data source products involved in the prior art, so no specific elaboration is made here.

[0073] In this way, the embodiments herein provide specific implementation solutions for the data source types that the present application may involve in practical applications. Moreover, it can also be seen that the solution of the present application can achieve good access effects for the above-mentioned series of data sources of different types, and has high compatibility and universality.

[0074] Step S102, collect the configuration information of the first data source;

[0075] After determining the first data source to be accessed currently, the corresponding configuration information can be collected. It can be understood that this application also involves the configuration work of the data source, and specifically, it is carried out based on the configuration information of the first data source. Or rather, the configuration information collected for the first data source here serves the subsequent data source configuration work. In specific applications, which configuration information of the first data source to collect will be determined according to the data source configuration work designed in this application.

[0076] As an exemplary embodiment here, the configuration information involved in this application may specifically include information such as database connection address, username, password, connection information, authentication credentials, and access permissions.

[0077] The configuration information collected for the first data source can be specifically stored in places such as a database or system cache.

[0078] Similar to the previous ones, the embodiments here provide a specific implementation solution for the configuration information that this application may involve in how to better complete the data source configuration work in actual applications, and have better practical significance.

[0079] Step S103, in the business system, through the data source management service, based on the configuration information of the first data source, create the corresponding second data source and data source connection, and put the data source connection into the data source connection pool;

[0080] It can be seen that the data source configuration work mentioned in this application before specifically refers to the creation process of the data source. That is to say, the data source management service configured in this application in the business system not only involves the creation work of the data source connection based on the data source connection pool, but also involves the data source creation work. And this means that for subsequent use of the data source, it is based on the data source connection in the data source connection pool to use the second data source created for the first data source, so as to achieve the original application goal of accessing the first data source for use.

[0081] Among them, it should be noted that the data source connection to be put into the data source connection pool for convenient invocation specifically targets the second data source, which corresponds to the implementation situation of the subsequent use of the second data source.

[0082] It can be learned that the improved data source connection pool mechanism of this application is based on the first data source to be accessed and used originally, and a convenient data source calling solution effect is carried out on the basis of creating a second data source and its corresponding data source connection. Among them, since each created second data source has its own corresponding data source connection, when calling the data source through the data source connection pool, it can be ensured that each data source can obtain the support of a pre-constructed and highly adapted data source connection, and there will be no situation where the pre-constructed data source connection to be called from the data source connection pool may not be adapted to the data source in some cases as in the prior art.

[0083] Moreover, in terms of details, since the data source connections in the data source connection pool of this application are specifically created for the created second data sources, they have the characteristics of being highly controllable and managed on the basis of high adaptability. When the second data sources are updated along with the first data source, the corresponding data source connections will be automatically updated, so that during the normal working process, the normal use of the second data sources can be ensured, and high data quality, processing efficiency, data security, flexibility and scalability can also be ensured, ensuring that the related services based on the data sources can be carried out with high quality and at a high level.

[0084] Regarding the convenience advantage in update and maintenance, specifically, this application can maintain the data source information through the web page. The system can synchronize the information of the current data source in real time according to the modified data source information from the first data source and other aspects. For example:

[0085] 1. If the data of a current data source is obtained in the form of a db2 database and is modified to a hive database, then the data is obtained from the specified hive database;

[0086] 2. When the address of the database needs to be changed to a new address, it can also be quickly switched on the web page without finding the corresponding table in the database and modifying it, because after the modification in the database, the service needs to be restarted, which has many adverse effects on the application after going online and will affect the normal use of users.

[0087] At the same time, in specific applications, this application can also involve a verification / validation mechanism for the second data source to monitor and ensure the data source quality.

[0088] In this regard, as an exemplary embodiment, the method of this application may further include:

[0089] Verifying the second data source and the data source connections in the data source connection pool by simulating the processing results of connection and data return to verify whether they are running normally.

[0090] It can be understood that in the embodiments herein, the present application initiates the verification process at regular intervals, randomly, or in other initiation manners in a way that simulates a real working scenario (such as a web service application scenario based on the web side involved in the specific application of the present application), and can mainly determine whether the second data source and / or the data source connection corresponding to the second data source is operating normally through two aspects: whether the data source can be normally connected and whether the data source can normally return data.

[0091] If it is not operating normally, or in other words, there are abnormalities, the second data source and / or the data source connection corresponding to the second data source can be reconfigured, so as to further ensure data quality and security.

[0092] As an example, in the specific verification process, specific factors such as the connection status and connection latency of the data source can be monitored. By regularly checking and handling errors and abnormal situations, the speed of locating the data source can be conveniently and effectively improved, and the data source access process can be optimized and improved to ensure the stability and reliability of data access.

[0093] In addition, to further simplify the application of the second data source and improve the corresponding processing efficiency, the present application can also perform corresponding data processing based on the assigned unique identifier (which can also be understood as the data source configuration name).

[0094] Correspondingly, as an exemplary embodiment, the method of the present application may further include:

[0095] Assigning a unique identifier to the second data source;

[0096] When there is a target business service related to the first data source, based on the second data source and the data source connection created in the data source connection pool, processing the target business service, including:

[0097] When there is a target business service related to the first data source, directly calling the second data source and the data source connection created in the data source connection pool with the assigned unique identifier to process the target business service.

[0098] It can be understood that the usually additionally assigned unique identifier can quickly locate the specifically pointed data source among the constructed different second data sources, thereby ensuring efficient and accurate processing effects.

[0099] In terms of details, all data sources in the system can be uniformly managed in a single table, further achieving centralized unification. This makes it convenient to obtain database connection information from this table even when the business becomes increasingly complex, eliminating the need to create additional custom data connections in the code logic and enabling efficient and convenient management of data sources. After creating the corresponding data connection each time, the connection is placed in the data connection pool cache. Subsequently, when used in the business logic, it can be read from the cache using a unique identifier (data source configuration name), and data can be obtained from the corresponding data source through SQL or API without repeated creation.

[0100] In addition, isolation can be achieved between the created and different second data sources to further facilitate management. When each data source is adjusted, it will not affect each other, and it is convenient for later expansion.

[0101] In this regard, as an exemplary embodiment, the second data source can be specifically encapsulated in the database connection pool in an independent encapsulation manner.

[0102] It can also be seen from another embodiment that the second data source can also be directly encapsulated in the database connection pool data. In this way, this modification to the database connection pool itself can more conveniently call and manage the second data source.

[0103] Step S104, when there is a target business service related to the first data source, based on the second data source and the data source connection created in the data source connection pool, process the target business service.

[0104] It can be understood that after the configuration of the second data source and its data source connection is completed, it can be put into actual use. When a target business service related to the first data source appears later and the first data source needs to be used, compared with originally calling the first data source, at this time, the corresponding second data source can be determined, and an adapted data source connection can be obtained from the data source connection pool to call the second data source to process the target business service, meeting the usage requirements of originally calling the first data source.

[0105] In addition, when a certain data source no longer needs to maintain its access service later, it can also be removed. Taking the example of configuring the second data source and the data source connection in the data source connection pool mentioned above, since the configuration related to the data source has been saved in the corresponding data connection pool, the removal process does not require operating on the data source itself. The data source can be removed directly by operating on the data connection pool, and the corresponding configuration information is deleted at the same time.

[0106] Thus, for the above solution content, generally speaking, aiming at the data access target of multiple data sources, this application provides a novel multi-data source integration access solution. Without the operation cost problem of the existing solution that still requires real-time establishment of data source connections in some cases, it can stably and efficiently achieve seamless integration of multiple data sources, with high compatibility, data quality, processing efficiency, data security, flexibility and scalability. Therefore, it can meet the high-quality data access requirements of multiple data sources.

[0107] The above is an introduction to the data access method of the data source provided by this application. To facilitate better implementation of the data access method of the data source provided by this application, this application also provides a data access device for the data source from the perspective of functional modules.

[0108] Refer to Figure 2 , Figure 2 FIG. is a schematic structural diagram of a data access device for the data source of this application. In this application, the data access device 200 for the data source may specifically include the following structure:

[0109] A determination unit 201, configured to determine a first data source to be accessed in the network architecture where it is located;

[0110] An acquisition unit 202, configured to acquire configuration information of the first data source;

[0111] A creation unit 203, configured to create a corresponding second data source and a data source connection in the business system through a data source management service based on the configuration information of the first data source, and put the data source connection into a data source connection pool;

[0112] A processing unit 204, configured to process a target business service based on the second data source and the data source connection created in the data source connection pool when there is a target business service related to the first data source.

[0113] In an exemplary embodiment, the specific data source types of the first data source include relational database type, data warehouse type, time series database type, in-memory database type, and file storage system type. The relational database type specifically involves oracle, mysql, db2, and sqlServer. The data warehouse type specifically involves hive and Olap. The time series database type specifically involves influxdb. The in-memory database type specifically involves Redis and Memcached. The file storage system type specifically involves fastDFS and OSS.

[0114] In another exemplary embodiment, the determination unit 201 is specifically configured to:

[0115] From the set of candidate data sources, based on application requirements, analyze and evaluate in combination with data structure, format, access method, and security requirements to determine the first data source to be accessed in the network architecture.

[0116] In yet another exemplary embodiment, the configuration information specifically includes a database connection address, a username, a password, connection information, authentication credentials, and access permissions.

[0117] In yet another exemplary embodiment, the apparatus further includes:

[0118] An allocation unit 205, configured to allocate a unique identifier for the second data source;

[0119] A processing unit 204, specifically configured to:

[0120] When there is a target business service related to the first data source, directly call the second data source and the data source connection created in the data source connection pool with the allocated unique identifier to process the target business service.

[0121] In yet another exemplary embodiment, the apparatus further includes:

[0122] A verification unit 206, configured to verify the second data source and the data source connections in the data source connection pool by simulating the processing results of connection and data return to verify whether they are operating properly.

[0123] In yet another exemplary embodiment, the second data source is encapsulated in the data source connection pool in an independent encapsulation manner.

[0124] This application also provides a processing device from the perspective of hardware structure. Refer to Figure 3 , Figure 3 shows a schematic structural diagram of the processing device of this application. Specifically, the processing device of this application may include a processor 301, a memory 302, and an input / output device 303. The processor 301 is configured to implement the steps of the data access method of the data source in the corresponding Figure 1 embodiment when executing the computer program stored in the memory 302; or, the processor 301 is configured to implement the functions of the respective units in the corresponding Figure 2 embodiment when executing the computer program stored in the memory 302. The memory 302 is configured to store the computer program required for the processor 301 to execute the data access method of the data source in the above Figure 1 corresponding embodiment.

[0125] Exemplarily, a computer program can be divided into one or more modules / units. One or more modules / units are stored in the memory 302 and executed by the processor 301 to complete the present application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the computer device.

[0126] The processing device may include, but is not limited to, the processor 301, the memory 302, and the input / output device 303. Those skilled in the art can understand that the schematic diagram is only an example of the processing device and does not constitute a limitation on the processing device. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the processing device may also include a network access device, a bus, etc. The processor 301, the memory 302, the input / output device 303, etc. are connected through the bus.

[0127] The processor 301 can be a central processing unit (CPU), or can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the processing device and connects various parts of the entire device through various interfaces and lines.

[0128] The memory 302 can be used to store computer programs and / or modules. The processor 301 realizes various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 302 and calling the data stored in the memory 302. The memory 302 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc.; the data storage area can store data created according to the use of the processing device, etc. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one magnetic disk storage device, flash device, or other volatile solid-state storage devices.

[0129] When the processor 301 is used to execute the computer program stored in the memory 302, the following functions can be specifically realized:

[0130] Determine the first data source to be accessed in the network architecture;

[0131] Collect the configuration information of the first data source;

[0132] In the business system, through the data source management service, based on the configuration information of the first data source, create the corresponding second data source and data source connection, and put the data source connection into the data source connection pool;

[0133] When there is a target business service related to the first data source, based on the second data source and the data source connection created in the data source connection pool, process the target business service.

[0134] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the data access device, processing device and their corresponding units of the data source described above can refer to, for example, Figure 1 The description of the data access method of the data source in the corresponding embodiment, which will not be elaborated here specifically.

[0135] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling relevant hardware through instructions. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0136] Therefore, the present application provides a computer-readable storage medium, in which multiple instructions are stored, and these instructions can be loaded by a processor to execute the steps of the data access method of the data source in the present application as Figure 1 described in the corresponding embodiment. The specific operation can refer to, for example, Figure 1 the description of the data access method of the data source in the corresponding embodiment, which will not be elaborated here.

[0137] Among them, the computer-readable storage medium may include: Read Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, etc.

[0138] Since the instructions stored in the computer-readable storage medium can execute the steps of the data access method of the data source in the present application as Figure 1 described in the corresponding embodiment, therefore, the beneficial effects that can be achieved by the data access method of the data source in the present application as Figure 1 described in the corresponding embodiment can be realized. For details, please refer to the previous description, which will not be elaborated here.

[0139] The above has introduced in detail the data access method, device, processing equipment and computer-readable storage medium provided by this application. Specific examples are used in this article to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the core idea of this application; at the same time, for those skilled in the art, according to the idea of this application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A data access method for a data source, characterized in that: The method comprises: Determine a first data source to be accessed in the network architecture; Collecting configuration information of the first data source; In the business system, through the data source management service, based on the configuration information of the first data source, a corresponding second data source and a data source connection are created, and the data source connection is placed in a data source connection pool; When there is a target business service related to the first data source, the target business service is processed based on the second data source and the data source connection created in the data source connection pool.

2. The method according to claim 1, characterized in that The specific data source types of the first data source include relational database type, data warehouse type, time series database type, memory database type and file storage system type. The relational database type specifically involves oracle, mysql, db2 and sqlServer, the data warehouse type specifically involves hive and Olap, the time series database type specifically involves influxdb, the memory database type specifically involves Redis and Memcached, and the file storage system type specifically involves fastDFS and OSS.

3. The method according to claim 1, characterized in that The determining of a first data source to be accessed in the network architecture includes: From the set of candidate data sources, analysis and evaluation are performed based on application requirements and in combination with data structure, format, access method and security requirements to determine the first data source to be accessed in the network architecture.

4. The method according to claim 1, characterized in that: The configuration information specifically includes database connection address, user name, password, connection information, authentication credentials and access rights.

5. The method according to claim 1, characterized in that The method further comprises: assigning a unique identifier to the second data source; When there is a target business service related to the first data source, processing the target business service based on the second data source and the data source connection created in the data source connection pool includes: When the target business service related to the first data source exists, the second data source and the data source connection created in the data source connection pool are directly called with the allocated unique identifier to process the target business service.

6. The method according to claim 1, characterized in that The method further comprises: By simulating the processing results of the connection and data return, the second data source and the data source connection in the data source connection pool are verified to verify whether they are operating normally.

7. The method according to claim 1, characterized in that The second data source is encapsulated in the database connection pool in an independently encapsulated manner.

8. A data access device for a data source, characterized in that: The device comprises: A determination unit, configured to determine a first data source to be accessed in the network architecture; A collection unit, used for collecting configuration information of the first data source; A creation unit, configured to create a corresponding second data source and a data source connection based on the configuration information of the first data source through a data source management service in a business system, and place the data source connection into a data source connection pool; The processing unit is used to process the target business service based on the second data source and the data source connection created in the data source connection pool when there is a target business service related to the first data source.

9. A processing device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor executes the method according to any one of claims 1 to 7 when calling the computer program in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method according to any one of claims 1 to 7.

Images