Multi-data source operation method and device, service equipment and storage medium

Abstract

The invention provides a multi-data-source operation method and device, service equipment and a storage medium, and relates to the field of distributed databases. The method is applied to service equipment deployed with a dynamic data source facade, and the dynamic data source facade comprises a routing section and a strategy executor. The method comprises the following steps: intercepting a target data request sent by an application program user based on a routing section, and determining N target data sources indicated by the target data request from a plurality of data sources; next, according to a preset data operation mode and a preset data operation range corresponding to the N target data sources, determining a target execution strategy corresponding to the N target data sources based on a strategy executor; and further performing operation processing on the N target data sources according to the target execution strategy to obtain an operation result, and returning the operation result to the application program user. According to the method, a plurality of different data sources can be connected, and the operability of the data sources is improved.

Description

Technical Field

[0001] The present application relates to the field of distributed databases, and in particular to a multi-data source operation method, apparatus, service equipment and storage medium. Background Art

[0002] With the popularity and development of distributed microservices and information-based innovation strategies, banks are gradually transforming to distributed microservices and information-based innovation architecture systems. This makes the calling relationship between services more complicated, and the dependency relationship becomes meshed and multi-layered. In other words, the current system faces the scenario of connecting to multiple data sources at the same time.

[0003] The prior art first connects to a data source through the Java Database Connectivity (JDBC) interface in the Java platform to interact, and then further operates and processes the connected data source. However, the data may be distributed in many data sources. Since JDBC is usually bound to a specific type of database, the application may not know which data source the required data is in, or in which data sources it is scattered. In addition, if the application needs complete data aggregated from multiple data sources, it will dynamically perform operations and merge the operation results in several data sources according to the data request, but JDBC does not have built-in operations and data merging functions across multiple data sources.

[0004] Therefore, the prior art has the technical problem that it is unable to connect multiple different data sources and the operations on the data sources are limited. Summary of the invention

[0005] The present application provides a multi-data source operation method, apparatus, service equipment and storage medium, which are used to solve the technical problems in the prior art that multiple different data sources cannot be connected and the operation of data sources is limited.

[0006] In a first aspect, the present application provides a multi-data source operation method, which is applied to a service device, wherein a dynamic data source facade is deployed in the service device, wherein the dynamic data source facade includes a routing aspect and a policy executor, and the dynamic data source facade is connected to multiple data sources, and the method includes:

[0007] Based on the routing aspect, intercepting the target data request sent by the application user; according to the target data request, determining N target data sources indicated by the target data request from the multiple data sources; wherein N is a positive integer greater than or equal to 1;

[0008] Determining, based on the policy executor, a target execution policy corresponding to the N target data sources according to a preset data operation mode and a preset data operation range corresponding to the N target data sources;

[0009] Based on the target execution strategy, the N target data sources are operated and processed to obtain operation results; and the operation results are returned to the application user.

[0010] In a possible implementation manner, the preset data operation mode is a fast return mode or an aggregation mode;

[0011] The preset data operation range indicates any one of the following situations: operating on the N target data sources, operating on a designated target data source among the N target data sources.

[0012] In a possible implementation manner, the preset data operation mode is a fast return mode, and the target execution strategy represents a data search process for a target data source indicated by the preset data operation range;

[0013] The N target data sources are operated and processed based on the target execution strategy to obtain operation results, including:

[0014] Performing data search processing on the target data source indicated by the preset data operation range based on the target execution strategy to obtain a first return value of the target data source;

[0015] If the first return value meets the first preset requirement, it is determined that the data search process is successful, and the operation result obtained by performing the data search process is obtained.

[0016] In a possible implementation manner, the preset data operation mode is an aggregation mode, and the target execution strategy represents data aggregation processing for the target data source indicated by the preset data operation range;

[0017] The N target data sources are operated and processed based on the target execution strategy to obtain operation results, including:

[0018] Performing data aggregation processing on the target data source indicated by the preset data operation range based on the target execution strategy to obtain a second return value of the target data source;

[0019] If the second return value meets the second preset requirement, it is determined that the data aggregation processing is successful, and the operation result obtained by performing the data aggregation processing is obtained.

[0020] In a possible implementation, the method further includes:

[0021] When the preset data operation mode is the fast return mode, if it is determined that the target data source has an abnormality, the target data source having the abnormality is skipped, and the operation processing continues on the remaining target data sources;

[0022] When the preset data operation mode is the aggregation mode, if it is determined that an exception occurs in the target data source, the operation processing of the target data source where the exception occurs is stopped, and aggregate exception information is sent to the application user; wherein the aggregate exception information indicates the target data source where the exception occurs and the cause of the exception.

[0023] In a possible implementation, the method further includes:

[0024] Get the custom policy executor created by the application user;

[0025] The custom policy executor is added to the policy executor based on a preset configuration method.

[0026] In a possible implementation, the dynamic data source facade further includes a data source selector, and the method further includes:

[0027] Grouping multiple data sources connected to the dynamic data source facade based on the data source selector;

[0028] The N target data sources are operated and processed based on the target execution strategy to obtain operation results, including:

[0029] In the process of operating the N target data sources based on the target execution strategy, the N target data sources are switched based on the data source selector to obtain an operation result.

[0030] In a second aspect, the present application provides a multi-data source operation device, which is applied to a service device, wherein a dynamic data source facade is deployed in the service device, wherein the dynamic data source facade includes a routing aspect and a policy executor, and the dynamic data source facade is connected to multiple data sources, and the device includes:

[0031] An interception module, used for intercepting a target data request sent by an application user based on the routing aspect;

[0032] A determination module, configured to determine, according to the target data request, N target data sources indicated by the target data request from the multiple data sources; wherein N is a positive integer greater than or equal to 1;

[0033] The determination module is further used to determine the target execution strategy corresponding to the N target data sources based on the strategy executor according to the preset data operation mode and the preset data operation range corresponding to the N target data sources;

[0034] A processing module is used to perform operation processing on the N target data sources based on the target execution strategy to obtain operation results; and return the operation results to the application user.

[0035] In a possible implementation manner, the preset data operation mode is a fast return mode or an aggregation mode;

[0036] The preset data operation range indicates any one of the following situations: operating on the N target data sources, operating on a designated target data source among the N target data sources.

[0037] In a possible implementation manner, the preset data operation mode is a fast return mode, and the target execution strategy represents a data search process for a target data source indicated by the preset data operation range;

[0038] The multi-data source operation device further includes: an acquisition module,

[0039] The processing module is further configured to perform data search processing on the target data source indicated by the preset data operation range based on the target execution strategy to obtain a first return value of the target data source;

[0040] The determination module is further configured to determine that the data search process is successful if the first return value meets a first preset requirement;

[0041] The acquisition module is used to acquire the operation result obtained by performing data search processing.

[0042] In a possible implementation manner, the preset data operation mode is an aggregation mode, and the target execution strategy represents data aggregation processing for the target data source indicated by the preset data operation range;

[0043] The processing module is further configured to perform data aggregation processing on the target data source indicated by the preset data operation range based on the target execution strategy to obtain a second return value of the target data source;

[0044] The determination module is further configured to determine that the data aggregation process is successful if the second return value meets a second preset requirement;

[0045] The acquisition module is also used to obtain the operation results obtained by performing data aggregation processing.

[0046] In a possible implementation manner, the processing module is further used to:

[0047] When the preset data operation mode is the fast return mode, if it is determined that the target data source has an abnormality, the target data source having the abnormality is skipped, and the operation processing continues on the remaining target data sources;

[0048] When the preset data operation mode is the aggregation mode, if it is determined that an exception occurs in the target data source, the operation processing of the target data source where the exception occurs is stopped, and aggregate exception information is sent to the application user; wherein the aggregate exception information indicates the target data source where the exception occurs and the cause of the exception.

[0049] In a possible implementation manner, the multi-data source operation device further includes: an adding module,

[0050] The acquisition module is further used to acquire the custom policy executor created by the application user;

[0051] The adding module is used to add the custom policy executor to the policy executor based on a preset configuration method.

[0052] In a possible implementation, the dynamic data source facade further includes a data source selector, and the multiple data source operation device further includes: a grouping module, configured to group the multiple data sources connected to the dynamic data source facade based on the data source selector;

[0053] The processing module further includes: a switching module for switching the N target data sources based on the data source selector during the process of operating the N target data sources based on the target execution strategy to obtain an operation result.

[0054] In a third aspect, the present application provides a service device, including:

[0055] Processor and memory;

[0056] Wherein, the memory stores computer-executable instructions;

[0057] The processor executes the computer-executable instructions stored in the memory to implement the multi-data source operating method as described in the first aspect and various possible implementations of the first aspect.

[0058] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the multi-data source operation method as described in the first aspect and various possible implementations of the first aspect.

[0059] In a fifth aspect, the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it is used to implement the multi-data source operation method as described in the first aspect and various possible implementations of the first aspect.

[0060] The multi-data source operation method, device, service device and storage medium provided by the present application are applied to a service device deployed with a dynamic data source facade, wherein the dynamic data source facade is connected to multiple databases, and the dynamic data source facade includes a routing aspect and a policy executor. Further, the routing aspect is used to intercept the target data request sent by the application user, and determine the N target data sources indicated by the target data request from the dynamic data source facade. Next, according to the preset data operation mode and preset data operation range corresponding to the N target data sources, the target execution policy corresponding to the N target data sources is determined based on the policy executor. The connection of multiple databases is realized through the dynamic data source facade, and the required N target data sources are dynamically selected through the high flexibility and scalability of the routing aspect and the policy executor, and then the appropriate target execution policy is formulated to operate and process the N target data sources, obtain the operation result, and finally return the operation result to the application user. It can be seen that the present application is no longer limited to operating and processing a specific type of target data source, but can dynamically operate and process one or more target data sources from multiple data sources according to the target data request, and obtain the final operation result, which not only realizes the connection of multiple different data sources, but also improves the operability of the data source. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0062] Figure 1 An overall architecture diagram of the multi-data source operation method provided in an embodiment of the present application;

[0063] Figure 2 A flowchart of a multi-data source operation method provided in an embodiment of the present application;

[0064] Figure 3 A schematic diagram of the structure of a multi-data source operation device provided in an embodiment of the present application;

[0065] Figure 4 A hardware structure diagram of the service device provided in an embodiment of the present application.

[0066] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0067] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0068] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein, for example.

[0069] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0070] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0071] Terminology explanation:

[0072] Data source: The source of data generation. In computer architecture, it generally refers to various data storage middleware, including databases, file systems, message queues, etc. The data source in this application refers only to the database.

[0073] Dynamic Data Source Facade: It is the interface or entry point for applications to access data sources. It is used to manage the configuration and connection of multiple data sources. All data requests are processed and routed through the dynamic data source facade.

[0074] Routing aspect: Responsible for intercepting data requests and providing necessary context information for the data source selection process.

[0075] Policy executor: A component responsible for executing data source selection policies. It applies specific policies to select appropriate data sources based on the context information provided by the routing aspect.

[0076] Data source selector: The component responsible for actually selecting and configuring the data source based on the output of the policy executor. It converts the identifier returned by the policy executor into a specific data source instance.

[0077] With the popularity and development of distributed microservices and information-based innovation strategies, banks are gradually transforming to distributed microservices and information-based innovation architecture systems. In a microservice architecture, services communicate through the network. This communication may be synchronous or asynchronous. As the number of services increases, the call relationship between services forms a complex mesh structure. This complexity makes system management and debugging more difficult. At the same time, the dependency between services is not only direct, but also indirect, that is, one service depends on another service, which in turn depends on a third service. This multi-level dependency also increases the complexity of the system.

[0078] In addition, under the Xinchuang strategy, in order to enhance independent controllability and reduce dependence on foreign technology, traditional database middleware may need to be replaced by domestic databases. During the system transformation process, in order to ensure business continuity, the old and new systems may need to run in parallel, so multiple data sources need to be connected at the same time. Different data sources may use different database technologies (such as relational databases, non-relational databases, etc.), further increasing the complexity of data management.

[0079] In the prior art, JDBC is a set of application programming interfaces in the Java platform, which provides a standardized method to connect to a single data source, that is, a specific database instance, so that an application can interact with the data source.

[0080] However, each service in the microservice architecture has an independent data source, and different data sources use a variety of database technologies, which may result in data being distributed in multiple different data sources. Since JDBC is usually bound to a specific type of database and lacks unified management and access capabilities for multiple different data sources, the application may not know which data source the required data is in, or in which data sources it is scattered.

[0081] Similarly, when multiple data sources need to be dynamically selected and operated, JDBC's functionality will be insufficient because it does not have built-in operation processing and data merging capabilities across multiple data sources, especially when it is necessary to integrate the operation processing of multiple data sources.

[0082] As can be seen from the above, the prior art has the technical problems of being unable to connect multiple different data sources and having limited operations on data sources.

[0083] In response to the above technical problems, the inventors discovered during the process of studying the connection of multiple data sources that the existing interfaces are not flexible enough when connecting different data sources that use multiple database technologies, and cannot cope with the complex scenarios of operations and aggregation when the application connects to multiple data sources at the same time. Therefore, the inventors designed a dynamic data source facade that can aggregate multiple data sources together as the entrance to multiple data sources. At the same time, a routing aspect for intercepting data requests and a policy executor that can formulate relevant policies based on data requests are added to the dynamic data source facade. The policy executor analyzes the data requests intercepted by the routing aspect, determines the operations that the data request wants to achieve, and formulates appropriate strategies to operate and process multiple data sources to meet the complex scenarios of operations and aggregation of multiple data sources.

[0084] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0085] Figure 1 This is an overall architecture diagram of the multi-data source operation method provided in the embodiment of the present application. Figure 1 As shown, the method is applied to a service device, wherein a dynamic data source facade is deployed in the service device, and the dynamic data source facade includes a routing aspect, a policy executor, and a data source selector.

[0086] Specifically, the dynamic data source facade is used to connect to the default data source in the application running on the service device, as well as multiple other data sources; the routing aspect is used to intercept data requests sent by application users; the policy executor is used to formulate execution strategies corresponding to the data source based on data requests; and the data source selector is used to group and switch data sources.

[0087] based on Figure 1 In addition to the architecture shown, the embodiment of the present application also provides a multi-data source operation method. Figure 2 A flowchart of a multi-data source operation method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method includes:

[0088] S201. Intercept a target data request sent by an application user based on a routing aspect, and determine N target data sources indicated by the target data request from multiple data sources according to the target data request.

[0089] It is understood that application users generally refer to entities that directly use or interact with the application, which can be people, other software systems or devices. Specifically, application users can include end users, customers, client applications, other systems or services, and devices.

[0090] A target data request is a request sent by an application user to obtain certain specific data. The target data request sent by the application user will be intercepted by the routing aspect, which analyzes the target data request and selects one or more (N) target data sources from multiple connected data sources to meet the target data request based on the content and requirements of the target data request. Here, N is a positive integer greater than or equal to 1, which means that at least one target data source will be selected.

[0091] It can be seen that this design architecture is highly flexible and scalable, and is suitable for scenarios where data sources are dynamically selected, such as microservice architecture, distributed database systems, or multi-tenant environments.

[0092] S202: Determine a target execution strategy corresponding to the N target data sources based on a strategy executor according to preset data operation modes and preset data operation ranges corresponding to the N target data sources.

[0093] The preset data operation range indicates any one of the following situations: operating on N target data sources, or operating on a designated target data source among the N target data sources.

[0094] It can be understood that since N is a positive integer greater than or equal to 1, if the value of N is 1, it indicates that only one target data source is operated. If the value of N is the same as the number of data sources connected to the dynamic data source facade, it indicates that all data sources are target data sources, that is, all target data sources are operated. If the value of N is greater than 1 and less than the number of data sources connected to the dynamic data source facade, it indicates that some target data sources are operated.

[0095] The preset data operation mode is the quick return mode or the aggregation mode. If the preset operation mode is the quick return mode, the target execution strategy represents the data search processing for the target data source indicated by the preset data operation range, and immediately returns when the target data required by the target data request is found in the target data source. If the preset operation mode is the aggregation mode, the target execution strategy represents the data aggregation processing for the target data source indicated by the preset search range, that is, the target data source is traversed and operated, and after all processing is completed, all operation results are aggregated.

[0096] Exemplarily, when the value of N is 1 and the target data source is the default data source in the application, the target execution strategy is to perform operation processing (data search processing or data aggregation processing) on ​​the default data source. The target execution strategy in this case can be named the default execution strategy.

[0097] When the value of N is 1 and the target data source is a data source other than the default data source, the target execution strategy is to perform operations on the specified target data source. The target execution strategy in this case can be named a single target data source execution strategy.

[0098] When the value of N is the same as the number of data sources connected to the dynamic data source facade, and the data operation mode is selected as the fast return mode, the target execution strategy is to perform data search processing on all connected data sources and quickly return the searched target data. The target execution strategy in this case can be named the fast return execution strategy.

[0099] When the value of N is greater than 1 and less than the number of data sources connected to the dynamic data source facade, and the data operation mode is selected as the quick return mode, the target execution strategy is to perform search operations on some target data sources and quickly return the searched target data. The target execution strategy in this case can be named the in-range quick return execution strategy.

[0100] When the value of N is the same as the number of data sources connected to the dynamic data source facade, and the data operation mode is selected as the aggregation mode, the target execution strategy is to operate on all connected data sources and aggregate the operation results. The target execution strategy in this case can be named the aggregation execution strategy.

[0101] According to the above-mentioned preset data operation mode and preset data operation scope, the target execution strategy corresponding to N target data sources is determined based on the policy executor. In order to ensure that the policy executor can understand the needs of the application user, the target execution strategy can be designed as a string expression that the policy executor can understand: dataOps = "operationMode | operationScope". Among them, dataOps is the target execution strategy, operationMode is the preset data operation mode, and operationScope is the preset data operation scope.

[0102] For example, dataOps="FAST_RETURN|ds1" means performing data search processing on the target data source ds1, and returning the searched target data quickly, where FAST_RETURN is the fast return mode. dataOps="FAST_RETURN|ALL" means performing data search processing on all connected data sources, and returning the searched target data quickly. dataOps="FAST_RETURN|ds1,ds2" means performing data search processing on the target data sources ds1 and ds2, and returning the searched target data quickly. dataOps="AGGREGATE|ALL" means performing operations on all connected data sources and aggregating the operation results, where AGGREGATE is the aggregation mode. In addition, in one possible implementation, dataOps is not configured or is designed to perform search operations on the default data source and return the searched target data quickly.

[0103] It is worth noting that the policy executor also supports custom extensions by application users. Application users can create custom policy executors and add them to the existing policy executors through preset configurations. For example, custom policy executors can be added to the policy executors through Extensible Markup Language (XML) files.

[0104] Explanatory, when creating a custom policy executor, the application user needs to define the corresponding match method and invoke method in the custom policy executor. Among them, the match method is usually used to determine whether a certain execution policy or operation is applicable to the current context or condition. In the custom policy executor, the match method can be used to select the appropriate execution policy. The invoke method is used to execute the logic related to the execution policy or operation, and it is called when the match method returns true.

[0105] S203: Based on the target execution strategy, N target data sources are operated and processed to obtain operation results; and the operation results are returned to the application user.

[0106] Specifically, if the preset data operation mode is the fast return mode, a data search process is performed on the target data source indicated by the preset data operation range to obtain a first return value of the target data source. If the first return value meets the first preset requirement, it is determined that the data search process is successful, and the operation result obtained by performing the data search process is obtained.

[0107] Exemplarily, in the quick return mode, the first return value of the target data source is checked. If any one of the following three scenarios is met (the first preset requirement), the data search process is determined to be successful, and the operation result is quickly returned to the application user: (1) The first return value type is an integer and the first return value is greater than 0; (2) The first return value type is a non-empty collection class, which may specifically include all subclasses of non-empty collection classes, mapping classes (map), list classes (list), etc.; (3) The first return value type is void, that is, the target data source does not return any value, and no exception is thrown during the call to the target data source.

[0108] If the preset data operation mode is the aggregation mode, data aggregation processing is performed on the target data source indicated by the preset data operation range to obtain a second return value of the target data source. If the second return value meets the second preset requirement, it is determined that the data aggregation processing is successful, and the operation result obtained by performing the data aggregation processing is obtained.

[0109] Exemplarily, in the aggregation mode, the second return value of the target data source is checked, and the second return value of the target data source needs to meet the following requirements: (1) When performing a query operation, the second return value type must be a list type, that is, the query result of each target data source is a list; (2) When performing a modification operation (update, delete, insert, etc.), the second return value type is an integer, where the integer represents the number of records affected by the modification operation. For example, if 2, 3, and 5 records are updated in the three target databases respectively, then the second return value is 10, indicating that a total of 10 records are updated.

[0110] Furthermore, in the aggregation mode, if a query operation is performed, in order to obtain a complete query result, the lists from different target data sources need to be merged into a unified list. If a modification operation is performed, the number of records affected by the modification operation from different target data sources needs to be accumulated.

[0111] It is understandable that since there are N target data sources that need to be operated, when N is a positive integer greater than 1, it indicates that multiple target data sources need to be operated. After the operation processing of one of the target data sources is completed, it is necessary to switch to another target data source to continue the operation processing. Therefore, the dynamic data source facade includes a data source selector, which is used to implement the process of operating the N target data sources based on the target execution strategy, and then switch the N target data sources through the data source selector to obtain the operation result.

[0112] In addition to switching between N target data sources, the data source selector also provides another function, which is to group multiple data sources connected to the dynamic data source facade. Specifically, you can group them according to the data source type or other requirements. For example, group the same type of data source into a group, including oracle group, mysql group, etc. The string expression corresponding to the data source selector is com.bocsoft.bfwj.datasource.GroupedDataSourceSelector.

[0113] Exemplarily, a user-defined grouping method can be used to divide the data sources into a first group and a second group, where the first group includes data source ds1 and data source ds2, and the second group includes data source ds3; or the first group includes a default data source, and the second group includes data sources ds1 and ds2. The grouping method can be determined according to actual conditions and is not specifically limited.

[0114] It should be noted that in the process of operating and processing N target data sources based on the target execution strategy, there may be abnormal target data sources, and under different preset operation modes, the exception handling method is also different. Specifically, when the preset data operation mode is the quick return mode, if it is determined that the target data source has an exception, the target data source where the exception occurs is skipped, and the data search and processing of the remaining target data sources continues. When the preset data operation mode is the aggregation mode, if it is determined that the target data source has an exception, the data aggregation processing of the target data source where the exception occurs is stopped, and the aggregation exception information is sent to the application user; wherein the aggregation exception information indicates the target data source where the exception occurs and the cause of the exception.

[0115] In a possible implementation, the application user can also obtain the location of the target data source through the policy executor, and use the direct positioning mode to operate and process the target data source. The execution strategy in this case can be named the positioning execution strategy. In the positioning execution strategy, the return value of the target data source must be a map, and the located target data source is stored in the locatedDS key of the map, where the locatedDS key is an inherent key in the map. In addition, the user can also customize the addition of keys for storing business data (such as the result key) in the map. While storing the located target data source in the locatedDS key, the target data source is also stored in the result key, which is convenient for direct calling next time without having to search again. It should be noted that if the required target data source is not located, the corresponding return value should be a null value (null) or an empty map.

[0116] For example, the string expression corresponding to the location execution policy that can be understood by the policy executor is dataOps = "LOCATE|ALL", which means locating the target data source in all connected data sources, where LOCATE is the location execution policy. Further, the string of the location execution policy is converted into an instruction that can be executed by the policy executor, Rpc Context.getContext().setAttachment("dataOps", "LOCATE|ALL").

[0117] If the target data source is located, the instructions executed by the policy executor are as follows:

[0118] System.out.println(map); / / Indicates that the return value of the target data source is map

[0119] Object bizData = map.get("result"); / / Get the result key customized by the application user

[0120] String locatedDS = (String) map.get ("locatedDS"); / / indicates obtaining the locatedDS key in the map. Further, the instruction corresponding to storing the target data source in the result key is map.put ("result", name).

[0121] The multi-data source operation method provided by the present application is applied to a service device deployed with a dynamic data source facade, wherein the dynamic data source facade is connected to multiple databases, and the dynamic data source facade includes a routing aspect, a policy executor, and a data source selector. Further, the routing aspect is used to intercept the target data request sent by the application user, and to determine the N target data sources indicated by the target data request from the dynamic data source facade. Next, according to the preset data operation mode and the preset data operation range corresponding to the N target data sources, the target execution strategy corresponding to the N target data sources is determined based on the policy executor. Among them, the preset data operation mode includes a fast return mode or an aggregation mode, and the preset data operation range includes operating on the N target data sources or operating on the specified target data sources among the N target data sources. The connection of multiple databases is realized through the dynamic data source facade, and the required N target data sources are dynamically selected through the high flexibility and scalability of the routing aspect and the policy executor, and then a suitable target execution strategy is formulated to perform data search processing or data aggregation processing on the N target data sources or the specified target data sources among the N target data sources. At the same time, N target data sources are switched based on the data source selector, and then the operation is determined to be successful based on the return value of the target data source. After the operation is successful, the corresponding operation result is obtained, and finally the operation result is returned to the application user. It can be seen that this application is no longer limited to operating a specific type of target data source, but can dynamically perform data search processing or data aggregation processing from multiple data sources to one or more target data sources according to the target data request, and obtain the final data search result or data aggregation result, which not only realizes the connection of multiple different data sources, but also improves the operability of the data source.

[0122] Figure 3 This is a schematic diagram of the structure of the multi-data source operation device provided by this application. Figure 3 As shown, the multi-data source operation device 300 includes: an interception module 301, a determination module 302, and a processing module 303;

[0123] The interception module 301 is used to intercept the target data request sent by the application user based on the routing aspect;

[0124] A determination module 302 is used to determine, according to the target data request, N target data sources indicated by the target data request from multiple data sources; wherein N is a positive integer greater than or equal to 1;

[0125] The determination module 302 is further configured to determine, based on the policy executor, a target execution policy corresponding to the N target data sources according to the preset data operation mode and the preset data operation range corresponding to the N target data sources;

[0126] The processing module 303 is used to perform operation processing on N target data sources based on the target execution strategy to obtain operation results; and return the operation results to the application user.

[0127] In a possible implementation, the preset data operation mode is a fast return mode or an aggregation mode;

[0128] The preset data operation range indicates any one of the following situations: operating on N target data sources, operating on a designated target data source among the N target data sources.

[0129] In a possible implementation, the preset data operation mode is a fast return mode, and the target execution strategy represents a data search process for a target data source indicated by the preset data operation range;

[0130] The multi-data source operation device 300 further includes: an acquisition module 304,

[0131] The processing module 303 is further used to perform data search processing on the target data source indicated by the preset data operation range based on the target execution strategy to obtain a first return value of the target data source;

[0132] The determination module 302 is further configured to determine that the data search process is successful if the first return value meets the first preset requirement;

[0133] The acquisition module 304 is used to acquire the operation result obtained by performing data search processing.

[0134] In a possible implementation, the preset data operation mode is an aggregation mode, and the target execution strategy represents data aggregation processing for a target data source indicated by the preset data operation range;

[0135] The processing module 303 is further used to perform data aggregation processing on the target data source indicated by the preset data operation range based on the target execution strategy to obtain a second return value of the target data source;

[0136] The determination module 302 is further configured to determine that the data aggregation process is successful if the second return value meets a second preset requirement;

[0137] The acquisition module 304 is also used to obtain the operation result obtained by performing data aggregation processing.

[0138] In a possible implementation manner, the processing module 303 is further configured to:

[0139] When the preset data operation mode is the fast return mode, if it is determined that an abnormality occurs in the target data source, the target data source where the abnormality occurs is skipped, and the operation processing continues on the remaining target data sources;

[0140] When the preset data operation mode is the aggregation mode, if it is determined that an exception occurs in the target data source, the operation processing of the target data source where the exception occurs is stopped, and the aggregate exception information is sent to the application user; wherein the aggregate exception information indicates the target data source where the exception occurs and the cause of the exception.

[0141] In a possible implementation, the multi-data source operation device 300 further includes: an adding module 305,

[0142] The acquisition module 304 is also used to acquire a custom policy executor created by the application user;

[0143] The adding module 305 is used to add the custom policy executor to the policy executor based on a preset configuration method.

[0144] In a possible implementation, the dynamic data source facade further includes a data source selector, and the multiple data source operation device 300 further includes: a grouping module 306, configured to group multiple data sources connected to the dynamic data source facade based on the data source selector;

[0145] The processing module 303 further includes: a switching module 307 for switching the N target data sources based on the data source selector during the process of operating the N target data sources based on the target execution strategy to obtain an operation result.

[0146] The multi-data source operation device provided in the embodiment of the present application can be used to execute the multi-data source operation method in any of the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0147] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0148] Figure 4 The hardware structure diagram of the service equipment provided for this application. Figure 4 As shown, the service device 400 includes:

[0149] Processor 401 and memory 402;

[0150] Memory stores computer-executable instructions;

[0151] The processor executes the computer execution instructions stored in the memory 402, so that the service device executes the multi-data source operation method as described above.

[0152] It should be understood that the processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASIC). A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor. The memory 402 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a disk, or an optical disk.

[0153] The service device provided in the embodiment of the present application can be used to execute the multi-data source operation method provided in any of the above method embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0154] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the multi-data source operation method as described above.

[0155] The embodiment of the present application also provides a computer program product accordingly. The computer program product includes a computer program. When the computer program is executed by a processor, it is used to implement the multi-data source operation method as described above.

[0156] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0157] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0158] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0159] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0160] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0161] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0162] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0164] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for operating multiple data sources, characterized in that: The method is applied to a service device, wherein a dynamic data source facade is deployed in the service device, wherein the dynamic data source facade includes a routing aspect and a policy executor, and wherein the dynamic data source facade is connected to multiple data sources. The method includes: Based on the routing aspect, intercepting the target data request sent by the application user; according to the target data request, determining N target data sources indicated by the target data request from the multiple data sources; wherein N is a positive integer greater than or equal to 1; Determining, based on the policy executor, a target execution policy corresponding to the N target data sources according to a preset data operation mode and a preset data operation range corresponding to the N target data sources; Based on the target execution strategy, the N target data sources are operated and processed to obtain operation results; and the operation results are returned to the application user.

2. The method according to claim 1, characterized in that The preset data operation mode is a fast return mode or an aggregation mode; The preset data operation range indicates any one of the following situations: operating on the N target data sources, operating on a designated target data source among the N target data sources.

3. The method according to claim 2, characterized in that The preset data operation mode is a fast return mode, and the target execution strategy represents a data search process for a target data source indicated by the preset data operation range; The operating and processing the N target data sources based on the target execution strategy to obtain the operating results includes: Performing data search processing on the target data source indicated by the preset data operation range based on the target execution strategy to obtain a first return value of the target data source; If the first return value meets the first preset requirement, it is determined that the data search process is successful, and the operation result obtained by performing the data search process is obtained.

4. The method according to claim 2, characterized in that: The preset data operation mode is an aggregation mode, and the target execution strategy represents data aggregation processing for the target data source indicated by the preset data operation range; The N target data sources are operated and processed based on the target execution strategy to obtain operation results, including: Performing data aggregation processing on the target data source indicated by the preset data operation range based on the target execution strategy to obtain a second return value of the target data source; If the second return value meets the second preset requirement, it is determined that the data aggregation processing is successful, and the operation result obtained by performing the data aggregation processing is obtained.

5. The method according to claim 1, characterized in that The method further comprises: When the preset data operation mode is the fast return mode, if it is determined that an abnormality occurs in the target data source, the target data source where the abnormality occurs is skipped, and the operation processing continues on the remaining target data sources; When the preset data operation mode is the aggregation mode, if it is determined that an exception occurs in the target data source, the operation processing of the target data source where the exception occurs is stopped, and aggregate exception information is sent to the application user; wherein the aggregate exception information indicates the target data source where the exception occurs and the cause of the exception.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Get the custom policy executor created by the application user; The custom policy executor is added to the policy executor based on a preset configuration method.

7. The method according to any one of claims 1 to 5, characterized in that The dynamic data source facade further includes a data source selector, and the method further includes: Grouping multiple data sources connected to the dynamic data source facade based on the data source selector; The N target data sources are operated and processed based on the target execution strategy to obtain operation results, including: In the process of operating the N target data sources based on the target execution strategy, the N target data sources are switched based on the data source selector to obtain an operation result.

8. A multi-data source operation device, characterized in that: The device is applied to a service device, in which a dynamic data source facade is deployed, the dynamic data source facade includes a routing aspect and a policy executor, the dynamic data source facade is connected to multiple data sources, and the device includes: An interception module, used for intercepting a target data request sent by an application user based on the routing aspect; A determination module, configured to determine, according to the target data request, N target data sources indicated by the target data request from the multiple data sources; wherein N is a positive integer greater than or equal to 1; The determination module is further configured to determine, based on the policy executor, a target execution policy corresponding to the N target data sources according to a preset data operation mode and a preset data operation range corresponding to the N target data sources; A processing module is used to perform operation processing on the N target data sources based on the target execution strategy to obtain operation results; and return the operation results to the application user.

9. A service device, characterized in that: Applied to a security platform, comprising: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the multi-data source operating method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the multi-data source operation method according to any one of claims 1 to 7.

Images