Data query method and device and computer equipment

Abstract

The invention relates to a data query method and device and computer equipment. The method comprises the following steps: monitoring each data source corresponding to a service system, and acquiring a target update data set of target data source update under the condition that at least one target data source update exists; performing uniform format transcoding on the target update data set, and storing the transcoded target update data set into a memory database; receiving a query request sent by a service system, and generating a target query statement according to a request parameter in the query request; and querying the target query data set in the memory database according to the target query statement. By adopting the method, the data query efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, and particularly relates to a data query method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] In the execution process of a business system, a target update data set is often obtained from a data source. The target update data set is used to support the operation of the business system. For example, in order to construct a user portrait, attribute information, transaction data and browsing data of a target user are obtained from different data sources, and data analysis is performed on the data to obtain the user portrait of the target user.

[0003] In the traditional technology, the business system is associated with each data source, and the data query requirements of each data source are different. In order to obtain target data, the developer needs to write a SQL (Structured Query Language) statement and an ETL (Extract: data extraction, Transform: data conversion, Load: data loading) template corresponding to each data source according to the business requirements, and obtain the target data from each data source according to the SQL statement and the ETL template.

[0004] However, in the traditional technology, because the data formats of the data sources are different and the data acquisition requirements of each data source are different, a large amount of time is needed to write complex SQL statements and ETL templates, which leads to slow speed of obtaining target data. Therefore, the efficiency of the current data query method is low. SUMMARY

[0005] Therefore, it is necessary to provide a data query method, device, computer equipment, computer readable storage medium and computer program product to solve the above technical problems.

[0006] In a first aspect, the present application provides a data query method, comprising:

[0007] listening to each data source corresponding to a business system, and obtaining a target update data set updated by at least one target data source in a case where the target data source exists;

[0008] performing uniform format transcoding on the target update data set, and storing the transcoded target update data set into an in-memory database;

[0009] receiving a query request sent by the business system, and generating a target query statement according to a request parameter in the query request;

[0010] querying a target query data set in the in-memory database according to the target query statement.

[0011] In one of the embodiments, the obtaining the target update data set in the presence of at least one target data source update comprises:

[0012] In the presence of at least one target data source update, obtaining the to-be-updated data set from the target data source in response to update information of the target data source;

[0013] Determining a target update template corresponding to the target data source, and performing data processing on the to-be-updated data set based on the target update template to obtain the target update data set.

[0014] In one of the embodiments, the obtaining the to-be-updated data set from the target data source in the presence of at least one target data source update in response to update information of the target data source comprises:

[0015] In the presence of at least one target data source update, determining a target collection tool corresponding to the target data source in response to change information of the target data source;

[0016] Obtaining full-amount data from the target data source by using the target collection tool to obtain an initial data set;

[0017] Based on a historical update time and a current time, filtering a to-be-updated data set from the initial data set.

[0018] In one of the embodiments, the determining a target update template corresponding to the target data source, and performing data processing on the to-be-updated data set based on the target update template to obtain the target update data set comprises:

[0019] According to a data source identifier of the target data source, determining each target update template corresponding to the target data source;

[0020] Based on a target parsing tool identifier in the target update template, determining a target parsing tool, and based on a target processing function identifier in the target update template, determining a target processing function;

[0021] Based on the target parsing tool, the target processing function, and a target parsing rule in the target update template, performing data processing on the to-be-updated data set to obtain the target update data set.

[0022] In one of the embodiments, the performing data processing on the to-be-updated data set based on the target parsing tool, the target processing function, and a target parsing rule in the target update template to obtain the target update data set comprises:

[0023] The target analysis tool and target analysis rules in the target update template are used to analyze the to-be-updated data set, to obtain an initial target update data set; the target analysis tool is consistent with the format of the update data set;

[0024] The initial target update data set is processed according to the target processing function, to obtain a target update data set.

[0025] In one of the embodiments, the uniform format transcoding of the target update data set and the storage of the transcoded target update data set into the in-memory database include:

[0026] A target data table associated with the target update data set is determined in the in-memory database;

[0027] The target update data set is uniformly format-converted according to the format of the in-memory database, and the format-converted target update data set is updated into the target data table of the in-memory database.

[0028] In one of the embodiments, before the listening business system corresponds to each data source, the method further includes:

[0029] For each business interface in the business system, feature information of the business interface is acquired, and full-amount initial data is acquired from a corresponding data source according to a data source identifier in the feature information; the feature information includes a positioning time, an analysis rule and a processing function identifier;

[0030] The initial data is analyzed according to the data format of the initial data, the positioning time and the analysis rule, to obtain each analyzed initial data, and a business data set is determined according to the processing function identifier and each processed initial data;

[0031] The business data set is stored into the in-memory database according to the format of the in-memory database.

[0032] In one of the embodiments, the initial data is analyzed according to the data format of the initial data, the positioning time and the analysis rule, to obtain each analyzed initial data, and a business data set is determined according to the processing function identifier and each processed initial data, including:

[0033] A target analysis tool corresponding to the data format of the initial data is determined based on the data format of the initial data, and each initial data is analyzed based on the positioning time, the analysis rule and the target analysis tool;

[0034] A processing function corresponding to the initial data is determined according to the processing function identifier, and each analyzed initial data is processed according to the processing function, to obtain a business data set.

[0035] According to the parsing rule, the parsing tool and the processing function, a data update template is constructed, and the data source and the data update template are associated.

[0036] In a second aspect, the present application further provides a data query device, comprising:

[0037] A listening module is configured to listen to each data source corresponding to a business system, and acquire a target update data set of a target data source update in a case where there is at least one target data source update;

[0038] A transcoding module is configured to perform uniform format transcoding on the target update data set, and store the transcoded target update data set into an in-memory database;

[0039] A receiving module is configured to receive a query request sent by the business system, and generate a target query statement according to a request parameter in the query request;

[0040] A querying module is configured to query a target query data set in the in-memory database according to the target query statement.

[0041] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor realizes the following steps when executing the computer program:

[0042] The business system corresponding to each data source is listened to, and a target update data set of a target data source update is acquired in a case where there is at least one target data source update;

[0043] The target update data set is subjected to uniform format transcoding, and the transcoded target update data set is stored into an in-memory database;

[0044] A query request sent by the business system is received, and a target query statement is generated according to a request parameter in the query request;

[0045] A target query data set is queried in the in-memory database according to the target query statement.

[0046] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program realizes the following steps when executed by a processor:

[0047] The business system corresponding to each data source is listened to, and a target update data set of a target data source update is acquired in a case where there is at least one target data source update;

[0048] uniformly format-encode the target update data set, and store the uniformly format-encoded target update data set into the in-memory database;

[0049] receive a query request sent by the business system, and generate a target query statement according to a request parameter in the query request;

[0050] query a target query data set in the in-memory database according to the target query statement.

[0051] In a fifth aspect, the present application further provides a computer program product, comprising a computer program which, when executed by a processor, implements the following steps:

[0052] monitor each data source corresponding to the business system, and acquire a target update data set of target data source update in a case where there is at least one target data source update;

[0053] uniformly format-encode the target update data set, and store the uniformly format-encoded target update data set into the in-memory database;

[0054] receive a query request sent by the business system, and generate a target query statement according to a request parameter in the query request;

[0055] query a target query data set in the in-memory database according to the target query statement.

[0056] The data query method, device, computer device, computer readable storage medium and computer program product described above monitor each data source corresponding to the business system, acquire a target update data set of target data source update in a case where there is at least one target data source update, uniformly format-encode the target update data set, store the uniformly format-encoded target update data set into the in-memory database, receive a query request sent by the business system, generate a target query statement according to a request parameter in the query request, and query a target query data set in the in-memory database according to the target query statement. By using the method, in a case where there is target data source update, the updated target update data set is uniformly format-encoded, and the uniformly format-encoded target update data set is stored into the in-memory database, so that the business system can directly acquire the target query data set in the in-memory database through a simple query statement, the target query statement is simplified, the generation time of the target query statement is reduced, and the efficiency of the data query method is improved. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 An application environment diagram of a data query method in an embodiment;

[0059] Figure 2 A flowchart of a data query method in an embodiment;

[0060] Figure 3 A flowchart of determining a target update data set in an embodiment;

[0061] Figure 4 A flowchart of obtaining a to-be-updated data set in an embodiment;

[0062] Figure 5 A flowchart of obtaining a target update data set in an embodiment;

[0063] Figure 6 A flowchart of parsing and processing a to-be-updated data set in an embodiment;

[0064] Figure 7 A flowchart of storing a target update data set in an embodiment;

[0065] Figure 8 A flowchart of storing a business data set in an embodiment;

[0066] Figure 9 A flowchart of parsing and processing each initial data in an embodiment;

[0067] Figure 10 A data transformation diagram of a data query method in an exemplary embodiment;

[0068] Figure 11 A structural block diagram of a data query device in an embodiment;

[0069] Figure 12 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0070] In order to make the purposes, technical solutions and advantages of the present application clearer, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0071] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0072] The data query method provided in this application embodiment can be applied to, for example... Figure 1 The data query system 100 shown includes a business system 110, a server 120, and various data sources 130. The business system 110 executes various business operations, interacting with the server 120 during execution. The server 120 interacts with each data source 130, retrieving the target query dataset required by the business system from each data source 130. The business system 110 is a server cluster or distributed system composed of multiple physical servers. The server 120 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Data sources can be, but are not limited to, different types of databases or file storage systems.

[0073] In one exemplary embodiment, such as Figure 2 As shown, a data query method is provided, which can be applied to... Figure 1 Taking server 120 (hereinafter referred to as the server) as an example, the explanation includes the following steps 202 to 208. Wherein:

[0074] Step 202: Monitor each data source corresponding to the business system. If at least one target data source is updated, obtain the target update dataset of the target data source.

[0075] In implementation, the business system is associated with various data sources. For each data source, the server places a listener. This listener monitors for updates to the data in the data source. If at least one target data source is updated, the listener in that target data source sends an update notification to the server. In response to this update, the server uses the corresponding target acquisition tool to process and retrieve the updated target dataset from the target data source.

[0076] Specifically, the business system includes different business modules. Each business module generates different data and stores the data in different data sources. Therefore, in order to monitor the changes of the data sources, the server needs to place a listener at each data source. When the listener detects a change in the data source, the listener sends information about the data source update to the server. The server receives the information about the data source change and determines the changed data source as a target data source. Then, the server determines a target collection tool corresponding to the target data source, and acquires a to-be-updated data set from the target data source according to the target collection tool, and performs data processing on the to-be-updated data set according to a target update template corresponding to the target data source, to obtain a target update data set.

[0077] Optionally, the data source can be, but is not limited to, various types of databases or file storage systems. The database includes a relational database and a non-relational database. The file types in the file storage system include, but are not limited to, table files or CSV (Comma-Separated Values) files. The embodiments of the present application do not limit the data source.

[0078] In step 204, the target update data set is uniformly format-converted, and the converted target update data set is stored in the in-memory database.

[0079] In implementation, the server uniformly format-converts the target update data set according to the format of the in-memory database, to obtain a converted target update data set. Then, the server stores the target update data set in the target database.

[0080] Specifically, the server determines a target data table corresponding to the target update data set in the in-memory database. Then, the server uniformly format-converts the target update data in the target update data set according to the format of the in-memory database, to obtain a converted target update data set. The server stores the to-be-updated data in the converted target update data set in the target data table in batches through a database plug-in.

[0081] In step 206, a query request sent by the business system is received, and a target query statement is generated according to request parameters in the query request.

[0082] In implementation, when the business system needs to use data in the data source, the business system sends a query request to the server. The server receives the query request sent by the business system. Then, the server determines a target query template from the query templates according to a query target and a query type in the query request. Then, the server generates a target query statement according to the target query template and the request parameters in the query request.

[0083] In an example embodiment, in the data query stage, when the business system needs to use the data in the data source, the business system sends a query request to the server. The query request contains the query target and the query type that need to be queried. The query target is the data table in the in-memory database, and the query type includes but is not limited to filtering, sorting, and aggregation. Then, the server determines the target query template from each query template according to the query target and the query type in the query request. Then, the server generates the target query statement according to the target query template and each request parameter in the query request. Each request parameter is a detailed parameter required by the business system to query. For example, the target query statement can filter the sales data in a specific time period, or sort and group the customer information. These operations can extract valuable information from a large amount of data to support decision-making. Through the powerful functions of SQL, combined with advanced features such as window functions and subqueries, complex data conversion is performed to ensure the accuracy and efficiency of the conversion results.

[0084] In an example embodiment, the business system is a bank system. The server receives the query request sent by the business system. The query request contains the query target transformed_person, the query type (filtering), and each query parameter. The query request represents that the business system needs to create a mail domain name statistics table. The server determines the target query template from each query template according to the query target and the query type. Among them, the target query template is CREATE TABLE XXXX AS

[0085] SELECT

[0086] domain,

[0087] COUNT(*) AS XXXX

[0088] FROM transformed_person

[0089] GROUP BY domain

[0090] HAVING XXXX;

[0091] Among them, in the above target query template, XXXX is a placeholder. The server updates the placeholder in the target query template based on the request parameter to obtain the target query statement. Among them, the target query statement is as follows:

[0092] CREATE TABLE domain_stats AS

[0093] SELECT

[0094] domain,

[0095] COUNT(*) AS user_count

[0096] FROM transformed_person

[0097] GROUP BY domain

[0098] HAVING COUNT(*) > 0.

[0099] Step 208, querying the target query data set in the in-memory database according to the target query statement.

[0100] In implementation, the server queries data in the in-memory database according to the target query statement, and obtains the target query data set.

[0101] In an exemplary embodiment, the target query statement is a precompiled statement, thereby preventing SQL injection. Massive data retrieval is processed by a paging query ROWNUM mechanism (a pseudo-column mechanism in a database) to retrieve the required data of the business system, i.e., the target query data set, from the in-memory database. The target query data set can be directly used for subsequent data processing or provided to the business system to support various business services. The flexibility and powerful data processing capability of SQL query make it the preferred tool for data retrieval. Through the target query statement, combined with index and query optimization techniques, the required data can be quickly and accurately obtained to meet business requirements.

[0102] In the above data query method, in the case where there is a target data source update, by uniformly format-converting the updated target update data set and storing the format-converted target update data set in the in-memory database, the business system can directly obtain the target query data set in the in-memory database through a simple query statement, simplifying the target query statement and reducing the generation time of the target query statement, thereby improving the efficiency of the data query method.

[0103] In an exemplary embodiment, as shown in FIG. 3, the specific process of obtaining the target update data set of the target data source update in step 202 in the case where there is at least one target data source update includes steps 302 to 304. Wherein: Figure 3 Step 302, in the case where there is at least one target data source update, obtaining the to-be-updated data set from the target data source in response to the update information of the target data source.

[0104] Step 302, in the case where there is at least one target data source update, obtaining the to-be-updated data set from the target data source in response to the update information of the target data source.

[0105] Step 304, in the case where there is at least one target data source update, format-converting the to-be-updated data set to obtain the target update data set of the target data source update.In implementation, in case that there is at least one target data source update, the server receives an update message of the target data source, and determines a target collection tool corresponding to the target data source and a historical update time. Then, the server acquires a to-be-updated data set from the target data source according to the historical update time and the target collection tool.

[0106] Specifically, in case of data source update, a listener in the data source sends information of the data source update to the server. The server receives the information of the data source update, and determines the updated data source as a target data source. Then, the server determines a target collection tool corresponding to the target data source and a historical update time according to the type of the target data source and the data table identifier of the target data source. The historical update time is the time of the nearest update from the current time. The server collects full-amount data from the target data source according to the target collection tool, to obtain an initial data set, and filters a to-be-updated data set in the initial data set according to the historical update time and the current time.

[0107] In an optional embodiment, if the data source is a file in a file storage system, the server does not need to place a listener at the data source. When the file is updated, the file sends information of the file update to the server.

[0108] Step 304, determining a target update template corresponding to the target data source, and performing data processing on the to-be-updated data set based on the target update template to obtain a target update data set.

[0109] In implementation, the server is provided with an association relationship between each data source and each update template. The server determines a target update template associated with the target data source based on the association relationship between each data source and each update template. Then, the server performs data processing on the to-be-updated data set according to the target parsing tool identifier, the target parsing rule and the target processing function identifier in the target update template, to obtain a target update data set.

[0110] Specifically, in the process of acquiring an initial data from each data source and storing the initial data to the in-memory database, a data update template corresponding to the data source is constructed, and an association relationship between the data source and the data update template is established. The server determines an update template associated with the target data source based on the association relationship between each data source and each update template, and determines the template as a target update template. The target update template contains a target parsing tool identifier, a target parsing rule and a target processing function identifier. The server determines a target parsing tool based on the target parsing tool identifier and a target processing function based on the target processing function identifier for each target update template. Then, the server performs data processing on the to-be-processed data set according to the target parsing tool, the target processing function and the target parsing rule, to obtain a target update data set.

[0111] Optionally, one data source can be associated with multiple data templates, and embodiments of the present application do not limit the data source.

[0112] In the embodiment, in the case of target data source update, the uniform analysis and format conversion of the to-be-updated data of the update are performed, the uniform conversion of different format heterogeneous data is realized, the efficiency and accuracy of data processing are improved, the target query data is obtained by simple query statement in the subsequent, the target query statement is checked, the generation time of the target query statement is reduced, and the efficiency of the data query method is improved.

[0113] In one exemplary embodiment, as shown in FIG. 4, Figure 4 the specific processing process of step 302 includes steps 402 to 406. Among them:

[0114] In step 402, in the case of at least one target data source update, the target acquisition tool corresponding to the target data source is determined in response to the change information of the target data source.

[0115] In the implementation, in the case of at least one target data source update, the target acquisition tool corresponding to the target data source is determined according to the type of the target data source in response to the change information of the target data source.

[0116] Specifically, in the case of data source update, the data source sends a message of data source update to the server. The server receives the information of data source update, and determines the updated data source as the target data source. Then, the server determines the target acquisition tool corresponding to the target data source according to the type of the target data source for each target data source.

[0117] In an example embodiment, the business system is associated with a plurality of different data sources, including but not limited to file systems (such as HDFS (Hadoop Distributed File System), FTP (File Transfer Protocol) servers) (notification mechanism), databases (relational MySQL, non-relational MongoDB) (listening), APIs (Application Programming Interface) (RESTful interface, SOAP (Simple Object Access Protocol) service). And the data format of the data source includes but is not limited to CSV, Excel, JSON (JavaScript Object Notation), XML (eXtensible Markup Language), Parquet (Apache Parquet, an open source file format). In the case of data source update, the data source sends a message of data source update to the server. The server receives the information of the data source update, and determines the updated data source as a target data source. Then, the server determines the target collection tool corresponding to the target data source according to the type of the target data source for each target data source. For example, if the type of the target data source is an Excel file, the server determines the target collection tool as OpenCSV, an open source Java library (a development tool library specially processing CSV). If the data format in the target data source is JSON, the server determines the target collection tool as jackson-databind library (a development tool library).

[0118] Step 404, obtaining full data from the target data source by the target collection tool to obtain an initial data set.

[0119] In implementation, the server collects full data of the target data source from the target data source based on the target collection tool to obtain an initial data source.

[0120] In an example embodiment, the target data source is an Excel file. The server obtains full data of the Excel file from the Excel file by OpenCSV, an open source Java library, to obtain an initial data set.

[0121] Step 406: Based on the historical update time and the current time, filter the dataset to be updated in the initial dataset.

[0122] The historical update time is the time when the target data source was last updated.

[0123] In implementation, the server determines the historical update time as the start time and the current time as the determined end time. Then, using the start and end times as filtering criteria, the server filters the initial dataset for each piece of data that has been updated, thus obtaining the dataset to be updated.

[0124] Specifically, the server determines the start time based on historical update times and the end time based on the current time. Each data point in the initial dataset has a corresponding data update record, which contains the update time for that data. For each data point in the initial dataset, the server determines whether its corresponding update time falls between the start and end times. If the update time falls between the start and end times, the server identifies that data as data to be updated. The server then constructs a dataset to be updated based on each piece of data to be updated.

[0125] In this embodiment, a suitable target acquisition tool is matched to the target data source based on its type, and the dataset to be updated is acquired using the target acquisition tool, thereby accelerating the data acquisition speed and improving the data acquisition efficiency.

[0126] In one exemplary embodiment, such as Figure 5 As shown, the specific processing procedure of step 304 includes steps 502 to 506. Wherein:

[0127] Step 502: Determine the target update templates corresponding to the target data source based on the data source identifier of the target data source.

[0128] In practice, the server selects the target update template that corresponds to the target data source from among the various update templates based on the data source identifier of the target data source.

[0129] Specifically, during the initial data retrieval from the data source and storage in the in-memory database, the server constructs an update template for that data source to facilitate subsequent data updates. Therefore, the server sets up various update templates and their associations with different data sources. Since a data source can generate multiple in-memory tables, it can be associated with multiple update templates. Based on the associations between data sources and update templates, the server determines the update templates associated with the target data source and designates each associated update template as a target update template.

[0130] At step 504, the target parsing tool is determined based on the target parsing tool identifier in the target update template, and the target processing function is determined based on the target processing function identifier in the target update template.

[0131] The target update template includes the target parsing tool identifier, the target processing function identifier, and the target parsing rule.

[0132] In implementation, the server determines the target parsing tool according to the target parsing tool identifier, and calls the target processing function according to the target processing function identifier.

[0133] In an exemplary embodiment, the type of the data set to be updated is a CSV file or an Excel file. The target parsing tool identifier is the name of the CSVReader method (a function for reading the content of a CSV file) in the OpenCSV library. The server determines the CSVReader method as the target parsing tool in the tool library according to the name of the CSVReader method in the OpenCSV library. Then, the server calls the target processing function according to the name of the target processing function. Specifically, the target processing function can filter, sort, and aggregate the data to be updated in each data set to be updated, thereby obtaining a brand-new target update data set.

[0134] In an exemplary embodiment, the type of the data set to be updated is a data table. The target parsing tool identifier is the name of the XSSFWorkbook (a function for processing a table) in the Apache POI (an open source tool library). The server determines the XSSFWorkbook of the Apache POI as the target parsing tool in the tool library according to the name of the XSSFWorkbook of the Apache POI. Then, the server calls the target processing function according to the name of the target processing function. Specifically, the target processing function can filter, sort, and aggregate the data to be updated in each data set to be updated, thereby obtaining a brand-new target update data set.

[0135] Optionally, one target update template can include multiple target parsing tool identifiers, that is, one data set to be updated can be parsed by multiple target parsing tools. One target update template can include multiple target processing function identifiers, that is, one data set to be updated can be parsed by multiple target processing functions. The number of target processing functions and target parsing tools in the embodiments of the present application is not limited.

[0136] At step 506, the data set to be updated is processed based on the target parsing tool, the target processing function, and the target parsing rule in the target update template, thereby obtaining a target update data set.

[0137] In implementation, the server processes the dataset to be updated using the target parsing tool and target parsing rules to obtain the initial target update dataset. Then, the server processes the initial target update dataset using the target processing function to obtain the final target update dataset.

[0138] In this embodiment, by determining the target parsing tool, target processing function, and target parsing rules that match the dataset to be updated, the dataset to be updated can be quickly parsed and processed according to the target parsing tool, target processing function, and target parsing rules to obtain the stored target updated dataset, thus improving data processing efficiency. Simultaneously, it can accurately parse the dataset to be updated, improving data processing accuracy.

[0139] In one exemplary embodiment, such as Figure 6 As shown, the specific processing procedure of step 506 includes steps 602 to 604. Wherein:

[0140] Step 602: Based on the target parsing rules in the target parsing tool and the target update template, the dataset to be updated is parsed to obtain the initial target update dataset.

[0141] The target parsing tool is compatible with the format of the updated dataset.

[0142] In implementation, the server runs a target parsing tool to parse the dataset to be updated according to the target parsing rules in the target update template, resulting in the parsed initial target update dataset. The server employs a multi-layered parsing strategy, combining regular expressions for pattern matching, data cleaning techniques to handle missing values, and data validation techniques to ensure type safety, ensuring that every detail of the data is accurately captured and understood.

[0143] In one exemplary embodiment, the dataset to be updated is a CSV file or an Excel file. The target parsing rule is a custom delimiter. The target parsing tool is the CSVReader method from the OpenCSV library. The server runs the CSVReader method to process special characters in the dataset to be updated according to the custom delimiter and parse the dataset to be updated to obtain the parsed initial target updated dataset.

[0144] In one exemplary embodiment, the dataset to be updated is a data table dataset. The target parsing tool is Apache POI's XSSFWorkbook. The server uses XSSFWorkbook to parse the dataset to be updated, obtaining the parsed initial target update dataset.

[0145] In an example embodiment, the data format of the data set to be updated is JSON format. The target parsing tool is a JSON Schema in a json library. The server verifies the data set to be updated by the JSON Schema in the json library, and performs structured parsing to obtain the parsed initial target update data set. The parsing work is crucial because it determines whether the server can accurately understand the meaning of the data.

[0146] Optionally, the target parsing rule can be, but is not limited to, a regular expression or a target matching rule, and the embodiments of the present application do not limit the target parsing rule.

[0147] In step 604, the initial target update data set is processed according to the target processing function to obtain the target update data set.

[0148] In implementation, the server filters, sorts, and aggregates the data to be updated in the initial target update data set according to the target processing function to obtain the target update data set.

[0149] In an example embodiment, the initial target update data set is a mailbox data set. The target processing function is to filter the mailbox data created on July 21, 2025 18:39:59 to August 21, 2025 18:39:52. The server filters the mailbox data created on July 21, 2025 18:39:59 to August 21, 2025 18:39:52 in the mailbox data set by the target processing function to obtain the target update data set.

[0150] In this embodiment, the data set to be updated is parsed and processed according to the target parsing tool, the target processing function, and the target parsing rule to obtain the stored target update data set, which realizes efficient data conversion and improves data processing efficiency. At the same time, the data set to be updated can be accurately parsed, which improves the accuracy of data processing.

[0151] In an example embodiment, as shown in FIG. 7, Figure 7 the specific processing process of step 204 includes steps 702 to 704. Among them:

[0152] In step 702, the target data table associated with the target update data set is determined in the in-memory database.

[0153] Among them, the target data table related to the target update data set is a data table storing data in the target data source.

[0154] In implementation, the server filters each data table storing data in the target data source in each data table of the in-memory database, and determines these data tables as the target data table associated with the target update data set.

[0155] Optionally, the in-memory database can be, but is not limited to, H2 in-memory database (an open source relational database), and the embodiments of the present application do not limit the in-memory database.

[0156] In step 704, the target update data set is uniformly format-converted according to the format of the in-memory database, and the format-converted target update data set is updated into the target data table of the in-memory database.

[0157] In implementation, the server uniformly format-converts the to-be-updated data in the target update data set according to the relational model of the in-memory database, to obtain each converted to-be-updated data. Each converted to-be-updated data exists in a normalized data table structure. Then, the server updates each format-converted to-be-updated data table into the target data table of the in-memory database through a database plug-in.

[0158] In an exemplary embodiment, the server converts the target update data table into a normalized data table structure according to the relational model of the in-memory database (for example, H2, Apache Derby (an open source relational data management system)), establishes a primary-foreign key constraint through entity relationship mapping, to obtain the format-converted target data table. In this process, the original meaning and correlation of the data need to be preserved, and a data dictionary containing version control is designed, and a data type conversion matrix (for example, string to TIMESTAMP (a time data format), numerical value precision adjustment) is implemented, to ensure the integrity and consistency of the data. The conversion work needs to accurately map the data structure, and nested JSON objects are processed by flattening, and layered XML (Extensible Markup Language) data is processed by node splitting, so that the in-memory database can efficiently process these data. The server loads the converted target update data set into the in-memory database through batch insertion of JDBC (a standard application program interface for standardizing client programs to access databases) or using the CrudRepository interface (a data processing interface) of Spring Data (a way to simplify the data access layer). The purpose of this is to facilitate subsequent data processing work, so that the data can be quickly accessed and operated. The high-speed access characteristics (for example, MVCC (Multi-Version Concurrency Control) transaction mechanism of H2) of the in-memory database make data processing more efficient, which is particularly important for real-time analysis, online transaction processing and other application scenarios that require fast response. Through the high-performance characteristics of the in-memory database, a two-level cache strategy is implemented through a persistence framework such as MyBatis (a persistence framework), and resource utilization is optimized by combining database connection pool, to ensure that the data loading process is both fast and stable, laying a solid foundation for subsequent real-time query and complex analysis.

[0159] In the embodiment, the updated target update data set is uniformly transcoded, and the transcoded target update data set is stored in the in-memory database, so that the business system can directly obtain the target query data set in the in-memory database through a simple query statement, the target query statement is simplified, the generation time of the target query statement is reduced, and the efficiency of the data query method is improved.

[0160] In an exemplary embodiment, before listening to each data source, the data in the data source needs to be stored in the in-memory database. As shown in FIG. 8, before step 202 is executed, the specific processing process of the data query method includes steps 802 to 806. Among them: Figure 8

[0161] Step 802, for each business interface in the business system, obtaining the feature information of the business interface, and obtaining the full initial data from the data source corresponding to the data source identifier in the feature information.

[0162] Among them, the feature information includes positioning time, parsing rule and processing function identifier.

[0163] In implementation, the server obtains the feature information of each business interface in the business system. The feature information includes the data source identifier of the data to be obtained. The server determines the data source corresponding to the data source identifier, and obtains the full initial data of the data source from the data source.

[0164] Specifically, the business system interacts with the server through the business interface, and obtains the data required by the business through the business interface. In order to quickly provide the data required by the business system to the business system, the server obtains the feature information of each business interface. The feature information of the business interface includes the data source identifier of the data source to be obtained. The server determines the data source corresponding to each data source identifier, and obtains the full initial data from the data source.

[0165] In an optional embodiment, a development complete proprietary system API is set in the server to cover as many data sources as possible, so as to ensure the comprehensiveness and diversity of data collection.

[0166] Optionally, the feature information can be but is not limited to the data source identifier of one data source, and the number of data source identifiers in the feature information is not limited in the embodiment of the application.

[0167] Step 804, according to the data format, positioning time and parsing rule of the initial data, the initial data is parsed to obtain each parsed initial data, and the business data set is determined according to the processing function identifier and each processed initial data.

[0168] ​In implementation, the server parses each initial data according to the data format and the parsing rule, and performs data processing on each parsed initial data according to the initial function identifier, to obtain the business data set.

[0169] Specifically, the server determines the parsing tool corresponding to the data format based on the data format of the initial data, and parses each initial data based on the positioning time, the parsing rule and the parsing tool. Then, the server determines the processing function corresponding to the initial data according to the processing function identifier, and performs data processing on each parsed initial data according to the processing function, to obtain the business data set. The server constructs the data update template according to the parsing rule, the parsing tool and the processing function, and associates the data source and the data update template.

[0170] In step 806, the business data set is stored into the in-memory database in the format of the in-memory database.

[0171] In implementation, the server uniformly transcodes the business data set according to the data format of the in-memory database, and stores the transcoded business data set into the in-memory database. The process of storing the business data set is similar to the process of storing the target update data set, and details are described in steps 702 to 704. The embodiments of the present application will not be described here.

[0172] In an exemplary embodiment, the business interface of the business system is designed according to business requirements and data access logic. The purpose of interface design is to effectively interact with the upper layer application, and the data access logic is responsible for implementing data addition, deletion, modification and query operations. A well-designed service layer can simplify the complexity of the application layer and improve the maintainability and scalability of the system. The present application adopts the design style of RESTful API (Representational State Transfer Application Programming Interface), combined with Swagger (an open source tool set) or OpenAPI document (a specification for describing RESTful API), to ensure the flexibility and scalability of the service layer business interface to adapt to changing business requirements.

[0173] The server further processes and processes the target query data set according to business requirements. For example, the server calculates the total sales, generates customer reports or makes product recommendations, etc. These processing and processing work can provide more rich and useful information for the upper layer application. The server can use the Pandas library (a function library) and NumPy library (a function library) of Python (a programming language) and other data processing tools, combined with machine learning algorithms, to ensure the efficiency and accuracy of data processing, thereby providing high-quality data support for the upper layer application.

[0174] Through the RESTful API interfaces provided by the service layer, the server offers real-time and consistent data services to upper-layer applications. The service layer directly accesses data in the in-memory database, ensuring the real-time nature and consistency of the data service. This architectural design enables the application layer to respond quickly to user requests while guaranteeing efficient and accurate data processing. We optimize the performance of the service layer using technologies such as load balancing and microservice architecture to ensure stable and reliable data services that meet the real-time requirements of the business. The rapid response and efficient provision of interface data services by the service layer meet the needs of upper-layer applications. Furthermore, because the service layer directly accesses data in the in-memory database, it avoids the cumbersome process of data export and import, reducing the risks of data latency and inconsistency.

[0175] In this embodiment, a customized business dataset is created for each business interface based on its feature information and stored in an in-memory database. This allows the business system to directly retrieve the target query dataset from the in-memory database using a simple query statement, simplifying the target query statement, reducing the generation time of the target query statement, and thus improving the efficiency of the data query method.

[0176] In one exemplary embodiment, such as Figure 9 As shown, the specific processing procedure of step 804 includes steps 902 to 906. Wherein:

[0177] Step 902: Based on the data format of the initial data, determine the parsing tool corresponding to the data format, and parse each initial data based on the positioning time, parsing rules, and parsing tool.

[0178] In implementation, the server maps the parsing tool corresponding to the data format of the initial data to the parsing tool corresponding to the initial data. Then, the server filters the initial data for each data point that matches the location time using the location time and the current time. Finally, the server parses each data point according to the parsing rules and the parsing tool to obtain the parsed initial data.

[0179] Specifically, the server maps the parsing tool corresponding to the data format of the initial data to the parsing tool corresponding to the initial data. The server determines the positioning time as the start time and the current time as the determined end time. Then, the server uses the start and end times as filtering conditions to filter each piece of first data that has changed from each piece of initial data. Then, the server executes the parsing tool so that the parsing tool processes each piece of first data according to the parsing rules to obtain each piece of parsed initial data. The data parsing process is similar to the data parsing process in step 602. For the specific processing process, please refer to step 602 above. This embodiment of the application will not be described again here.

[0180] In an example embodiment, if the data format of the initial data is CSV or Excel, the server determines that the parsing tool of the initial data is the CSVReader method in the OpenCSV library. If the data format of the initial data is a data table, the server determines that the parsing tool is XSSFWorkbook of Apache POI. If the data format of the initial data is JSON format, the server determines that the parsing tool is JSON Schema in the json library.

[0181] At step 904, the processing function corresponding to the initial data is determined according to the processing function identifier, and the parsed initial data is processed according to the processing function to obtain the business data set.

[0182] In implementation, the server determines the processing function corresponding to the initial data in each processing function according to the processing function identifier. Then, the server processes the parsed initial data according to the processing function corresponding to the initial data to obtain the business data set. The process of processing the initial data is the same as the process of processing the data to be updated, and details are described above with reference to step 604. Therefore, the example embodiment of the present application will not be described here.

[0183] At step 906, the data update template is constructed according to the parsing rule, the parsing tool and the processing function, and the data source and the data update template are associated.

[0184] In implementation, in order to facilitate subsequent updating of data in the in-memory database, the server constructs the data update template according to the parsing rule, the parsing tool identifier of the parsing tool and the processing function identifier of the processing function. Then, the server associates the update template and the data source. If there are multiple data sources, the server associates the update template and each data source.

[0185] In the example embodiment, by constructing the data update template corresponding to the data source and associating the data source and the data update template, the updated data is synchronized to the in-memory database based on the information in the data update template in the subsequent data source update process, and the update efficiency is improved.

[0186] In an example embodiment, Figure 10 FIG. 2 is a schematic diagram of data transformation of the data query method in an example embodiment. As shown in FIG. 2, the data query method includes the following steps. Figure 10As shown, the server obtains an initial data set from each data source, and performs heterogeneous data conversion on the initial data set to obtain a business data set. The process of heterogeneous data conversion includes data collection, data parsing and data conversion. Then, the server stores the business data set into an H2 database. The business system sends a query request to the server through a business interface. The server generates a target query statement according to the query request. The target query statement can be a SQL data conversion statement and a SQL data query data. The types of the SQL data conversion statement include filtering, sorting and collection. Then, the server retrieves a target query data set according to the target query statement. The target query data set is fed back to the server from the in-memory database through the interface designed by the API service. The server can reprocess the target query data set according to the demand of the business interface, and feed back to the business system through the business interface.

[0187] It should be understood that, although each step in the flowchart involved in the above embodiments is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, as described above, at least part of the steps in the flowchart involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by combination are within the scope of protection of the present application.

[0188] Based on the same inventive concept, the embodiments of the present application also provide a data query device for implementing the above-mentioned data query method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more data query device embodiments provided below can refer to the limitations of the data query method in the above text, which will not be repeated here.

[0189] In one exemplary embodiment, as shown in Figure 11 A data query device 1100 is provided, comprising a listening module 1101, a transcoding module 1102, a receiving module 1103 and a query module 1104, wherein:

[0190] The listening module 1101 is configured to listen to each data source corresponding to the business system, and obtain a target update data set of a target data source update in the case that there is at least one target data source update.

[0191] The transcoding module 1102 is configured to uniformly transcode the target update dataset and store the transcoded target update dataset into the in-memory database.

[0192] The receiving module 1103 is configured to receive a query request sent by the business system and generate a target query statement according to a request parameter in the query request.

[0193] The query module 1104 is configured to query a target query dataset in the in-memory database according to the target query statement.

[0194] In an example embodiment, the listening module 1101 includes a first listening sub-module and a first obtaining sub-module. The first obtaining sub-module includes:

[0195] The second obtaining sub-module is configured to, in a case where there is at least one target data source update, obtain a to-be-updated dataset from the target data source in response to update information of the target data source.

[0196] The first updating sub-module is configured to determine a target update template corresponding to the target data source and perform data processing on the to-be-updated dataset based on the target update template to obtain the target update dataset.

[0197] In an example embodiment, the second obtaining sub-module includes:

[0198] The first determining sub-module is configured to, in a case where there is at least one target data source update, determine a target collection tool corresponding to the target data source in response to change information of the target data source.

[0199] The third obtaining sub-module is configured to obtain full-amount data from the target data source by using the target collection tool to obtain an initial dataset.

[0200] The first screening sub-module is configured to screen the to-be-updated dataset from the initial dataset based on a historical update time and a current time.

[0201] In an example embodiment, the first updating sub-module includes:

[0202] The second determining sub-module is configured to determine each target update template corresponding to the target data source according to a data source identifier of the target data source.

[0203] The third determining sub-module is configured to determine a target parsing tool based on a target parsing tool identifier in the target update template and determine a target processing function based on a target processing function identifier in the target update template.

[0204] The first processing submodule is configured to perform data processing on the to-be-updated data set based on the target analysis tool, the target processing function, and the target analysis rule in the target update template, to obtain the target update data set.

[0205] In an example embodiment, the first processing submodule includes:

[0206] The second processing submodule is configured to perform analysis processing on the to-be-updated data set based on the target analysis tool and the target analysis rule in the target update template, to obtain the initial target update data set; the target analysis tool is consistent with the format of the update data set.

[0207] The third processing submodule is configured to perform data processing on the initial target update data set according to the target processing function, to obtain the target update data set.

[0208] In an example embodiment, the transcoding module 1102 includes:

[0209] The fourth determining submodule is configured to determine, in the in-memory database, a target data table associated with the target update data set.

[0210] The first conversion module is configured to perform uniform format conversion on the target update data set according to the format of the in-memory database, and update the target update data set after the format conversion into the target data table of the in-memory database.

[0211] In an example embodiment, the data query apparatus 1100 further includes:

[0212] The first obtaining module is configured to, for each business interface in the business system, obtain feature information of the business interface, and obtain full-amount initial data from a corresponding data source according to a data source identifier in the feature information; the feature information includes a positioning time, an analysis rule, and a processing function identifier.

[0213] The first determining module is configured to parse the initial data according to the data format of the initial data, the positioning time, and the analysis rule, to obtain each parsed initial data, and determine a business data set according to the processing function identifier and each processed initial data.

[0214] The first storage module is configured to store the business data set into the in-memory database according to the format of the in-memory database.

[0215] In an example embodiment, the first determining module includes:

[0216] The fifth determining submodule is configured to determine, based on the data format of the initial data, an analysis tool corresponding to the data format, and parse each initial data based on the positioning time, the analysis rule, and the analysis tool.

[0217] The fourth processing sub-module is configured to determine a processing function corresponding to the initial data according to the processing function identifier, and perform data processing on each parsed initial data according to the processing function, to obtain a service data set.

[0218] The first association sub-module is configured to construct a data update template according to the parsing rule, the parsing tool and the processing function, and associate the data source and the data update template.

[0219] The modules in the data query device can be implemented by software, hardware or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be called and executed by the processor to perform operations corresponding to the modules.

[0220] In an exemplary embodiment, a computer device, which can be a server, has an internal structure as shown in Figure 12 The computer device includes a processor, a memory, an input / output interface and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store data used by the data query method. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a data query method.

[0221] Those skilled in the art can understand that Figure 12 The structure shown in the above

[0222] In an embodiment, a computer device is also provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0223] In an embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the above method embodiments.

[0224] In an embodiment, a computer program product is provided, which includes a computer program. The computer program is executed by a processor to implement the steps of the above method embodiments.

[0225] A person of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the computer program can include the processes of the above method embodiments. Any reference to a memory, a database or other medium in each embodiment provided in the present application can include at least one of a non-volatile memory and a volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive random access memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc. The database involved in each embodiment provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., and is not limited thereto. The processor involved in each embodiment provided in the present application can be a general processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., and is not limited thereto.

[0226] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, any combination of these technical features is deemed to be within the scope of the present application.

[0227] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A data query method, characterized in that, The method includes: Monitor each data source corresponding to the business system, and if at least one target data source is updated, obtain the target update dataset of the target data source update; The target update dataset is transcoded into a unified format, and the transcoded target update dataset is stored in an in-memory database; Receive query requests sent by the business system, and generate target query statements based on the request parameters in the query request; The target query dataset is queried from the in-memory database based on the target query statement.

2. The method according to claim 1, characterized in that, The step of obtaining the target update dataset when at least one target data source is updated includes: If at least one target data source is updated, in response to the update information of the target data source, the dataset to be updated is obtained from the target data source; Determine the target update template corresponding to the target data source, and perform data processing on the dataset to be updated based on the target update template to obtain the target update dataset.

3. The method according to claim 2, characterized in that, In the event that at least one target data source is updated, in response to the update information of the target data source, obtaining the dataset to be updated from the target data source includes: In the event that at least one target data source is updated, the target acquisition tool corresponding to the target data source is determined in response to the change information of the target data source; The target acquisition tool is used to obtain full data from the target data source to obtain an initial dataset; Based on historical update time and current time, the dataset to be updated is filtered from the initial dataset.

4. The method according to claim 2, characterized in that, The step of determining the target update template corresponding to the target data source and processing the dataset to be updated based on the target update template to obtain the target update dataset includes: Based on the data source identifier of the target data source, determine each target update template corresponding to the target data source; The target parsing tool is determined based on the target parsing tool identifier in the target update template, and the target processing function is determined based on the target processing function identifier in the target update template; Based on the target parsing tool, the target processing function, and the target parsing rules in the target update template, the dataset to be updated is processed to obtain the target update dataset.

5. The method according to claim 4, characterized in that, The process of processing the updated dataset based on the target parsing tool, the target processing function, and the target parsing rules in the target update template to obtain the target update dataset includes: Based on the target parsing tool and the target parsing rules in the target update template, the dataset to be updated is parsed to obtain the initial target update dataset; the target parsing tool is in a format consistent with the updated dataset. The initial target update dataset is processed according to the target processing function to obtain the target update dataset.

6. The method according to claim 1, characterized in that, The step of transcoding the target update dataset to a unified format and storing the transcoded target update dataset in an in-memory database includes: Determine the target data table associated with the target updated dataset in the memory database; The target update dataset is uniformly converted according to the format of the in-memory database, and the converted target update dataset is then updated to the target data table of the in-memory database.

7. The method according to claim 2, characterized in that, Before each data source corresponding to the monitoring business system, the method further includes: For each business interface in the business system, obtain the feature information of the business interface, and obtain the full initial data from the data source corresponding to the data source identifier in the feature information; the feature information includes the positioning time, parsing rules and processing function identifier; The initial data is parsed according to the data format, the positioning time, and the parsing rules to obtain each parsed initial data, and the business dataset is determined according to the processing function identifier and each processed initial data; The business dataset is stored in the in-memory database according to the format of an in-memory database.

8. The method according to claim 7, characterized in that, The process of parsing the initial data according to the data format, the positioning time, and the parsing rules to obtain each parsed initial data, and determining the business dataset according to the processing function identifier and each processed initial data, includes: Based on the data format of the initial data, a parsing tool corresponding to the data format is determined, and each of the initial data is parsed based on the positioning time, the parsing rules, and the parsing tool; The processing function corresponding to the initial data is determined according to the processing function identifier, and the parsed initial data is processed according to the processing function to obtain the business dataset; Based on the parsing rules, the parsing tool, and the processing function, a data update template is constructed, and the data source and the data update template are associated.

9. A data query device, characterized in that, The device includes: The monitoring module is used to monitor each data source corresponding to the business system, and when there is an update of at least one target data source, it obtains the target update dataset of the target data source. The transcoding module is used to transcode the target update dataset in a unified format and store the transcoded target update dataset in an in-memory database. The receiving module is used to receive query requests sent by the business system and generate target query statements based on the request parameters in the query request. The query module is used to query the target query dataset in the in-memory database based on the target query statement.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images