Data query method and device, electronic equipment and storage medium

Abstract

This application discloses a data query method, apparatus, electronic device, and storage medium, relating to the field of data management technology. The method includes: determining the dataset corresponding to the query parameter based on a dataset identifier in the query parameters; if the dataset corresponding to the query parameter is not a related dataset, obtaining the target data corresponding to the query parameter from the dataset corresponding to the query parameter, wherein the related dataset includes a primary data subset and a secondary data subset; if the dataset corresponding to the query parameter is a related dataset, obtaining first intermediate data from the primary data subset based on the query parameter; obtaining second intermediate data from the secondary data subset based on the query parameter and the first intermediate data; and obtaining the target data corresponding to the query parameter based on the first and second intermediate data. This solves the problems of data query methods being difficult to use for cross-database queries, resulting in incomplete data and poor scalability.

Description

Technical Field

[0001] This invention relates to the field of data management technology, specifically to data query methods, devices, electronic equipment, and storage media. Background Technology

[0002] With the rapid development of cloud computing technology and the deepening of enterprise digital transformation, cloud platforms have become a core infrastructure supporting business operations, resource management, and data analysis. Within a cloud platform, users frequently need to query data to monitor resource usage and ensure system stability.

[0003] Currently, when querying data for custom reports, a data query interface is developed based on business needs. Data is then assembled and combined within this interface for report display. Alternatively, a generic data query interface is defined based on flexibly configured datasets from different data sources. This interface is used to query SQL (Structured Query Language) or call APIs (Application Programming Interfaces) in the database to retrieve report data. However, current data query methods require customized interface and backend logic development, compilation, packaging, deployment, and testing for different databases or reports, resulting in poor scalability. Furthermore, data queries often need to cross databases, making it impossible to retrieve complete data from a single data source dataset. The current data query methods struggle to query data across databases, leading to incomplete data retrieval. Summary of the Invention

[0004] This invention provides a data query method, apparatus, electronic device, and storage medium to solve the problems that data query methods are difficult to use to query data across databases, resulting in incomplete data and poor scalability.

[0005] Firstly, this application provides a data query method, which includes: Obtain the query parameters and determine the dataset corresponding to the query parameters based on the dataset identifier in the query parameters; If the dataset corresponding to the query parameters is not a related dataset, the target data corresponding to the query parameters is obtained from the dataset corresponding to the query parameters based on the query parameters. The related dataset contains a subset of primary data and a subset of secondary data. If the dataset corresponding to the query parameters is a related dataset, the first intermediate data is obtained from the main data subset according to the query parameters. Based on the query parameters and the first intermediate data, the second intermediate data is obtained from the data subset, and the target data corresponding to the query parameters is obtained based on the first and second intermediate data.

[0006] Secondly, this application provides a data query device, which includes: The acquisition module is used to acquire query parameters and determine the dataset corresponding to the query parameters based on the dataset identifier in the query parameters; The first query module is used to retrieve the target data corresponding to the query parameters from the dataset corresponding to the query parameters, when the dataset corresponding to the query parameters is not a related dataset. The related dataset includes a subset of main data and a subset of secondary data. The second query module is used to obtain the first intermediate data from the main data subset based on the query parameters when the dataset corresponding to the query parameters is a related dataset. The third query module is used to obtain second intermediate data from a data subset based on the query parameters and the first intermediate data, and to obtain the target data corresponding to the query parameters based on the first intermediate data and the second intermediate data.

[0007] Thirdly, this application provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the data query method described in the first aspect or any corresponding embodiment.

[0008] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the data query method described in the first aspect or any corresponding embodiment.

[0009] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the data query method described in the first aspect or any corresponding embodiment thereof.

[0010] This application addresses the following issues: if the dataset corresponding to the query parameters is not a related dataset, the target data corresponding to the query parameters is retrieved from the dataset corresponding to the query parameters; if the dataset corresponding to the query parameters is a related dataset, first intermediate data is retrieved from the main data subset of the related dataset based on the query parameters; second intermediate data is retrieved from the secondary data subset based on the query parameters and the first intermediate data; and the target data corresponding to the query parameters is obtained based on the first and second intermediate data. This solves the problems of incomplete data retrieval and poor scalability of data query methods due to difficulties in cross-database queries. By supporting dataset association, this method solves problems such as the inability to retrieve complete data from a single data source dataset, the inability to combine and output cached data, and the inability to extend existing customized interfaces when data queries require cross-database access. It supports configuring datasets using existing interfaces and data sources through the user interface. In most common reporting scenarios, additions and modifications do not require recompilation and packaging; configuration is complete and the dataset is ready to use, reducing the time cost of compilation, packaging, deployment, testing, and integration. It also supports integration testing and replacement verification in production environments, quickly resolving on-site issues. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this application, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the data query method according to an embodiment of this application; Figure 2 This is a schematic diagram of the cloud platform report data query process according to an embodiment of this application; Figure 3 This is a schematic diagram of the interface data query process according to an embodiment of this application; Figure 4 This is a schematic diagram of data matching and assembly according to an embodiment of this application; Figure 5 This is a structural block diagram of a data query device according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0014] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0015] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] With the rapid development of cloud computing technology and the deepening of enterprise digital transformation, cloud platforms have become the core infrastructure supporting business operations, resource management, and data analysis. In a cloud environment, users frequently need to monitor resource usage (such as computing, storage, and network consumption), cost distribution, service performance, and security logs to optimize resource allocation, control costs, and ensure system stability. However, the standardized report templates provided by cloud platforms often fail to meet the personalized needs of different industries, business scenarios, and management roles. For example, finance teams focus on cost allocation and budget comparisons, operations personnel need to focus on real-time performance indicators, while management tends to favor comprehensive business health analysis. Furthermore, cross-service data silos, dynamically changing business indicators, and diverse data visualization needs further highlight the limitations of existing reporting systems. To improve the efficiency of data-driven decision-making, users urgently need a flexible, customizable reporting function that supports on-demand configuration of data sources, filtering dimensions, aggregation logic, and visualization formats, thereby quickly generating insightful analyses tailored to business objectives.

[0017] Currently, custom report data query solutions include: developing report data query interfaces based on business needs, assembling and combining data within the interface for report display; configuring data sources, flexibly configuring SQL-based or API-based datasets based on the data source, defining a general data query interface, and flexibly querying SQL in the database or calling the API to obtain report data based on the configuration information. However, customized development and integration costs are high. Adding a new report requires developing, compiling, packaging, deploying, and testing customized interfaces and backend logic. Configuration based on a single data source dataset cannot adapt to the following business scenarios: data queries need to cross databases, making it impossible to obtain complete data from a single data source dataset via SQL; data in relational databases is incomplete, and for business performance considerations, much business data is cached in a Redis database, which cannot be directly retrieved from the relational database and needs to be combined with cached data in the Redis database for output; modifying existing customized reports, some customized reports need to extend one or more columns, which cannot be extended based on the original customized interface.

[0018] This application provides a data query method and designs a standardized dataset configuration method that supports single dataset configuration and associated dataset configuration. It shields customization differences and designs a configuration management UI (user interface) for unified maintenance and management. Based on the standardized dataset configuration, a standard dataset data query interface is defined, which flexibly collects and assembles data according to the dataset configuration and returns it. It supports relational database datasets (such as MySQL datasets), Redis datasets, API datasets, and pre-configured associated datasets, combining any two to form a new associated dataset.

[0019] According to the embodiments of this application, a data query method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be run in the above-mentioned data query system, or can be executed in a computer system such as a set of executable instructions, for example, a computer, a server, etc. Although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0020] This embodiment provides a data query method. Figure 1 This is a flowchart of a data query method according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain the query parameters and determine the dataset corresponding to the query parameters based on the dataset identifier in the query parameters.

[0021] Specifically, this involves designing a standardized dataset configuration method that supports both single dataset configuration and associated dataset configuration. Customization differences will be shielded, and a configuration management UI will be designed for unified maintenance and management.

[0022] Data query methods for associated datasets. Based on standardized dataset configurations, a standard dataset data query interface is defined. This interface flexibly collects and assembles data according to the dataset configuration and returns it. It supports relational database datasets (such as MySQL datasets), Redis datasets, API datasets, and pre-configured associated datasets, allowing pairwise combinations to create new associated datasets.

[0023] Obtain query parameters through the data query interface. For example, the data query interface URL is: / cloud / report / datas / {dataSetID}, and the request method is: GET. Query parameters include: the dataset ID (IDentification), such as dataSetID; other parameters: key and value, which are customized according to the placeholders configured in the dataset configuration. The dataset identifier is, for example, the dataset ID. The dataset ID determines the dataset corresponding to the query parameters.

[0024] Step S102: If the dataset corresponding to the query parameter is not a related dataset, obtain the target data corresponding to the query parameter from the dataset corresponding to the query parameter according to the query parameter. The related dataset includes a primary data subset and a secondary data subset.

[0025] Specifically, when querying data from non-related datasets such as MySQL, Redis, and APIs, the SQL configuration supports configuring query parameters for placeholder substitution. First, placeholders are replaced based on the query parameters; if none exist, no replacement is needed. Then, it checks for related data in the primary subset. If so, the current dataset query is targeting the secondary subset, requiring it to iterate through the related data in the primary subset and replace values ​​matching the placeholders. After replacement, the query statement with the replaced parameters retrieves the target data corresponding to the query parameters from the dataset containing those parameters. Related datasets contain both primary and secondary subsets. For example, a MySQL dataset and a Redis dataset are related; the MySQL dataset is the primary subset, and the Redis dataset is the secondary subset.

[0026] Step S103: If the dataset corresponding to the query parameters is a related dataset, obtain the first intermediate data from the main data subset according to the query parameters.

[0027] Specifically, when the dataset corresponding to the query parameters is a related dataset, the dataset configuration information is first queried based on the IDs of the primary and secondary data subsets. The query parameters of the interface are then used as the query parameters for the primary data subset to retrieve data. The primary data subset then retrieves its first intermediate data according to the process described above, depending on its dataset type. Afterwards, the query parameters of the interface and the data retrieved from the primary data subset are used as parameters to sequentially query the secondary data subset. The secondary data subset retrieves its data according to the process described above, depending on its dataset type.

[0028] Step S104: Based on the query parameters and the first intermediate data, obtain the second intermediate data from the data subset, and based on the first intermediate data and the second intermediate data, obtain the target data corresponding to the query parameters.

[0029] Specifically, based on the query parameters and the first intermediate data, the second intermediate data is obtained from the data subset. After obtaining the data for each dataset, the first and second intermediate data are combined according to the association relationship configured in the associated datasets and returned to obtain the target data. The target data includes data mapping matching and alias conversion.

[0030] The data query method provided in this embodiment, if the dataset corresponding to the query parameter is not a related dataset, retrieves the target data corresponding to the query parameter from the dataset corresponding to the query parameter; if the dataset corresponding to the query parameter is a related dataset, retrieves the first intermediate data from the main data subset of the related dataset according to the query parameter; retrieves the second intermediate data from the secondary data subset according to the query parameter and the first intermediate data, and obtains the target data corresponding to the query parameter based on the first and second intermediate data. This method, by supporting dataset association, solves problems such as the inability to retrieve complete data from a single data source dataset using query statements when data queries need to cross databases, the inability to combine and output with cached data, and the inability to extend based on existing customized interfaces. It supports configuring datasets using existing interfaces and data sources through the UI. In most common reporting scenarios, adding or modifying data does not require recompiling and packaging; it can be used immediately after configuration, reducing the time cost of compilation, packaging, deployment, testing, and integration debugging. It also supports integration testing and replacement verification in the production environment, quickly resolving on-site issues. This method solves the problems of data query methods being difficult to use for cross-database queries, resulting in incomplete data and poor scalability.

[0031] As an optional embodiment, before determining the dataset corresponding to the query parameter based on the dataset identifier in the query parameter, the method further includes: Determine the first preset number of data sources, and generate the configuration items and configuration item descriptions for the dataset based on the data sources; Based on the data source and configuration item description information, generate dataset configuration information, and based on the dataset configuration information, create a second preset number of datasets; In the second preset number of datasets, determine the master data subset and the slave data subset, and generate the associated dataset configuration information of the associated dataset based on the dataset configuration information of the master data subset and the dataset configuration information of the slave data subset, and create the associated dataset based on the associated dataset configuration information.

[0032] Specifically, the first preset quantity represents one or more. For example... Figure 2 As shown, the data sources include MySQL data sources, Redis data sources, and API data sources, corresponding to MySQL databases, Redis databases, and API databases, respectively. These data sources are used to generate corresponding datasets, including MySQL datasets, Redis datasets, and API datasets. Related datasets can be generated by configuring the MySQL dataset, Redis dataset, and API dataset. Furthermore, data from these datasets can be queried through the standardized data query interface in the report backend module and displayed on the report user interface.

[0033] Configure MySQL, Redis, and API data sources, and configure individual datasets for each data source. Individual datasets can directly collect data and return it to the UI via a standardized data query interface. Data sets can also be linked together to form associated datasets, which return data to the UI via the same standardized data query interface. A data source is the origin of the report data, such as a MySQL database or a business system address. A dataset represents a collection of data, configured to query data based on the data source, such as an SQL database query or a Hypertext Transfer Protocol (HTTP) interface.

[0034] The dataset configuration items and their descriptions are generated based on the data source. The dataset configuration information, including the configuration items, descriptions, and dataset configuration details, is generated, as shown in Tables 1, 2, 3, and 4.

[0035] Table 1 Dataset Configuration Information Table

[0036] Table 2 MySQL Dataset Configuration Table

[0037] Table 3 API Dataset Configuration Table

[0038] Table 4 Redis Dataset Configuration Table

[0039] Based on the dataset configuration information, create a second preset number of datasets, where the second preset number represents one or more. When configuring a single dataset of MySQL, Redis, or API types, placeholders can be used instead of variables, for example: `select ID, name, description from user where name={userName}`. When calling the standard data query interface to query data, passing `userName=xxx` can replace the placeholder in the SQL. The configuration of API and Redis datasets is similar.

[0040] In a second preset number of datasets, a primary data subset and a secondary data subset are determined. For example, a MySQL dataset and a Redis dataset are related; the MySQL dataset is the primary data subset, and the Redis dataset is the secondary data subset. Based on the dataset configuration information of the primary and secondary data subsets, the related dataset configuration information is generated, and the related dataset is created based on this configuration information. If a dataset is referenced as a secondary data subset by a related dataset, dynamic replacement of interface parameters is supported. If the interface does not have a corresponding parameter, the parameter value of the corresponding field in the return value of the primary data subset is replaced. For example, if the secondary data subset is: `select ID, name, description from user where ID={userID}`, and the primary data subset returns: `[{"resourceID":"aa","userID":"xxx"}]`, and the data query interface does not pass the `userID` parameter, then `userID` is matched and replaced in the return value of the primary data subset. This method can be used to implement scenarios where query parameters are passed between primary and secondary data subsets.

[0041] This application presents a standardized dataset configuration method that supports both single dataset configuration and associated dataset configuration. It shields customization differences, designs a configuration management UI, and enables unified maintenance and management. This addresses the challenges of custom development for new custom reports on cloud platforms, incomplete data retrieval from single datasets on cloud platforms, and the reuse and expansion of customized report interfaces.

[0042] As an optional embodiment, generating associated dataset configuration information for an associated dataset based on the dataset configuration information of the master data subset and the dataset configuration information of the slave data subset includes: obtaining a first dataset identifier, a first return value, a second dataset identifier, and a second return value of the master data subset from the dataset configuration information of the master data subset and the dataset configuration information of the slave data subset; determining the fields to be matched and the associated fields of the slave data subset based on the first return value and the second return value; and generating associated dataset configuration information based on the first dataset identifier, the second dataset identifier, the fields to be matched, and the associated fields.

[0043] Specifically, from the dataset configuration information of the master data subset and the dataset configuration information of the slave data subset, the first dataset identifier of the master data subset, the first return value, the second dataset identifier of the slave data subset, and the second return value are obtained. The first dataset identifier is, for example, the master data subset ID. The first return value is, for example, the cloud host ID, name, organization ID, and user ID. The second dataset identifier of the slave data subset is, for example, the slave data subset ID. The second return value is, for example, ID, name, and account. See Tables 5 and 6.

[0044] Table 5. Related Dataset Information Table

[0045] Table 6 Information from Data Subset Table

[0046] Based on the first and second return values, determine the fields to be matched and the associated fields from the data subset. Fields to be matched include, for example, the fields to be matched from the data subset. Associated fields include, for example, the fields from the main data subset and the associated fields from the data subset.

[0047] Fields used for matching from a data subset: These are the "connection identifiers / matching keys" belonging to the data subset itself. They are the fields used by the data subset to connect with the main data subset. For example, in the example, if data subset 1 queries the user table using `select ID,name,description from user where ID = {userID}`, its matching field is `ID`. This means that the `ID` field from the data subset is used to match a field in the main data subset to find the corresponding data.

[0048] Associated master data subset fields: These are the "interconnection identifiers / matching keys" belonging to the master data subset. They are fields that correspond one-to-one with the matching fields in the slave data subset, and are crucial for linking the two. For example, in the example, the associated field for the master data subset is userID. This means pairing the userID of the master data subset with the ID of slave data subset 1. This ensures that "the data with userID xxx in the master data subset corresponds to the data with ID xxx in the slave data subset," preventing data cross-referencing.

[0049] Fields associated from a data subset: These are the "valid content fields" from the data subset. Besides the matching fields, they are the core data fields that are actually extracted from the subset and incorporated into the final report. They represent the content users truly want to see in the report. For example, in the example, the associated fields for data subset 1 are "name" and "description." These two fields are not used for matching; rather, after a successful match, they are extracted from the subset and added to the final assembled result. These fields can also be renamed (e.g., "name" becomes "userName"), and will ultimately appear directly in the report data.

[0050] Generate associated dataset configuration information based on the first dataset identifier, the second dataset identifier, the fields to be matched, and the associated fields.

[0051] Based on the configuration model above, a dataset configuration table is created in the database, and a UI configuration page is designed to support visual addition, modification, query, and deletion of configurations. When a dataset configuration is added or modified, the configuration information is updated in the table.

[0052] As an optional embodiment, based on the query parameters, the target data corresponding to the query parameters is obtained from the dataset corresponding to the query parameters, including: Based on the dataset type and dataset configuration information, obtain the first initial retrieval information, which includes the first data to be replaced; Obtain the first target parameter from the query parameters, and replace the first data to be replaced in the first initial search information according to the first target parameter to obtain the first target search information; Based on the dataset type, determine the first target data source corresponding to the query parameters, and retrieve the target data corresponding to the query parameters from the first target data source based on the first target retrieval information.

[0053] Specifically, based on the dataset type and configuration information, the initial retrieval information is obtained. For example, for a MySQL dataset, the initial retrieval information is `select ID,name,description from user where ID={userID}`; for a Redis dataset, it's `GET{userID}`, `HGET {userID}`, or `HGETALL{userID}`; and for an API dataset, it's ` / v1 / user-userID={userID}`. The first data to be replaced is `{userID}`.

[0054] The first target parameter is obtained from the query parameters, and the first data to be replaced in the first initial search information is replaced according to the first target parameter to obtain the first target search information. For example, if the first target parameter is aaa, then the first target search information is / v1 / user-userID=aaa, GET aaa, or HGET aaa,name, or HGETALL aaa.

[0055] Based on the first target retrieval information, an array of SQL queries to be executed is obtained (which may contain one or more records). This array is deduplicated to avoid duplicate queries. Then, based on the dataset type, the first target data source corresponding to the query parameters is determined. Based on the first target retrieval information, the target data corresponding to the query parameters is retrieved from the first target data source. For example, the data source connection strings corresponding to the datasets are combined to execute SQL queries sequentially in the database, and the retrieved data is merged and returned.

[0056] The above content is as follows Figure 3 As shown. After receiving the query parameters, the query interface first queries the dataset configuration information based on the dataset ID and performs parameter validation on the dataset configuration information, including: whether the dataset exists, whether the data source associated with the dataset is available, and if it is a related dataset, whether each related dataset exists and whether the data source is available. After passing the validation, the data query is performed. The dataset type is determined, and the data query is performed based on the dataset type. For example, if the dataset type is [dataset type missing], the placeholders in the SQL configuration are replaced with query parameters. It then checks for related data with a subset of the master data. If not, it executes "combining the database connection information in the data source and querying the database using SQL"; if so, it replaces the placeholders in the SQL configuration with the related data and executes "combining the database connection information in the data source and querying the database using SQL". See also: [link to other content]. Figure 3When performing data queries, four query methods are defined for datasets: MySQL, Redis, API, and joins. The appropriate query method is selected based on the type of dataset being queried. The data query methods are explained below: MySQL Dataset: When configuring SQL for a dataset, dynamic parameters for placeholder replacement interfaces are supported. First, the placeholders are replaced based on the interface parameters; if none exist, no replacement is needed. Then, it checks for related data from the main dataset subset. If so, the current dataset query is being performed as a query on the subset, requiring it to iterate through the related data of the main dataset subset and replace each value matching the placeholder. Example SQL configuration: `select ID,name,description from user where ID = {userID}`; Interface parameter replacement: If the interface parameter is `url: / cloud / report / datas / dataSetID-userID=aaa`, the SQL needs to be replaced with: `select ID,name,description from user where ID = aaa`. The associated data of the master data subset is: [{"resourceID":"re-a","userID":"user-a"},{"resourceID":"re-b","userID":"user-b"},{"resourceID":"re-c","userID":"user-b"}]. The SQL statements need to be replaced sequentially as follows: select ID,name,description from user where ID =user-a; select ID,name,description from user where ID = user-b; select ID,name,description from user where ID = user-b; After the SQL replacement, an array of SQL statements to be executed (possibly one or more) is obtained. The array is deduplicated to avoid duplicate queries. Then, the data source connection strings corresponding to the dataset are combined to execute the SQL queries sequentially in the database, and the queried data is merged and returned.

[0057] Redis Datasets: When configuring dataset keys and fields, placeholders can be used to replace dynamic parameters. First, placeholders are replaced based on the API parameters and the associated data of the main dataset subset, similar to the MySQL example. After replacement, the SQL will look like the following example: GET aaa or HGET aaa,name, or HGETALL aaa. API Datasets: When configuring dataset APIs, placeholders can be used to replace dynamic parameters of the API. First, placeholders are replaced based on the API parameters and the associated data of the main dataset subset, similar to the MySQL example. After replacement, the API will look like the following example: / v1 / user-userID=aaa, / v1 / user-userID=bbb. The array is deduplicated to avoid duplicate queries. Then, the data source connection string corresponding to the dataset is combined and sent as an HTTP request to query data. The queried data is then merged and returned.

[0058] In this embodiment, multiple dataset queries are supported, adapting to MySQL, Redis, API, and related scenarios; placeholders are dynamically replaced to adapt to interface parameters and master data subset related data; deduplication avoids duplicate queries, and returned data is merged to improve query efficiency and accuracy.

[0059] As an optional embodiment, based on the first target retrieval information, the target data corresponding to the query parameters is obtained from the first target data source, including: From the first target retrieval information, obtain the first search statement and the second search statement; Based on the first search statement, retrieve the first returned data from the first target data source; Based on the second search statement, retrieve the second returned data from the first target data source; According to the preset format, the first and second returned data are converted to obtain the target data.

[0060] Specifically, in the first target retrieval information, a first search statement and a second search statement are obtained. The first search statement is, for example, GET aaa, HGET aaa,name, etc.; the second search statement is, for example, HGETALL aaa. The first search statement returns a string value as its first return data, and the second search statement returns a map object (e.g., {"key":"value"}).

[0061] When querying data using the first and second search statements, the array is deduplicated to avoid duplicate queries. Then, the data source connection string corresponding to the dataset is combined to execute the SQL query data in the database sequentially. The GET command and HGET return a string value, while HGETALL returns a map object (example: {"key":"value"}).

[0062] Preset format example: [{"key":"value"}]. Based on the preset format, the first and second returned data are converted to obtain the target data. For example, when returning data to the interface, the unified format is [{"key":"value"}]. GET aaa: [{"key":"aaa","value":"xxx"}], HGET aaa,name: [{"key":"aaa","field":"name","value":"xxx"}], HGETALL aaa: [{"key":"aaa",……}].

[0063] In this embodiment, Redis supports queries using multiple types of search statements. Before querying, the statement array is deduplicated to avoid duplicate queries and improve query efficiency. The string and map format data returned by different search statements are unified to a specified format to ensure that the data returned by the interface is consistent.

[0064] As an optional embodiment, the first intermediate data is obtained from the main data subset according to the query parameters, including: Obtain the identifier of the first dataset from the query parameters, and determine the primary data subset based on the identifier of the first dataset; Based on the associated dataset configuration information of the associated dataset, obtain the second initial retrieval information; Obtain the first target parameter from the query parameters, and replace the second data to be replaced in the second initial search information according to the first target parameter to obtain the second target search information; Based on the information retrieved for the second objective, the first intermediate data is obtained from the data source corresponding to the subset of master data.

[0065] Specifically, the first dataset identifier is, for example, the ID of a primary data subset. If the dataset corresponding to the query parameters is a related dataset, the dataset configuration information is first queried based on the ID of the primary data subset to determine the primary data subset. Second initial retrieval information is then obtained from the related dataset configuration information of the related dataset.

[0066] Retrieve the first target parameter from the query parameters, for example: the first target parameter is "aaa". The second initial search information is, for example: GET {userID}. The second data to be replaced is, for example: {userID}. Replace the second data to be replaced in the second initial search information with the first target parameter to obtain the second target search information, for example: GET aaa.

[0067] Based on the second target retrieval information, first intermediate data is obtained from the data source corresponding to the main data subset. For example, the query parameters of the interface are used as query parameters for querying data in the main data subset to obtain the data of the main data subset. The main data subset obtains data according to the process described in the above embodiment, depending on the dataset type. Then, the query parameters of the interface and the data queried from the main data subset are used as parameters to query the data of the secondary data subset sequentially. The secondary data subset obtains data according to the process described in the above embodiment, depending on the dataset type. At this point, the data of each dataset has been obtained. Then, the data is assembled according to the association relationship configured for the associated datasets and returned to obtain the first intermediate data. The first intermediate data includes data mapping matching and alias conversion.

[0068] In this embodiment, it supports querying related datasets. First, it obtains the subset data of the master data based on the query parameters, and then combines the parameters and the master data to query the subset data of the slave data. It adapts to different types of datasets, assembles data according to the relationship, including mapping matching and alias conversion, to ensure that the data query is complete and well-organized.

[0069] As an optional embodiment, based on the query parameters and the first intermediate data, second intermediate data is obtained from a data subset, and based on the first and second intermediate data, the target data corresponding to the query parameters is obtained, including: Obtain the second dataset identifier from the query parameters, and determine the data subset based on the second dataset identifier; Based on the associated dataset configuration information, obtain the third initial retrieval information, the first associated field of the main data subset, the second associated field of the secondary data subset, and the reference field; If the query parameters do not contain the second target parameter corresponding to the data subset, the second target parameter is obtained from the first intermediate data based on the comparison result of the first associated field and the second associated field. Based on the second target parameters, the referenced fields, and the third initial search information, the third target search information is obtained; Based on the information retrieved from the third target, second intermediate data is obtained from the data source corresponding to the data subset; Assemble the first and second intermediate data to obtain the target data.

[0070] Specifically, the second dataset identifier is obtained from the query parameters, and the subset of data is determined based on the second dataset identifier. For example, the subset of data is determined based on the subset ID in the query parameters, and the second dataset identifier is the subset ID. Additionally, the primary data subset is, for example, / v1 / resource / datas.

[0071] Combination Figure 4 This embodiment will be described below. Based on the associated dataset configuration information, the following are obtained: third initial retrieval information, first associated fields of the main data subset, second associated fields of the secondary data subset, and reference fields. Examples of the third initial retrieval information include: `select ID, name, description from user where ID = {userID}`, `GET {resourceID}`, etc. Examples of the first associated fields of the main data subset include: fields associated with the main data subset such as `userID`, fields associated with the main data subset such as `resourceID`, etc. Examples of the second associated fields of the secondary data subset include: fields matched with the secondary data subset such as `ID`, fields matched with the secondary data subset such as `key`, etc. Examples of reference fields include: fields referenced from the secondary data subset such as `name` (alias `userName`), `description` (alias `userDesc`), and `value` (alias `status`).

[0072] For example, the second target parameter is {userID}. If the query parameters do not contain the second target parameter corresponding to the data subset, the second target parameter needs to be obtained from the first associated field.

[0073] Based on the comparison results of the first and second associated fields, it is known that the first associated field contains the second target parameter, which is "aaa". Therefore, the second target parameter is retrieved from the first intermediate data based on the comparison results. Using the second target parameter and the reference field, the corresponding parameter in the third initial retrieval information is replaced to obtain the third target retrieval information. The third target retrieval information has the same format and function as the second target retrieval information; the difference between the two is only in their retrieval methods. Examples of second and third target retrieval information include: / v1 / user-userID=aaa, GET aaa, HGET aaa, name, HGETALL aaa, etc.

[0074] Based on the information retrieved from the third target, second intermediate data is obtained from the data source corresponding to the data subset. For example: from data subset 1: [{"ID":"user-a","name":"aaaa","description":"desA"},{"ID":"user-b", "name":"bbbb" "description":"desB"}], from data subset 2: [{"key":"resID-a","value":"normal"},{"key":"resID-b","value":"normal"},{"key":"resID-c","value":"normal"}].

[0075] Additionally, the first intermediate data is, for example, a subset of the master data: [{"resourceID":"resID-a","resourceName":"res-a", "userID":"user-a"},{"resourceID":"resID-b","resourceName":"res-b", "userID":"user-b"}, {"resourceID":"resID-c","resourceName":"res-c", "userID":"user-b"}].

[0076] Assemble the first and second intermediate data to obtain the target data, for example: Figure 4 As shown, the first and second intermediate data are concatenated to obtain [{"resourceID":"resID-a", "resourceName":"res-a", "userID":"user-a", "userName":"aaaa", "userDesc":"desA", "status":"normal"},{"resourceID":"resID-b", "resourceName":"res-b", "userID":"user-b", "userName":"bbbb", "userDesc":"desB", "status":"normal"},{"resourceID":"resID-c", "resourceName":"res-c", "userID":"user-b"}, "userName":"bbbb", "userDesc":"desB", "status":"normal"}].

[0077] In this embodiment, the query parameters of the secondary data subset can be completed from the primary data subset, the associated fields of the primary and secondary data subsets can be accurately matched, the secondary data can be obtained by the referenced fields with aliases, and the intermediate data of the primary and secondary data can be assembled to realize cross-dataset data integration, ensuring accurate data association, complete query results and regular fields.

[0078] This embodiment also provides a data query device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0079] This embodiment provides a data query device, such as... Figure 5 As shown, it includes: The acquisition module 501 is used to acquire query parameters and determine the dataset corresponding to the query parameters based on the dataset identifier in the query parameters. The first query module 502 is used to obtain the target data corresponding to the query parameters from the dataset corresponding to the query parameters, when the dataset corresponding to the query parameters is not a related dataset, wherein the related dataset includes a primary data subset and a secondary data subset. The second query module 503 is used to obtain the first intermediate data in the main data subset according to the query parameters when the dataset corresponding to the query parameters is a related dataset. The third query module 504 is used to obtain second intermediate data from a data subset based on the query parameters and the first intermediate data, and to obtain the target data corresponding to the query parameters based on the first intermediate data and the second intermediate data.

[0080] In some alternative embodiments, the device further includes: The determination module is used to determine a first preset number of data sources and generate configuration items and configuration item description information for the dataset based on the data sources; The first generation module is used to generate dataset configuration information based on the data source and configuration item description information, and to create a second preset number of datasets based on the dataset configuration information; The second generation module is used to determine the master data subset and the slave data subset in the second preset number of datasets, and generate the associated dataset configuration information of the associated dataset based on the dataset configuration information of the master data subset and the dataset configuration information of the slave data subset, and create the associated dataset based on the associated dataset configuration information.

[0081] In some alternative implementations, the second generation module includes: The first acquisition unit is used to acquire, from the dataset configuration information of the main data subset and the dataset configuration information of the secondary data subset, the first dataset identifier of the main data subset, the first return value, the second dataset identifier of the secondary data subset, and the second return value. The first determining unit is used to determine the field to be matched and the associated field from the data subset based on the first return value and the second return value; The generation unit is used to generate associated dataset configuration information based on the first dataset identifier, the second dataset identifier, the field to be matched, and the associated field.

[0082] In some optional implementations, the first query module 502 includes: The second acquisition unit is used to acquire first initial retrieval information based on the dataset type and dataset configuration information, wherein the first initial retrieval information includes first data to be replaced; The first obtaining unit is used to obtain the first target parameter from the query parameters, and replace the first data to be replaced in the first initial search information according to the first target parameter to obtain the first target search information; The second determining unit is used to determine the first target data source corresponding to the query parameters according to the dataset type of the dataset, and to obtain the target data corresponding to the query parameters in the first target data source according to the first target retrieval information.

[0083] In some optional implementations, the second determining unit includes: The first acquisition submodule is used to acquire the first search statement and the second search statement from the first target search information; The second acquisition submodule is used to acquire the first returned data from the first target data source according to the first search statement; The third acquisition submodule is used to retrieve the second returned data from the first target data source according to the second search statement; The format conversion submodule is used to convert the first and second returned data according to a preset format to obtain the target data.

[0084] In some optional implementations, the second query module 503 includes: The third determining unit is used to obtain the identifier of the first dataset from the query parameters and determine the master data subset based on the identifier of the first dataset. The third acquisition unit is used to acquire the second initial retrieval information based on the associated dataset configuration information of the associated dataset; The second obtaining unit is used to obtain the first target parameter from the query parameters, and replace the second data to be replaced in the second initial search information according to the first target parameter to obtain the second target search information; The fourth acquisition unit is used to retrieve the first intermediate data from the data source corresponding to the main data subset based on the second target retrieval information.

[0085] In some optional implementations, the third query module 504 includes: The fourth determining unit is used to obtain the identifier of the second dataset from the query parameters and determine the subset of data based on the identifier of the second dataset. The fifth acquisition unit is used to acquire the third initial retrieval information, the first associated field of the main data subset, the second associated field of the secondary data subset, and the reference field based on the associated dataset configuration information; The sixth acquisition unit is used to acquire the second target parameter from the first intermediate data based on the comparison result of the first associated field and the second associated field when the query parameters do not contain the second target parameter corresponding to the data subset. The third obtaining unit is used to obtain the third target retrieval information based on the second target parameters, the referenced fields, and the third initial retrieval information; The seventh acquisition unit is used to retrieve second intermediate data from the data source corresponding to the data subset based on the information retrieved from the third target. The assembly unit is used to assemble the first intermediate data and the second intermediate data to obtain the target data.

[0086] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0087] In this embodiment, the data query device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0088] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0089] The following is a detailed reference. Figure 6This diagram illustrates a suitable structural design for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of the electronic device. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0090] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0091] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the data query method of the embodiments of the present invention.

[0092] Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0093] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the data query method shown in the above embodiments is implemented.

[0094] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0095] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A data query method, characterized in that, The method includes: Obtain the query parameters, and determine the dataset corresponding to the query parameters based on the dataset identifier in the query parameters; If the dataset corresponding to the query parameter is not a related dataset, the target data corresponding to the query parameter is obtained from the dataset corresponding to the query parameter according to the query parameter, wherein the related dataset includes a primary data subset and a secondary data subset; If the dataset corresponding to the query parameter is a related dataset, the first intermediate data is obtained from the main data subset according to the query parameter. Based on the query parameters and the first intermediate data, the second intermediate data is obtained from the data subset, and the target data corresponding to the query parameters is obtained based on the first intermediate data and the second intermediate data.

2. The method according to claim 1, characterized in that, Before determining the dataset corresponding to the query parameter based on the dataset identifier in the query parameter, the method further includes: Determine a first preset number of data sources, and generate configuration items and configuration item description information for the dataset based on the data sources; Based on the data source and the configuration item description information, generate dataset configuration information, and based on the dataset configuration information, create a second preset number of datasets; The master data subset and the slave data subset are determined from a second preset number of datasets. Based on the dataset configuration information of the master data subset and the dataset configuration information of the slave data subset, the associated dataset configuration information of the associated dataset is generated, and the associated dataset is created based on the associated dataset configuration information.

3. The method according to claim 2, characterized in that, The step of generating associated dataset configuration information for the associated dataset based on the dataset configuration information of the master data subset and the dataset configuration information of the slave data subset includes: From the dataset configuration information of the master data subset and the dataset configuration information of the slave data subset, obtain the first dataset identifier of the master data subset, the first return value, the second dataset identifier of the slave data subset, and the second return value. Based on the first return value and the second return value, determine the fields to be matched and the associated fields from the data subset; The associated dataset configuration information is generated based on the first dataset identifier, the second dataset identifier, the field to be matched, and the associated field.

4. The method according to claim 2, characterized in that, The step of obtaining the target data corresponding to the query parameters from the dataset corresponding to the query parameters includes: Based on the dataset type and dataset configuration information, first initial retrieval information is obtained, wherein the first initial retrieval information includes first data to be replaced; Obtain the first target parameter from the query parameters, and replace the first data to be replaced in the first initial search information according to the first target parameter to obtain the first target search information; Based on the dataset type, determine the first target data source corresponding to the query parameter, and obtain the target data corresponding to the query parameter from the first target data source based on the first target retrieval information.

5. The method according to claim 4, characterized in that, The step of obtaining the target data corresponding to the query parameters from the first target data source based on the first target retrieval information includes: From the first target retrieval information, obtain the first retrieval statement and the second retrieval statement; Based on the first search statement, obtain the first returned data from the first target data source; According to the second search statement, the second returned data is obtained from the first target data source; According to a preset format, the first returned data and the second returned data are converted to obtain the target data.

6. The method according to claim 2, characterized in that, The step of obtaining the first intermediate data from the main data subset according to the query parameters includes: Obtain the first dataset identifier from the query parameters, and determine the master data subset based on the first dataset identifier; Based on the associated dataset configuration information of the associated dataset, obtain the second initial retrieval information; Obtain the first target parameter from the query parameters, and replace the second data to be replaced in the second initial search information according to the first target parameter to obtain the second target search information; Based on the second target retrieval information, the first intermediate data is obtained from the data source corresponding to the main data subset.

7. The method according to claim 6, characterized in that, The step of obtaining second intermediate data from the data subset based on the query parameters and the first intermediate data, and obtaining the target data corresponding to the query parameters based on the first intermediate data and the second intermediate data, includes: Obtain the second dataset identifier from the query parameters, and determine the subset of data based on the second dataset identifier; Based on the associated dataset configuration information, obtain the third initial retrieval information, the first associated field of the main data subset, the second associated field of the secondary data subset, and the reference field; If the query parameters do not include the second target parameter corresponding to the subset of data, the second target parameter is obtained from the first intermediate data based on the comparison result of the first associated field and the second associated field. Based on the second target parameter, the reference field, and the third initial search information, the third target search information is obtained; Based on the third target retrieval information, the second intermediate data is obtained from the data source corresponding to the data subset; The target data is obtained by assembling the first intermediate data and the second intermediate data.

8. A data query device, characterized in that, The device includes: The acquisition module is used to acquire query parameters and determine the dataset corresponding to the query parameters based on the dataset identifier in the query parameters. The first query module is used to obtain the target data corresponding to the query parameter from the dataset corresponding to the query parameter in the case that the dataset corresponding to the query parameter is not a related dataset, wherein the related dataset includes a primary data subset and a secondary data subset; The second query module is used to obtain first intermediate data in the main data subset according to the query parameters when the dataset corresponding to the query parameters is a related dataset. The third query module is used to obtain second intermediate data from the data subset based on the query parameters and the first intermediate data, and to obtain the target data corresponding to the query parameters based on the first intermediate data and the second intermediate data.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the data query method of any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the data query method according to any one of claims 1 to 7.

Images