Data query method and device and electronic equipment

Abstract

The embodiment of the invention provides a data query method and device and electronic equipment, and relates to the field of big data. The method comprises the following steps: in response to an obtained data query task, analyzing the query task to obtain a query target, a query condition and an output requirement corresponding to the query task; determining target data according to the query target and the query condition; the data format of the target data is determined, whether the data format of the target data is matched with the output requirement or not is judged, and the output requirement is used for indicating the data format of output data; and under the condition that the data format is not matched with the output requirement, performing conversion processing on the target data to obtain the target data matched with the output requirement, and outputting the target data, so that the data query efficiency is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of big data, and in particular to a data query method, device, and electronic device. Background Art

[0002] In financial systems, as business scale continues to expand, the system contains a large amount of data from diverse sources, complex structures, and non-standard or heterogeneous formats. By optimizing data query methods, the response speed of data queries and the accuracy of results can be improved.

[0003] Currently, computing engines have specific requirements for data formats, which may not match the data formats in storage systems. When a computing engine reads data in an incompatible format, it typically uses full data format conversion to convert the data into a processable format, thus avoiding errors that may be caused by inconsistent data formats.

[0004] However, full data format conversion, that is, converting the entire query result dataset from one format to another, requires a large amount of computing and storage resources, and the conversion process is time-consuming, resulting in low data query efficiency. Summary of the Invention

[0005] The present application provides a data query method, device and electronic device to solve the problems caused by full data conversion when the data format stored in the existing data query process does not match the data format processed by the computing engine, especially the technical difficulties caused by long processing time and low efficiency in the case of full conversion.

[0006] In a first aspect, the present application provides a data query method, comprising:

[0007] In response to the acquired data query task, the query task is parsed and processed to obtain the query target, query conditions and output requirements corresponding to the query task;

[0008] Determining target data according to the query target and the query condition;

[0009] Determining a data format of the target data, and judging whether the data format of the target data matches the output requirement, where the output requirement is used to indicate the data format of the output data;

[0010] In the case that the data format does not match the output requirement, the target data is converted to obtain target data that matches the output requirement, and the target data is output.

[0011] Optionally, the query target includes: a query data table and at least one query field, and determining target data according to the query target and the query condition includes:

[0012] Determine a query data table according to the query target;

[0013] Determine whether the query data table exists in the database;

[0014] If the query data table exists in the database, determining whether at least one query field exists in the query data table;

[0015] In a case where at least one query field exists in the query data table, determining a query row and a query column for each query field in the query data table;

[0016] For any one of the at least one query field, target data matching the query condition is screened from the query rows and query columns in the query data table.

[0017] Optionally, filtering target data matching the query condition from the query rows and query columns in the query data table includes:

[0018] Extracting data from query rows and query columns in the query data table to obtain an original data set;

[0019] Based on the query condition, each raw data in the raw data set is traversed, and the raw data is filtered to obtain at least one target data.

[0020] Optionally, the target data has multiple data formats, and determining whether the data format of the target data matches the output requirement includes:

[0021] Determining whether the target data corresponding to each data format matches the output requirement;

[0022] The converting the target data to obtain target data matching the output requirement includes:

[0023] Format conversion processing is performed on target data corresponding to at least one abnormal data format that does not match the output requirement to obtain target data that matches the output requirement.

[0024] Optionally, the data format includes at least one of the following: a field type, an encoding method, and a data structure. The performing format conversion processing on the target data corresponding to the at least one abnormal data format that does not match the output requirement to obtain the target data that matches the output requirement includes:

[0025] determining the type of discrepancy between each abnormal data format and the output requirements;

[0026] If the difference type is a field type, the target data corresponding to the abnormal data format is converted based on the field conversion rule;

[0027] If the difference type is encoding, then based on the encoding conversion rules, the target data corresponding to the abnormal data format is converted;

[0028] If the difference type is a data structure, the target data corresponding to the abnormal data format is converted based on a structure conversion rule.

[0029] Optionally, the method further includes: outputting the target data when the data format matches the output requirement.

[0030] Optionally, the method further comprises: encapsulating the target data after the conversion process, and storing the encapsulated data in a cache database;

[0031] Before determining the query data table according to the query target, the method further includes:

[0032] Based on at least one of the query fields and the query condition, it is determined whether corresponding target data exists in the cache database.

[0033] In a second aspect, the present application provides a data query device, comprising:

[0034] A processing module is used to analyze and process the acquired data query task in response to the query task, and obtain the query target, query conditions and output requirements corresponding to the query task;

[0035] A determination module, configured to determine target data according to the query target and the query condition;

[0036] The determining module is further configured to determine the data format of the target data;

[0037] a judging module, configured to judge whether the data format of the target data matches the output requirement, wherein the output requirement is used to indicate the data format of the output data;

[0038] The processing module is further configured to convert the target data to obtain target data that matches the output requirement when the data format does not match the output requirement;

[0039] An output module is used to output the target data.

[0040] Optionally, the determining module is further configured to determine a query data table according to the query target;

[0041] The judgment module is further used to judge whether the query data table exists in the database;

[0042] The judgment module is further configured to judge whether at least one query field exists in the query data table when the query data table exists in the database;

[0043] The determining module is further configured to determine a query row and a query column for each query field in the query data table if at least one query field exists in the query data table;

[0044] The processing module is further configured to filter target data matching the query condition from query rows and query columns in the query data table for any one of the at least one query field.

[0045] Optionally, the processing module is further configured to extract data from query rows and query columns in the query data table to obtain an original data set;

[0046] The processing module is further configured to traverse each raw data in the raw data set based on the query condition, filter the raw data, and obtain at least one target data.

[0047] Optionally, the judgment module is further configured to respectively judge whether the target data corresponding to each data format matches the output requirement;

[0048] The processing module is further configured to perform format conversion processing on target data corresponding to at least one abnormal data format that does not match the output requirement, to obtain target data that matches the output requirement.

[0049] Optionally, the determining module is further configured to determine a difference type between each abnormal data format and the output requirement;

[0050] The processing module is further configured to convert the target data corresponding to the abnormal data format based on the field conversion rule when the difference type is a field type;

[0051] The processing module is further configured to convert the target data corresponding to the abnormal data format based on the encoding conversion rules when the difference type is encoding mode;

[0052] The processing module is further configured to convert the target data corresponding to the abnormal data format based on a structure conversion rule when the difference type is a data structure.

[0053] Optionally, the output module is further configured to output the target data when the data format matches the output requirement.

[0054] Optionally, the processing module is further configured to encapsulate the target data after the conversion process and store the encapsulated data in a cache database;

[0055] The determining module is further configured to determine whether corresponding target data exists in the cache database based on at least one query field and the query condition before determining the query data table according to the query target.

[0056] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0057] The memory stores computer-executable instructions;

[0058] The processor executes the computer-executable instructions stored in the memory to implement the data query method as described in the first aspect and various possible implementations of the first aspect.

[0059] In a fourth aspect, the present application provides a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a processor, are used to implement the data query method as described in the first aspect and various possible implementations of the first aspect.

[0060] In a fifth aspect, the present application provides a program product, including a computer program, which implements the data query method described above when executed by a processor.

[0061] The present application provides a data query method, device, and electronic device. The method, in response to an acquired data query task, parses and processes the query task to obtain a query target, query conditions, and output requirements corresponding to the query task; determines target data based on the query target and the query conditions; determines the data format of the target data, and judges whether the data format of the target data matches the output requirements, wherein the output requirements are used to indicate the data format of the output data; and, if the data format does not match the output requirements, converts and processes the target data to obtain target data that matches the output requirements, and outputs the target data, thereby effectively improving data query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0063] Figure 1 A schematic diagram of the process of a data query method provided in this application Figure 1 ;

[0064] Figure 2 A schematic diagram of the process of a data query method provided in this application Figure 2 ;

[0065] Figure 3 A schematic diagram of the process of a data query method provided in this application Figure 3 ;

[0066] Figure 4 A schematic diagram of the structure of a data query device provided by this application;

[0067] Figure 5 This is a structural diagram of a data query device provided in this application.

[0068] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0069] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0070] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0071] In addition, this application involves conducting big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and using artificial intelligence technology to make automated decisions, and making technical solutions that have a significant impact on personal rights and interests based on the results of automated decisions. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered.

[0072] It should be noted that the data query method, device and electronic equipment provided in this application can be used in the field of big data, and can also be used in any field other than big data. The application field of the data query method, device and electronic equipment in this application is not limited.

[0073] In financial systems, with the continuous expansion of business scale and the deepening of digital transformation, a large amount of data from diverse sources and complex structures has accumulated. This data often exists in non-standard or heterogeneous formats, such as log files, JSON text, and custom binary formats. Data query, as one of the primary methods of data processing in financial systems, completes query processing for financial business through automated query and data conversion.

[0074] Currently, different computing engines typically have specific requirements for the input data format. For example, Spark is more suitable for processing columnar storage formats (such as Parquet and ORC), while Presto is more adept at parsing row-oriented formats (such as JSON and CSV). However, in real-world applications, data in storage systems often exists in multiple formats, which may not match the format required by the computing engine. To address this problem, the traditional approach is to perform full data format conversion before writing or querying data. This converts the entire dataset to the target format to ensure that the computing engine can correctly read and process the data, thus avoiding parsing errors or task failures caused by format inconsistencies.

[0075] While full data format conversion can ensure normal task execution to a certain extent, it comes at the cost of additional computing resources and storage space. This efficiency issue is particularly prominent when dealing with large-scale data. Therefore, a more efficient and flexible data processing mechanism is urgently needed to achieve on-demand format conversion and improve data query efficiency.

[0076] In response to the above problems, the present application proposes a data query method, which determines the target data and the format of the target data in the query target that meet the query conditions based on the query target and query conditions, and judges whether the data format of the target data matches the output requirements. If the data format of the target data does not match the output requirements, the target data is converted instead of converting the entire query target, thereby improving the efficiency of data processing.

[0077] This application is specifically applied to efficient data query processing in financial systems, especially banking systems. For example, it can be applied to risk assessment, where banks need to frequently query and analyze large amounts of transaction records and customer information. This data may be stored in different formats. By using an on-demand conversion method, only the relevant data can be formatted and output according to specific query requirements, improving processing efficiency and ensuring compliance with data format requirements of different departments or external regulatory agencies.

[0078] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0079] Figure 1 A schematic diagram of a data query method provided in an embodiment of the present application Figure 1 .like Figure 1 As shown, the data query method provided in this embodiment includes:

[0080] S101: In response to the acquired data query task, the query task is parsed and processed to obtain the query target, query conditions and output requirements corresponding to the query task.

[0081] When the system receives a data query task, it parses it and extracts key information from the task, including the query target (i.e., the data to be queried, for example, a specific customer's transaction history); the query conditions (i.e., the conditions that must be met for the query, for example, a transaction amount greater than 1,000 yuan); and the output requirements (i.e., the format in which the query results should be returned, for example, a table or JSON-formatted data).

[0082] Specifically, the query task is: "Query all my transaction records with an amount greater than 1,000 yuan in the past month and return them in a table format." After receiving this task, the system will parse it and extract the following key information:

[0083] Query target: transaction records of the past month;

[0084] Query conditions: Time range: past month, transaction amount: greater than 1,000 yuan;

[0085] Output requirements: Return query results in tabular form.

[0086] The system analyzes and processes the query tasks in response to the acquired data, and obtains the query targets, query conditions and output requirements corresponding to the query tasks, so as to facilitate the subsequent screening of transaction records that meet these conditions from the database and output the results in a table form.

[0087] S102: Determine target data according to the query target and query conditions.

[0088] Here, according to the query target and query conditions specified by the user, specific data that meets the query target and query conditions are found from the database to obtain the target data. In other words, the data required by the user is found through screening and matching.

[0089] By clarifying the query target and query conditions, the system filters out specific data that meets these conditions from a large amount of data. This data is called the "target data." This process ensures the accuracy and relevance of the query results.

[0090] S103: Determine the data format of the target data, and judge whether the data format of the target data matches the output requirement.

[0091] The data format may include JSON, Parquet, CSV, or a database table structure, for example. It is determined whether the data format of the target data matches the data format required in the output requirements, that is, whether the format of the target data is consistent with the output format desired by the user.

[0092] Specifically, the data format of the query result may be the default format of the database, such as a table format. The user hopes to output the data in CSV format. It is determined that the data format of the table format does not match the CSV format, that is, the data format of the current target data does not match the output requirements, and further conversion processing is required to meet the user's needs.

[0093] S104: When the data format does not match the output requirement, the target data is converted to obtain target data that matches the output requirement, and the target data is output.

[0094] Among them, when the format of the target data obtained by the query does not meet the output requirements specified by the user, the system will convert the target data to make the target data format consistent with the data format in the output requirements, and then output the target data after format conversion to the user.

[0095] Specifically, the target data is converted from the current format to the user-specified format, which may involve operations such as field mapping, encoding conversion, and structure adjustment.

[0096] The target data is converted and processed to obtain target data that matches the output requirements, ensuring that users can obtain data formats that meet their needs and improving data availability. Users do not need to manually convert data formats, and the system automatically completes the conversion, saving time and improving efficiency.

[0097] This embodiment provides a data query method. The method, in response to an acquired data query task, parses the query task to obtain a query target, query conditions, and output requirements corresponding to the query task; determines target data based on the query target and the query conditions; determines the data format of the target data, and judges whether the data format of the target data matches the output requirements, wherein the output requirements are used to indicate the data format of the output data; and, if the data format does not match the output requirements, converts the target data to obtain target data that matches the output requirements, and outputs the target data, thereby effectively improving data query efficiency.

[0098] Figure 2 A schematic diagram of a data query method provided in an embodiment of the present application Figure 2 .like Figure 2 As shown, in Figure 1 Based on the embodiment, the data query method is described in detail, including:

[0099] S201: In response to the acquired data query task, the query task is parsed and processed to obtain the query target, query conditions and output requirements corresponding to the query task.

[0100] Among them, step S201 is similar to step S101 and will not be repeated here.

[0101] S202: Determine a query data table according to the query target.

[0102] When performing a data query, you first need to determine which data table or data source to retrieve the data from based on the query target, that is, the data content the user wants to obtain. In other words, you need to find the specific location where the relevant data is stored according to the user's needs.

[0103] A database typically contains multiple tables, each storing different types of data. To efficiently retrieve the required data, you must clearly identify which tables store the data relevant to your query. This step is fundamental to data querying and ensures that queries locate the correct data source.

[0104] By clearly defining the query target and the corresponding query data table, you can ensure that the query operation can accurately locate the specific location where the relevant data is stored. By clarifying the query target and the corresponding query data table, you can improve query efficiency and ensure data accuracy.

[0105] Optionally, before determining the query data table according to the query target, the method further includes:

[0106] Based on at least one of the query fields and the query condition, it is determined whether corresponding target data exists in the cache database.

[0107] A cache database is used to store temporary or frequently accessed data. Certain queries may be executed frequently, or certain data may be queried multiple times. By storing this frequently used data in a cache database, the system can retrieve it directly from the cache for subsequent queries, without having to access the main database each time. This not only improves query efficiency but also reduces the burden on the main database, improving overall system performance.

[0108] Therefore, before determining the query data table based on the query target, a query is first performed in the cached data based on at least one query field and the query condition. The cache database is first checked to see whether the target data that meets the query field and the query condition exists. If the target data exists in the cache database, the target data is directly retrieved from the cache database and output. If the target data does not exist in the cache database, the query data table is then determined in the database based on the query target.

[0109] By checking the cache database before querying the main database, the system can quickly retrieve frequently used data, improving query efficiency and reducing the burden on the main database. By properly utilizing the cache mechanism, the system performance and user experience can be significantly improved.

[0110] S203: Determine whether the query data table exists in the database. If so, execute step S204; if not, execute step S205.

[0111] Before attempting to retrieve data from the database, you need to first confirm whether the table containing the target data exists. If so, the query continues; if not, additional logic needs to be executed, such as reporting an error, prompting the user, or trying another data source.

[0112] In real-world applications, the database structure may change, for example, if a table is deleted, renamed, or not yet created. If you assume the table exists when querying, this may cause the query to fail. By checking whether the table exists in advance, you can avoid these problems and provide a more user-friendly experience.

[0113] Specifically, the query goal is to retrieve all order information for customer ID 101. Based on the query goal, it is determined that data needs to be retrieved from the Orders table. The query determines whether the query table exists in the database, that is, whether the Orders table exists in the database. If the Orders table exists, the subsequent query operation continues. If the Orders table does not exist, a prompt message is generated, indicating that the table does not exist. The user is advised to check whether the table name is correct or contact the administrator to confirm whether the table exists.

[0114] S204: Determine whether there is at least one query field in the query data table. If so, execute step S206; if not, execute step S207.

[0115] After determining the query data table, it is necessary to further check whether the query data table contains the user-specified query fields. The user may specify one or more fields, and the system needs to check whether at least one of the user-specified fields exists in the query data table. Specifically, after determining the query data table, it is determined whether at least one query field exists in the query data table. If so, step S206 is executed; if not, step S207 is executed.

[0116] One possible implementation method is that the query task is to query the consumption records of a certain bank user. By parsing the query task, the query target corresponding to the query task is obtained as follows: query all consumption records of customer ID 101, including consumption date and consumption amount. According to the query target, it is determined that data needs to be obtained from the Orders table. Determine whether there is an Orders data table in the database. For example, there are Orders, Customers, and Products data tables in the database. Since there is an Orders data table in the database, it is determined that there is an Orders query data table in the database. Further, determine whether there is at least one query field in the Orders query data table. For example, the fields specified by the user are OrderDate and OrderAmount, that is, determine whether there is at least one of the two fields OrderDate and OrderAmount in the Orders query data table.

[0117] Determining whether at least one query field exists in the query data table is a key checkpoint in the data query process. This ensures that the target field exists before executing the query, thus preventing the system from blindly executing the query and wasting resources.

[0118] S205: Prompt that the query failed.

[0119] S206: Determine the query row and query column of each query field in the query data table.

[0120] Among them, determining the query row and query column of each query field in the query data table is to determine the specific position of each query field in the data table, that is, to clarify which data rows and data columns are related to the query conditions, so as to accurately extract the required data.

[0121] Query row: The data row that meets the query conditions.

[0122] Query column: The column where the query field specified by the user is located.

[0123] Specifically, a query data table usually contains multiple rows and columns of data, but the user may only need certain rows and columns. By determining the query rows and columns, the data the user needs can be accurately located, improving query efficiency and reducing unnecessary data processing.

[0124] A possible implementation method is to query the fields: OrderDate and OrderAmount.

[0125] Query column: the column where OrderDate and OrderAmount are located.

[0126] Query rows: rows where CustomerID = 101.

[0127] Determine the positions of the OrderDate and OrderAmount query fields in the query data table: OrderDate is in the third column and OrderAmount is in the fourth column.

[0128] By specifying the query rows and columns, the data required by the user can be accurately located, unnecessary data scanning can be reduced, and query efficiency can be improved.

[0129] S207: Prompt that the query failed.

[0130] S208: For any one query field of the at least one query field, extract data from query rows and query columns in the query data table to obtain an original data set.

[0131] For any one of the at least one query field, data rows and data columns related to the query field are extracted from the query data table, thereby forming an original data set containing all relevant data.

[0132] In one possible implementation, the OrderDate and OrderAmount query fields are located in the query data table as follows: OrderDate is in the third column, and OrderAmount is in the fourth column. The query rows are: rows where CustomerID = 101.

[0133] Extract the data rows and columns related to the query fields from the query data table,

[0134] Original data 1: OrderID = 1, CustomerID = 101, OrderDate = 2024-05-10, OrderAmount = 500

[0135] Original data 2: OrderID = 2, CustomerID = 101, OrderDate = 2024-06-01, OrderAmount = 800

[0136] The original data 1 and the original data 2 constitute the original data set.

[0137] By accurately extracting data rows and columns related to the query field, unnecessary data processing is reduced and query efficiency is improved.

[0138] S209: Based on the query condition, traverse each original data in the original data set, filter the original data, and obtain at least one target data.

[0139] Among them, according to the query conditions specified by the user, the original data sets extracted from the query rows and query columns in the query data table are checked one by one, and the data that meets the query conditions is screened out to finally obtain the target data.

[0140] Query conditions: User-specified filter conditions used to determine which data the user needs, for example, consumption records with an amount greater than 500 yuan in the past month. Filter condition 1: past month, filter condition 2: consumption amount greater than 500 yuan.

[0141] One possible implementation method is as follows:

[0142] Original data 1: OrderID = 1, CustomerID = 101, OrderDate = 2024-05-10, OrderAmount = 500

[0143] Original data 2: OrderID = 2, CustomerID = 101, OrderDate = 2024-06-01, OrderAmount = 800

[0144] The query condition is for consumption records with an amount greater than 600 yuan. By filtering the raw data in the original dataset, it is determined that the consumption amount in raw data 1 is 500 yuan, which is lower than the consumption amount required by the query condition and is greater than 600 yuan. Therefore, raw data 1 does not meet the query condition and is filtered out. The consumption amount in raw data 2 is 800 yuan, which is higher than the consumption amount required by the query condition and is greater than 600 yuan. Therefore, raw data 2 meets the query condition and is retained. By traversing raw data 1 and raw data 2 in the original dataset, raw data 1 that does not meet the query condition is filtered out, and raw data 2 that meets the query condition is retained to obtain the target data.

[0145] The original dataset may contain a large amount of data, but the user may only need a portion of it. By filtering according to the query conditions, the data the user needs can be accurately screened out, improving query efficiency and reducing unnecessary data processing.

[0146] S210: Determine the data format of the target data.

[0147] S211: Determine whether the target data corresponding to each data format matches the output requirement. If so, execute step S212; if not, execute step S213.

[0148] S212: Output target data.

[0149] S213: Convert the target data to obtain target data that matches the output requirements, and output the target data.

[0150] During the data query and processing process, the target data format is checked and processed accordingly based on the output requirements. Specifically, the target data format is checked to see if it meets the user-specified output requirements. If it does, it is directly output; if not, the data is converted to meet the output requirements before being output.

[0151] The format in which data is stored may not match the output format required by users. For example, data in a database may be in tabular format, but users may need to export it to JSON format for further processing. By checking the data format and performing necessary conversions, you can ensure that users receive data in a format that meets their needs, thereby improving data usability and user experience.

[0152] This embodiment provides a data query method. The method parses and processes the query task in response to the acquired data query task to obtain the query target, query conditions, and output requirements corresponding to the query task; determines the query data table based on the query target and determines whether the query data table exists in the database. If the query data table exists in the database, determines whether at least one query field exists in the query data table; determines the query row and query column of each query field in the query data table, and extracts the data in the query row and query column to obtain the original data set; based on the query conditions, traverses each raw data in the original data set, filters it, obtains at least one target data, determines the data format of the target data, determines whether the data format of the target data matches the output requirements, and directly outputs it if it does; if it does not, converts the target data before outputting it. This method processes and converts only necessary data by gradually parsing and verifying the query task, avoiding full data operations and improving query efficiency.

[0153] Figure 3 A schematic diagram of a data query method provided in an embodiment of the present application Figure 3 .like Figure 3 As shown, in Figure 1 Based on the embodiment, a possible implementation method of converting the target data to obtain target data that matches the output requirements is described in detail, including:

[0154] S301: Determine the difference type between each abnormal data format and the output requirement.

[0155] S302: If the difference type is a field type, the target data corresponding to the abnormal data format is converted based on the field conversion rule.

[0156] S303: If the difference type is encoding, the target data corresponding to the abnormal data format is converted based on the encoding conversion rule.

[0157] S304: If the difference type is data structure, the target data corresponding to the abnormal data format is converted based on the structure conversion rule.

[0158] The types of format differences between the target data format and the output required data format are determined. The difference types may include, but are not limited to, field type differences, encoding method differences, and data structure differences. The format differences between the target data format and the output required data format may be one or more. For example, the encoding method and field type of the target data and the output required data format are different.

[0159] Determine the format difference type between the data format of the target data and the data format required for output, and convert the target data corresponding to the abnormal data format based on the conversion rules corresponding to different difference types to obtain the target data that meets the output condition format requirements.

[0160] By identifying and addressing discrepancies between data formats and output requirements, we ensure that data meets user needs. Through targeted conversion processing, we reduce unnecessary data processing steps and improve data query processing efficiency.

[0161] Optionally, the target data after the conversion process is encapsulated, and the encapsulated data is stored in a cache database.

[0162] Among them, a cache database is established to store temporary data or frequently accessed data, which is usually used to improve query efficiency.

[0163] After format conversion or other processing steps, data that meets user output requirements is structured to make it easier to store, manage, and use. Encapsulation can include adding metadata and formatting data structures. Encapsulated data is stored in a cache database, making it directly available for subsequent queries, avoiding repeated data conversion and processing, and conserving system resources.

[0164] This embodiment provides a batch data query method that analyzes the differences between each abnormal data format and the output requirements. The difference types include field type, encoding method, and data structure. According to the data conversion rules for different difference types, the target data is converted, packaged, and stored in a cache database for subsequent access. Through standardized conversion rules, the consistency of the data before and after conversion is ensured, while improving the efficiency of data query.

[0165] Figure 4 This is a structural diagram of a data query device provided by this application. Figure 4 As shown, the present application provides a data query device, which includes:

[0166] The processing module 401 is used to analyze the acquired data query task in response to the query task, and obtain the query target, query conditions and output requirements corresponding to the query task;

[0167] Determining module 402, configured to determine target data according to the query target and the query condition;

[0168] The determining module 402 is further configured to determine the data format of the target data;

[0169] A judgment module 403 is used to judge whether the data format of the target data matches the output requirement, where the output requirement is used to indicate the data format of the output data;

[0170] The processing module 401 is further configured to convert the target data to obtain target data that matches the output requirement when the data format does not match the output requirement;

[0171] The output module 404 is configured to output the target data.

[0172] Optionally, the determining module 402 is further configured to determine a query data table according to the query target;

[0173] The judgment module 403 is further used to judge whether the query data table exists in the database;

[0174] The judgment module 403 is further configured to, if the query data table exists in the database, judge whether at least one query field exists in the query data table;

[0175] The determining module 402 is further configured to determine a query row and a query column for each query field in the query data table if at least one query field exists in the query data table;

[0176] The processing module 401 is further configured to filter target data matching the query condition from query rows and query columns in the query data table for any one of the at least one query field.

[0177] Optionally, the processing module 401 is further configured to extract data from query rows and query columns in the query data table to obtain an original data set;

[0178] The processing module 401 is further configured to traverse each raw data in the raw data set based on the query condition, filter the raw data, and obtain at least one target data.

[0179] Optionally, the judging module 403 is further configured to judge whether the target data corresponding to each data format matches the output requirement;

[0180] The processing module 401 is further configured to perform format conversion processing on target data corresponding to at least one abnormal data format that does not match the output requirement, to obtain target data that matches the output requirement.

[0181] Optionally, the determining module 402 is further configured to determine a difference type between each abnormal data format and the output requirement;

[0182] The processing module 401 is further configured to convert the target data corresponding to the abnormal data format based on the field conversion rule when the difference type is a field type;

[0183] The processing module 401 is further configured to convert the target data corresponding to the abnormal data format based on the encoding conversion rule when the difference type is encoding mode;

[0184] The processing module 401 is further configured to convert the target data corresponding to the abnormal data format based on a structure conversion rule when the difference type is a data structure.

[0185] Optionally, the output module 404 is further configured to output the target data when the data format matches the output requirement.

[0186] Optionally, the processing module 401 is further configured to encapsulate the target data after the conversion process and store the encapsulated data in a cache database;

[0187] The determining module 402 is further configured to determine whether corresponding target data exists in the cache database based on at least one query field and the query condition before determining the query data table according to the query target.

[0188] The implementation principle and technical effects of the data query device provided in the embodiment of the present application are similar to the implementation methods of each part of the aforementioned data query method, and will not be repeated here.

[0189] Figure 5 This is a structural diagram of a data query device provided by this application. Figure 5 As shown, the present application provides a data query device, which includes a data query device 500 including a receiver 501 , a transmitter 502 , a processor 503 and a memory 504 .

[0190] Receiver 501, for receiving instructions and data;

[0191] Transmitter 502, used to send instructions and data;

[0192] Memory 504, for storing computer-executable instructions;

[0193] The processor 503 is configured to execute the computer-executable instructions stored in the memory 504 to implement the various steps of the data query method in the above embodiment. For details, please refer to the relevant description in the above embodiment of the data query method.

[0194] Optionally, the memory 504 may be independent or integrated with the processor 503 .

[0195] When the memory 504 is independently provided, the electronic device further includes a bus for connecting the memory 504 and the processor 503 .

[0196] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the aforementioned embodiments and will not be described in detail here.

[0197] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in any of the above embodiments is implemented.

[0198] An embodiment of the present application further provides a computer program product, including a computer program, which implements the method described in any of the aforementioned embodiments when executed by a processor.

[0199] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.

[0200] It should be further noted that, although the various steps in the flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0201] It should be understood that the above-described device embodiments are merely illustrative, and the device of the present application may also be implemented in other ways. For example, the division of units / modules in the above-described embodiments is merely a logical functional division, and actual implementations may employ other division methods. For example, multiple units, modules, or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0202] In addition, unless otherwise specified, the functional units / modules in the various embodiments of the present application may be integrated into a single unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The aforementioned integrated units / modules may be implemented in the form of hardware or software program modules.

[0203] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0204] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0205] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data query method, characterized in that: The method comprises: In response to the acquired data query task, the query task is parsed and processed to obtain the query target, query conditions and output requirements corresponding to the query task; Determining target data according to the query target and the query condition; Determining a data format of the target data, and judging whether the data format of the target data matches the output requirement, where the output requirement is used to indicate the data format of the output data; In the case that the data format does not match the output requirement, the target data is converted to obtain target data that matches the output requirement, and the target data is output.

2. The method according to claim 1, characterized in that The query target includes: a query data table and at least one query field, and determining target data according to the query target and the query condition includes: Determine a query data table according to the query target; Determine whether the query data table exists in the database; If the query data table exists in the database, determining whether at least one query field exists in the query data table; In a case where at least one query field exists in the query data table, determining a query row and a query column for each query field in the query data table; For any one of the at least one query field, target data matching the query condition is screened from the query rows and query columns in the query data table.

3. The method according to claim 2, characterized in that The step of screening target data matching the query condition from the query rows and query columns in the query data table includes: Extracting data from query rows and query columns in the query data table to obtain an original data set; Based on the query condition, each raw data in the raw data set is traversed, and the raw data is filtered to obtain at least one target data.

4. The method according to claim 3, characterized in that The target data may be in a variety of data formats, and determining whether the data format of the target data matches the output requirement includes: Determining whether the target data corresponding to each data format matches the output requirement; The converting the target data to obtain target data matching the output requirement includes: Format conversion processing is performed on target data corresponding to at least one abnormal data format that does not match the output requirement to obtain target data that matches the output requirement.

5. The method according to claim 4, characterized in that The data format includes at least one of the following: a field type, an encoding method, and a data structure. The format conversion processing is performed on the target data corresponding to the at least one abnormal data format that does not match the output requirement to obtain the target data that matches the output requirement, including: determining the type of discrepancy between each abnormal data format and the output requirements; If the difference type is a field type, the target data corresponding to the abnormal data format is converted based on the field conversion rule; If the difference type is encoding, then based on the encoding conversion rules, the target data corresponding to the abnormal data format is converted; If the difference type is a data structure, the target data corresponding to the abnormal data format is converted based on a structure conversion rule.

6. The method according to claim 4, characterized in that The method further comprises: When the data format matches the output requirement, the target data is output.

7. The method according to claim 2, characterized in that The method further comprises: Encapsulating the target data after the conversion process, and storing the encapsulated data in a cache database; Before determining the query data table according to the query target, the method further includes: Based on at least one of the query fields and the query condition, it is determined whether corresponding target data exists in the cache database.

8. A data query device comprising: A processing module is used to analyze and process the acquired data query task in response to the query task, and obtain the query target, query conditions and output requirements corresponding to the query task; A determination module, configured to determine target data according to the query target and the query condition; The determining module is further configured to determine the data format of the target data; a judging module, configured to judge whether the data format of the target data matches the output requirement, wherein the output requirement is used to indicate the data format of the output data; The processing module is further configured to convert the target data to obtain target data that matches the output requirement when the data format does not match the output requirement; An output module is used to output the target data.

9. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images