Data query method and device, electronic equipment and computer readable storage medium
By parsing and distinguishing overlapping and independent parts in data query commands, the query process is optimized, solving the problem of low query efficiency in existing technologies and achieving more efficient data querying.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING SELIS PHOENIX INTELLIGENT INNOVATION TECH CO LTD
- Filing Date
- 2023-06-19
- Publication Date
- 2026-08-04
AI Technical Summary
Existing data query methods are slow, costly, and inefficient due to the complexity of big data query systems.
By parsing the received query commands, we can distinguish between overlapping and independent data, determine the reusable datasets and independent datasets, optimize the query commands to reduce repetitive operations, and improve query efficiency by utilizing the reusable datasets.
It improves the efficiency of data querying, reduces the consumption of computing resources and storage space, and enhances the speed and cost-effectiveness of data querying.
Smart Images
Figure CN116910090B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data query technology, specifically to a data query method, apparatus, and computer-readable storage medium. Background Technology
[0002] With the continuous development of new energy vehicles, vehicle intelligence has become a key development direction for major automakers. As a result, a large amount of data is generated in the production, testing and operation of new energy vehicles, and business personnel need to extract target data from the large amount of data to set up and analyze the vehicles.
[0003] Existing data query methods typically utilize big data technology, but the current big data query systems are extremely cumbersome, slow, and costly, resulting in very low data query efficiency. Summary of the Invention
[0004] In view of the above problems, embodiments of this application provide a data query method, apparatus and computer-readable storage medium to solve the problem of low data query efficiency in the prior art.
[0005] According to one aspect of the embodiments of this application, a data query method is provided, the method comprising:
[0006] Parse at least two received query commands to obtain the query data corresponding to each query command; distinguish the overlapping and independent data portions of the query data between the query commands; during the execution of each query command, determine a reused dataset based on the overlapping data portion and an independent dataset for each query command from a preset database, respectively; the data in the reused dataset is used by the query commands that have the overlapping data portion; obtain the target data corresponding to each query command based on the reused dataset and the independent dataset.
[0007] In an optional approach, the step of determining the reused dataset from a preset database based on the overlapping data portion during the execution of each query instruction further includes:
[0008] Identify the currently executed query instruction to obtain the current query instruction; determine whether there is data corresponding to the data overlap portion of the current query instruction in the current reused dataset; if not, obtain the data corresponding to the data overlap portion of the current query instruction from the database based on the current query instruction, and store the obtained data in the reused dataset; if it exists, obtain the data corresponding to the data overlap portion of the current query instruction from the reused dataset based on the current query instruction.
[0009] In an alternative approach, the step of determining the independent datasets for each query instruction based on the data independence portion further includes:
[0010] Identify the currently executed query instruction to obtain the current query instruction; based on the current query instruction, retrieve the data corresponding to the data-independent part of the current query instruction from the database, and store the retrieved data in the independent dataset of the current query instruction.
[0011] In one optional approach, the step of parsing at least two received query commands to obtain the query data corresponding to each query command, and distinguishing between the overlapping and independent data portions of the query data between the query commands, further includes:
[0012] Based on preset metadata, the query commands are parsed to obtain the data to be queried in the query commands; the data to be queried that has a matching relationship with each query command is identified as the data overlap part; the data to be queried that does not have a matching relationship with each query command is identified as the data independent part.
[0013] In an optional manner, the step of obtaining the target data corresponding to each query instruction based on the reused dataset and the independent dataset further includes:
[0014] Based on the overlapping data portion of the currently executed query instruction, the reused data is queried in the reused dataset; based on the independent data portion of the currently executed query instruction, the independent data is queried in the independent dataset; the reused data and / or the independent data are packaged to obtain the target data of the currently executed query instruction.
[0015] In one alternative approach, after the step of parsing the received at least two query instructions to obtain the data to be queried corresponding to each query instruction, the method includes:
[0016] The task complexity of the query task corresponding to the query instruction is identified, and the execution load of each query node is detected. If the task complexity is greater than or equal to the execution load of each query node, the query task is divided into at least one query subtask and a subtask complexity based on the task complexity and the execution load. The query subtask and the subtask complexity correspond to each other. Based on the matching degree between the subtask complexity and the execution load of each query node, the query subtask is allocated to each query node.
[0017] In an alternative approach, after the step of dispatching the query subtasks to the respective query nodes, the method further includes:
[0018] Determine whether the query instruction corresponding to the query subtask in the query node includes the overlapping data portion and / or the independent data portion; if it includes the overlapping data portion, query the data corresponding to the query instruction of the query subtask in the reused dataset; if it includes the independent data portion, query the data corresponding to the query instruction of the query subtask in the independent dataset.
[0019] According to another aspect of the embodiments of this application, a data query apparatus is provided, comprising:
[0020] The instruction parsing module is used to parse at least two received query instructions to obtain the query data corresponding to each query instruction;
[0021] The data differentiation module is used to distinguish between the overlapping and independent data portions of the data to be queried among the various query instructions;
[0022] The data determination module is used to determine a reused dataset and an independent dataset for each query instruction from a preset database based on the overlapping data portion and the independent data portion, respectively, during the execution of each query instruction. The data in the reused dataset is used by the query instructions that have the overlapping data portion.
[0023] The target data acquisition module is used to acquire the target data corresponding to each query instruction based on the reused dataset and the independent dataset.
[0024] According to another aspect of the embodiments of this application, an electronic device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;
[0025] The memory is used to store at least one executable instruction that causes the processor to perform the data query method as described in any of the foregoing embodiments.
[0026] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the storage medium storing at least one executable instruction, which, when executed on an electronic device / data querying device, causes the electronic device / data querying device to perform the operation of the data querying method as described in any of the foregoing embodiments.
[0027] This application embodiment parses at least two received query instructions to obtain the query data corresponding to each query instruction; it distinguishes the overlapping and independent data portions of the query data between the query instructions, thereby determining the data that can be reused during the execution of the query instructions and the data that can only be used independently; during the execution of the query instructions, based on the overlapping and independent data portions between the query instructions, it determines the reuse dataset and the independent dataset respectively, so that the data in the reuse dataset can be reused during the execution of the query instructions, thereby improving the data query efficiency; based on the query instructions, it obtains the target data from the reuse dataset and the independent dataset respectively, thereby realizing the query of the target data and improving the data query efficiency.
[0028] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description
[0029] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0030] Figure 1 A flowchart illustrating the data query method provided in this application is shown;
[0031] Figure 2 This illustration shows a simplified diagram of how the query task is segmented and allocated in the data query method provided in this application;
[0032] Figure 3 A schematic diagram of the structure of a first embodiment of the data query device provided in this application is shown;
[0033] Figure 4 A schematic diagram of an embodiment of the electronic device provided in this application is shown. Detailed Implementation
[0034] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein.
[0035] Figure 1 A flowchart illustrating a first embodiment of the data query method of this application is shown, which can be executed by an electronic device and / or a data query apparatus. Figure 1 As shown, the method includes the following steps:
[0036] Step S110: Parse the received at least two query commands to obtain the query data corresponding to each query command.
[0037] Among them, query instructions refer to instructions used to perform data query tasks, such as Structured Query Language (SQL), which is a database query and programming language that can be used to access, query, update and manage database systems. Several SQL instructions can be nested and related, which makes the data query method extremely flexible and powerful. This application uses SQL as an example for illustrative explanation.
[0038] For example, at least two received query commands can be parsed based on the acquired metadata to obtain the query data corresponding to each query command. Metadata describes data properties and can be used to support functions such as indicating storage location, historical data, resource lookup, and file records, thereby assisting in data retrieval. In short, metadata is data about data, essentially assigning attributes or types to the data. The acquired metadata can be pre-defined or obtained from database tables and / or project-developed Excel spreadsheets; no limitation is made here. Taking automotive-related data as an example, technicians typically query the database for the vehicle identification number (VIN), signal value, and signal acquisition time. "VIN," "signal value," and "signal acquisition time" can all be considered metadata. Data retrieved based on metadata, such as the VIN...
[0039] The specific data retrieved based on the query command are “LM***********0001”, the signal value “signal1”, and the signal acquisition time “May 1, 2023”.
[0040] Specifically, the query command text usually needs to include metadata in order to retrieve the data corresponding to the metadata. Therefore, the data type to be queried by the query command can be analyzed based on the obtained metadata. For example, the SQL statement "SELECT VINFROM TABLE_A" selects VIN data from table A, so the data to be queried corresponding to this query command is VIN data.
[0041] Step S120: Distinguish between the overlapping and independent data portions of the data to be queried between each query instruction.
[0042] It should be noted that different query commands may retrieve the same data. When dealing with a large amount of data, it is necessary to repeatedly scan, open, and parse the data file.
[0043] It should also be noted that the data to be queried between at least two query commands can have only overlapping data or independent data, or both overlapping data and independent data.
[0044] In this context, data overlap refers to the presence of identical or partially identical data between different query commands. For example, consider query command 1 "SELECT VINFROMTABLE_A" and query command 2 "SELECT VINFROMTABLE_AWHEREDATE = 20230501". Query command 1 retrieves all VIN data from table A, while query command 2 retrieves VIN data from table A with a date of May 1, 2023. Clearly, the data overlap between the two query commands is the VIN data with a date of May 1, 2023, while VIN data with a date other than May 1, 2023 is the independent data from query command 1. It can be understood that regardless of the execution order of query commands 1 and 2, after either query command 1 or any query command in query command 2 is executed, at least the VIN data with a date of May 1, 2023, will be retrieved—the data corresponding to the data overlap. Therefore, this data can be fed back into the execution of another query command to reduce redundant operations on the data file and improve data query efficiency.
[0045] Specifically, the query data between different query commands can be matched, and the matching query data can be identified as the overlapping data, while the non-matching query data can be identified as the independent data.
[0046] Optionally, after determining the overlapping and independent data portions of the data to be queried among the various query commands, pre-defined SQL syntax rules can be used to merge query commands with overlapping data portions to obtain optimized query commands. Executing the optimized query command can retrieve the union of the data to be queried from the query commands that participated in the command merging. For example, query command 2 "SELECT VINFROM TABLE_AWHEREDATE = 20230501" and query command 3 "SELECT VINFROM The query "TABLE_AWHEREDATE = 20230501 OR 20230502" is merged to obtain the optimized query command "SELECT VINFROM TABLE_AWHEREDATE = 20230501 OR DATE = 20230502". This optimized query command retrieves the VIN data for both dates 20230501 and 20230502 from table A. It also feeds back the VIN data for date 20230501 to query commands 2 and 3, and the VIN data for date 20230502 to query command 3. This improves data retrieval efficiency during the execution of query commands 2 and 3.
[0047] Furthermore, since the data retrieved by query instructions executed earlier in the execution order can be reused by query instructions executed later in the execution order, the execution priority of query instructions can be set according to the data range of the data to be queried in the query instructions. This allows the query instructions executed earlier to retrieve as much data as possible that all query instructions need to query, thereby reducing the query efficiency of the execution process of query instructions executed later.
[0048] In step S130, during the execution of each query instruction, a reuse dataset is determined from the preset database based on the overlapping data portion, and an independent dataset for each query instruction is determined based on the independent data portion. The data in the reuse dataset is used by the query instructions that have overlapping data portions.
[0049] In conjunction with the steps described above, a reused dataset refers to a collection of data used to store overlapping data during the execution of various query commands. For ease of explanation, the data in the reused dataset can be called reused data. Reused data can be used or reused by query commands that have overlapping data.
[0050] Correspondingly, an independent dataset refers to the collection of data used to store the independent parts of the data during the execution of each query command. For ease of explanation, the data in an independent dataset can be called independent data. Independent data is the data to be queried that is unique to each query command. Taking query command 2 "SELECT VINFROMTABLE_AWHEREDATE = 20230501" and query command 3 "SELECT VINFROMTABLE_AWHEREDATE = 20230501 OR 20230502" as examples, the data part corresponding to "DATE = 20230502" is the independent part of the data to be queried in query command 3. That is, the dataset of VIN data with date 20230502 obtained from table A by executing query command 3 is the independent dataset of query command 3.
[0051] Step S140: Obtain the target data corresponding to each query instruction based on the reused dataset and the independent dataset.
[0052] Specifically, during the execution of a query instruction, it can first be determined whether there are overlapping and / or independent data parts in the query execution. Then, based on the overlapping and / or independent data parts of the query instruction, data queries are performed in the current reused dataset and independent dataset to obtain the target data to be queried corresponding to the query instruction.
[0053] As can be seen, this application obtains the query data corresponding to each query command by parsing at least two received query commands; it distinguishes the overlapping and independent data parts of the query data between each query command, thereby determining the data that can be reused during the execution of the query commands and the data that can only be used independently; during the execution of the query commands, based on the overlapping and independent data parts between each query command, it determines the reuse dataset and the independent dataset respectively, so that the data in the reuse dataset can be reused during the execution of the query commands, thereby improving the data query efficiency; based on the query commands, it obtains the target data from the reuse dataset and the independent dataset respectively, thereby realizing the query of the target data and improving the data query efficiency.
[0054] Based on the above embodiments, this embodiment provides an exemplary description of the steps for determining a reused dataset from a preset database based on overlapping data portions during the execution of each query instruction. Specifically, the method includes the following steps:
[0055] Identify the currently executed query command and obtain the current query command; determine whether the current reused dataset contains data corresponding to the data overlap of the current query command; if not, retrieve the data corresponding to the data overlap of the current query command from the database based on the current query command and store the retrieved data in the reused dataset; if it exists, retrieve the data corresponding to the data overlap of the current query command from the reused dataset based on the current query command.
[0056] It should be noted that since the complexity and priority of the query tasks corresponding to the received query instructions may be different, the execution order of the query instructions and / or the execution progress of the query instructions may be different.
[0057] Specifically, for the currently executed query command, the command text in the current query command is analyzed to identify the data overlap portion of the data to be queried in the current query command. The data to be queried in the data overlap portion of the current query command is compared with the reused data in the current reused dataset to determine whether there is data in the current reused dataset that can be reused by the current query command. If it does not exist, it means that the data overlap portion to be queried by the current query command is being queried for the first time. Therefore, it needs to be queried from the preset database, and the data retrieved from the database is stored in the reused dataset so that subsequent query commands will have the same data overlap portion as the current query command. That is, subsequent query commands need to query the data to be queried that is wholly or partially the same as the current query command. If it exists, it means that the data overlap portion to be queried by the current query command has been queried by the previously executed query command. Therefore, the reused data in the reused dataset can be directly called during the execution of the current query command.
[0058] As can be seen, this step reduces the data processing steps required to repeatedly scan, open, and parse data files during the execution of query commands, greatly reducing the time consumption caused by data operations, as well as the number of data parsing operations and the storage space consumed, saving computing resources and improving the efficiency of data query.
[0059] Based on the above embodiments, this embodiment provides an exemplary description of the steps for determining the independent datasets for each query instruction based on the data independence component. Specifically, the steps of this embodiment include:
[0060] Identify the currently executed query command and obtain the current query command; based on the current query command, retrieve the data corresponding to the data-independent part of the current query command from the database, and store the retrieved data into the independent dataset of the current query command.
[0061] In conjunction with the foregoing embodiments, for the currently executed query instruction, the instruction text in the current query instruction is analyzed to identify the independent data part of the data to be queried in the current query instruction. Based on the independent data part of the current query instruction, the independent data in the data to be queried by the current query instruction is obtained, and the obtained independent data is stored in the independent dataset of the current query instruction. The independent dataset obtained based on each query instruction corresponds one-to-one with each query instruction.
[0062] Based on the above embodiments, this embodiment provides an illustrative description of the steps for parsing at least two received query commands to obtain the query data corresponding to each query command, and distinguishing the overlapping and independent data portions of the query data between the query commands. Specifically, the steps of this embodiment include:
[0063] Based on the preset metadata, the query commands are parsed to obtain the data to be queried in the query commands; the data to be queried that has a matching relationship between the query commands is identified as the data overlap part; the data to be queried that does not have a matching relationship between the query commands is identified as the data independent part.
[0064] Based on the foregoing embodiments, it can be understood that the instruction text of the query instruction usually includes metadata, that is, the data to be queried corresponding to the query instruction. For the explanation of the metadata, please refer to step S110, which will not be repeated here. The metadata can be obtained from a preset or from a database, which is not limited here.
[0065] Specifically, parsing each query command based on metadata can be achieved by matching the parsed command text with the metadata. The command text that matches the metadata is the data to be queried by the query command. If the data to be queried between at least two query commands matches each other, for example, if executing two query commands respectively yields two sets of data that are wholly or partially identical, then the two query commands can be considered to have a matching relationship, and the data to be queried between the two query commands matches each other. Otherwise, the data to be queried by the two query commands is considered to be independent of each other, and there is no matching relationship between the two query commands. The data to be queried for the two query commands are used independently.
[0066] Based on the above embodiments, this embodiment provides an exemplary description of the steps for obtaining the target data corresponding to each query instruction based on a reused dataset and an independent dataset. Specifically, the steps of this embodiment include:
[0067] Based on the overlapping data of the currently executed query instruction, the reused data is queried in the reused dataset; based on the independent data of the currently executed query instruction, the independent data is queried in the independent dataset; the reused data and / or independent data are packaged to obtain the target data of the currently executed query instruction.
[0068] It should be noted that during the execution of a query command, the system determines whether the currently executed query command has overlapping and / or independent data components. If the currently executed query command has overlapping data components, it queries the reused dataset based on the overlapping data components. If the reused data corresponding to the query command is not found in the reused dataset, it queries the database for the data corresponding to the query command and then stores the queried data in the reused dataset. If the currently executed query command has independent data components, it queries the independent dataset based on the independent data components. Each query command has corresponding reused and / or independent data. The reused and / or independent data corresponding to the query command are packaged and processed to obtain the target data corresponding to the query command.
[0069] It should also be noted that, in addition to retrieving data from the default database, it can also be obtained from the InfluxDB time-series database and the On-Board System (OBS). The process specifically includes: identifying the data type of the currently queried data, which includes cold data and hot data. Cold and hot data are categorized based on the access frequency and importance of the data. Hot data refers to data that is accessed frequently and is critical to business and applications. This data usually needs to be accessed and processed quickly and efficiently, so it is usually stored on high-performance, low-latency storage devices, such as retrieving hot data from InfluxDB. Cold data refers to data that is accessed less frequently and is less important to business and applications. This data usually needs to be stored for a long time but does not require frequent access and processing, so it can be stored on lower-cost, higher-capacity storage devices, such as retrieving cold data from OBS.
[0070] To further explain, because there is too much and too complex data related to automobiles, and the original data files are inconvenient to use—if the files are too small, there will be a large number of files, requiring frequent file operations; if the files are too large, they will be very slow to open or even impossible to open—therefore, after the data is retrieved, it can be packaged. Specifically, the method for packaging reusable data and / or independent data can include: obtaining the data volume of each data point; packaging each data point based on a preset data packaging threshold and the data volume of each data point to obtain at least one data packet, wherein the total data volume of each data point in the data packet should be less than or equal to the data packaging threshold; and during the packaging process, the data can also be sorted according to the data information; the data packets can be labeled according to the data information to distinguish the data contained in the data packets; furthermore, at least one data packet can be compressed separately or together to obtain a compressed package, i.e., the target data.
[0071] Based on the above embodiments, this embodiment provides an exemplary description of the steps after distinguishing the overlapping and independent data portions of the data to be queried between various query instructions. Specifically, the steps of this embodiment include:
[0072] The system identifies the task complexity of the query task corresponding to the query command and detects the execution load of each query node. If the task complexity is greater than or equal to the execution load of each query node, the query task is divided into at least one query subtask and its complexity based on the task complexity and the system execution load. The query subtasks and their complexities are corresponding. Based on the matching degree between the subtask complexity and the execution load of each query node, the query subtasks are allocated to each query node.
[0073] The task complexity can be calculated by weighting indicators such as the number of signals, signal types, time, number of data files, and number of VINs required for the query task.
[0074] The execution load may include, but is not limited to, querying data such as the CPU usage, memory usage, and disk I / O of the node.
[0075] Understandably, in a common query system, after receiving a query instruction from a query task, the task scheduler in the query system will allocate the query task to the task executor. If the task executor receives the query task allocated by the task scheduler, it will execute the query instruction of the query task to query the data. The task executor is equivalent to a query node.
[0076] Therefore, after receiving and parsing the query command of the query task, the task complexity of the query task corresponding to the query command is compared with the execution load of each query node to filter out the query nodes that can execute the query task, so as to facilitate the allocation of query tasks and improve data query efficiency.
[0077] Specifically, query nodes with task complexity lower than execution load are selected. The query task can be arbitrarily allocated among the selected query nodes, or allocated to query nodes with execution load lower than a preset load threshold. If the task complexity of the query task is greater than or equal to the execution load of each query node, it indicates that no query node can completely execute the query task. Therefore, the query task can be divided into at least one query subtask and its complexity based on the query data file as the smallest granularity, so that the query subtask can be allocated to query nodes for execution. Similarly, by comparing the subtask complexity of the query subtask with the execution load of each query node, the divided query task can be allocated to each query node for execution.
[0078] For example, refer to Figure 2 , Figure 2 This is a simplified diagram illustrating the segmentation and allocation of query tasks in this application. Figure 2 This includes data tables for query nodes and data tables for query tasks. The query system has three query nodes, and the execution load of these three query nodes is as follows: Figure 2 As shown in the diagram, the received query task requires querying six data files. The complexity of querying each data file is determined based on the number of VINs and signals in each file. It should be noted that during the actual task allocation and execution, tasks in a waiting state can exist within the query node. This means that the complexity of the subtasks actually assigned to the query subtasks can appropriately exceed the complexity that the query node can execute. This can be determined by a preset over-limit threshold, which determines the order of magnitude of the complexity exceeding the query node's execution capability. This operation ensures data integrity during task splitting and allocation. Therefore, it can be determined by... Figure 2 As shown in the table, the complexity of the entire query task is 49. Based on the idle percentage of each query node relative to the query system, query node 1 can execute 5.39 (≈5), query node 2 can execute 20.09 (≈20), and query node 3 can execute 23.52 (≈24). Therefore, the query task is divided into 3 parts, namely 3 query subtasks, and the query subtasks are scheduled to the corresponding executors for execution according to their task complexity.
[0079] Based on the above embodiments, this embodiment provides an exemplary description of the steps after the query subtasks are allocated to each query node. Specifically, the steps in this embodiment include:
[0080] Determine whether the query instruction corresponding to the query subtask in the query node includes overlapping data and / or independent data; if it includes overlapping data, query the data corresponding to the query instruction of the query subtask in the reused dataset; if it includes independent data, query the data corresponding to the query instruction of the query subtask in the independent dataset.
[0081] Referring to the foregoing embodiments, after the query subtasks are allocated to each query node, each query node executes its received query subtask. The execution process of each query node is similar to the execution process of the undivided query task in the foregoing embodiments. Each query node determines the data overlap part and / or data independence part of the query instruction corresponding to its respective query subtask. If the data overlap part is included, the data corresponding to the query instruction of the query subtask is queried in the reused dataset. If the data independence part is included, the data corresponding to the query instruction of the query subtask is queried in the independent dataset.
[0082] Therefore, by dividing and allocating complex query tasks based on the task complexity and execution load of query nodes, adaptive scheduling of query tasks is achieved, thereby improving query efficiency.
[0083] Figure 3 A schematic diagram of an embodiment of the data query device of this application is shown. Figure 3 As shown, the device 300 includes: an instruction parsing module 310, used to parse at least two received query instructions to obtain the query data corresponding to each query instruction; a data differentiation module 320, used to differentiate the overlapping and independent data portions of the query data between each query instruction; a data determination module 330, used to determine a reused dataset based on the overlapping data portions and an independent dataset based on the independent data portions from a preset database during the execution of each query instruction, wherein the data in the reused dataset is used by query instructions with overlapping data portions; and a target data acquisition module 340, used to acquire the target data corresponding to each query instruction based on the reused dataset and the independent dataset.
[0084] As can be seen, the data query device provided in this application obtains the query data corresponding to each query instruction by parsing at least two received query instructions; it distinguishes the overlapping and independent data parts of the query data between each query instruction, thereby determining the data that can be reused during the execution of the query instructions and the data that can only be used independently; during the execution of the query instructions, based on the overlapping and independent data parts between each query instruction, it determines the reuse dataset and the independent dataset respectively, so that the data in the reuse dataset can be reused during the execution of the query instructions, thereby improving the data query efficiency; based on the query instructions, it obtains the target data from the reuse dataset and the independent dataset respectively, thereby realizing the query of the target data and improving the data query efficiency.
[0085] In one alternative approach, the step of determining the reused dataset from a pre-defined database based on the overlapping data portion during the execution of each query instruction further includes:
[0086] Identify the currently executed query command and obtain the current query command; determine whether the current reused dataset contains data corresponding to the data overlap of the current query command; if not, retrieve the data corresponding to the data overlap of the current query command from the database based on the current query command and store the retrieved data in the reused dataset; if it exists, retrieve the data corresponding to the data overlap of the current query command from the reused dataset based on the current query command.
[0087] In one alternative approach, the step of determining the independent datasets for each query instruction based on the data-independent components further includes:
[0088] Identify the currently executed query command and obtain the current query command; based on the current query command, retrieve the data corresponding to the data-independent part of the current query command from the database, and store the retrieved data into the independent dataset of the current query command.
[0089] In one alternative approach, parsing at least two received query commands to obtain the query data corresponding to each query command; the step of distinguishing the overlapping and independent data portions of the query data between the query commands further includes:
[0090] Based on the preset metadata, the query commands are parsed to obtain the data to be queried in the query commands; the data to be queried that has a matching relationship between the query commands is identified as the data overlap part; the data to be queried that does not have a matching relationship between the query commands is identified as the data independent part.
[0091] In one alternative approach, the step of obtaining the target data corresponding to each query instruction based on the reused dataset and the independent dataset further includes:
[0092] Based on the overlapping data of the currently executed query instruction, the reused data is queried in the reused dataset; based on the independent data of the currently executed query instruction, the independent data is queried in the independent dataset; the reused data and / or independent data are packaged to obtain the target data of the currently executed query instruction.
[0093] In one alternative approach, after parsing at least two received query commands to obtain the query data corresponding to each query command, the method includes:
[0094] The system identifies the task complexity of the query task corresponding to the query command and detects the execution load of each query node. If the task complexity is greater than or equal to the execution load of each query node, the query task is divided into at least one query subtask and its complexity based on the task complexity and the system execution load. The query subtasks and their complexities are corresponding. Based on the matching degree between the subtask complexity and the execution load of each query node, the query subtasks are allocated to each query node.
[0095] In one alternative approach, after the step of dispatching query subtasks to each query node, the method further includes:
[0096] Determine whether the query instruction corresponding to the query subtask in the query node includes overlapping data and / or independent data; if it includes overlapping data, query the data corresponding to the query instruction of the query subtask in the reused dataset; if it includes independent data, query the data corresponding to the query instruction of the query subtask in the independent dataset.
[0097] Figure 4 The diagram shows a structural schematic of an embodiment of the electronic device of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.
[0098] like Figure 4 As shown, the electronic device may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.
[0099] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other network elements such as clients or other servers. The processor 402 executes program 410, specifically performing the relevant steps described in the data query method embodiment.
[0100] Specifically, program 410 may include program code, which includes computer-executable instructions.
[0101] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The electronic device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.
[0102] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0103] Specifically, program 410 can be called by processor 402 to cause the electronic device to perform the following operations:
[0104] Parse at least two received query commands to obtain the query data corresponding to each query command; distinguish between the overlapping and independent data parts of the query data between each query command; during the execution of each query command, determine a reuse dataset based on the overlapping data parts and an independent dataset based on the independent data parts from a preset database, and use the data in the reuse dataset together with the query commands that have overlapping data parts; obtain the target data corresponding to each query command based on the reuse dataset and the independent dataset.
[0105] As can be seen, the electronic device provided in this application obtains the query data corresponding to each query command by parsing at least two received query commands; it distinguishes the overlapping and independent data parts of the query data between each query command, thereby determining the data that can be reused during the execution of the query command and the data that can only be used independently; during the execution of the query command, based on the overlapping and independent data parts between each query command, it determines the reuse dataset and the independent dataset respectively, so that the data in the reuse dataset can be reused during the execution of the query command, thereby improving the data query efficiency; based on the query command, it obtains the target data from the reuse dataset and the independent dataset respectively, thereby realizing the query of the target data and improving the data query efficiency.
[0106] In an alternative manner, program 410 is invoked by processor 402 to cause the electronic device to perform the following operations:
[0107] Identify the currently executed query command and obtain the current query command; determine whether the current reused dataset contains data corresponding to the data overlap of the current query command; if not, retrieve the data corresponding to the data overlap of the current query command from the database based on the current query command and store the retrieved data in the reused dataset; if it exists, retrieve the data corresponding to the data overlap of the current query command from the reused dataset based on the current query command.
[0108] In an alternative manner, program 410 is invoked by processor 402 to cause the electronic device to perform the following operations:
[0109] Identify the currently executed query command and obtain the current query command; based on the current query command, retrieve the data corresponding to the data-independent part of the current query command from the database, and store the retrieved data into the independent dataset of the current query command.
[0110] In an alternative manner, program 410 is invoked by processor 402 to cause the electronic device to perform the following operations:
[0111] Based on the preset metadata, the query commands are parsed to obtain the data to be queried in the query commands; the data to be queried that has a matching relationship between the query commands is identified as the data overlap part; the data to be queried that does not have a matching relationship between the query commands is identified as the data independent part.
[0112] In an alternative manner, program 410 is invoked by processor 402 to cause the electronic device to perform the following operations:
[0113] Based on the overlapping data of the currently executed query instruction, the reused data is queried in the reused dataset; based on the independent data of the currently executed query instruction, the independent data is queried in the independent dataset; the reused data and / or independent data are packaged to obtain the target data of the currently executed query instruction.
[0114] In an alternative manner, program 410 is invoked by processor 402 to cause the electronic device to perform the following operations:
[0115] The system identifies the task complexity of the query task corresponding to the query command and detects the execution load of each query node. If the task complexity is greater than or equal to the execution load of each query node, the query task is divided into at least one query subtask and its complexity based on the task complexity and the system execution load. The query subtasks and their complexities are corresponding. Based on the matching degree between the subtask complexity and the execution load of each query node, the query subtasks are allocated to each query node.
[0116] In an alternative manner, program 410 is invoked by processor 402 to cause the electronic device to perform the following operations:
[0117] Determine whether the query instruction corresponding to the query subtask in the query node includes overlapping data and / or independent data; if it includes overlapping data, query the data corresponding to the query instruction of the query subtask in the reused dataset; if it includes independent data, query the data corresponding to the query instruction of the query subtask in the independent dataset.
[0118] This application provides a computer-readable storage medium storing at least one executable instruction that, when executed on an electronic device / data querying device, causes the electronic device / data querying device to perform the data querying method in any of the above method embodiments.
[0119] Specifically, the executable instructions can be used to cause the electronic device / data query device to perform the following operations:
[0120] Parse at least two received query commands to obtain the query data corresponding to each query command; distinguish between the overlapping and independent data parts of the query data between each query command; during the execution of each query command, determine a reuse dataset based on the overlapping data parts and an independent dataset based on the independent data parts from a preset database, and use the data in the reuse dataset together with the query commands that have overlapping data parts; obtain the target data corresponding to each query command based on the reuse dataset and the independent dataset.
[0121] As can be seen, the computer-readable storage medium provided in this application obtains the query data corresponding to each query instruction by parsing at least two received query instructions; it distinguishes the overlapping and independent data parts of the query data between each query instruction, thereby determining the data that can be reused during the execution of the query instructions and the data that can only be used independently; during the execution of the query instructions, based on the overlapping and independent data parts between each query instruction, it determines the reuse dataset and the independent dataset respectively, so that the data in the reuse dataset can be reused during the execution of the query instructions, thereby improving the data query efficiency; based on the query instructions, it obtains the target data from the reuse dataset and the independent dataset respectively, thereby realizing the query of the target data and improving the data query efficiency.
[0122] In one alternative approach, the executable instructions cause the electronic device / data querying device to perform the following operations:
[0123] Identify the currently executed query command and obtain the current query command; determine whether the current reused dataset contains data corresponding to the data overlap of the current query command; if not, retrieve the data corresponding to the data overlap of the current query command from the database based on the current query command and store the retrieved data in the reused dataset; if it exists, retrieve the data corresponding to the data overlap of the current query command from the reused dataset based on the current query command.
[0124] In one alternative approach, the executable instructions cause the electronic device / data querying device to perform the following operations:
[0125] Identify the currently executed query command and obtain the current query command; based on the current query command, retrieve the data corresponding to the data-independent part of the current query command from the database, and store the retrieved data into the independent dataset of the current query command.
[0126] In one alternative approach, the executable instructions cause the electronic device / data querying device to perform the following operations:
[0127] Based on the preset metadata, the query commands are parsed to obtain the data to be queried in the query commands; the data to be queried that has a matching relationship between the query commands is identified as the data overlap part; the data to be queried that does not have a matching relationship between the query commands is identified as the data independent part.
[0128] In one alternative approach, the executable instructions cause the electronic device / data querying device to perform the following operations:
[0129] Based on the overlapping data of the currently executed query instruction, the reused data is queried in the reused dataset; based on the independent data of the currently executed query instruction, the independent data is queried in the independent dataset; the reused data and / or independent data are packaged to obtain the target data of the currently executed query instruction.
[0130] In one alternative approach, the executable instructions cause the electronic device / data querying device to perform the following operations:
[0131] The system identifies the task complexity of the query task corresponding to the query command and detects the execution load of each query node. If the task complexity is greater than or equal to the execution load of each query node, the query task is divided into at least one query subtask and its complexity based on the task complexity and the system execution load. The query subtasks and their complexities are corresponding. Based on the matching degree between the subtask complexity and the execution load of each query node, the query subtasks are allocated to each query node.
[0132] In one alternative approach, the executable instructions cause the electronic device / data querying device to perform the following operations:
[0133] Determine whether the query instruction corresponding to the query subtask in the query node includes overlapping data and / or independent data; if it includes overlapping data, query the data corresponding to the query instruction of the query subtask in the reused dataset; if it includes independent data, query the data corresponding to the query instruction of the query subtask in the independent dataset.
[0134] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Furthermore, the embodiments in this application are not directed to any particular programming language.
[0135] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. Similarly, for the sake of brevity and to aid in understanding one or more aspects of the application, in the above description of exemplary embodiments of this application, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the application.
[0136] Those skilled in the art will understand that the modules in the device of the embodiment can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiment can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.
[0137] It should be noted that the above embodiments are illustrative of this application and not restrictive, and those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.
Claims
1. A data query method, characterized in that, The method includes: Parse at least two received query commands to obtain the query data corresponding to each query command; Distinguish between the overlapping and independent data portions of the data to be queried among the various query instructions, and set the execution priority of each query instruction according to the data range of the data to be queried in each query instruction; During the execution of each query instruction according to the execution priority, a reuse dataset is determined from a preset database based on the data overlap portion, and an independent dataset for each query instruction is determined based on the data independence portion. The data in the reuse dataset is used by the query instructions that have the data overlap portion. The target data corresponding to each query instruction is obtained based on the reused dataset and the independent dataset.
2. The method according to claim 1, characterized in that, The step of determining the reused dataset from a preset database based on the overlapping data portion during the execution of each query instruction further includes: Identify the currently executing query command and obtain the current query command; Determine whether there is data in the current reused dataset that corresponds to the data overlap portion of the current query instruction; If it does not exist, then based on the current query instruction, retrieve the data corresponding to the data overlap of the current query instruction from the database, and store the retrieved data in the reused dataset; If it exists, then the data corresponding to the data overlap of the current query instruction is obtained from the reused dataset based on the current query instruction.
3. The method according to claim 1, characterized in that, The step of determining the independent datasets for each query instruction based on the data independence portion further includes: Identify the currently executing query command and obtain the current query command; Based on the current query instruction, retrieve the data corresponding to the data-independent portion of the current query instruction from the database, and store the retrieved data in the independent dataset of the current query instruction.
4. The method according to claim 1, characterized in that, The parsing process receives at least two query commands to obtain the query data corresponding to each query command. The step of distinguishing between the overlapping and independent data portions of the data to be queried among the various query instructions further includes: The query instructions are parsed based on the preset metadata to obtain the data to be queried in the query instructions; Data to be queried that has a matching relationship among the various query commands is identified as overlapping data; data to be queried that does not have a matching relationship among the various query commands is identified as independent data.
5. The method according to claim 1, characterized in that, The step of obtaining the target data corresponding to each query instruction based on the reused dataset and the independent dataset further includes: Based on the data overlap portion of the currently executed query instruction, query the reused data in the reused dataset; Based on the data-independent portion of the currently executed query instruction, independent data is queried in the independent dataset; The reused data and / or the independent data are packaged and processed to obtain the target data of the currently executed query instruction.
6. The method according to claim 1, characterized in that, After the step of parsing at least two received query commands to obtain the query data corresponding to each query command, the method includes: Identify the task complexity of the query task corresponding to the query instruction, and detect the execution load of each query node at present; If the task complexity is greater than or equal to the execution load of each query node, then the query task is divided into at least one query subtask and a subtask complexity based on the task complexity and the execution load, and the query subtask and the subtask complexity correspond to each other. Based on the matching degree between the complexity of the subtask and the execution load of each query node, the query subtask is allocated to each query node.
7. The method according to claim 6, characterized in that, After the step of allocating the query subtasks to the respective query nodes, the method further includes: Determine whether the query instruction corresponding to the query subtask in the query node includes the overlapping data portion and / or the independent data portion; If the overlapping data portion is included, then the data corresponding to the query instruction of the query subtask is queried in the reused dataset; If the independent data portion is included, then the data corresponding to the query instruction of the query subtask is queried in the independent dataset.
8. A data query device, characterized in that, The device includes: The instruction parsing module is used to parse at least two received query instructions to obtain the query data corresponding to each query instruction; The data differentiation module is used to differentiate between the overlapping and independent data portions of the data to be queried among the various query instructions, and to set the execution priority of each query instruction according to the data range of the data to be queried in each query instruction; The data determination module is used to determine a reused dataset and an independent dataset for each query instruction from a preset database based on the data overlap portion and the data independence portion, respectively, during the execution of each query instruction according to the execution priority. The data in the reused dataset is used by the query instructions that have the data overlap portion. The target data acquisition module is used to acquire the target data corresponding to each query instruction based on the reused dataset and the independent dataset.
9. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the data query method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on the electronic device / data query device, causes the electronic device / data query device to perform the operation of the data query method as described in any one of claims 1-7.