Data query caching method and device, equipment and storage medium

Abstract

The invention discloses a data query caching method, device and equipment and a storage medium, and is applied to the field of big data, and the method comprises the following steps: obtaining a query request, and analyzing the query request to obtain a query condition; judging whether a matched query record matched with the query condition exists in the cache library, and if yes, directly returning the matched query record as a query result; otherwise, executing the query task based on the query condition to obtain a query result, and updating the cache library according to the query condition and the query result. By analyzing the query condition, the intention of the user can be more accurately understood, and a foundation is laid for cache matching and query execution. When the cache library is hit, the result can be quickly returned without repeated calculation, so that the waiting time of the user is shortened, the query efficiency is improved, and the calculation resource consumption is reduced. When the query result is not hit, the query task is executed to obtain the query result, and the new result is stored in the cache library, so that cache support is provided for subsequent same query, effective cache data is accumulated step by step, and the query efficiency is continuously optimized.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a data query caching method, device, equipment and storage medium. Background Art

[0002] In big data applications, financial institutions need to handle the storage and computing needs of massive amounts of data. With the rapid growth of data volumes and the frequent occurrence of query requests, efficiently processing data queries has become a pressing issue.

[0003] In existing technologies, query processing on big data platforms typically involves directly executing queries across massive amounts of data based on user query requests. This approach consumes significant computing resources when faced with numerous repeated queries, increasing user wait times. Furthermore, repeated query requests can cause the same computational task to be executed multiple times, wasting computing resources. Furthermore, high levels of concurrent query requests can strain the performance of the big data platform, impacting overall response efficiency. Summary of the Invention

[0004] The present invention provides a data query caching method, apparatus, device and storage medium, which improve query efficiency and reduce computing resource consumption by recording and storing query requests and results.

[0005] According to one aspect of the present invention, a data query caching method is provided, the method comprising:

[0006] Obtain a query request, and parse the query request to obtain query conditions;

[0007] Determine whether there is a matching query record matching the query condition in the cache library. If so, directly return the matching query record as the query result, wherein the cache library includes various historical query records;

[0008] Otherwise, the query task is executed based on the query conditions to obtain the query results, and the cache library is updated according to the query conditions and query results.

[0009] Optionally, a query request is obtained and the query request is parsed to obtain query conditions, including: receiving a query request initiated by a user through a client, wherein the query request includes a structured query statement or a semi-structured query statement; performing grammatical and semantic analysis on the query request through a parsing engine to extract various query parameters; performing parameter standardization and denoising on various query parameters, retaining key query parameters, and using each key query parameter as a query condition.

[0010] The benefits of this setting are: achieving accurate analysis and optimization of query requests, filtering out invalid information, improving the accuracy and efficiency of subsequent query matching, and reducing the system processing burden.

[0011] Optionally, the historical query records include historical query key values ​​and corresponding historical query results, and determining whether there are matching query records that match the query conditions in the cache library includes: performing hash calculations on each key query parameter to generate a hash value of a fixed length as a cache index; determining whether there is a target query key value consistent with the cache index in each historical query key value, and if so, taking the historical query record where the target query key value is located as a matching query record; otherwise, determining that there is no matching query record.

[0012] The benefits of this setting are: implementing an efficient cache matching mechanism, converting complex parameter comparisons into simple hash value comparisons, significantly improving matching speed, reducing computing overhead, and optimizing query response efficiency.

[0013] Optionally, query tasks are executed based on the query conditions to obtain query results, including: calling the distributed computing engine in the computing cluster to decompose the query conditions into distributed query tasks; distributing each distributed query task to each node in the computing cluster, and reading shard data from the distributed file system in parallel through each node based on the corresponding distributed query task; integrating the shard data of each node to generate query results.

[0014] The benefits of this setup are: enabling parallel processing of large-scale data, fully utilizing cluster resources to accelerate query execution, significantly improving system throughput and response speed, and effectively responding to massive data query needs.

[0015] Optionally, updating the cache according to the query conditions and query results includes: performing hash calculation on the query conditions to generate a fixed-length hash value as an update query key; combining the update query key and the query result to generate an update query record, and inserting the update query record into the cache.

[0016] The benefits of this setting are: implementing an efficient cache update mechanism, ensuring key value uniqueness through hash values, and reducing response time and resource consumption for subsequent queries.

[0017] Optionally, the method further includes: setting an expiration time for each historical query record, and obtaining the storage time corresponding to each historical query record according to a specified period; when the storage time reaches the expiration time, deleting the corresponding historical query record.

[0018] The benefits of this setting are: dynamic cleaning and optimization of the cache library, timely elimination of expired data, avoiding invalid records occupying cache space, and improving cache utilization and query matching efficiency.

[0019] Optionally, the method also includes: obtaining the non-access time corresponding to each historical query record, and sorting each historical query record in descending order according to the non-access time to generate an non-access queue; obtaining the remaining space of the cache library, and when the remaining space is less than the preset capacity, selecting the historical query records in the non-access queue in turn for deletion.

[0020] The advantage of this setting is that it prioritizes cleaning invalid data that has not been used for a long time, effectively alleviating cache space pressure and improving cache hit rate.

[0021] According to another aspect of the present invention, an electronic device is provided, comprising:

[0022] at least one processor;

[0023] and a memory communicatively coupled to the at least one processor;

[0024] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a data query caching method described in any embodiment of the present invention.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a data query caching method according to any embodiment of the present invention when executed.

[0026] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the computer program implements a data query caching method according to any embodiment of the present invention.

[0027] The technical solution of the embodiment of the present invention can more accurately understand the user's query intention by parsing the query conditions, extract key parameters for subsequent processing, and lay the foundation for cache matching and query execution. When a cache library is hit, the result can be quickly returned without repeated calculations, which significantly reduces user waiting time, improves query response efficiency, and reduces computing resource consumption. In the event of a miss, results are generated through distributed computing to ensure the accuracy of the query, and the new results are stored in the cache to provide cache support for subsequent identical queries, gradually accumulate effective cache data, and continuously optimize query efficiency.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 This is a flowchart of a data query caching method provided according to the first embodiment of the present invention;

[0031] Figure 2 This is a flowchart of another data query caching method provided according to the second embodiment of the present invention;

[0032] Figure 3 This is a structural diagram of a data query cache device provided according to a third embodiment of the present invention;

[0033] Figure 4 The present invention is a schematic diagram of the structure of an electronic device for implementing a data query caching method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] Example 1

[0037] Figure 1A flowchart of a data query caching method is provided for the first embodiment of the present invention. This embodiment is applicable to data query scenarios. The method can be executed by a data query caching device. The data query caching device can be implemented in the form of hardware and / or software. The data query caching device can be configured in a computer controller. Figure 1 As shown, the method includes:

[0038] S110: Obtain a query request, and parse the query request to obtain query conditions.

[0039] A query request is a request from a user to the big data platform for specific data, containing information about the data the user is looking for. Parsing involves the big data platform analyzing and processing the query request to understand the information contained within. Query conditions are the specific criteria set by the user in the query request to limit the query scope and filter the target data, such as the data's time range, type, and keywords. Query conditions are the key basis for determining the query target, and the big data platform can execute specific query operations based on the query conditions.

[0040] Specifically, when a user initiates a data query request to the big data platform, the platform will first receive the query request. The query request is essentially a user's retrieval instruction for the target data, which may contain structured or unstructured information. The platform will perform semantic and syntactic analysis on the request through a parser, such as extracting key fields, time ranges, data types, filtering conditions, and sorting requirements in the query statement, thereby clarifying the specific characteristics of the data required by the user. For example, if a user queries "credit card consumption data of Bank A's Beijing branch in May 2025", the query conditions obtained after parsing include the time range "May 2025", the institution "Beijing branch", and the data type "credit card consumption".

[0041] Optionally, a query request is obtained and the query request is parsed to obtain query conditions, including: receiving a query request initiated by a user through a client, wherein the query request includes a structured query statement or a semi-structured query statement; performing grammatical and semantic analysis on the query request through a parsing engine to extract various query parameters; performing parameter standardization and denoising on various query parameters, retaining key query parameters, and using each key query parameter as a query condition.

[0042] The query request contains a structured or semi-structured query statement. The parsing engine then performs syntactic and semantic analysis on the query request. Syntactic analysis checks the statement format for correctness, such as ensuring the correct keywords and closed brackets. If there are any syntactic errors, a warning message will be displayed. Semantic analysis understands the actual meaning of the statement, extracting parameters such as table names and filter conditions from the statement, or parsing the key-value pairs or fields corresponding to tags from the statement. After extracting the query parameters, the platform standardizes and denoises the parameters. Standardization converts parameters with different formats to a unified format. For example, standardizing the time parameter "May 2025" to "2025-05" ensures the accuracy of subsequent matching. Denoising filters out invalid information, such as misspelled parameters, duplicate fields, or irrelevant conditions. For example, if a query request contains "branch = Beijing Branch", denoising will modify or exclude this parameter, ultimately retaining parameters such as "branch = Beijing Branch" and "time range = 2025-05". The parsing engine's lexical analysis, syntax tree construction, and parameter cleaning algorithms can ensure the accuracy and effectiveness of query conditions, laying the foundation for subsequent cache hits or efficient queries.

[0043] S120 , determining whether there is a matching query record matching the query condition in the cache library, if so, executing S130 , otherwise executing S140 .

[0044] The cache refers to a database within the bank's big data platform that stores historical query records and their corresponding results. It efficiently stores and manages massive amounts of query cache data. Historical query records contain the query conditions and corresponding results of previous user requests. Matching query records are stored in the cache to ensure rapid responses to subsequent identical or similar query requests. Matching query records are historical query records in the cache that exactly match the current query conditions. The presence of matching query records in the cache indicates that the same query request has been previously processed and the results have been saved.

[0045] Specifically, the cache library is a cache space built by the platform based on a distributed storage architecture, which stores a complete record of historical queries, including the specific conditions of each query and the corresponding result data. After obtaining the current query conditions, the platform will quickly retrieve the cache library through a hash algorithm or index mechanism. If there is a historical record in the cache that is exactly the same as the current query conditions, for example, a user has previously queried "Beijing Branch Credit Card Consumption Data in May 2025" and the result has been cached, the cache hit mechanism will be triggered. At this time, the platform does not need to re-execute the underlying data calculation, but directly retrieves the result from the cache and returns it to the user.

[0046] Optionally, the historical query records include historical query key values ​​and corresponding historical query results, and determining whether there are matching query records that match the query conditions in the cache library includes: performing hash calculations on each key query parameter to generate a hash value of a fixed length as a cache index; determining whether there is a target query key value consistent with the cache index in each historical query key value, and if so, taking the historical query record where the target query key value is located as a matching query record; otherwise, determining that there is no matching query record.

[0047] It should be noted that in the big data platform, historical query records are composed of historical query keys and corresponding query results. After the platform extracts key query parameters through the parsing engine, it performs a hash calculation.

[0048] Specifically, the platform combines key query parameters into a string according to specific rules, then applies a hash algorithm to this string to generate a fixed-length binary hash value, such as a 32-bit string. This calculated hash value is then used as a cache index to quickly locate records in the cache. Each historical query record in the cache stores a corresponding historical query key value, which is an index generated by applying the same hash algorithm to the historical query parameters. When a new cache index is generated, the platform iterates through all historical query key values ​​in the cache to check whether there is a target query key value that is exactly the same as the current index. For example, if a user has previously searched for data with the same criteria in a historical query, the corresponding historical query key value will be the same as the current cache index. In this case, the platform directly locates the historical query record containing that key value and returns the corresponding historical query result to the user as a matching result, without having to re-perform the underlying calculations. If, after iterating through all historical query key values, no target key value is found that matches the current cache index, the cache is determined to contain no matching record. The matching method through hash index takes advantage of the efficiency and uniqueness of the hash algorithm, ensures the accuracy and efficiency of cache matching, can quickly locate targets in massive historical records, and improve query response speed.

[0049] S130: directly return matching query records as query results, wherein the cache includes various historical query records.

[0050] S140: Execute the query task based on the query condition to obtain the query result, and update the cache library according to the query condition and the query result.

[0051] Among them, when there are no records matching the current query conditions in the cache, the big data platform needs to perform retrieval, calculation and other operations in the massive data according to the query conditions to generate the query results required by the user. The query result refers to the data result obtained by the big data platform after executing the query task according to the query conditions, which contains the information that the user ultimately wants to obtain. Updating the cache refers to adding the query conditions and corresponding query results of this query to the cache, so that subsequent identical query requests can directly obtain results from the cache, improving query efficiency. During the update process, the space management and data validity of the cache also need to be considered. The cache replacement algorithm can be used to clear expired or invalid cache records to ensure the efficient use of the cache.

[0052] Specifically, if no matching record is found in the cache, a cache miss, the platform converts the query into a computational task for the underlying computing cluster. The platform can then invoke a distributed computing framework to scan, filter, and aggregate the massive amount of data stored in the distributed file system. For example, for the query above, the platform can traverse log files corresponding to the time and institution, filter out credit card purchase records, and aggregate them. Once the query results are generated, the platform stores them in the cache along with the query conditions.

[0053] Optionally, the method further includes: setting an expiration time for each historical query record, and obtaining the storage time corresponding to each historical query record according to a specified period; when the storage time reaches the expiration time, deleting the corresponding historical query record.

[0054] It's known that in big data platforms, to ensure the validity of cached data and the rational use of cache space, an expiration mechanism is implemented for historical query records. Specifically, the platform sets an expiration period for each historical query record, which can be determined based on factors such as the frequency of data updates and business scenario requirements. For example, for daily updated transaction data, the expiration period for its cached records might be set to 24 hours; for monthly statistical data, the expiration period might be extended to 30 days. The expiration period is designed to ensure that cached records remain valid for a reasonable period of time, preventing inconsistencies between cached results and the latest data due to data updates.

[0055] Specifically, the platform will automatically scan the cache library according to the specified period to obtain the storage duration of each historical query record. The storage duration refers to the time interval from the time the record is stored in the cache library to the current scan time point. For example, a record is stored in the cache at 10:00 on June 10, 2025. If the scanning cycle is every hour, when it is scanned at 10:00 on June 11, its storage duration is 24 hours. When the storage duration of a historical query record reaches the preset expiration duration, the platform will automatically delete the record from the cache library. For example, if the expiration duration of a record is 24 hours, when the storage duration exceeds 24 hours, the deletion operation will be triggered at the next scan. By deleting expired records, you can avoid expired data occupying cache space and ensure that valid query results are always stored in the cache library.

[0056] Optionally, the method also includes: obtaining the non-access time corresponding to each historical query record, and sorting each historical query record in descending order according to the non-access time to generate an non-access queue; obtaining the remaining space of the cache library, and when the remaining space is less than the preset capacity, selecting the historical query records in the non-access queue in turn for deletion.

[0057] It's known that, to optimize cache space utilization, big data platforms also manage historical query records by their inaccessibility time. The platform tracks the inaccessibility time for each historical query record in real time, representing the time interval from the last time the record was accessed to the current time. For example, if a record was accessed at 10:00 AM on June 10, 2025, and has not been accessed since, the system will calculate its inaccessibility time at 10:00 AM on June 11 as 24 hours.

[0058] Specifically, the platform sorts all historical query records by their inaccessibility time from longest to shortest, creating an unaccessed queue. Records at the top of the unaccessed queue indicate they have been accessed the longest and are the least used cached data. The platform also continuously monitors the remaining storage space in the cache. The preset capacity is a threshold set based on platform hardware resources and business needs. When the remaining space falls below this threshold, the cache is nearing capacity and needs to be freed up to store new query records. At this point, the platform selects and deletes historical query records sequentially, starting from the head of the unaccessed queue. For example, the record with the longest inaccessibility time in the unaccessed queue is deleted first, followed by the next longest, and so on. By sorting by inaccessibility time and deleting records in order, cache records that have not been used for a long time are prioritized, freeing up space for new query results. This maintains the cache's efficient operation in high-volume query scenarios, improving overall query response speed and resource utilization.

[0059] The technical solution of the embodiment of the present invention can more accurately understand the user's query intention by parsing the query conditions, extract key parameters for subsequent processing, and lay the foundation for cache matching and query execution. When a cache library is hit, the result can be quickly returned without repeated calculations, which significantly reduces user waiting time, improves query response efficiency, and reduces computing resource consumption. In the event of a miss, results are generated through distributed computing to ensure the accuracy of the query, and the new results are stored in the cache to provide cache support for subsequent identical queries, gradually accumulate effective cache data, and continuously optimize query efficiency.

[0060] Example 2

[0061] Figure 2 This is a flowchart of a data query cache method provided by the second embodiment of the present invention. This embodiment adds a specific process of executing a query task based on the query conditions to obtain the query results, and updating the cache library according to the query conditions and query results. Among them, the specific content of steps S250-S260 is roughly the same as that of steps S120-S130 in the first embodiment, so they will not be repeated in this embodiment. Figure 2 As shown, the method includes:

[0062] S210: Obtain a query request, and parse the query request to obtain query conditions.

[0063] Optionally, a query request is obtained and the query request is parsed to obtain query conditions, including: receiving a query request initiated by a user through a client, wherein the query request includes a structured query statement or a semi-structured query statement; performing grammatical and semantic analysis on the query request through a parsing engine to extract various query parameters; performing parameter standardization and denoising on various query parameters, retaining key query parameters, and using each key query parameter as a query condition.

[0064] S220: Determine whether there is a matching query record matching the query condition in the cache library. If so, execute S230; otherwise, execute S240-S280.

[0065] Optionally, the historical query records include historical query key values ​​and corresponding historical query results, and determining whether there are matching query records that match the query conditions in the cache library includes: performing hash calculations on each key query parameter to generate a hash value of a fixed length as a cache index; determining whether there is a target query key value consistent with the cache index in each historical query key value, and if so, taking the historical query record where the target query key value is located as a matching query record; otherwise, determining that there is no matching query record.

[0066] S230: directly return matching query records as query results, wherein the cache includes various historical query records.

[0067] S240: Call the distributed computing engine in the computing cluster to decompose the query conditions into distributed query tasks.

[0068] Specifically, when the cache library does not hit the query conditions, the platform needs to execute the query task through distributed computing. Specifically, it can call the distributed computing engine in the computing cluster to break down the parsed query conditions into multiple distributed query tasks. For example, if a user queries "credit card consumption data of the Beijing branch in May 2025", the platform will split the query into subtasks for different dimensions such as "May 2025" and "Beijing branch" according to the sharding rules of the data storage, such as partitioning by time or institution. Each subtask corresponds to specific data processing logic, such as filtering, aggregation and other operations, to ensure that the query conditions are broken down into units that can be executed in parallel.

[0069] S250: Distribute each distributed query task to each node in the computing cluster, and read the shard data from the distributed file system in parallel based on the corresponding distributed query task through each node.

[0070] Specifically, the platform distributes distributed query tasks to the nodes of a computing cluster. A computing cluster consists of multiple data nodes and compute nodes, each responsible for processing its assigned subtasks. Upon receiving a task, a node reads the corresponding sharded data from the distributed file system. Distributed file systems store massive amounts of data in blocks across different nodes. For example, data for "May 2025" might be sharded and stored across multiple nodes. Each node reads its own sharded data in parallel, achieving parallel data processing.

[0071] S260: Integrate the shard data of each node and generate query results.

[0072] Specifically, after each node processes the sharded data, the platform aggregates the processing results from each node through an integration mechanism. For example, credit card consumption records from the Beijing branch, filtered by each node, are transmitted to a designated node for data merging, deduplication, sorting, or further aggregation to ultimately generate complete query results. By utilizing distributed computing and parallel processing of data across multiple nodes in the cluster, the efficiency of massive data queries is significantly improved, performance bottlenecks encountered during single-node processing are avoided, and complex query tasks can be completed within a reasonable timeframe, providing users with timely query responses.

[0073] S270: Perform hash calculation on the query condition to generate a hash value of a fixed length as an update query key value.

[0074] Specifically, after obtaining query results through distributed computing, the cache needs to be updated. The platform performs a hash calculation on the current query conditions. This means the platform combines the extracted key query parameters into a string according to specific rules. The resulting string is then hashed using a hash algorithm to generate a fixed-length binary hash value, which serves as the updated query key. The query key identifies the current query conditions, ensuring that identical query conditions generate the same key value, while different conditions generate different key values.

[0075] S280: Combine the update query key value and the query result to generate an update query record, and insert the update query record into the cache.

[0076] Specifically, the platform will combine the update query key value with the query result to generate a complete update query record. The query result may be structured data or semi-structured data, and the update query key value serves as the index identifier of the record. After the record is generated, the platform will insert it into the cache. At the same time, the insertion of new records follows the cache space management strategy. For example, when the cache space is insufficient, the old records will be eliminated in combination with the expiration time or non-access time strategy to make room for new records, thereby ensuring that the cache always stores efficient and effective query results, and continuously improving the platform's query response speed and resource utilization.

[0077] Optionally, the method further includes: setting an expiration time for each historical query record, and obtaining the storage time corresponding to each historical query record according to a specified period; when the storage time reaches the expiration time, deleting the corresponding historical query record.

[0078] Optionally, the method also includes: obtaining the non-access time corresponding to each historical query record, and sorting each historical query record in descending order according to the non-access time to generate an non-access queue; obtaining the remaining space of the cache library, and when the remaining space is less than the preset capacity, selecting the historical query records in the non-access queue in turn for deletion.

[0079] The technical solution of the embodiment of the present invention, by utilizing the parallel processing capabilities of the distributed computing engine, splits complex queries into multiple subtasks, avoids single-node computing bottlenecks, and improves the efficiency of large-scale data processing. By reading sharded data in parallel through cluster nodes, the concurrency advantages of the distributed storage architecture are fully utilized to shorten data reading time and accelerate the execution of query tasks. The sharded data processed by each node is integrated into a complete result to ensure the accuracy and completeness of the query results and meet the user's needs for massive data queries. By storing new query records in the cache library and expanding the amount of cached data, subsequent identical queries can directly hit the cache, reducing repeated calculations and continuously improving query response speed and system resource utilization.

[0080] Example 3

[0081] Figure 3 This is a structural diagram of a data query cache device provided by the third embodiment of the present invention. Figure 3 As shown, the apparatus includes: a query condition acquisition module 310, configured to: acquire a query request, and parse the query request to acquire a query condition;

[0082] The query record matching module 320 is used to determine whether there is a matching query record matching the query condition in the cache library, and if so, directly return the matching query record as the query result, wherein the cache library includes various historical query records;

[0083] The cache library updating module 330 is configured to: otherwise, execute the query task based on the query condition to obtain the query result, and update the cache library according to the query condition and the query result.

[0084] Optionally, the query condition acquisition module 310 is specifically used to: receive a query request initiated by a user through a client, wherein the query request includes a structured query statement or a semi-structured query statement; perform grammatical and semantic analysis on the query request through a parsing engine to extract various query parameters; perform parameter standardization and denoising on each query parameter, retain key query parameters, and use each key query parameter as a query condition.

[0085] Optionally, the query record matching module 320 is specifically used to: perform hash calculation on each key query parameter to generate a hash value of a fixed length as a cache index; determine whether there is a target query key value consistent with the cache index in each historical query key value, and if so, use the historical query record where the target query key value is located as a matching query record; otherwise, determine that there is no matching query record.

[0086] Optionally, the cache library update module 330 specifically includes: a query result acquisition unit, which is used to: call the distributed computing engine in the computing cluster to decompose the query conditions into distributed query tasks; distribute each distributed query task to each node in the computing cluster, and read the shard data from the distributed file system in parallel through each node based on the corresponding distributed query task; integrate the shard data of each node to generate query results.

[0087] Optionally, the cache library update module 330 specifically includes: a cache library update unit, which is used to: perform hash calculation on the query conditions to generate a hash value of a fixed length as an update query key value; combine the update query key value and the query result to generate an update query record, and insert the update query record into the cache library.

[0088] Optionally, the device also includes: an invalid record deletion module, which is used to: set an expiration time for each historical query record, and obtain the storage time corresponding to each historical query record according to a specified period; when the storage time reaches the expiration time, the corresponding historical query record is deleted.

[0089] Optionally, the device also includes: an unaccessed queue determination module, which is used to: obtain the unaccessed time corresponding to each historical query record, and sort each historical query record in descending order according to the unaccessed time to generate an unaccessed queue; obtain the remaining space of the cache library, and when the remaining space is less than the preset capacity, select the historical query records in the unaccessed queue in turn for deletion.

[0090] The technical solution of the embodiment of the present invention can more accurately understand the user's query intention by parsing the query conditions, extract key parameters for subsequent processing, and lay the foundation for cache matching and query execution. When a cache library is hit, the result can be quickly returned without repeated calculations, which significantly reduces user waiting time, improves query response efficiency, and reduces computing resource consumption. In the event of a miss, results are generated through distributed computing to ensure the accuracy of the query, and the new results are stored in the cache to provide cache support for subsequent identical queries, gradually accumulate effective cache data, and continuously optimize query efficiency.

[0091] A data query caching device provided by an embodiment of the present invention can execute a data query caching method provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0092] Example 4

[0093] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0094] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0095] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0096] The processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a data query caching method.

[0097] In some embodiments, a data query caching method can be implemented as a computer program that is tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of a data query caching method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform a data query caching method in any other appropriate manner (e.g., by means of firmware).

[0098] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0099] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0100] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0101] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0102] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0103] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem, addressing the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0104] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0105] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A data query caching method, characterized in that: include: Obtaining a query request, and parsing the query request to obtain query conditions; Determine whether there is a matching query record matching the query condition in the cache library, and if so, directly return the matching query record as the query result, wherein the cache library includes various historical query records; Otherwise, the query task is executed based on the query condition to obtain a query result, and the cache library is updated according to the query condition and the query result.

2. The method according to claim 1, characterized in that The obtaining of the query request and parsing the query request to obtain the query condition include: Receiving a query request initiated by a user through a client, wherein the query request includes a structured query statement or a semi-structured query statement; Performing grammatical and semantic analysis on the query request through a parsing engine to extract query parameters; Parameter standardization and noise removal are performed on each of the query parameters, key query parameters are retained, and each of the key query parameters is used as the query condition.

3. The method according to claim 2, characterized in that The historical query record includes a historical query key value and a corresponding historical query result. The determining whether there is a matching query record matching the query condition in the cache library includes: Performing hash calculation on each of the key query parameters to generate a hash value of fixed length as a cache index; Determine whether there is a target query key value consistent with the cache index in each historical query key value, and if so, use the historical query record where the target query key value is located as a matching query record; Otherwise, it is determined that no matching query record exists.

4. The method according to claim 1, wherein The executing the query task based on the query condition to obtain the query result includes: Call the distributed computing engine in the computing cluster to break down the query conditions into distributed query tasks; Distribute each of the distributed query tasks to each node in the computing cluster, and read the shard data from the distributed file system in parallel based on the corresponding distributed query task through each node; Integrate the shard data of each node to generate query results.

5. The method according to claim 4, characterized in that The updating of the cache library according to the query condition and the query result includes: Performing hash calculation on the query condition to generate a hash value of fixed length as an update query key value; The update query key value and the query result are combined to generate an update query record, and the update query record is inserted into the cache library.

6. The method according to claim 1, characterized in that The method further comprises: Set an expiration time for each historical query record, and obtain the storage time corresponding to each historical query record according to the specified period; When the storage time reaches the expiration time, the corresponding historical query record will be deleted.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining the non-access duration corresponding to each historical query record, and sorting the historical query records in descending order of the non-access duration to generate a non-access queue; The remaining space of the cache library is obtained. When the remaining space is less than a preset capacity, the historical query records in the unaccessed queue are sequentially selected for deletion.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

9. A computer storage medium, characterized in that The computer storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 7 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images