A database cache query method, device, equipment and storage medium

Abstract

The application discloses a database cache query method and device, equipment and storage medium, belongs to computer technical field, the technical scheme is: the received query request is parsed, according to the analysis result, the ID identification of each data set to be queried is determined, and the corresponding query condition of each data set is determined according to the query request;The ID identification of data set is matched with the ID identification of data set saved in the cache queue one by one;If the matching degree of the ID identification of data set meets the preset condition, the record mapping information corresponding to the data set ID identification saved in the cache queue is read from the database, the data record mapped by the record mapping information is read, and the data record meeting the corresponding query condition of the current data set is matched from the data record, as the query result and return;Solve the technical problem of improving the hit rate of cache and reducing the query pressure of underlying database under the condition of limited cache space.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a database caching query method, apparatus, device, and storage medium. Background Technology

[0002] With the application of IoT technology and the increase in interactive IoT devices, the demand for real-time data is becoming increasingly higher. Traditional databases can no longer meet the requirements for real-time data processing. Therefore, for IoT applications, sensing databases are often used for management. Data and transactions in sensing database systems have time-related characteristics, meaning they can be managed according to traditional structured data relationships, or they can store data with real-time characteristics online.

[0003] The significant characteristics of sensing data are large data volume, short effective period, and the operation of this data in business is often read more and write less, that is, the data is written once and read frequently. For example, various sensing data generated by various IoT devices in security scenarios, or various sensing data generated by IoT vehicles, etc.

[0004] For perception data, databases typically store it in partitions, which can be roughly divided into two parts: one part is historical data that is not updated, and the other part is data that is in the updated partition range, but updates are also infrequent.

[0005] Currently, a common problem encountered when querying most perception data is the high concurrency, complex, and repetitive queries for frequently read but unmodified data. Therefore, how to design a cache for this type of database, improve the cache hit rate, and reduce the query pressure on the underlying database under limited cache space is a technical problem that needs to be solved. Summary of the Invention

[0006] This invention provides a database cache query method that can improve the cache hit rate and reduce the query pressure on the underlying database when the cache space is limited.

[0007] In a first aspect, the present invention provides a database caching query method, comprising:

[0008] The received query request is parsed, and based on the parsing result, the ID identifier of each dataset to be queried is determined, and the corresponding query conditions for each dataset are determined based on the query request.

[0009] The ID identifier of the dataset to be queried is matched one by one with the ID identifier of the dataset stored in the cache queue;

[0010] If the matching degree of the ID identifier of the dataset to be queried meets the preset conditions, then according to the record mapping information corresponding to the ID identifier of the dataset to be queried stored in the cache queue, the data record mapped by the record mapping information is read from the database, and the data record that meets the corresponding query conditions of the current dataset is matched from the data record and returned as the query result;

[0011] The dataset contains data records in the database that meet specified query conditions; the ID of the dataset is generated according to a set rule, which and the specified query conditions are determined based on different granularities of each field in the database; the cache queue stores mapping information between the ID of the dataset and the corresponding records in the database that meet the specified query conditions of the dataset, and the mapping information has a mapping relationship with the records in the database that meet the specified query conditions of the dataset.

[0012] In embodiments of this invention, based on the query request and set rules, the query results required by the query request are parsed into various datasets, and the query results are returned according to the query conditions of the datasets. Compared with the method of obtaining query results solely based on the query request, this technical solution can simultaneously query multiple datasets in parallel, resulting in fast response speed and good concurrency performance. By parsing the query request into dataset form, it can be applied to queries of various complexities. By decomposing the query request, the query conditions of all datasets are determined, making the query simple and fast. The division of datasets can adopt different strategies according to different fields in the database, which is flexible and adaptable to various databases and various query scenarios. Furthermore, the set rules of embodiments of this invention are determined based on different division granularities of different fields in the database. The dataset IDs determined according to the set rules have a high degree of matching with the database. The dataset IDs generated according to the query request and set rules ensure a high hit rate of cached dataset IDs during actual querying. Query results can be directly obtained from the database based on the mapping relationship between the cached dataset IDs and the corresponding query results in the database, eliminating the need for repeated database queries and greatly reducing the query pressure on the underlying database.

[0013] Furthermore, in the technical solution of this invention, the cache queue can be stored inside the database, meaning it does not require additional technical components like a separate Redis database; only separate space is needed, resulting in low maintenance costs. The cache in this embodiment is pluggable, so users are unaware of its existence. The database usage remains unchanged regardless of the presence or absence of a cache, requiring no modifications when using the caching function. The cache size can be freely set as needed, and the categories stored in the cache do not need to be specified. Corresponding datasets can be generated and cached for all fields in the database, resulting in strong dataset versatility. Even if each received query request is different, parsing confirms that the queried dataset is the same, resulting in a high cache hit rate. Moreover, even if the cached dataset does not perfectly match the dataset to be queried, a cache hit is still possible. Subset queries of the cached dataset are also supported, further improving the query hit rate and significantly reducing the query pressure on the underlying database.

[0014] Optionally, in some embodiments, the step of "if the matching degree meets the preset conditions" further includes:

[0015] If the ID identifier of the dataset to be queried does not completely match the ID identifier of the dataset stored in the cache queue, then based on the mismatched identifier in the ID identifier of the dataset to be queried and the corresponding query conditions of the dataset to be queried, a new query statement is reassembled, and the data record corresponding to the mismatched identifier in the ID identifier of the dataset to be queried is read from the database according to the new query statement.

[0016] Based on the record mapping information corresponding to the ID identifier of the dataset to be queried, stored in the cache queue, the data records mapped by the record mapping information are read from the database. Data records that meet the corresponding query conditions of the current dataset are matched from the data records and merged with the data records corresponding to the identifiers that do not match the ID identifier of the dataset to be queried, obtained from the database. The merged data records are then returned as the query result.

[0017] In embodiments of the present invention, when the required data cannot be obtained through the mapping relationship between the dataset stored in the cache queue and the query results in the database, instead of directly sending the query request to the database for querying, a new query statement is reassembled based on the mismatched identifiers in the dataset's ID identifiers. The query is then performed on the database for the mismatched identifiers to obtain the query results. This technical solution can reduce the query pressure on the underlying database. It is not necessary to query the database based on all the query conditions of the query request to obtain the query results; it is only necessary to use the mismatched identifiers in the dataset's ID identifiers, thus reducing the query pressure on the underlying database.

[0018] Optionally, in some embodiments, if the matching degree of the ID identifier of the dataset to be queried does not meet the preset conditions, then according to the ID identifier of the dataset to be queried and the corresponding query conditions of the current dataset, data records that meet the query conditions of the current dataset are read from the database and returned as query results.

[0019] Optionally, in some embodiments, the matching degree meets preset conditions, specifically including:

[0020] The matching degree between the ID identifier of the dataset to be queried and the ID identifier of the dataset stored in the cache queue is greater than or equal to 2 / 3.

[0021] Optionally, in some embodiments, parsing the received query request specifically includes:

[0022] Based on the received query request, determine each query condition contained in the query request;

[0023] Based on the query conditions and set rules, determine the ID identifier of each dataset to be queried;

[0024] Based on the ID identifier of each dataset to be queried and the corresponding query conditions, the query conditions for each dataset to be queried are determined.

[0025] Wherein, the query conditions of the dataset to be queried are the same as the query conditions determined by the identifier of the corresponding ID of the dataset to be queried, or the query conditions of the dataset to be queried are a subset of the query conditions determined by the identifier of the corresponding ID of the dataset to be queried.

[0026] The technical solution of this invention determines the query conditions based on the query request, and then determines the ID of the dataset. The ID is determined according to a set rule, which is determined separately based on the different granularities of each field in the database. Therefore, the method of setting the ID of the dataset is flexible and can be applied to databases with various fields. Moreover, the ID of the dataset is related to the fields of the database, and the database query hits the fields related to the fields, making full use of the characteristics of the database. This method results in a high cache hit rate and a high matching degree.

[0027] Optionally, in some embodiments, the mapping information is stored using a map structure.

[0028] By utilizing the logical AND, OR, and XOR operations of the BitSet structure in the map structure, the time complexity can be reduced by tens of times. While meeting certain computational needs, this can significantly reduce time complexity and save storage space, allowing limited cache space to store more cache queue data and greatly improving query speed to obtain the required data.

[0029] Optionally, in some embodiments, after retrieving and returning data records that satisfy the query conditions of the current dataset from the database as query results, the method further includes:

[0030] Based on the ID of the dataset to be queried and the data records obtained from the database that meet the query conditions of the current dataset, a mapping information is generated between the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset.

[0031] Update the cache queue.

[0032] Optionally, in some embodiments, a historical query queue is also included;

[0033] If the matching degree does not meet the preset conditions, it also includes:

[0034] If the ID of the historical dataset stored in the historical query queue does not match the ID of the dataset to be queried, then the ID of the dataset to be queried is added to the historical query queue as the ID of the historical dataset, and the historical query queue is updated.

[0035] If the ID of the historical dataset matches the ID of the dataset to be queried, update the historical query queue;

[0036] After determining that the query count of the ID of the dataset to be queried meets the set condition, and after retrieving data records from the database that meet the query conditions of the current dataset as query results and returning them, the method further includes: generating mapping information between the ID of the dataset to be queried and the data records from the database that meet the query conditions of the current dataset, updating the cache queue, and deleting the historical dataset ID and the query count corresponding to the historical dataset ID from the historical query queue;

[0037] The historical query queue stores the ID identifier of a historical dataset and the number of queries for that historical dataset's ID identifier. The ID identifier of the historical dataset is the ID identifier of a dataset to be queried but not yet saved in the cache queue. The updating of the historical query queue includes updating the number of queries for the ID identifier of the historical dataset.

[0038] In embodiments of the present invention, when the dataset to be queried does not exist in the cache queue, after obtaining the query results from the database, the corresponding map structure is not immediately generated and stored in the cache queue. Instead, the queried records are first saved in the historical query records. Only when the historical query records exceed a preset value is the corresponding map structure generated and stored in the cache queue based on the database query results. This technical solution can effectively prevent the problem of low cache utilization caused by cache pollution. The LRU algorithm alone cannot avoid the storage space occupation caused by cache pollution and the additional operation time overhead caused by the eviction policy. The combination of this technical solution and the LRU algorithm can effectively avoid the problem of cache pollution and save the cache queue from the relative reduction of resource consumption caused by cache pollution.

[0039] Optionally, in some embodiments, the method further includes: if the time taken to obtain data records that meet the query conditions of the current dataset from the database exceeds a preset time, then the number of queries corresponding to the ID identifier of the historical dataset in the historical query queue is adjusted to N times the normal number of queries, where N is a positive integer greater than or equal to 2.

[0040] Secondly, embodiments of the present invention also provide a database cache query device, comprising:

[0041] The parsing module is used to parse the received query request, determine the ID identifier of each dataset to be queried based on the parsing result, and the query conditions corresponding to each dataset determined based on the query request.

[0042] The matching module is used to match the ID identifier of the dataset to be queried with the ID identifier of the dataset stored in the cache queue one by one; it is also used to, if the matching degree of the ID identifier of the dataset to be queried meets the preset conditions, read the data record mapped by the record mapping information corresponding to the ID identifier of the dataset to be queried from the database according to the record mapping information stored in the cache queue, and match the data record that meets the corresponding query conditions of the current dataset from the data record, and return it as the query result;

[0043] The dataset contains data records in the database that meet specified query conditions; the ID of the dataset is generated according to a set rule, which and the specified query conditions are determined based on different granularities of each field in the database; the cache queue stores mapping information between the ID of the dataset and the corresponding records in the database that meet the specified query conditions of the dataset, and the mapping information has a mapping relationship with the records in the database that meet the specified query conditions of the dataset.

[0044] Thirdly, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein the processor runs a program stored in the memory, and when the program is executed, the processor performs any of the methods described in the first aspect above.

[0045] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, the computer program including methods for performing any of the methods described in the first aspect.

[0046] The beneficial effects in the second, third, and fourth aspects can be found in the corresponding beneficial effects in the first aspect, and will not be repeated here. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A flowchart of a database cache query method provided in an embodiment of the present invention;

[0049] Figure 2 A flowchart of another database cache query method provided in an embodiment of the present invention;

[0050] Figure 3 A flowchart of another database cache query method provided in an embodiment of the present invention;

[0051] Figure 4 A flowchart for determining dataset query conditions is provided in an embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram of a map storage structure provided in an embodiment of the present invention;

[0053] Figure 6 A flowchart of another database cache query method provided in an embodiment of the present invention;

[0054] Figure 7 A flowchart of another database cache query method provided in an embodiment of the present invention;

[0055] Figure 8 A schematic block diagram of a database cache query device provided in an embodiment of the present invention;

[0056] Figure 9 This is a schematic block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] It should be understood that in the description of the embodiments of the present invention, the terms "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance, nor should they be construed as indicating or implying order.

[0059] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0060] Before introducing the embodiments of this application, some terms involved in the embodiments of this application will be explained. It should be noted that the following explanations are for the purpose of making the embodiments of this application easier to understand, and should not be regarded as a limitation on the scope of protection claimed by the embodiments of this application.

[0061] The Internet of Things (IoT) refers to the use of various information sensors, RFID technology, GPS, infrared sensors, laser scanners, and other IoT devices and technologies to collect real-time data on any object or process that needs to be monitored, connected, or interacted with. This data includes information on sound, light, heat, electricity, mechanics, chemistry, biology, location, and other relevant parameters. Through various possible network access methods, it achieves ubiquitous connectivity between things and between things and people, enabling intelligent perception, identification, and management of objects and processes. The IoT is an information carrier based on the internet and traditional telecommunications networks, enabling all independently addressable ordinary physical objects to form an interconnected network.

[0062] Sensing data refers to various types of data generated by IoT devices, including MAC, RFID, electronic fences, and data generated by various sensor devices. The data is described by some characteristic information.

[0063] File: can also be understood as a file block. Each File data file stores N data records of the corresponding data table. Each data table corresponds to M File data files, where M and N are both positive integers greater than or equal to 1.

[0064] In sensor databases, the data is typically structured, meaning the fields in the tables usually don't change, and the generated data records are read-only. However, due to the large amount of data triggered and recorded by various IoT devices, without data table partitioning, the data volume of any single table will grow excessively, eventually overwhelming the table. Therefore, reasonable data table partitioning is necessary. In practice, the common approach is to record the generated sensor data by creating file blocks. That is, each data table corresponds to M file blocks, and each file block stores N records from the corresponding data table, without needing to partition the data table itself.

[0065] Optionally, the File data file is read-only. A File data file is generated every second. When querying the data table, each File data file corresponding to the current data table will be searched in turn. Of course, if necessary, Files can be merged during the query to reduce the number of file blocks in the corresponding data table.

[0066] BitSet: A "binary number" of custom length. It uses a very compact format to indicate whether a given range of consecutive data has appeared or is correct. Each bit in a BitSet is a boolean value, represented by "0" or "1". Each bit only occupies 1 bit of storage space. Usually, "1" represents "yes" or TRUE, and "0" represents "no" or FALSE.

[0067] BitSet can be operated using logical AND, logical OR, and logical XOR, which can reduce time complexity by tens of times. Therefore, the advantage of BitSet is that it can greatly reduce time complexity and save storage space while meeting certain computational needs.

[0068] In practical applications, various IoT sensing devices generate a large amount of data, such as QR code tags and readers, RFID tags and readers, cameras, GPS, sensors, M2M terminals, sensor gateways, etc., enabling any object to connect to the Internet, i.e., the Internet of Things. The purpose of generating and recording data is to exchange information and communicate, so as to achieve intelligent identification, positioning, tracking, monitoring and management of objects. Therefore, for sensing databases, the following scenarios exist:

[0069] 1. High concurrency

[0070] The system is designed to receive and process many requests simultaneously and in parallel, and to respond to a large number of queries per second.

[0071] 2. Complex Queries

[0072] In various business scenarios, different access terminals propose various queries to meet various interaction and access needs. In some scenarios, it is also necessary to analyze one or more types of data and perform statistical analysis of various granular measurement indicators, such as the various indicator statistics required in the security business scenario. These queries need to flexibly specify the query range, and the query time can often be tens of seconds.

[0073] 3. Duplicate queries

[0074] Without a deduplication prevention mechanism, due to unreasonable program design or bugs, a large number of duplicate queries may be submitted.

[0075] Therefore, the IoT sensing database has the need to respond to high concurrency, complex queries, and repetitive queries. That is, within the same unit of time, it will face problems such as many query requests, many complex queries, and many potentially repetitive queries. Therefore, how to design the cache of the sensing database to make the cache hit rate high and reduce the query pressure on the underlying database is a technical problem that needs to be solved.

[0076] This invention provides a database cache query method, including:

[0077] Parse the received query request, and based on the parsing results, determine the ID identifiers of all datasets to be queried and the query conditions for the current dataset determined based on the query request;

[0078] Match the ID of the dataset to be queried one by one with the IDs of the datasets stored in the cache queue;

[0079] If the matching degree of the ID identifier of the dataset to be queried meets the preset conditions, then according to the record mapping information corresponding to the ID identifier of the dataset to be queried stored in the cache queue, the data record mapped by the record mapping information is read from the database, and the data record that meets the corresponding query conditions of the current dataset is matched from the data record and returned as the query result;

[0080] The dataset contains data records in the database that meet the specified query conditions; the dataset ID is generated according to the set rules, which are determined based on the different granularities of each field in the database; the cache queue stores the mapping information between the dataset ID and the corresponding records in the database that meet the specified query conditions of the dataset, and the record mapping information has a mapping relationship with the records in the database that meet the specified query conditions of the dataset.

[0081] The following description uses specific examples, see appendix. Figure 1 As shown.

[0082] S101: Parse the received query request, determine the ID of each dataset to be queried based on the parsing result, and determine the corresponding query conditions for each dataset based on the query request;

[0083] In practical applications, various IoT sensing devices generate a large amount of data. In order to exchange information and communicate, a large number of query requests will also be generated for various different needs. When a query request is received, it needs to be parsed. The parsing content can be defined by the user, such as determining the data table to be retrieved, the field information to be retrieved, and the query conditions contained in the query request. The parsing process is not limited here.

[0084] After parsing the query conditions, the ID identifier of each dataset to be queried is determined based on the parsed results, and the query conditions of each dataset are determined according to the query request.

[0085] A dataset contains data records in the database that meet the specified query conditions. It can also be understood as the smallest unit of division determined according to the set rules. The ID of the dataset is determined according to the set rules, that is, it is generated after the smallest unit of division is determined and the dataset is determined. The set rules and the specified query conditions are determined based on the different division granularity of each field in the database. The granularity of each field in the database varies. For example, in a perceptual database where time is critical, the granularity for query conditions related to time in a query request is determined to be in days. This can also be understood as the smallest unit of data being a dataset in days. Suppose a query request needs to retrieve data from June 14th to June 15th, 2022. According to the set rules, the granularity of the current field is in days, splitting it into two datasets with query times of June 14th and June 15th, 2022. If the granularity defined by the set rules is in hours, then the data is divided in hours. Using the above example, the two days will be divided into datasets with hourly granularity, resulting in 48 datasets.

[0086] The dataset ID is defined based on the specific dataset. Once the dataset is determined, its ID is assigned. For example, the dataset ID can be the name of the table to be queried plus the query conditions for the current dataset. Taking the granularity of days in the above embodiment as an example, if the data table to be queried is determined to be 'picRecord' based on the query request, and the field name for the date to be queried is determined to be capTime after parsing, then the dataset ID can be determined as "'picRecord' + 'capTime = 2022-4-25'". In this case, the dataset ID can be used as the query condition for the current dataset. Of course, the dataset ID can also be "table name" + "query conditions determined at the current query granularity". In this case, the "query conditions determined at the current query granularity" can be the same as or different from the query conditions for the current dataset. For example, taking a granularity of days as an example, in actual queries, there might be situations where only data for a specific time period within a particular day is needed. For instance, if the query needs to retrieve data from 6 AM to 8 AM on April 25, 2022, but according to the set rules, the granularity of the current field is days, meaning the granularity is measured in days, then the smallest unit of division is days. Therefore, the dataset ID is "table name" + "query condition determined by the current query granularity". Let's say the table to be queried in the query request is 'picRecord', and after parsing, the field name for the date to be queried is determined to be capTime. Then, the determined dataset ID can be "'picRecord' + 'capTime = 2022-4-25'". In this case, the query condition for the current dataset is different from the "query condition determined by the current query granularity". The query condition for the current dataset is a subset of the "query condition determined by the current query granularity", meaning it needs to query the date field in the data table 'picRecord' where 'capTime = The dataset is dated 2022-04-25, and the time period is from 6 AM to 8 AM on the current date. Therefore, based on the parsing results, it is necessary to determine the ID of all datasets to be queried and the query conditions for the current dataset determined according to the query request.

[0087] For example, when dealing with information condition fields in a database, if a field contains specified information, then the granularity of the partitioning is the current query condition and no further splitting is required. If a field contains two specified pieces of information, and the two pieces of information are in an OR relationship, then the two pieces of information in the OR relationship are split into two datasets, and the ID identifier of the dataset is determined based on the query conditions of the two split datasets.

[0088] The rules are set based on the different granularities of each field in the database and are used to generate the ID identifier of the dataset. Taking the granularity of days in the above embodiment as an example, the set rule can be "the name of the data table to be queried" and "the query condition determined by the current query granularity". The "data table to be queried" and "the query condition determined by the current query granularity" are connected by the symbol "+". Therefore, according to the set rule, the ID identifier of the dataset is "the name of the data table to be queried" + "the query condition determined by the current query granularity", that is, the ID of the dataset can be "'picRecord'+'capTime = 2022-4-25'". It can be determined that 'picRecord' is the name of the data table to be queried, and 'capTime = 2022-4-25' is the query condition, with the granularity of days.

[0089] Of course, the "data table to be queried" and the "query conditions determined by the current query granularity" can also be connected by the symbol "-", depending on the definition of the set rule. The set rule can also be in other ways; this is just an example.

[0090] The specified query conditions are also determined based on the different granularities of each field in the database. In the current embodiment, this can be understood as the specified query conditions being determined based on the current dataset. Taking the above embodiment as an example, when the dataset ID is "'picRecord'+'capTime = 2022-4-25'", the specified query condition is to determine the records in the 'picRecord' table whose field name is 'capTime' that meet the condition 2022-4-25. The specified query conditions are used to determine the query conditions for the current dataset based on the dataset. Therefore, the dataset ID can be defined according to the specific application scenario; the above is just an example.

[0091] Based on the set rules and query requests, the IDs of the datasets to be queried in the query request and the query conditions for each dataset can be determined.

[0092] S102: Match the ID of the dataset to be queried with the ID of the dataset stored in the cache queue one by one;

[0093] The cache queue stores the IDs of the datasets. The IDs of each determined dataset are matched one by one with the IDs of the datasets stored in the cache column. The matching method can be to match each determined dataset ID with the IDs of the datasets stored in the cache column individually, or to match each determined dataset ID with the sum of the IDs of all datasets stored in the cache column. The method can be defined according to the actual situation and needs.

[0094] To illustrate the difference between matching the identified datasets one by one with the IDs of the datasets stored in the cache queue, and matching the identified datasets with the sum of all dataset IDs stored in the cache queue, the following examples will be used to illustrate this.

[0095] Let the dataset to be queried be recorded('a','b','c').

[0096] The query conditions for this dataset are informational and contain specified information. Therefore, the granularity of the partition is determined to be the current query conditions, and no further splitting is performed.

[0097] Suppose that, based on the query request, the data table to be queried is picRecord, and the ID of the dataset is determined to be "'picRecord'+'record in('a','b','c')'". The query condition for the current dataset is that any field in the data table 'picRecord' satisfies 'record in('a','b','c')'.

[0098] For the sake of simplicity, in this embodiment, the dataset with the ID "'picRecord'+'record in('a','b','c')'" will be uniformly referred to as dataset 1.

[0099] Suppose that there are two datasets stored in the cache queue whose IDs are associated with dataset 1, namely dataset 2 and dataset 3. The ID of dataset 2 is “'picRecord'+'record in('b','c','d')'”, and the ID of dataset 3 is “'picRecord'+'record in('a','e','f')'”.

[0100] If the matching method is to match each dataset with the dataset ID stored in the cache queue one by one, that is, to match dataset 1 with dataset 2 and dataset 3 respectively.

[0101] The matching result between dataset 1 and dataset 2 is 2 / 3, meaning that in dataset 2, the dataset that can be recorded in ('b','c') can be determined, but the dataset that can be recorded in ('a') cannot be determined.

[0102] The matching result between dataset 1 and dataset 3 is 1 / 3, meaning that in dataset 2, the dataset that can be recorded in ('a') cannot be determined, but the dataset that can be recorded in ('b','c') cannot be determined.

[0103] Let the highest matching result be the final result. Dataset 1 is matched one by one with the dataset IDs stored in the cache queue, and the matching result is 2 / 3.

[0104] If the matching method is to match each determined dataset with the sum of all dataset IDs stored in the cache queue, that is, to merge the identifier of dataset 2 and the identifier of dataset 3 and then match it with the identifier of dataset 1, for ease of display, the merging here is a temporary merging of the identifiers for calculating the matching degree, and does not actually merge dataset 2 and dataset 3.

[0105] For the sake of simplicity and to illustrate the content to be expressed by this invention, the identifiers of the datasets are described here by the identifiers to be queried. The identifiers of dataset 1 are ('a','b','c'), the identifiers of dataset 2 are ('b','c','d'), and the identifiers of dataset 3 are ('a','e','f'). The sum of the identifiers of dataset 2 and dataset 3 is a temporary dataset ('b','c','d', 'a','e','f'). Therefore, the matching result between dataset 1 and the temporary dataset ('b','c','d', 'a','e','f') is 3 / 3.

[0106] The matching result indicates that the dataset with record in ('b','c') can be obtained from dataset 2, and the dataset with record in ('a') can be obtained from dataset 3. Only secondary filtering is needed to determine the required data. Therefore, the matching result of dataset 1 with the sum of all dataset IDs in the cache queue (datasets 2 and 3) is 3 / 3.

[0107] According to the above embodiments, it can be determined that the individual matching result of each dataset with the dataset ID identifier stored in the cache queue is different from the matching result of each dataset with the sum of all dataset ID identifiers stored in the cache queue. In practice, the choice can be made according to actual needs. Generally, the data can be filtered and filtered in a secondary manner. It is preferred to use the matching result of the current dataset with the sum of all dataset ID identifiers stored in the cache queue.

[0108] There is a mapping between the ID identifier of a dataset and the corresponding record in the database that matches the specified query conditions of the dataset. This mapping relationship is based on the query results obtained from the database that match the query conditions of the dataset's ID identifier and can be used to represent the dataset. When the cache queue stores the ID identifier of the dataset, it means that the database has been searched for the query conditions of the dataset's ID identifier and the corresponding query results have been obtained. Therefore, it can be determined which record in the corresponding data table in the database matches the query conditions of the dataset's ID identifier. The cache queue stores the dataset's ID identifier and the corresponding query results in the database.

[0109] Once the ID of the dataset is matched with the ID of the dataset stored in the cache queue, it can be determined whether the ID of the dataset to be queried can be obtained directly from the cache queue based on the mapping relationship without having to query the database.

[0110] S103: If the matching degree of the ID identifier of the dataset to be queried meets the preset conditions, then according to the record mapping information corresponding to the dataset ID identifier stored in the cache queue, the data record mapped by the record mapping information is read from the database, and the data record that meets the corresponding query conditions of the current dataset is matched from the data record and returned as the query result;

[0111] The dataset contains data records in the database that meet the specified query conditions; the dataset ID is generated according to the set rules, and the set rules and the specified query conditions are determined based on the different granularity of each field in the database; the cache queue stores the mapping information between the dataset ID and the corresponding records in the database that meet the specified query conditions of the dataset, and the mapping information has a mapping relationship granularity with the records in the database that meet the specified query conditions of the dataset.

[0112] The preset conditions for matching degree can be defined by the user, such as matching degree greater than a set threshold, or being a subset.

[0113] Taking the above embodiment as an example, the preset condition is that the matching degree is greater than 2 / 3. The matching degree is determined as the sum of the ID of the current dataset and the ID identifiers of all datasets stored in the cache queue.

[0114] Based on the above conditions, the matching result between dataset 1 and the ID identifier of the dataset stored in the cache queue is 3 / 3, which satisfies the preset conditions.

[0115] Based on the mapping information between the ID identifiers of datasets stored in the cache queue and the corresponding records in the database that meet the specified query conditions of the datasets, data is read from the database. This means reading records that meet the query conditions of dataset 2 from the corresponding database based on the ID identifiers of dataset 2 stored in the cache. This can also be understood as reading records that satisfy record in('b','c','d')' from the database based on the mapping relationship. Similarly, data that meets the query conditions of dataset 3 is read from the corresponding database based on the ID identifiers of dataset 3 stored in the cache. This can also be understood as reading data that satisfies record in('a','e','f')' from the database based on the mapping relationship.

[0116] The query process involves matching data records that match the corresponding query conditions of the current dataset and returning them as query results. This can be understood as follows: if the query condition for dataset 1 is 'picRecord' + 'record in('a','b','c'), then based on the defined query condition for dataset 1, the data from datasets 2 and 3 is determined. Specifically, the query results of 'record in('d')' obtained from dataset 2 are filtered out, and the query results of 'record in('e','f')' obtained from dataset 3 are filtered out, resulting in the query result 'record in('a','b','c')'. The data matching the dataset 'record in('a','b','c')' is then identified and returned as the query result.

[0117] If a certain ID of a dataset stored in the cache queue matches the ID of dataset 1 exactly, for example, if the cache queue stores a dataset with the ID 'record in('a','b','c')', as can be seen from the above embodiment, matching the data records that meet the corresponding query conditions of the current dataset from the data records and returning them as query results means that all records obtained from the database that meet the specified query conditions of the dataset meet the query conditions corresponding to the current dataset, without needing to filter any record data, and returning all record data as query results. In embodiments of this invention, based on the query request and set rules, the query results required by the query request are parsed into various datasets, and the query results are returned according to the query conditions of the datasets. Compared with the method of obtaining query results solely based on the query request, this technical solution can simultaneously query multiple datasets in parallel, resulting in fast response speed and good concurrency performance. By parsing the query request into dataset form, it can be applied to queries of various complexities. By decomposing the query request, the query conditions of all datasets are determined, making the query simple and fast. The division of datasets can adopt different strategies according to different fields in the database, which is flexible and adaptable to various databases and various query scenarios. Furthermore, the set rules of embodiments of this invention are determined based on different division granularities of different fields in the database. The dataset IDs determined according to the set rules have a high degree of matching with the database. The dataset IDs generated according to the query request and set rules ensure a high hit rate of cached dataset IDs during actual querying. Query results can be directly obtained from the database based on the mapping relationship between the cached dataset IDs and the corresponding query results in the database, eliminating the need for repeated database queries and greatly reducing the query pressure on the underlying database.

[0118] Furthermore, in the technical solution of this invention, the cache queue can be stored inside the database, meaning it does not require additional technical components like a separate Redis database; only separate space is needed, resulting in low maintenance costs. The cache in this embodiment is pluggable, so users are unaware of its existence. The database usage remains unchanged regardless of the presence or absence of a cache, requiring no modifications when using the caching function. The cache size can be freely set as needed, and the categories stored in the cache do not need to be specified. Corresponding datasets can be generated and cached for all fields in the database, resulting in strong dataset versatility. Even if each received query request is different, parsing confirms that the queried dataset is the same, resulting in a high cache hit rate. Moreover, even if the cached dataset does not perfectly match the dataset to be queried, a cache hit is still possible. Subset queries of the cached dataset are also supported, further improving the query hit rate and significantly reducing the query pressure on the underlying database.

[0119] Optionally, as one embodiment, if the ID identifier of the dataset to be queried does not completely match the ID identifier of the dataset stored in the cache queue, then based on the mismatched identifier in the ID identifier of the dataset to be queried and the corresponding query conditions of the dataset to be queried, a new query statement is reassembled, and the data record corresponding to the mismatched identifier in the ID identifier of the dataset to be queried is read from the database according to the new query statement.

[0120] Based on the record mapping information corresponding to the ID identifier of the dataset to be queried, stored in the cache queue, the data records mapped by the record mapping information are read from the database. The data records that match the corresponding query conditions of the current dataset are matched from the data records and merged with the data records corresponding to the identifiers that do not match the ID identifier of the dataset to be queried, which are obtained from the database. The merged data records are then returned as the query result.

[0121] The following description uses specific examples, see appendix. Figure 2 As shown.

[0122] Appendix Figure 2 With appendix Figure 1 The same process applies here, so it will not be described again; only the process described above will be discussed. Figure 2 and Figure 1 The differences between the two will be described, among which... Figure 2 Steps S201-S202 and Figure 1 S101-S102 are the same.

[0123] S203: Determine that the matching degree of the ID identifier of the dataset to be queried meets the preset conditions.

[0124] This step is essentially the same as the method used in step S103 to determine the matching degree of the ID identifier of the dataset to be queried, and will not be described again here.

[0125] S204: Are there any incompletely matching identifiers? If there are incompletely matching identifiers, proceed to step S205 for the incompletely matching identifiers and to step S206 for the matching identifiers.

[0126] Based on the matching degree, the matching and non-matching identifiers in the ID identifiers of the dataset to be queried can be determined. If it is a non-matching identifier, step S205 is executed; if it is a matching identifier, step S206 is executed.

[0127] S205: Based on the mismatched identifiers in the ID identifiers of the dataset to be queried and the corresponding query conditions of the current dataset, reassemble a new query statement, and read the data corresponding to the mismatched identifiers in the ID identifiers of the dataset to be queried from the database according to the new query statement.

[0128] If the ID identifiers of the dataset to be queried contain mismatched identifiers, it means that the current cache queue does not store the mapping information between the identifier and the database. Therefore, it is necessary to query the database to obtain the record data with the mismatched identifiers. Thus, it is necessary to reassemble a new query statement based on the query conditions of the current dataset, and obtain the corresponding query results from the database based on the new query statement.

[0129] Taking the above embodiment as an example, let the preset condition be that the matching degree is greater than or equal to 2 / 3, the dataset to be queried is dataset 1, the query condition of dataset 1 is the same as in the above embodiment, and let the dataset 3 not exist in the current cache queue, and the dataset related to dataset 2 be dataset 2.

[0130] According to the aforementioned embodiment, the ID of the dataset 1 to be queried can be determined as "'picRecord'+'recordin('a','b','c')'".

[0131] The ID of dataset 2 stored in the cache queue is “'picRecord'+'record in('b','c','d')'”.

[0132] The matching degree between dataset 1 and dataset 2 is 2 / 3, which meets the preset conditions. However, the identifier of dataset 1 is a perfect match with record in('b','c') in dataset 2 in the cache queue. The identifier record in('a') in dataset 1 does not match the cache queue. Therefore, the identifier that does not match the ID of dataset 1 with the dataset stored in the cache queue is record in('a').

[0133] As for the query results in the database corresponding to the identifier 'record in('b','c')' in dataset 1, they have already been directly obtained according to the mapping relationship stored in the cache queue. There is no need to query the data of 'record in('b','c')' from the database again. It is only necessary to obtain the query results of 'record in('a')' from the database.

[0134] Therefore, based on the mismatched identifier 'record in('a')' in the ID identifiers of dataset 1, and the query condition "'picRecord'+'record in('a','b','c')'" determined by the current dataset, a new query statement is reassembled. The new query statement can be a select statement. The command `from picRecord record in ('a')` retrieves the query results from the `picRecord` table containing the record in ('a').

[0135] The reassembled new query statement will vary depending on the database. The above is an example of SQL language, but it is not the only form of this language. You only need to retrieve the data corresponding to the IDs that do not match the IDs in the dataset from the database based on the new query conditions.

[0136] S206: Based on the record mapping information corresponding to the dataset ID identifier to be queried stored in the cache queue, read the data record mapped by the record mapping information from the database, and match the data record that meets the corresponding query conditions of the current dataset from the data record.

[0137] This step is essentially the same as step S103, and will not be described again here.

[0138] S207: Match the data records that meet the corresponding query conditions of the current dataset from the data records, merge them with the data records obtained from the database that do not match the dataset ID, and return the merged data records as the query result.

[0139] The data obtained from step S205 will be merged with the data obtained from step S206 to obtain the data required for the current request, and the query result will be returned.

[0140] In embodiments of the present invention, when the required data cannot be obtained through the mapping relationship between the dataset stored in the cache queue and the query results in the database, instead of directly sending the query request to the database for querying, a new query statement is reassembled based on the mismatched identifiers in the dataset's ID identifiers. The query is then performed on the database for the mismatched identifiers to obtain the query results. This technical solution can reduce the query pressure on the underlying database. It is not necessary to query the database based on all the query conditions of the query request to obtain the query results; it is only necessary to use the mismatched identifiers in the dataset's ID identifiers, thus reducing the query pressure on the underlying database.

[0141] Optionally, as one embodiment, if the matching degree of the ID identifier of the dataset to be queried does not meet the preset conditions, then according to the ID identifier of the dataset to be queried and the corresponding query conditions of the current dataset, the data records that meet the query conditions of the current dataset are read from the database and returned as the query results.

[0142] The following description uses specific examples, see appendix. Figure 3 As shown.

[0143] Appendix Figure 3 With appendix Figure 1 The same process applies here, so it will not be described again; only the process described above will be discussed. Figure 3 and Figure 1The differences between the two will be described, among which... Figure 3 Steps S301-S302 and Figure 1 S101-S102 are the same.

[0144] S303: Determine whether the matching degree of the ID identifier of the dataset to be queried meets the preset conditions. If yes, proceed to step S304; otherwise, proceed to step S305.

[0145] S304: Based on the record mapping information corresponding to the ID identifier of the dataset to be queried stored in the cache queue, read the data record mapped by the record mapping information from the database, and match the data record that meets the corresponding query conditions of the current dataset from the data record, and return it as the query result.

[0146] The above steps are essentially the same as step S103, and will not be repeated here.

[0147] S305: Based on the ID of the dataset to be queried and the corresponding query conditions of the current dataset, read the data records that meet the query conditions of the current dataset from the database, and return them as the query results.

[0148] Taking the above embodiment as an example, suppose there is a dataset 1 to be queried, with the preset condition that the matching degree is greater than or equal to 2 / 3, and there is no dataset 2 or dataset 3 in the current cache queue.

[0149] Based on the above conditions, the matching degree between dataset 1 and the ID identifier of the dataset stored in the cache queue is determined to be 0. Therefore, the matching degree of the ID identifier of dataset 1 does not meet the preset conditions.

[0150] The ID of dataset 1 is 'picRecord'+'record in('a','b','c')'. The query conditions of dataset 1 are the query conditions determined by the current query granularity. Therefore, based on the query conditions of dataset 1, the corresponding database query statement is generated, and the database returns the corresponding query results based on the corresponding query statement.

[0151] Optionally, as one embodiment, the matching degree meets preset conditions, specifically including:

[0152] The matching degree between the ID of the dataset to be queried and the ID of the dataset stored in the cache queue is greater than or equal to 2 / 3.

[0153] The following is an explanation through specific examples.

[0154] Taking the above example, dataset 1 is the dataset to be queried, and the ID of dataset 1 is “'picRecord'+'record in('a','b','c')'”.

[0155] Dataset 2 and Dataset 3 are any two datasets stored in the cache queue. The ID of Dataset 2 is “'picRecord'+'record in('b','c','d')'”, and the ID of Dataset 3 is “'picRecord'+'recordin('a','e','f')'”.

[0156] Based on the above embodiments and the foregoing description, it can be determined that the ID identifier of dataset 1 is a subset of the ID identifiers of all datasets stored in the cache queue. This means that dataset 1 has a 100% matching degree with the cache queue, but data needs to be filtered from it.

[0157] Alternatively, suppose that dataset 3 is not in the cache queue, and only dataset 2 is related to dataset 1.

[0158] Therefore, the matching degree between the dataset 1 to be queried and the dataset ID stored in the queue is 2 / 3.

[0159] When the matching degree between the ID of the dataset and the ID of the dataset stored in the cache queue is greater than or equal to 2 / 3, the matching degree is considered to meet the preset condition.

[0160] In other words, if the cache queue contains only dataset 3 and not dataset 2, which is related to dataset 1 to be queried, the matching degree between the ID of dataset 1 and the ID of the dataset stored in the cache queue is 1 / 3. Even if some data in dataset 1 to be queried can be determined through the cache queue, the required data cannot be obtained through the cache because the data obtained through the cache does not meet the preset conditions. Therefore, the required query results are no longer obtained through the cache, but by querying the database.

[0161] Extensive data experiments have verified that if the matching degree between the dataset ID and the dataset ID stored in the cache queue is less than 2 / 3, the time and resources required to retrieve data from the cache and then reassemble the query statement to obtain results from the database with other identifiers are higher than the time and resources required to obtain them directly from the database. Therefore, when the matching degree between the dataset ID and the dataset ID stored in the cache queue is greater than or equal to 2 / 3, the matching degree is considered to meet the preset conditions; otherwise, the required query results are read directly from the database.

[0162] Optionally, as one embodiment, parsing the received query request specifically includes:

[0163] Based on the received query request, determine the query conditions contained in the query request;

[0164] Based on the query conditions and set rules, determine the ID identifier of each dataset to be queried;

[0165] Based on the ID of each dataset to be queried and the corresponding query conditions, determine the query conditions for each dataset to be queried.

[0166] In this case, the query conditions of the dataset to be queried are the same as the query conditions determined by the identifier of the corresponding dataset ID, or the query conditions of the dataset to be queried are a subset of the query conditions determined by the identifier of the corresponding dataset ID.

[0167] The process from receiving a query request to determining the dataset ID is illustrated below with specific examples. See Appendix. Figure 4 .

[0168] Let the rules be set as follows: for time-related fields in the database, the smallest granularity of division is the number of days; for information-related fields in the database, the smallest granularity of division is the current query condition and no further splitting is required; and the ID of the dataset is "the name of the table to be queried" + "the query condition determined by the current query granularity".

[0169] The following description uses SQL as an example. In practical applications, the corresponding query requests can be determined according to different languages ​​and usage environments.

[0170] S401: Based on the received query request, determine the query conditions contained in the query request;

[0171] Suppose that there are two received query requests, namely query request 1 and query request 2.

[0172] Query request 1 is as follows:

[0173] select from picRecord where capTime between 2022-4-25 and 2022-5-10 and record in ('a', 'b', 'c');

[0174] Query request 2 is as follows:

[0175] select from picRecord where record in ('c', 'b', 'd') and capTimebetween 2022-4-26 and 2022-5-11 and channelId in ('1', '2', '3');

[0176] Based on the received query request 1 and query request 2, the process of determining each query condition contained in the query request is as follows:

[0177] The query conditions included in query request 1 are as follows:

[0178] Condition 11: capTime between 2022-4-25 and 2022-5-10;

[0179] Condition 12: record in ('a', 'b', 'c');

[0180] The query conditions included in query request 2 are as follows:

[0181] Condition 21: record in ('c', 'b', 'd');

[0182] Condition 22: capTime between 2022-4-26 and 2022-5-11;

[0183] Condition 23: channelId in ('1', '2', '3');

[0184] Therefore, it can be determined that all query conditions contained in query request 1 are condition 11 and condition 12, and all query conditions contained in query request 2 are condition 21, condition 22 and condition 23.

[0185] S402: Determine the ID identifier of each dataset to be queried based on the query conditions and set rules;

[0186] Continuing with the above embodiment as an example, the process of determining the ID identifier of each dataset to be queried based on the query conditions and the above-defined rules is as follows:

[0187] According to the established rules, the smallest granularity for time fields is the number of days. Therefore, by dividing the dataset into daily units, the dataset to be queried can be split for condition 11 as follows:

[0188] Dataset 11-1: capTime = 2022-4-25;

[0189] Dataset 11-2: capTime = 2022-4-26;

[0190] ...

[0191] Dataset 11-16capTime = 2022-5-10;

[0192] According to the established rules, the smallest unit of division for fields related to information is the current query condition, without further subdivision. Therefore, for condition 12, the dataset to be queried can be subdivided as follows:

[0193] Dataset 12-1: record in ('a', 'b', 'c');

[0194] Therefore, in query request 1, there are a total of 17 datasets to be queried.

[0195] Regarding condition 21, the dataset to be queried can be split as follows:

[0196] Dataset 21-1: record in ('c', 'b', 'd');

[0197] Regarding condition 22, the dataset to be queried can be split as follows:

[0198] Dataset 22-1: capTime = 2022-4-26;

[0199] ...

[0200] Dataset 22-16: capTime = 2022-5-11;

[0201] Regarding condition 23, the dataset to be queried can be split as follows:

[0202] Dataset 23-1: channelIdin('1','2','3');

[0203] Therefore, in query request 2, there are a total of 18 datasets to be queried.

[0204] Based on the foregoing description, one of the rules is set as follows: the ID of the dataset is "the name of the table to be queried" + "the query conditions determined by the current query granularity". The process of determining the ID of the dataset is as follows:

[0205] The query condition for dataset 11-1 is based on the table named picRecord. The current query granularity is days. Therefore, the query condition determined by the current query granularity is data that meets the query condition for a specific day, divided into units of days. Specifically, according to this embodiment, the query condition for dataset 11-1 is data in the field that meets the condition capTime = 2022-4-25. Therefore, the ID of dataset 11-1 is:

[0206] 'picRecord'+'capTime = 2022-4-25';

[0207] Similarly, the ID identifier for dataset 11-2 is determined as follows:

[0208] 'picRecord'+'capTime = 2022-4-26';

[0209] ...

[0210] For the dataset identified in query request 2, the ID of dataset 12-1 is as follows:

[0211] The query condition for dataset 12-1 is to query the table named picRecord. The current query granularity is the data that meets the query condition of the current dataset. Specifically, according to this embodiment, the query condition for dataset 12-1 is the data in the field that meets the record in ('a', 'b', 'c'). Therefore, the ID of dataset 12-1 is 'picRecord'+'recordin ('a', 'b', 'c')'.

[0212] ...

[0213] The ID process for other datasets will not be listed here. The ID identifier for a dataset is determined as follows:

[0214] The ID identifiers for dataset 22-1 are as follows:

[0215] 'picRecord'+'capTime = 2022-4-26';

[0216] The ID identifiers for dataset 22-2 are as follows:

[0217] 'picRecord'+'capTime = 2022-4-27';

[0218] ...

[0219] The ID identifiers for dataset 23-1 are as follows:

[0220] 'picRecord'+'channelId in ('1', '2', '3')';

[0221] As can be seen from the ID identifiers of the datasets above, the ID identifiers determined by dataset 11-2 and dataset 22-1 are the same. The reason for this is that the query conditions of the two datasets are completely consistent. Therefore, the corresponding query results obtained by the two datasets from the database are also completely identical. Thus, dataset 11-2 and dataset 22-1 can share the same cache queue to store the corresponding database mapping relationship.

[0222] Similarly, the IDs for datasets 11-3 and 22-2 are also the same for the batch. Therefore, the method used in this application to determine datasets allows different query requests to share the corresponding database mapping stored in the cache queue if the datasets they determine have the same query conditions. This can be simply understood as sharing the cached content in the intersection of query requests. This method of determining datasets results in a high cache hit rate, thereby reducing the query pressure on the database.

[0223] S403: Determine the query conditions for each dataset based on its ID and corresponding query conditions.

[0224] Based on the ID of each dataset, the query conditions for the dataset with a specific query granularity can be determined. For example, if the ID of dataset 11-1 is 'picRecord'+'capTime = 2022-4-25', the query condition for the current dataset can be determined as data from the picRecord table that meets the condition 2022-4-25 in the field named capTime. This query condition is determined based on the dataset ID. Once the dataset ID is determined, the query conditions for the dataset can be determined.

[0225] The ID of the current dataset 11-1 is “'picRecord'+'capTime = 2022-4-25'”.

[0226] The query condition 11 is "capTime between 2022-4-25 and 2022-5-10". Therefore, it can be determined that the query condition for the current dataset 11-1 is also the data in the data table 'picRecord' whose field name is capTime and which meets the condition 2022-4-25.

[0227] Therefore, based on the dataset ID and query conditions, the query conditions for the current dataset 11-1 are determined to be the same as those for the dataset “'picRecord'+'capTime = 2022-4-25'”. That is, based on query condition 11, the query conditions for the current dataset 11-1 are the same as those for the dataset with the ID “'picRecord'+'capTime = 2022-4-25'”, and all query results for the dataset with the ID “'picRecord'+'capTime = 2022-4-25'” are required.

[0228] The following is an example illustrating that the query conditions for the dataset are a subset of the query conditions determined by the dataset's ID.

[0229] Suppose query request 3 is as follows:

[0230] select from picRecord where capTime between 2022-4-25 6:00 and2022-5-10 20:00 and record in ('a', 'b', 'c');

[0231] Compared to query request 1, query request 3 searches for data after 6:00 AM on April 25, 2022, and data before 8:00 PM on May 10, 2022. Data from April 26 to May 9, 2022, is still considered as data for the entire day. The difference from query request 1 is that for April 25 and May 9, 2022, only data from 6:00 AM to 12:00 AM on April 25 and from 12:00 AM to 8:00 PM on May 9, 2022, is not required for the entire day.

[0232] Taking the aforementioned embodiment as an example, when the smallest division granularity is the number of days, the cache queue stores the smallest unit determined by the number of days as the division granularity, and cannot be further subdivided into hours or minutes.

[0233] In this situation, the dataset is also divided at the smallest granularity according to the query conditions and set rules, and cannot be further subdivided to hours or minutes. The process of determining the query conditions according to the query request is the same, and will not be described here. The content described here is different from the previous embodiment.

[0234] Suppose that, according to query request 3, the determined dataset 31-1 is capTime = 2022-4-25, and according to the query conditions determined by the query request, the query condition for the current dataset is to obtain data from the data table 'picRecord' whose field name is capTime and which meets the condition 2022-4-256:00-24:00.

[0235] According to the established rules, the dataset ID for dataset 31-1 is determined as follows:

[0236] 'picRecord'+'capTime = 2022-4-25';

[0237] Based on the query conditions, the query conditions for dataset 31-1 are determined as follows:

[0238] From the data table 'picRecord', the column named capTime contains data that meets the condition 2022-4-25 6:00-24:00.

[0239] In other words, the query conditions for a dataset are a subset of the specific query conditions identified by the corresponding dataset ID.

[0240] The technical solution of this invention determines the query conditions based on the query request, and then determines the ID of the dataset. The ID is determined according to a set rule, which is determined separately based on the different granularities of each field in the database. Therefore, the method of setting the ID of the dataset is flexible and can be applied to databases with various fields. Moreover, the ID of the dataset is related to the fields of the database, and the database query hits the fields related to the fields, making full use of the characteristics of the database. This method results in a high cache hit rate and a high matching degree.

[0241] Optionally, as one embodiment, the mapping information is stored using a map structure.

[0242] The following is an explanation through specific embodiments. It should be noted that this embodiment is described using a map as an example. In actual use, other structures can also be used to store the mapping relationship between the ID identifier of the dataset and the corresponding query results in the database.

[0243] A Map structure is a data structure that stores data in the form of key-value pairs. In this application, the cache queue uses a map structure to store the mapping relationship between the ID of the dataset and the result of the dataset in the current cache.

[0244] The Map structure has two parameters, expressed as map<key,value> The parameter key can be understood as a keyword and cannot be empty. One key corresponds to one value. Values ​​can be the same or different, or they can be empty.

[0245] In this application, specifically, the map structure is a nested map structure, and the parameter value is a map structure.

[0246] The mapping relationship between dataset IDs and corresponding query results in the database is represented by the following map structure:

[0247] Map <Result_Tag,Map<File, RecordIdSet> >;

[0248] In this structure, the key value of the outer map structure, namely Result_Tag, is the ID identifier of the dataset. The key value of the inner map structure is a string consisting of the names of all file blocks that meet the query conditions and make up the corresponding data table. The value of the inner map, RecordIdSet, is an identifier that represents whether each record in all file blocks meets the requirements. It is recorded using the BitSet data structure.

[0249] As described above, each data table is composed of multiple corresponding file blocks. Therefore, the ID of the same dataset will correspond to a data table composed of multiple file blocks.

[0250] Taking dataset 11-1 as an example, the ID of dataset 11-1 is 'picRecord' + 'capTime = 2022-4-25'. Assume the data table picRecord consists of ten file blocks, but only three file blocks contain data records that match the current query conditions. These three file blocks are picRecord01, picRecord02, and picRecord03. In the cache queue, they are stored in the following format:

[0251] <'picRecord'+'capTime=2022-4-25',<'picRecord01'+'picRecord02'+'picRecord03',010100…00100110>>;

[0252] RecordIdSet is stored using the BitSet data structure, which has been described previously. It consists of binary boolean values, representing whether each record in 'picRecord01' + 'picRecord02' + 'picRecord03' meets the query criteria. 1 indicates that it meets the criteria, and 0 indicates that it does not. The number of boolean values ​​corresponds to the number of records in 'picRecord01' + 'picRecord02' + 'picRecord03'.

[0253] When reading RecordIdSet bits, see Figure 5 As shown, reading from the least significant bit to the most significant bit indicates whether the data from the first record in file block picRecord01 to the last record in file block picRecord03 meets the query conditions.

[0254] For example, if the first bit of RecordIdSet is 0, it means that the first data record in file block picRecord01 does not meet the query conditions. If the second bit of RecordIdSet is 1, it means that the second data record in file block picRecord01 meets the query conditions. If the third bit of RecordIdSet is 1, it means that the third data record in file block picRecord01 meets the query conditions, and so on. If there are 5000 records in 'picRecord01' + 'picRecord02' + 'picRecord03', then RecordIdSet will have 5000 boolean values.

[0255] When using it, if the dataset ID matches the Result_Tag, the data can be directly read from the corresponding file block in the database based on the data with a boolean value of 1 in RecordIdSet, without having to search the database based on the query conditions of the dataset ID. That is, the current map structure has already saved the mapping relationship between the dataset ID and the corresponding query results in the database.

[0256] When the dataset ID is not found in the cache queue, the corresponding data in the database will be retrieved, and the query results that match the query conditions of the dataset ID will be obtained. Based on the query results retrieved from the database and the dataset ID, the corresponding map structure will be automatically generated and stored in the cache queue for reuse. This avoids retrieving the database multiple times for the same query conditions. Even if a large number of duplicate queries are generated, it will not affect the database and reduce the query pressure on the database.

[0257] Furthermore, the BitSet data structure can participate in logical AND, logical OR, and logical XOR operations. When the query conditions determined in the query request require AND, OR, NOT operations, the BitSet data structure can also participate in the operation. The following is an explanation through specific examples.

[0258] Taking query request 1 as an example, all the query conditions contained in query request 1 are as follows:

[0259] Condition 11: capTime between 2022-4-25 and 2022-5-10;

[0260] Condition 12: record in ('a', 'b', 'c');

[0261] Condition 11 and Condition 12 are related by AND, while the datasets in Condition 11, such as dataset 11-1: capTime = 2022-4-25, dataset 11-2: capTime = 2022-4-26, ..., dataset 11-16 capTime = 2022-5-10, are related by OR.

[0262] Therefore, when determining the query results, for example, for the query results required by query request 1, the RecordIdSet bits of the corresponding datasets in the cache queue of datasets 11-1 to 11-16 in condition 11 are first ORed, and then ANDed with the RecordIdSet bits of the dataset corresponding to dataset 12-1 in condition 12. The data in the RecordIdSet with a result of 1 after the operation is the final query result of query request 1. The record in the database corresponding to the RecordIdSet with a result of 1 is directly read, which is the query result of query request 1.

[0263] By using logical AND, logical OR, and logical XOR operations on the BitSet structure, the time complexity can be reduced by tens of times. While meeting certain computational needs, this can significantly reduce time complexity and save storage space, allowing limited cache space to store more cache queue data and greatly improving query speed to obtain the required data.

[0264] Optionally, as one embodiment, after retrieving and returning data records from the database that satisfy the query conditions of the current dataset as query results, the method further includes:

[0265] Based on the ID of the dataset to be queried and the data records in the database that meet the query conditions of the current dataset, generate mapping information between the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset; update the cache queue.

[0266] The following description uses specific examples; see [link to example]. Figure 6 As shown.

[0267] This embodiment only describes the... Figure 3 For the same content, please refer to other embodiments for differences, and this embodiment will not repeat them here.

[0268] S606: Based on the ID of the dataset to be queried and the data records in the database that meet the query conditions of the current dataset, generate mapping information between the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset.

[0269] If the matching degree of the dataset ID does not meet the preset conditions, the data in the corresponding database will be retrieved, and the query results corresponding to the query conditions that match the dataset ID will be obtained from the database.

[0270] Based on the query results retrieved from the database, the dataset ID, and the map structure defined in the cache queue, a corresponding map structure is automatically generated and stored in the cache queue. The generated map structure is the mapping information between the dataset ID and the data records that meet the query conditions of the current dataset.

[0271] S607: Update the cache queue.

[0272] Cache space is limited and cannot be occupied indefinitely. If a map structure stored in the cache queue is not used for a long time and is not replaced, the cache hit rate will be reduced. Therefore, how to replace the cache and improve the cache hit rate is a technical problem that needs to be considered.

[0273] There are many commonly used caching algorithms, such as LFU (Least Recently Used), LRU (Least Recently Used), ARC (Adaptive Cache Replacement), MRU (Most Recently Used), and FIFO (First In First Out). This embodiment uses the LRU algorithm as an example to describe how to update the cache queue.

[0274] The LRU cache eviction algorithm, also known as the Least Recently Used algorithm, prioritizes evicting data that is the furthest from the current time by determining the most recent time it was used.

[0275] The LRU algorithm process can be summarized as follows:

[0276] STEP01: When adding new cache, insert the cached data directly into the head of the list;

[0277] STEP02: Determine if the cached data is hit, and move the cached data to the head of the list;

[0278] STEP03: When the cache is full, remove the data at the end of the list.

[0279] Therefore, the cache queue is updated as follows: when it is determined that the ID of the dataset does not exist in the cache queue, the map structure containing the ID of the dataset is directly inserted into the head of the list; when it is determined that the ID of the cached dataset is matched, the map structure containing the ID of the dataset is moved to the head of the cache list; when it is determined that the cache is full, the map structure data at the tail of the cache list is removed.

[0280] Optionally, as one embodiment, a historical query queue is also included;

[0281] If the matching degree does not meet the preset conditions, it also includes:

[0282] If the ID of the historical dataset stored in the historical query queue does not match the ID of the dataset to be queried, the ID of the dataset to be queried is added to the historical query queue as the ID of the historical dataset, and the historical query queue is updated.

[0283] After determining that the query count of the ID of the dataset to be queried meets the set conditions, and after retrieving the data records that meet the query conditions of the current dataset from the database as the query results and returning them, the process also includes: generating mapping information between the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset based on the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset retrieved from the database; updating the cache queue; and deleting the historical dataset ID and the query count corresponding to the historical dataset ID from the historical query queue.

[0284] The historical query queue stores the ID identifier of the historical dataset and the number of queries for the corresponding historical dataset ID identifier. The ID identifier of the historical dataset is the ID identifier of the dataset to be queried but not saved in the cache queue. The update historical query queue includes the number of queries to update the ID identifier of the historical dataset.

[0285] The following description uses specific examples; see [link to example]. Figure 7 .

[0286] Steps S701-S705 are the same as steps S601-S605 in the above embodiment, and will not be repeated here. Only the differences from the above embodiment will be described.

[0287] S706: Check if the historical query queue matches the ID of the dataset to be queried. If not, proceed to step S707; if yes, proceed to step S708.

[0288] In addition to the cache queue, the technical solution of this application also includes a historical query queue. The historical query queue stores the ID identifier of the historical dataset and the number of queries corresponding to the ID identifier of the historical dataset. The ID identifier of the historical dataset is the ID identifier of the dataset to be queried but not saved in the cache queue, but the query results corresponding to the ID identifier of the dataset are not saved.

[0289] If the matching degree of the dataset ID does not meet the preset conditions, in addition to obtaining the query results from the database, it is also necessary to query the historical query queue to determine whether the ID of the historical dataset stored in the historical query queue matches the ID of the dataset.

[0290] S707: Add the ID of the dataset to be queried to the historical query queue as the ID of the historical dataset.

[0291] If the ID of the historical dataset stored in the historical query queue does not match the ID of the dataset, it means that there is no corresponding ID in the historical query queue, and the ID is added to the historical query queue.

[0292] S708: Update the historical query queue;

[0293] Updating the historical query queue includes updating the query count of the ID identifier of the historical dataset. If the ID identifier of the historical dataset exists, the query count of the ID identifier of the current historical dataset will be displayed. If it is added to the historical query queue for the first time, the query count will be 1.

[0294] If the ID of a historical dataset matches the ID of the dataset stored in the historical query queue, it means that the ID of the dataset already exists in the historical query queue. Therefore, it is necessary to update the query count of the dataset's ID. Typically, the query count starts from 1 and gradually increases.

[0295] The historical query queue can also be stored using the caching algorithms mentioned above, such as LFU (Least Recently Used), LRU (Least Recently Used), ARC (Adaptive Cache Replacement), MRU (Most Recently Used), and FIFO (First In First Out), to eliminate unnecessary data. If the LRU algorithm is used, the historical query queue also stores the query time of the ID of the current historical dataset.

[0296] S709: When the number of queries for the ID of the dataset to be queried meets the set conditions, generate mapping information between the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset obtained from the database, and update the cache queue.

[0297] This step is similar to step S606. The same parts will not be described here, only the differences will be described.

[0298] The historical query queue stores the query counts for the IDs of historical datasets, but it does not store the corresponding query results in the database. Simply put, the historical query queue stores the dataset IDs, but not the map structure. Therefore, the query counts for dataset IDs stored in the historical query queue can indirectly reflect whether the dataset IDs are frequently used or accessed. When certain conditions are met, it indicates that the dataset is frequently queried, and it is necessary to generate mapping information and add it to the cache queue to speed up the query. If the conditions are not met, the dataset is not frequently queried, only occasionally, and it is unnecessary to generate mapping information and save it to the cache queue, thus consuming cache space.

[0299] Optionally, the preset value can be set to 2, meaning that the set condition is met when the ID of the dataset is queried for the second time.

[0300] S710: Delete the ID identifier of the historical dataset and the query count corresponding to the ID identifier of the historical dataset from the historical query queue.

[0301] The ID of the historical dataset is deleted from the historical query queue. Once the ID of the dataset enters the cache queue, the corresponding query result can be obtained directly through the cache. Therefore, it is not necessary to query the historical query queue again, nor is it necessary to retain the query record of the historical dataset ID in the current historical queue. In order to save FS205, it can effectively prevent the problem of low cache utilization caused by cache pollution. The LRU algorithm alone cannot avoid the storage space occupation caused by cache pollution and the additional operation time overhead caused by the eviction policy. This technical solution, combined with the LRU algorithm, can effectively avoid the problem caused by cache pollution and save the cache queue, thus reducing the resource consumption caused by cache pollution.

[0302] Optionally, as one embodiment, if the time taken to retrieve data records that satisfy the query conditions of the current dataset from the dataset ID and the corresponding current database exceeds a preset time, the number of queries corresponding to the ID of the historical dataset in the historical query queue is adjusted to N times the normal number of queries, where N is a positive integer greater than or equal to 2.

[0303] The following is an explanation through specific examples.

[0304] Taking the above embodiment as an example, for the information condition field in the database, the granularity of the division is the current query condition and it is no longer split. If the content queried by the query condition is large, the query time obtained from the database will be relatively long. Therefore, for this case, a preset duration is set, such as 10 seconds. When the query time obtained from the database exceeds 10 seconds, the number of queries in the historical query queue is adjusted by N times, where N is a positive integer greater than or equal to 2. That is to say, it is equivalent to increasing the weight of the number of queries in the historical query queue. If N is 2, although the current dataset is only queried once, when recording the number of queries, since the query time exceeds the preset duration, it is equivalent to being queried twice to increase the weight, so that the dataset can enter the cache queue faster. If we continue to take the above embodiment as an example, and assume that the number of queries reaches 2 to enter the cache queue, then when N is 2, for datasets with a query time exceeding 10 seconds, they can directly enter the cache queue to avoid the dataset occupying more system resources and query time when it is queried again.

[0305] Optionally, embodiments of the present invention also provide a database cache query device 800, see [link to documentation]. Figure 8 As shown, it includes:

[0306] The parsing module 801 is used to parse the received query request, determine the ID identifier of each dataset to be queried based on the parsing result, and the query conditions corresponding to each dataset determined based on the query request.

[0307] The matching module 802 is used to match the ID identifier of the dataset to be queried with the ID identifier of the dataset stored in the cache queue one by one; it is also used to, if the matching degree of the ID identifier of the dataset to be queried meets the preset conditions, read the data record mapped by the record mapping information corresponding to the ID identifier of the dataset to be queried from the database according to the record mapping information stored in the cache queue, and match the data record that meets the corresponding query conditions of the current dataset from the data record, and return it as the query result.

[0308] The dataset contains data records in the database that meet the specified query conditions; the dataset ID is generated according to the set rules, which are determined based on the different granularities of each field in the database; the cache queue stores the mapping information between the dataset ID and the corresponding records in the database that meet the specified query conditions of the dataset, and the record mapping information has a mapping relationship with the records in the database that meet the specified query conditions of the dataset.

[0309] Optionally, embodiments of the present invention also provide an electronic device 900, see Figure 9 As shown, it includes:

[0310] It includes a processor 901 and a memory 902. The processor 901 runs a program stored in the memory 902. When the program is executed, the processor 901 performs any of the methods described in the above embodiments.

[0311] Optionally, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, the computer program including methods for performing any of the above embodiments.

[0312] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0313] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0314] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0315] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0316] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A database caching query method, characterized in that, The method includes: Parse the received query request and, based on the parsing results, determine the ID identifier of each dataset to be queried; The corresponding query conditions for each dataset are determined based on the query request; The ID identifier of the dataset to be queried is matched one by one with the ID identifier of the dataset stored in the cache queue; If the matching degree of the ID identifier of the dataset to be queried meets the preset conditions, then according to the record mapping information corresponding to the ID identifier of the dataset to be queried stored in the cache queue, the data record mapped by the record mapping information is read from the database, and the data record that meets the corresponding query conditions of the current dataset is matched from the data record and returned as the query result; If the matching degree of the ID identifier of the dataset to be queried does not meet the preset conditions, then: Based on the ID of the dataset to be queried and the corresponding query conditions of the current dataset, data records that meet the query conditions of the current dataset are read from the database and returned as query results. Furthermore, the historical query queue is queried. If the ID of a historical dataset stored in the historical query queue does not match the ID of the dataset to be queried, the ID of the dataset to be queried is added to the historical query queue as the ID of the historical dataset, and the historical query queue is updated. If the ID of the historical dataset matches the ID of the dataset to be queried, the historical query queue is updated. When it is determined that the number of queries for the ID of the dataset to be queried meets the set conditions, the mapping information between the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset obtained from the database is generated according to the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset. The cache queue is updated, and the historical dataset ID and the number of queries corresponding to the historical dataset ID are deleted from the historical query queue. The historical query queue stores the ID identifier of a historical dataset and the corresponding query count of that historical dataset ID identifier. The ID identifier of the historical dataset is the ID identifier of a dataset to be queried but not yet saved in the cache queue. Updating the historical query queue includes updating the query count of the ID identifier of the historical dataset.

2. The method according to claim 1, characterized in that, If the matching degree of the ID identifiers of the dataset to be queried meets the preset conditions, the method further includes: If the ID identifier of the dataset to be queried does not completely match the ID identifier of the dataset stored in the cache queue, then based on the mismatched identifier in the ID identifier of the dataset to be queried and the corresponding query conditions of the dataset to be queried, a new query statement is reassembled, and the data record corresponding to the mismatched identifier in the ID identifier of the dataset to be queried is read from the database according to the new query statement. Based on the record mapping information corresponding to the ID identifier of the dataset to be queried, stored in the cache queue, the data records mapped by the record mapping information are read from the database. Data records that meet the corresponding query conditions of the current dataset are matched from the data records and merged with the data records corresponding to the identifiers that do not match the ID identifier of the dataset to be queried, obtained from the database. The merged data records are then returned as the query result.

3. The method according to claim 1, characterized in that, The matching degree meets preset conditions, including: The matching degree between the ID identifier of the dataset to be queried and the ID identifiers of all datasets stored in the cache queue is greater than or equal to 2 / 3.

4. The method according to claim 1, characterized in that, The parsing of the received query request includes: Based on the received query request, determine each query condition contained in the query request; Based on the query conditions and set rules, determine the ID identifier of each dataset to be queried; Based on the ID identifier of each dataset to be queried and the corresponding query conditions, the query conditions for each dataset to be queried are determined. Wherein, the query conditions of the dataset to be queried are the same as the query conditions determined by the identifier of the corresponding ID of the dataset to be queried, or the query conditions of the dataset to be queried are a subset of the query conditions determined by the identifier of the corresponding ID of the dataset to be queried.

5. The method according to claim 1, characterized in that, The mapping information is stored using a map structure.

6. The method according to claim 1, characterized in that, After retrieving and returning data records from the database that satisfy the query conditions of the current dataset as query results, the method further includes: Based on the ID of the dataset to be queried and the data records obtained from the database that meet the query conditions of the current dataset, a mapping information is generated between the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset. Update the cache queue.

7. The method according to claim 1, characterized in that, The method further includes: if the time taken to obtain data records that meet the query conditions of the current dataset from the database exceeds a preset time, then the number of queries corresponding to the ID identifier of the historical dataset in the historical query queue is adjusted to N times the normal number of queries, where N is a positive integer greater than or equal to 2.

8. A database cache query device, characterized in that, The device includes: The parsing module is used to parse the received query request, determine the ID identifier of each dataset to be queried based on the parsing result, and the query conditions corresponding to each dataset determined based on the query request. The matching module is used to match the ID identifier of the dataset to be queried one by one with the ID identifiers of the datasets stored in the cache queue; it is also used to: If the matching degree of the ID identifier of the dataset to be queried meets the preset conditions, then according to the record mapping information corresponding to the ID identifier of the dataset to be queried stored in the cache queue, the data record mapped by the record mapping information is read from the database, and the data record that meets the corresponding query conditions of the current dataset is matched from the data record and returned as the query result; If the matching degree of the ID identifier of the dataset to be queried does not meet the preset conditions, then: Based on the ID of the dataset to be queried and the corresponding query conditions of the current dataset, data records that meet the query conditions of the current dataset are read from the database and returned as query results. Furthermore, the historical query queue is queried. If the ID of a historical dataset stored in the historical query queue does not match the ID of the dataset to be queried, the ID of the dataset to be queried is added to the historical query queue as the ID of the historical dataset, and the historical query queue is updated. If the ID of the historical dataset matches the ID of the dataset to be queried, the historical query queue is updated. When it is determined that the number of queries for the ID of the dataset to be queried meets the set conditions, the mapping information between the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset obtained from the database is generated according to the ID of the dataset to be queried and the data records that meet the query conditions of the current dataset. The cache queue is updated, and the historical dataset ID and the number of queries corresponding to the historical dataset ID are deleted from the historical query queue. The historical query queue stores the ID identifier of a historical dataset and the corresponding query count of that historical dataset ID identifier. The ID identifier of the historical dataset is the ID identifier of a dataset to be queried but not yet saved in the cache queue. Updating the historical query queue includes updating the query count of the ID identifier of the historical dataset.

9. An electronic device, characterized in that, The device includes a processor and a memory, wherein the processor runs a program stored in the memory, and when the program is executed, causes the processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The device contains a computer program that includes methods for performing any one of claims 1-7.

Images