A data query method, apparatus, device, medium and product

Abstract

The application provides a data query method and device, equipment, medium and product, which can be applied to the fields of artificial intelligence and financial technology. The method comprises the following steps: determining a target data query request for a target database; the target data query request is used to query first data; the target database comprises N data tables; a preset storage area outside the target database is determined; N data sets are stored in the preset storage area, different data sets correspond to different data tables in the N data tables respectively, and any data set is used to store data in the corresponding data table; a first data table to which the first data belongs is determined; in the case that the first data is queried in the data set corresponding to the first data table, the first data queried is fed back for the target data query request; in the case that the first data is not queried in the data set corresponding to the first data table, the first data is queried in the first data table of the target database.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence technology and fintech, specifically to a data query method, apparatus, device, medium, and product. Background Technology

[0002] With the widespread adoption of information technology, data retrieval plays a crucial role in many business operations. For example, in user authentication, it is typically necessary to retrieve pre-stored user authentication information from the database for authentication; similarly, in video playback, it is necessary to retrieve the desired video from the database for transmission.

[0003] Currently, data is usually queried directly from the database based on the data query request. This results in a large query load on the database and relatively low database stability. Summary of the Invention

[0004] In view of the above problems, embodiments of this application provide a data query method, apparatus, device, medium, and product that can improve the stability of the database.

[0005] According to a first aspect of this application, a data query method is provided, comprising: determining a target data query request for a target database; the target data query request being used to query first data; the target database containing N data tables, where N is a positive integer; determining a preset storage area outside the target database; the preset storage area storing N data sets, each data set corresponding to a different data table in the N data tables, each data set being used to store data in the corresponding data table; determining the first data table to which the first data belongs; querying the first data in the data set corresponding to the first data table in the preset storage area according to the target data query request; if the first data is found in the data set corresponding to the first data table in the preset storage area, providing feedback on the found first data for the target data query request; if the first data is not found in the data set corresponding to the first data table in the preset storage area, querying the first data in the first data table of the target database according to the target data query request.

[0006] According to an embodiment of this application, querying the first data in the first data table of the target database according to the target data query request includes: if the first data is found in the first data table of the target database, storing the found first data in the data set corresponding to the first data table, and providing feedback on the found first data in response to the target data query request; if the first data is not found in the first data table of the target database, providing feedback on the target data query request to indicate that the query for the first data failed.

[0007] According to an embodiment of this application, the step of querying the first data in the data set corresponding to the first data table in the preset storage area according to the target data query request includes: if it is determined that the first data table belongs to the N data tables, querying the first data in the data set corresponding to the first data table in the preset storage area according to the target data query request; if it is determined that the first data table does not belong to the N data tables, providing feedback on the target data query request to indicate that the query for the first data failed.

[0008] According to an embodiment of this application, the preset storage area is used to store the value range of the primary keys in the N data tables; the step of querying the first data in the data set corresponding to the first data table in the preset storage area according to the target data query request includes: if it is determined that the primary key value in the target data query request belongs to the value range of the primary key in the first data table, querying the first data in the data set corresponding to the first data table in the preset storage area according to the target data query request; if it is determined that the primary key value in the target data query request does not belong to the value range of the primary key in the first data table, providing a result indicating that the query for the first data failed.

[0009] According to an embodiment of this application, the method further includes: predicting the future data query probability for each of the N data tables based on a pre-trained query probability prediction model; and selecting data from data tables whose future data query probability is greater than a preset query probability threshold and storing the data in the corresponding data set.

[0010] According to an embodiment of this application, the query probability prediction model based on pre-trained data predicts the future data query probability for each of the N data tables, including: based on the pre-trained query probability prediction model, predicting the future data query probability for each of the N data tables according to at least one of the following: the historical data query frequency of the data set corresponding to the targeted data table; the historical data query frequency of the targeted data table; and historical data query requests where no data was found in the data set corresponding to the targeted data table, but data was found in the targeted data table of the target database.

[0011] According to an embodiment of this application, the method further includes: if the number M of M data query requests for which no data was found in the target database within a preset time period is greater than a preset request number threshold, generating filtering rules based on the M data query requests; where M is a positive integer; and for any new data query request that conforms to any of the generated filtering rules, providing feedback to indicate that the data query failed.

[0012] According to an embodiment of this application, querying the first data in the first data table of the target database according to the target data query request includes: combining the target data query request and other data query requests for querying other data in the first data table to generate a comprehensive data query request; querying the first data and the other data in the first data table of the target database according to the comprehensive data query request; wherein the other data is not stored in the data set corresponding to the first data table.

[0013] According to an embodiment of this application, the method further includes: for any one of the N data tables, determining the upper limit of the storage volume of the data set corresponding to the targeted data table in the preset storage area based on at least one of the following: the data query frequency of the targeted data table; the data query frequency of the data set corresponding to the targeted data table; wherein the determined upper limit of storage volume is positively correlated with the data query frequency of the targeted data table, and the determined upper limit of storage volume is positively correlated with the data query frequency of the data set corresponding to the targeted data table.

[0014] According to embodiments of this application, the data update method of the preset storage area includes at least one of the following: the data stored in the preset storage area has an expiration date; in the preset storage area, data that has exceeded its expiration date is deleted; any one of the N data sets corresponds to a storage capacity limit; for any one of the N data sets, if the proportion of the currently stored data in the corresponding storage capacity limit exceeds a preset proportion threshold, data is selected from the targeted data set for deletion; in the preset storage area, data with a query frequency lower than a preset idle frequency threshold is selected for deletion; data is selected from the target database and stored in the preset storage area; in response to modification of the second data in the target database, the modified second data is synchronized to the second data stored in the preset storage area.

[0015] According to an embodiment of this application, the method further includes: when storing the third data in the target database into the preset storage area, determining the effective storage duration of the third data based on the data query frequency and / or data modification frequency of the third data; the determined effective storage duration is positively correlated with the data query frequency; and the determined effective storage duration is negatively correlated with the data modification frequency.

[0016] According to a second aspect of this application, a data query apparatus is provided, comprising: a determining module, configured to determine a target data query request for a target database; the target data query request is used to query first data; the target database contains N data tables, where N is a positive integer; determining a preset storage area outside the target database; the preset storage area stores N data sets, different data sets corresponding to different data tables in the N data tables, each data set being used to store data in the corresponding data table; determining a first data table to which the first data belongs; a first query module, configured to query the first data in the data set corresponding to the first data table in the preset storage area according to the target data query request; a feedback module, configured to, if the first data is found in the data set corresponding to the first data table in the preset storage area, provide feedback on the first data found according to the target data query request; and a second query module, configured to, if the first data is not found in the data set corresponding to the first data table in the target database according to the target data query request.

[0017] According to a third aspect of this application, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0018] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program or instructions are stored, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0019] According to a fifth aspect of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0020] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0021] Figure 1 This illustration schematically depicts an application scenario of a data query method according to an embodiment of this application.

[0022] Figure 2 A flowchart illustrating a data query method according to an embodiment of this application is shown schematically;

[0023] Figure 3 This schematic diagram illustrates a structural block diagram of a data query device according to an embodiment of the present application;

[0024] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a data query method according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0029] With the widespread adoption of information technology, data querying plays a crucial role in many business processes. For example, user authentication typically requires retrieving pre-stored user authentication information from the database for authentication; similarly, video playback requires retrieving the desired video from the database for transmission. Currently, data is usually queried directly from the database based on query requests. This results in a heavy query load on the database, potentially causing it to crash or freeze, leading to relatively low database stability.

[0030] To address the aforementioned technical problems, embodiments of this application provide a data query method. In this method, an external storage area (referred to as a preset storage area for convenience) can be deployed for the database. Specifically, the preset storage area can be an external cache or other storage medium. A portion of the data from the database can be stored in the preset storage area. Therefore, for a data query request, the system first checks if the required data exists in the preset storage area. If the required data exists, the retrieved data can be directly returned. If the required data does not exist, further data querying can be performed in the database.

[0031] This method can improve database stability by deploying an additional pre-defined storage area to offload the data query load on the database, reducing the number and frequency of data queries.

[0032] In the above method, the data query request can be in the form of a data query statement, which makes it easy to determine the location of the data to be queried from the data query statement.

[0033] In the above method, the database can contain multiple data tables. Correspondingly, multiple data sets can be deployed in a preset storage area. The number of data sets can be the same as the number of data tables, and each data set can correspond one-to-one with a data table. The data sets can be used to store data from the corresponding data tables. This allows for the determination of the data set corresponding to the data table to be queried in the preset storage area when a data query request is performed. Further data querying can then be performed within the determined data set, thus improving the efficiency of data querying.

[0034] This allows users to select a portion of data from a data table and store it in a corresponding data set, thereby increasing the success rate of data queries in the preset storage area and improving the efficiency of data retrieval.

[0035] In the above method, specifically during the data query process in the database, if the data to be queried exists in the database, the queried data can be stored in a preset storage area first, and then the queried data can be returned from the preset storage area.

[0036] If the requested data does not exist in the database, a result indicating query failure can be returned. Specifically, this can be achieved by first storing the data query request and the result indicating query failure in a pre-defined storage area, and then retrieving the result from the pre-defined storage area for the same query request. The pre-defined storage area can also store a dataset for storing query failure results, which can contain the data query request (which can be in the form of a query statement) and the result indicating query failure, allowing for convenient direct return of the result indicating query failure for the same data query request.

[0037] In the above method, data in the preset storage area can be updated accordingly to adapt to specific query requirements. Specifically, data retrieved from the database can be stored in the preset storage area, and data stored in the preset storage area can be added, deleted, or modified.

[0038] When adding data to a preset storage area, data can be selected from the database and stored in the preset storage area. Specifically, the selection can be based on data query requests, or on the query popularity or frequency of the data tables. Data from tables with high query popularity or frequency can be selected and stored in the preset storage area.

[0039] When deleting data stored in a preset storage area, you can set an expiration date for the data and delete data that has exceeded the expiration date. Alternatively, you can consider deleting data based on the storage limit of the preset storage area. Specifically, you can allocate a storage limit for each data set corresponding to each data table, so that if the amount of data in the data set exceeds the allocated storage limit, you can select some data to delete from the preset storage area. You can also select data with low query frequency in the preset storage area for deletion.

[0040] When modifying data stored in a preset storage area, the same modification can be performed on the same data stored in the preset storage area in response to data modifications in the database.

[0041] By updating the data stored in the preset storage area, it is possible to easily adapt to the current query requirements in real time, improve the success rate of data queries in the preset storage area, and improve data query efficiency.

[0042] In the above method, during the initialization of the preset storage area, data can be selected from the database and stored in the preset storage area, or data can be selected from the database periodically or at any time and stored in the preset storage area. To improve the success rate of data queries in the preset storage area, the query frequency of data in each data table and the query frequency of data in each data set can be determined based on recent data query requests. This allows data from data tables with high query frequencies or those exceeding a threshold to be selected and stored in the preset storage area.

[0043] In a specific example, based on recent historical data query requests, it can be determined that the data in data table 1 is queried frequently, and the data in the corresponding data set of data table 1 is queried frequently. Therefore, it can be determined that the data in data table 1 has a high query popularity. More data can be selected from data table 1 and stored in the corresponding data set of data table 1 to improve the success rate of data query in the preset storage area and improve the efficiency of data query.

[0044] Correspondingly, the future query frequency of data in the data table can be predicted based on changes in query frequency (e.g., query frequency change sequence). This allows for the selection of data from the data table whose future query frequency is higher than a threshold, and storage of the corresponding data set within a preset storage area.

[0045] Of course, it is also possible to predict the future query probability of data in the data table, and select some data in the data table whose future query probability is higher than the threshold to store in the corresponding data set in the preset storage area, so as to improve the success rate of data query in the preset storage area and improve the efficiency of data query.

[0046] The specific method for selecting data can be random selection, or it can be to further determine the query probability or query frequency of each data in the selected data table, and select data whose query probability or query frequency is higher than the threshold.

[0047] The specific method for predicting the future query probability of a data table is not limited. In particular, a pre-trained machine learning model can be used to predict the future query probability for each data table in the database.

[0048] The information used to predict the future query probability of a data table may include: the historical data query frequency of the data table (which may be in the form of a query frequency sequence), and the historical data query frequency of the corresponding data set of the data table (which may be in the form of a query frequency sequence).

[0049] Furthermore, considering the success rate of data queries in the preset storage area, data query requests that fail to be queried in the preset storage area but succeed in the database can be further utilized. If such data query requests occur frequently and are concentrated in a certain number of data tables, it can be determined that there is a large amount of data in these data tables that has query requirements. Therefore, data can be selected from these data tables and stored in the preset storage area.

[0050] In a specific example, the data in Table 2 is queried infrequently and contains a lot of cold data. However, the number of query requests for data in Table 2 has increased recently. Specifically, this can be reflected in the "increase in the number of data query requests that fail to retrieve data in the preset storage area but succeed in the database". Therefore, by statistically analyzing the data tables where the data queried by these data query requests is located, it can be determined that there is a high demand for data in Table 2. Further, data can be selected from Table 2 and stored in the preset storage area.

[0051] Therefore, the information used to predict the future query probability of a data table can also include: data query requests that did not find data in the corresponding dataset but did find data in the target database. Specifically, the prediction of the future query probability can be made based on the number of such query requests and the distribution of the queried data within the data table, thus improving the accuracy of the prediction.

[0052] In the above method, it is considered that some data query requests cannot be successfully retrieved in the preset storage area and the database. The reason for such data query requests may be that the query information is incorrect, or the attacker may want to occupy the database resources through such invalid query requests.

[0053] In a specific example, the attacker may randomly generate a large number of invalid data query requests. These invalid data query requests cannot be successfully retrieved in the preset storage area and database, thus consuming a lot of database resources and potentially causing the database to crash or malfunction.

[0054] Therefore, the above method can further identify such invalid data query requests, intercept them outside the database, reduce the impact of invalid data query requests on the database, and reduce the invalid occupation of database resources.

[0055] One way to identify invalid data query requests is to determine whether the data to be queried in the data query request belongs to a table in the database.

[0056] Specifically, by utilizing the data sets corresponding to each data table in a preset storage area, for any data query request, the system first determines the data table to which the requested data belongs. The preset storage area can store the identification information of each data table, thus allowing the system to determine whether the database contains the data table to which the requested data belongs. If the database does not contain the data table to which the requested data belongs, the data query request can be directly determined as invalid, and the query will not succeed in either the preset storage area or the database. Therefore, the system can directly return a query failure result without querying the preset storage area or the database.

[0057] For example, the database can contain data tables 1 to 9, and a preset storage area can deploy the data sets corresponding to each of the 9 tables, as well as store the identification information for the 9 tables. For any data query request, this can include the data table to which the data to be queried belongs, such as the query range in the query statement, specifically data table 10. By comparing the identification information of the data table, it can be determined that data table 10 does not belong to the database, and therefore the query cannot succeed. Thus, the query failure result can be directly returned without querying the preset storage area and the database.

[0058] Of course, if the database contains the table to which the data requested in the data query belongs, then the corresponding data query operation can be executed.

[0059] Another way to identify invalid data query requests is to determine whether the primary key value of the data to be queried in the data query request falls within the range of primary key values ​​in the database.

[0060] Understandably, a database table can have a primary key and a range of values ​​for that primary key. For example, a table might store 100 records, and the corresponding primary key value could be between 1 and 100. If a data query request targeting that table uses a primary key value of 101, it can be determined that the query is invalid and will fail.

[0061] To facilitate the identification of invalid data query requests, the primary key value ranges of each data table in the database can be stored in a preset storage area, specifically in the data collection corresponding to each data table.

[0062] For any data query request, after determining the data table to which the data to be queried belongs, the primary key value range stored in the corresponding data set can be compared with the primary key value in the data query request. If the primary key value in the data query request is not within the range of primary key values ​​stored in the corresponding data set, the data query request can be determined as an invalid data query request, and there is no need to query the preset storage area and database; the query failure result will be directly returned.

[0063] The range of primary key values ​​can be a set of primary key values ​​that includes all primary key values ​​in the data table, or it can be the maximum and minimum values ​​of the primary key values ​​in the data table.

[0064] Furthermore, if the primary key value in the data query request is within the range of primary key values ​​stored in the corresponding data set, then the corresponding data query operation can be executed.

[0065] Another way to identify invalid data query requests is to build filtering rules and match them with the data query requests, identifying those that match the filtering rules as invalid data query requests.

[0066] For example, if it is determined that the same user frequently initiates data query requests that fail in the database, a filtering rule can be constructed to determine all data query requests initiated by that user within a specified future period as invalid data query requests, without needing to query the preset storage area and database, and directly return the result of the query failure.

[0067] If it is determined that all data query requests that have recently failed in the database contain the same identifier, a filtering rule can be constructed to identify all data query requests containing this identifier within a specified future period as invalid data query requests. This eliminates the need to query the preset storage area and database, and directly returns the result of the query failure.

[0068] Furthermore, if the number of recently failed data query requests in the database exceeds a threshold, filtering rules can be constructed based on these requests. Specifically, filtering rules can be built based on common or frequently occurring characteristics among the recently failed data query requests. This can be achieved using large-scale machine learning models for analysis and rule construction.

[0069] Of course, any other methods can be used to identify invalid data query requests that cannot be successfully retrieved in the database, and the embodiments of this application are not limited thereto.

[0070] By employing the aforementioned methods, invalid data query requests can be identified and intercepted before data queries are performed in the preset storage area. This eliminates the need to query the preset storage area and the database, directly reporting the query failure result, thereby reducing the impact of invalid data query requests on the database and minimizing the unnecessary use of database resources.

[0071] In the above method, to reduce the database query load, multiple failed data query requests in the preset storage area can be merged into a new comprehensive data query request. This merged request reduces the database query load. The retrieved results can then be stored in the preset storage area for feedback.

[0072] This could involve merging different data query requests targeting the same data table into a single comprehensive data query request. This would allow for data queries on the same data table to be performed using a single comprehensive data query request, thereby improving data query efficiency.

[0073] In the above method, for the data sets corresponding to each data table in the preset storage area, the upper limit of the storage volume of each data set can be allocated. Specifically, the upper limit of the storage volume of each data set can be pre-allocated, or the upper limit of the storage volume of each data set can be adjusted periodically or in real time, or the upper limit of the storage volume of each data set can be dynamically adjusted.

[0074] Understandably, for data tables with high query frequency or high query volume, the upper limit of storage size for the corresponding dataset can be increased accordingly to improve the success rate of data retrieval in the preset storage area. For data tables with low query frequency or low query volume, the upper limit of storage size for the corresponding dataset can be decreased accordingly to improve the utilization rate of storage resources in the preset storage area.

[0075] In the above method, a validity period can be set for data stored from the database to the preset storage area, facilitating updates to the data in the preset storage area based on the validity period. The validity period can be set according to the data's query and modification frequency.

[0076] If the data is queried frequently, a longer validity period can be set to facilitate querying the data in the preset storage area. If the data is modified frequently, a shorter validity period can be set. For the same data stored in both the database and the preset storage area, if the data is modified in the database, the data in the preset storage area also needs to be modified accordingly. Therefore, a shorter validity period can be set to facilitate data modification.

[0077] Therefore, the effective duration of data in the preset storage area can be positively correlated with the query frequency; the effective duration of data in the preset storage area can be negatively correlated with the modification frequency.

[0078] It should be noted that the data query method and apparatus provided in the embodiments of this application can be applied to the fields of artificial intelligence technology and fintech. For example, for data query requests from financial institutions such as banks, the data query method provided in the embodiments of this application can be used to perform data queries. The data query method provided in the embodiments of this application can also be applied to any field other than fintech. For example, for multimedia data query requests in the audio and video field, the data query method provided in the embodiments of this application can also be used to perform data queries. The application fields of the data query method and apparatus provided in the embodiments of this application are not limited.

[0079] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0080] It's important to note that the term "neural network" can refer to a machine learning network based on deep learning. A neural network processes input and provides corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between them. Neural networks used in deep learning applications often include many hidden layers, increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer serves as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output becomes the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each processing the input from the layer above.

[0081] It should be understood that machine learning generally includes three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0082] Figure 1 The illustration shows an application scenario diagram of a data query method according to an embodiment of this application.

[0083] like Figure 1 As shown, application scenario 100 according to an embodiment of this application may include a first terminal device 101, a second terminal device 102, a database 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the database 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. For example, a user can use the first terminal device 101 and the second terminal device 102 to interact with the server 105 and / or the database 103 through the network 104 to receive or send information, etc.

[0084] The first terminal device 101 and the second terminal device 102 can be electronic devices such as smartphones, wearable devices, personal computers, intelligent voice interaction devices, smart home appliances, intelligent vehicles, in-vehicle terminals, aircraft, unmanned vending terminals, and extended reality devices. Extended reality devices can include virtual reality devices, augmented reality devices, and mixed reality devices. A client application for the target application can be installed and run on the terminal device. This target application can include, but is not limited to, financial transaction applications, payment applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, and social media platform software (these are just examples). Furthermore, this application embodiment does not limit the form of the target application, including but not limited to applications, mini-programs, etc., installed on the terminal device, and can also be in web page form.

[0085] Server 105 can be a server providing various services, such as a backend management server supporting websites browsed by users using the first terminal device 101 and the second terminal device 102 (this is just an example). The backend management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated based on user requests) to the terminal devices. Server 105 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Specifically, it can be a cloud server providing basic cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data. Server 105 can be the backend server for the aforementioned target application, used to provide backend services to the clients of the target application.

[0086] Database 103 is a professional storage system for storing and managing data. It can store various types of data related to the target application, such as user account information, business transaction records, and content resource data. It supports structured, semi-structured, or unstructured data storage and has management capabilities such as data creation, deletion, modification, querying, backup, and recovery. In this application scenario, database 103 can be connected to server 105 via a communication link. Server 105 can retrieve the required data from database 103 for processing based on requests from the first terminal device 101 and the second terminal device 102. It can also synchronously store new data generated by the operations of the first terminal device 101 and the second terminal device 102 into database 103, thereby achieving data persistence and efficient retrieval.

[0087] It should be noted that the data query method provided in this application embodiment can generally be executed by server 105 and / or terminal devices 101-102. Accordingly, the data query device provided in this application embodiment can generally be set in server 105 and / or terminal devices 101-102.

[0088] It should be understood that Figure 1 The number of terminal devices, networks, databases, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, databases, and servers can be included.

[0089] Figure 2 A flowchart illustrating a data query method according to an embodiment of this application is shown schematically. Figure 2 As shown, a data query method 200 according to an embodiment of this application may include steps S210 to S240.

[0090] The embodiments of this application do not limit the executing entity of a data query method. Optionally, a data query method can be applied to any electronic device or any software application, specifically to a server, client, or query application.

[0091] In step S210, a target data query request for the target database is determined; the target data query request is used to query the first data; the target database contains N data tables, where N is a positive integer; a preset storage area outside the target database is determined; the preset storage area stores N data sets, and different data sets correspond to different data tables in the N data tables, with each data set used to store data in the corresponding data table; the first data table to which the first data belongs is determined.

[0092] In step S220, according to the target data query request, the first data is queried from the data set corresponding to the first data table in the preset storage area.

[0093] In step S230, if the first data is found in the data set corresponding to the first data table in the preset storage area, the first data found is returned in response to the target data query request.

[0094] In step S240, if the first data is not found in the data set corresponding to the first data table in the preset storage area, the first data is queried in the first data table of the target database according to the target data query request.

[0095] This method can deploy a preset storage area outside the target database. Based on the data query request, the data query is first performed in the preset storage area. If the data to be queried is not found in the preset storage area, the data query is performed in the target database. In this way, the data query load on the target database can be distributed through the deployed preset storage area, reducing the number of data queries and the data query frequency on the target database, thereby improving the stability of the database.

[0096] Furthermore, by storing the data sets corresponding to each data table in the preset storage area, it is convenient to perform data queries in the corresponding data sets according to the data table to be queried, which can narrow the data query range in the preset storage area and improve the data query efficiency in the preset storage area.

[0097] The embodiments of this application do not limit the target database. Optionally, the target database can be any database, and the embodiments of this application do not limit the specific architecture or form of the target database. Optionally, the target database can be a distributed database, a relational database, a key-value database, etc.

[0098] Optionally, the target database can contain N tables to store data. Each table can contain one or more records, and different records can be distinguished by a primary key for easy database management.

[0099] The embodiments of this application do not limit the preset storage area. Optionally, the preset storage area can be any storage area or any storage space outside the target database. The embodiments of this application do not limit the storage medium form of the preset storage area, specifically it can be in the form of cache or disk.

[0100] In one specific example, the target database and the preset storage area can be deployed on the same device, with the preset storage area storing a portion of the data from the target database. For data query requests targeting the target database, the query can first be performed in the preset storage area. If the query is successful in the preset storage area, there's no need to query the target database again. This allows the preset storage area to offload the data query load on the target database, reducing its workload and improving database stability. Alternatively, the target database and the preset storage area can be deployed on different devices, intercepting data query requests targeting the target database and performing the query in the preset storage area of ​​another device.

[0101] Optionally, N data sets can be deployed in the preset storage area, and the number of data sets can be consistent with the number of data tables in the target database. Different data sets can correspond one-to-one with different data tables in the target database, and each data set can be used to store data from its corresponding data table.

[0102] In a specific example, the target database may contain 10 tables. Therefore, 10 data sets can be deployed in a pre-defined storage area, each corresponding to one of the 10 tables. Each data set can store a portion of the data from its corresponding table for data retrieval. Deploying different data sets based on the tables facilitates identifying the appropriate data set for the query request, narrowing the scope of the query and improving efficiency.

[0103] It is understandable that, in response to the addition of a data table in the target database, a corresponding data set can also be added in the preset storage area; in response to the deletion of a data table in the target database, a corresponding data set can also be deleted in the preset storage area; and in response to the modification of a data table in the target database, a corresponding data set can also be modified in the preset storage area.

[0104] The embodiments of this application do not limit the data storage method in the preset storage area. Optionally, the data storage method in the database can be adopted, using the primary key as an index for data storage, which can also facilitate data querying based on data query requests; or key-value pairs can be used for data storage, or information from data query requests can be used as an index for data storage.

[0105] For example, if the desired data is found in the target database for any data query request, the retrieved data can be stored in a preset storage area, and the information from the data query request can be used as an index to facilitate subsequent data queries for the same data request in the preset storage area. The data query request can specifically take the form of a database query statement, which can then be used as an index.

[0106] Optionally, data stored in a preset storage area can be updated. The preset storage area can store one or more data items from the target database, and each data set can store one or more data items from a corresponding data table. The data stored in the preset storage area can be updated according to actual needs.

[0107] For example, for any data query request, if the required data is not found in the preset storage area but is found in the target database, the queried data can be added to the preset storage area so that subsequent data query requests for the same data can find that data in the preset storage area.

[0108] This includes updating the data stored in the preset storage area, which can specifically include adding, deleting, and modifying the data stored in the preset storage area.

[0109] For example, data can be selected from the target database and added to a preset storage area to improve the success rate of data queries in the preset storage area; data that has not been queried for a long time or has become invalid can also be deleted from the preset storage area; or, in response to data modification operations in the target database, the same modifications can be performed on the same data in the preset storage area.

[0110] By updating the data in the preset storage area, the system can adapt to current query requirements, improve the success rate of data queries in the preset storage area, increase the overall efficiency of data queries, and enhance the stability of the database.

[0111] In addition, the storage capacity limit of multiple data sets in the preset storage area can be allocated and adjusted to facilitate data management in the preset storage area.

[0112] For a detailed explanation of the preset storage area, please refer to the explanations in other embodiments.

[0113] The embodiments of this application are not limited to data query requests. Optionally, a data query request may specifically be a data query statement, which may be performed using a query statement, such as a database query statement; a data query request may also be performed using a network request.

[0114] In a specific example, a data query request may include a query scope, primary key values, and query objects. For example, the query scope may be the target database or a specific table within the target database. The query objects may be one or more records corresponding to the primary key values, or the values ​​of certain fields corresponding to the primary key values.

[0115] For example, a product data table in the target database can be used to store data about product entities. This table can include a primary key for the product data, "Product Identifier," as well as multiple fields for the product data, such as "Product Name," "Product Batch," and "Product Description." The query scope in a data query request can be the product data table, the primary key value can be a specific product identifier (e.g., 598), and the query object can be the value of the "Product Description" field in the product data corresponding to identifier 598.

[0116] The embodiments of this application do not limit the target data query request. Optionally, the target data query request can be any data query request, specifically any data query request targeting a target database. The data query request targeting a target database can be a data query request sent to the target database, or it can be a data query request used to request data to be retrieved from the target database.

[0117] The embodiments of this application do not limit the method of determining the target data query request. Optionally, any data query request sent to the target database can be determined as the target data query request, or the query scope of any data query request can be determined. If the query scope includes the target database, then the data query request is determined as the target data query request. That is, any data query request whose query scope includes the target database is determined as the target data query request. Optionally, the executing entity of the above method flow can intercept the target data query request and execute the above method flow, first performing a data query in a preset storage area according to the target data query request.

[0118] In a specific example, the server could intercept a target data query request sent to the target database and first perform a data query in a preset storage area. If the query fails in the preset storage area, then the data query can be performed in the target database.

[0119] For ease of description, the data that the target data query request needs to retrieve can be referred to as the first data. Optionally, a single target data query request can be used to query one or more first data. Different first data can belong to the same data table or different data tables. Accordingly, for each piece of first data, the data table to which each piece of first data belongs can be determined (they can all be referred to as the first data table). For any piece of first data and the first data table to which it belongs, the corresponding step S220 can be executed to perform a data query in a preset storage area. For each piece of first data, the first data table to which it belongs can be determined, and the existence of the targeted first data can be checked in the data set corresponding to the determined first data table in the preset storage area.

[0120] The above method flow mainly explains the data query process for a single target data query request. It can be understood that for different data query requests, they can be identified as target data query requests, and the above method flow can be executed to complete the data query. For a detailed explanation, please refer to the explanation of the above method flow.

[0121] The embodiments of this application do not limit the specific method of determining the first data table to which the first data belongs. Optionally, the first data table to which the first data belongs can be determined according to the target data query request. Specifically, it can be determined according to the query range in the target data query request, identifying the data table within the query range as the first data table; or it can be determined according to the information in the target data query request, specifically, it can be determined according to the primary key field information in the target data query request, identifying the data table containing the primary key field information as the first data table; of course, if it is difficult to determine the first data table according to the target data query request, all N data tables in the target database can be directly identified as the first data table.

[0122] Therefore, optionally, determining the first data table to which the first data belongs may specifically include at least one of the following: (1) determining the first data table to which the first data belongs based on the target data query request; (2) determining the data table included in the query range of the target data query request as the first data table; (3) determining the data table whose primary key field information is the same as the primary key field information in the target data query request as the first data table.

[0123] The embodiments of this application do not limit the overall data query process. Optionally, a data query can first be performed in a preset storage area based on the target data query request. If the query fails in the preset storage area, the data query can continue in the target database. The preset storage area can store data from the target database, and the amount of data stored in the preset storage area is less than the amount of data stored in the target database.

[0124] The embodiments of this application are not limited to the specific process of querying data in the target database according to the target data query request. Optionally, the target data query request can be directly forwarded to the target database for data query, or the executing entity of the above method process can reconstruct the data query request for querying the first data and send it to the target database for data query. Optionally, if the query in the target database is successful, the first data found can be returned accordingly; if the query in the target database fails, a result indicating the query failure can be returned accordingly.

[0125] Optionally, based on the target data query request, the first data is queried in the first data table of the target database. Specifically, this may include: if the first data is found in the first data table of the target database, storing the queried first data in the data set corresponding to the first data table, and providing feedback on the found first data in response to the target data query request; if the first data is not found in the first data table of the target database, providing feedback on the target data query request to indicate that the query for the first data failed. This embodiment allows data querying to be performed in the first data table of the target database, which can narrow the query scope to the first data table, improve data query efficiency, and provide corresponding feedback based on different data query results, thereby improving the stability of the data query process.

[0126] Optionally, if the first data is found in the first data table of the target database, the found first data can be stored in the data set corresponding to the first data table. The retrieved first data can then be retrieved from the data set corresponding to the first data table in response to the target data query request. This transforms the data query on the target database into a task of pulling data from the target database to a preset storage area. In other words, once the retrieved first data is stored in the data set corresponding to the first data table, the data query process in the target database can end, confirming that the preset storage area has successfully stored the retrieved first data. There is no need to wait for feedback on the success of the target data query request, thereby reducing the load on the target database and improving its stability.

[0127] Optionally, if the first data is not found in the first data table of the target database, a result indicating the failure of the first data query can be returned for the target data query request. Specifically, the result indicating the failure of the first data query can be returned directly to the initiator of the target data query request, or the result indicating the failure of the first data query can be stored in a preset storage area, and the result indicating the failure of the first data query can be returned for the target data query request from the preset storage area.

[0128] Optionally, the results indicating the failure of querying the first set of data, along with information about the target data query request, can be stored in a preset storage area. The target data query request information can serve as an index, allowing direct retrieval of the results indicating the failure of querying the first set of data for the same data query request. Specifically, the target data query request information can be in the form of a query statement, or it can include primary key values ​​and information about the query object.

[0129] Alternatively, a set of failed data query requests can be deployed to store the target data query requests that failed to be queried in the target database.

[0130] Accordingly, for any new data query request, it can be first determined whether the target data query request is included in the set of failed requests. If the target data query request is included in the set of failed requests, a result indicating that the data query failed can be directly fed back. If the target data query request is not included in the set of failed requests, the corresponding data query process can be executed.

[0131] The embodiments of this application do not limit the storage location of the failed request set. Specifically, it can be stored in a preset storage area or in other storage areas.

[0132] The embodiments of this application are not limited to the specific form of the result used to characterize the failure of querying the first data. Optionally, a specified identifier can be used as the result used to characterize the failure of querying data. For example, null can be used as the result used to characterize the failure of querying data. In a preset storage area, information about data query requests that failed to query the target database (e.g., data query statements) and the corresponding null query results can be stored to characterize the failure of data queries.

[0133] In an optional embodiment, in order to reduce the data query load of the target database, multiple data query requests that failed to be queried in a preset storage area can be merged to construct a new comprehensive data query request and then query the target database.

[0134] The embodiments of this application do not limit the specific method of merging data query requests. Specifically, it can merge the query ranges between different data query requests and add the primary key value and the query object to the comprehensive data query request.

[0135] It is understandable that for different data query requests with the same query scope, the same primary key value, and the same query object, merging the data query requests can reduce the data query load on the target database.

[0136] Optionally, different data query requests for querying the same data table can be merged, thereby reducing the data query load for the same data table.

[0137] In a specific example, one data query request can be used to query records 1 to 10 in data table 1, and another data query request can be used to query records 2 to 40 in data table 1. These two data query requests are merged to obtain a combined data query request that can be used to query records 1 to 40 in data table 1. It is understandable that by merging data query requests, the data query load on the target database can be reduced.

[0138] Optionally, based on the target data query request, the first data is queried in the first data table of the target database. Specifically, this may include: combining the target data query request and other data query requests for querying other data in the first data table to generate a comprehensive data query request; and based on the comprehensive data query request, querying the first data and other data in the first data table of the target database; the other data is not stored in the data set corresponding to the first data table. This embodiment can reduce the number of data queries against the target database, reduce the data query load of the target database, improve the stability of the target database, and improve the efficiency of data query by merging different data query requests that query the same data table.

[0139] It is understandable that if a target data query request fails to retrieve data in the preset storage area, and other data is not stored in the data set corresponding to the first data table, then other data query requests will also fail to retrieve data in the preset storage area. Thus, different data query requests that failed to retrieve data in the preset storage area can be combined to obtain a new comprehensive data query request, which can reduce the number of queries against the first data table.

[0140] It should be noted that by merging data query requests, at least the different data query requests that require the same data can be simplified, and the different data query requests that require overlapping data can also be simplified, thereby improving the efficiency of data query.

[0141] For comprehensive data query requests, the query results in the target database can be stored in a preset storage area, and the results can be fed back for different comprehensive data query requests respectively. For a detailed explanation, please refer to the explanations in other embodiments.

[0142] In one optional embodiment, when performing a specific data query in a preset storage area, the data query can be performed according to the specific data index format in the preset storage area.

[0143] In a specific example, the data stored in the preset storage area can use the primary key value as an index. Accordingly, when performing a data query based on the target data query request, the primary key value in the target data query request can be matched with the primary key value index in the preset storage area to achieve the data query.

[0144] In another specific example, data can be stored in the preset storage area in the form of key-value pairs. Accordingly, when performing a data query based on a target data query request, the primary key value in the target data query request can be matched with the key-value pairs in the preset storage area to achieve the data query.

[0145] Of course, you can also use the query statement or query request as an index to store the data in a preset storage area.

[0146] In a specific example, a comprehensive caching fallback mechanism can be provided, adding a caching fallback strategy before database access, and implementing unified processing strategies for cache lookup, cache addition, cache cleanup, and cache breakdown prevention. The cache here can correspond to a preset storage area in other embodiments.

[0147] The specific process may include: (1) Database script collection and processing. Data query scripts (data query statements) accessing the database from the data layer can be intercepted; the data query scripts are processed into unique and fixed-length cache indexes. Because data query scripts vary in length, and some are quite long, a hash algorithm can be used to hash the data query scripts to obtain a fixed-length hash value. To further reduce collisions, the first T bits of the data query script are truncated and concatenated with the aforementioned hash value to obtain a fixed-length and unique cache index. T can be a positive integer.

[0148] (2) Cache processing logic. Based on the cache index obtained in (1), query the cache. If the cache contains the corresponding value, return the queried value directly without querying the database. If the cache does not contain the corresponding value, the database can be accessed to query the data, and the retrieved data can be written into the cache and returned. If there is no value in the database either, write an anti-cache penetration value into the cache. The anti-cache penetration value can be a fixed identifier used to indicate that the data query failed. When writing data into the database in the cache, an expiration time can be set. An additional random value can be set for the expiration time to reduce the probability of multiple cached data expiring at the same time.

[0149] Regarding the data query process described above, the embodiments of this application do not limit the specific method of returning the query results. Optionally, the query results can be returned to the party that initiated the target data query request, or the query results can be displayed for the target data query request.

[0150] The above embodiments explain the data query process. However, when querying a target database, query failures may occur. For ease of description, a data query request that fails to retrieve data from the target database can be referred to as an invalid data query request.

[0151] Invalid data query requests may occur due to errors in the query information, such as incorrect primary key values ​​or incorrect query ranges. Alternatively, attackers may send a large number of invalid data query requests to target the database, consuming its resources and reducing its stability.

[0152] Understandably, an increase in the number of invalid data query requests can reduce the stability of the target database and increase its query load. Therefore, it is advisable to identify and intercept invalid data query requests as early as possible before querying the target database, thereby reducing the data query failure rate and improving the stability of the target database.

[0153] The embodiments of this application do not limit the method of identifying invalid data query requests.

[0154] Optionally, information from the target database can be used for identification. For example, if the first data table requested by the target data query request does not exist in the target database, it can be directly identified as an invalid data query request. Similarly, if the primary key value requested by the target data query request does not exist in the first data table, or if the primary key field information requested does not exist in the target database, it can be directly identified as an invalid data query request. Furthermore, multiple invalid data query requests can be analyzed to determine common or frequently occurring characteristics among them. For example, multiple invalid data query requests may be initiated by the same user, or they may be querying the same data, or they may contain the same identifier information. Filtering rules can be constructed using these determined characteristics to identify data query requests that match the filtering rules as invalid data query requests. Specifically, the data query requests in the failed request set in the above embodiments can be analyzed to construct filtering rules.

[0155] The embodiments of this application do not limit the specific timing of identifying invalid data query requests. Optionally, invalid data query requests can be identified in a preset storage area. Specifically, this can be done before performing a data query in the preset storage area, identifying whether the target data query request is invalid. Of course, invalid data query requests can also be identified when the query fails in the preset storage area, that is, when the first data is not found in the preset storage area.

[0156] Optionally, invalid data query requests can be intercepted. The embodiments of this application do not limit the specific interception process; it may involve not providing a result, or it may directly provide a result indicating that the data query failed.

[0157] In one optional embodiment, according to the target data query request, the first data is queried in the data set corresponding to the first data table within a preset storage area. Specifically, this may include at least one of the following: (1) if it is determined that the first data table belongs to N data tables, the first data is queried in the data set corresponding to the first data table within the preset storage area according to the target data query request; (2) if it is determined that the first data table does not belong to N data tables, a result indicating the failure to query the first data is fed back to the target data query request. This embodiment can identify invalid data query requests by determining whether the first data table belongs to a data table in the target database, thereby reducing invalid data queries against the target database, reducing the data query load of the target database, and improving the stability of the target database.

[0158] The embodiments of this application do not limit the specific method for determining whether the first data table belongs to the N data tables in the target database. Optionally, data table identification information of the N data tables in the target database can be stored in a preset storage area, and further, the existence of data table identification information of the first data table can be determined from the data table identification information of the N data tables.

[0159] It is understandable that if the table identifier information of the first table is not found among the table identifier information of the N tables, it can be determined that the first table does not belong to the N tables in the target database, and the target data query request can be determined to be an invalid data query request.

[0160] In another optional embodiment, a preset storage area can be used to store the value range of primary keys in N data tables; according to the target data query request, the first data is queried in the data set corresponding to the first data table in the preset storage area, which may specifically include at least one of the following: (1) if it is determined that the primary key value in the target data query request belongs to the value range of the primary key in the first data table, the first data is queried in the data set corresponding to the first data table in the preset storage area according to the target data query request; (2) if it is determined that the primary key value in the target data query request does not belong to the value range of the primary key in the first data table, a result is fed back to the target data query request to indicate that the query for the first data failed. This embodiment can identify invalid data query requests by judging whether the primary key value in the target data query request is within the value range of the primary key in the first data table, which can reduce invalid data queries for the target database, reduce the data query load of the target database, and improve the stability of the target database.

[0161] The embodiments of this application do not limit the range of primary key values ​​in the N data tables of the target database. Optionally, for any data table in the target database, all values ​​of the primary key in the target data table can be added to the primary key value set corresponding to the target data table, and the primary key value set can be stored in a preset storage area. This facilitates identification of whether the primary key value in the target data query request is in the primary key value set corresponding to the first data table, thereby improving identification accuracy. For any data table in the target database, the maximum and minimum values ​​of the primary key values ​​in the target data table can be used as the primary key value range corresponding to the target data table. This facilitates identification of whether the primary key value in the target data query request exceeds the primary key value range corresponding to the first data table. By using the maximum and minimum values, the security of the primary key values ​​can be improved, and the storage resources occupied by the primary key value range in the preset storage area can be reduced. The maximum value of the primary key in the same data table can be greater than the minimum value of the primary key.

[0162] Specifically, for N tables in the target database, the primary key value ranges can be stored in preset storage areas. The specific storage method is not limited; it can be that for any table, the corresponding primary key value range is stored in the corresponding data set, facilitating comparison by querying the corresponding primary key value range within the data set.

[0163] Understandably, if it is determined that the primary key value in the target data query request is not within the range of the primary key values ​​corresponding to the first data table, then even in the data set corresponding to the first data table and the first data table in the target database, the same primary key value cannot be found. Therefore, the target data query request can be determined as an invalid data query request, and the query failure result can be directly returned without querying the preset storage area and the target database.

[0164] In a specific example, the range of primary key values ​​corresponding to the first data table can include the maximum and minimum primary key values. It can be determined whether the primary key value in the target data query request is greater than the minimum primary key value and less than the maximum primary key value in the first data table. If the primary key value in the target data query request is less than the minimum primary key value or greater than the maximum primary key value in the first data table, then the target data query request can be determined to be an invalid data query request.

[0165] In another alternative embodiment, filtering rules can be constructed to filter the received data query requests, and data query requests that meet the filtering rules can be identified as invalid data query requests.

[0166] The embodiments of this application do not limit the filtering rules. For example, the filtering rules may be that the data query request contains specified information, or that the initiator of the data query request is a specified user, or that the data query request is used to query specified data.

[0167] The embodiments of this application do not limit the specific construction method of the filtering rules. Optionally, for multiple invalid data query requests, common features or frequently occurring features can be determined, and filtering rules can be constructed based on the determined features; alternatively, a machine learning model can be used to generate filtering rules based on invalid data query requests, specifically, a large model can be used to generate filtering rules.

[0168] The embodiments of this application do not limit the timing of constructing filtering rules. Optionally, filtering rules can be constructed in real time based on the identified invalid data query requests; filtering rules can also be constructed when there is an abnormal increase in invalid data query requests; or filtering rules can be constructed periodically based on newly added invalid data query requests.

[0169] In an optional embodiment, the above method flow may further include: if the number M of M data query requests for which no data was found in the target database within a preset time period is greater than a preset request number threshold, generating filtering rules based on the M data query requests; M is a positive integer; and for any new data query request that conforms to any of the generated filtering rules, providing feedback to characterize the failure of the data query. This embodiment can generate filtering rules when the number of invalid data query requests exceeds a threshold within a preset time period, thereby identifying new invalid data query requests. This can improve the accuracy and real-time performance of identifying invalid data query requests, reduce invalid data queries against the target database, lower the data query load on the target database, and improve the stability of the target database.

[0170] The embodiments of this application are not limited to a preset time period. Optionally, the preset time period may be a time period of a specified duration before the current time, or any other time period.

[0171] The embodiments of this application are not limited to the method of generating filtering rules based on M data query requests. Optionally, filtering rules can be generated using machine learning models such as large models, based on common features among the M data query requests or features with a proportion greater than a threshold.

[0172] Optionally, filtering rules can be pre-built. Accordingly, for a target data query request, it can be determined whether it meets any of the filtering rules. For a target data query request that meets any of the filtering rules, a result can be returned to indicate that the data query failed.

[0173] The above embodiments explain the identification of invalid data query requests. It is understood that different embodiments can be combined to comprehensively identify invalid data query requests through multiple methods. For example, it can first determine whether the first data table belongs to N data tables in the target database. If it is determined that the first data table belongs to N data tables in the target database, the corresponding primary key value range can be used for identification. Alternatively, data query requests can be filtered using filtering rules first, and then identification can be performed based on the data table and the primary key value range. The embodiments of this application do not limit the combination of different embodiments.

[0174] The following is a detailed explanation of the preset storage area.

[0175] The embodiments of this application are not limited to the specific method of updating data stored in the preset storage area. Optionally, data in the target database stored in the preset storage area can be added, deleted, or modified. This can be done to improve the success rate of data queries in the preset storage area or to improve the utilization rate of storage resources in the preset storage area.

[0176] For example, you can add data that is frequently queried or has a high probability of being queried in the target database to the preset storage area, or you can delete data that has not been queried for a long time in the preset storage area, or you can perform the same modification on the same data in the preset storage area in response to data modifications in the target database in order to achieve data consistency.

[0177] Optionally, the data update method of the preset storage area may include at least one of the following: (1) the data stored in the preset storage area has an expiration date, and data that has exceeded its expiration date is deleted in the preset storage area; (2) any of the N data sets has a storage limit, and for any of the N data sets, if the proportion of the currently stored data in the corresponding storage limit exceeds a preset proportion threshold, data is selected from the targeted data set for deletion; (3) in the preset storage area, data with a query frequency lower than a preset idle frequency threshold is selected for deletion; (4) data is selected from the target database and stored in the preset storage area; (5) in response to modification of the second data in the target database, the modified second data is synchronized to the second data stored in the preset storage area. This embodiment can update the data stored in the preset storage area, which can improve the utilization rate of storage resources in the preset storage area, improve the data consistency between the preset storage area and the target database, and improve the efficiency of data query.

[0178] The embodiments of this application do not limit the specific method of deleting data in a preset storage area. Optionally, since the storage resources in the preset storage area are limited, some data can be selected for deletion to save storage resources and improve storage resource utilization. The specific method of selecting the data to be deleted is not limited; it can be based on the data query frequency or the data query probability. For example, data in the preset storage area with a data query frequency or a data query probability lower than a threshold can be selected for deletion, or it can be based on the data's validity period.

[0179] Accordingly, if the remaining storage in the preset storage area is insufficient, some data can be selected for deletion. For example, data can be sorted from high to low according to its query frequency, and data ranked lower can be deleted. Specifically, data ranked higher than D can be deleted, where D can be a positive integer. Specifically, data can be deleted when the proportion of the currently stored data in the preset storage area to the maximum storage capacity of the preset storage area exceeds a threshold. Alternatively, the data can be divided into N data sets, each allocated a maximum storage capacity. If the proportion of the currently stored data in any data set to the corresponding maximum storage capacity exceeds a threshold, data can be selected from that data set for deletion to save storage resources. The threshold for the proportion can be 95% or 98%, etc., and the embodiments of this application do not limit the preset proportion threshold. Alternatively, data can be selected from any data set when the currently stored data in any data set equals the corresponding maximum storage capacity.

[0180] Furthermore, the embodiments of this application are not limited to the specific method of setting storage capacity limits for each of the N data sets. By setting storage capacity limits for each of the N data sets in a preset storage area, it is possible to conveniently manage the N data sets and improve the utilization rate of storage resources. Moreover, different storage capacity limits can be flexibly and dynamically set for different data sets to adapt to specific data query needs.

[0181] In one optional embodiment, the N data sets can each correspond to N data tables in the target database, thereby determining the upper limit of storage capacity for the corresponding data set based on the relevant information of the data tables. For example, if the data in a data table is queried frequently or with a high probability of being queried, a larger upper limit of storage capacity can be allocated to the data set corresponding to that data table, so as to store more data from that data table in the data set, thereby improving the success rate of data retrieval in the preset storage area and improving the efficiency of data retrieval.

[0182] Therefore, optionally, the above method flow may further include: for any one of the N data tables, determining the upper limit of the storage volume of the data set corresponding to the targeted data table in the preset storage area based on at least one of the following information: (1) the data query frequency of the targeted data table; (2) the data query frequency of the data set corresponding to the targeted data table. Wherein, the determined upper limit of storage volume is positively correlated with the data query frequency of the targeted data table, and the determined upper limit of storage volume is positively correlated with the data query frequency of the data set corresponding to the targeted data table. This embodiment can determine the upper limit of the storage volume of the data set based on the data query frequency, which can improve the accuracy of the upper limit of the data set storage volume, improve the adaptability of the upper limit of storage volume to data query needs, improve the success rate of data query in the preset storage area, and improve the efficiency of data query.

[0183] Optionally, the future query probability of the data table and the future query probability of the data set can also be considered. For any of the N data tables, the upper limit of the storage volume of the data set corresponding to the data table in the preset storage area can be determined based on at least one of the following: (1) the data query frequency of the data table; (2) the data query frequency of the data set corresponding to the data table; (3) the future data query probability of the data table; (4) the future data query probability of the data set corresponding to the data table. The determined upper limit of storage volume is positively correlated with the future data query probability of the data table and the future data query probability of the data set corresponding to the data table.

[0184] The embodiments of this application do not limit the data query frequency; specifically, the data query frequency can be determined comprehensively based on the data query frequency of historical periods. Optionally, the storage limit of the corresponding data set can be determined based on the average data query frequency of the data table within a historical period or other statistical values; alternatively, the future data query frequency of future periods can be predicted based on the data query frequency of the data table within a historical period to determine the storage limit of the corresponding data set.

[0185] The embodiments of this application do not limit the probability of future data queries. Optionally, the probability of future data queries can be predicted using a machine learning model to determine the upper limit of the storage capacity of the dataset. For example, based on the data query frequency of the data table in a historical period (which can be in the form of a time series), the probability of future data queries for the data table in a future period can be predicted and determined using a machine learning model. For a more detailed explanation, please refer to other embodiments.

[0186] The embodiments of this application do not limit the method of adding data to the preset storage area. Optionally, a portion of data may be selected from the target database and stored in the preset storage area. Specifically, data with high query frequency or high query probability may be selected and stored in the preset storage area to improve the success rate of data query in the preset storage area and improve the efficiency of data query.

[0187] The embodiments of this application do not limit the timing of selecting data from the target database to the preset storage area. Specifically, the data can be selected from the target database to the preset storage area when initializing the preset storage area, or it can be selected periodically. Alternatively, it can be selected based on actual query needs, such as when the number of query failures in the preset storage area exceeds a threshold, or when the query failure rate in the preset storage area exceeds a threshold.

[0188] The embodiments of this application do not limit the specific data selection method for selecting data from the target database and storing it in the preset storage area. Optionally, data can be randomly selected from the target database, or data with a query frequency higher than a threshold can be selected from the target database, or data with a query probability higher than a threshold can be selected from the target database based on N data tables, and then data can be selected from the selected data tables.

[0189] Optionally, the above method may further include: based on a pre-trained query probability prediction model, predicting the future data query probability for each of the N data tables; and selecting data from data tables whose future data query probability is greater than a preset query probability threshold and storing the data in the corresponding data set. This embodiment can select data from data tables whose future data query probability is greater than the threshold and store it in a preset storage area, which can improve the success rate of data query in the preset storage area and improve the efficiency of data query.

[0190] The embodiments of this application are not limited to a specific method of selecting data from a data table and storing it in a corresponding data set. Optionally, data from a data table whose future query probability is greater than a preset query probability threshold can be randomly selected and stored in the corresponding data set. Alternatively, data with a query frequency higher than a threshold can be selected and stored in the corresponding data set. Furthermore, the future query probability of each data item in the data table can be predicted, and data with a future query probability higher than a threshold can be selected and stored in the corresponding data set. Alternatively, data with a strong correlation higher than a threshold can be selected and stored in the corresponding data set based on the relationships between data items in the data table. The embodiments of this application are not limited to a method of predicting the future query probability for each data item in the data table. Optionally, a machine learning model can be used to predict the future query probability based on the historical query frequency of the data. For a detailed explanation, please refer to the explanation of query probability prediction models.

[0191] The embodiments of this application do not limit the preset query probability threshold. Optionally, the preset query probability threshold can be a fixed value or a dynamically adjustable value. For example, N data tables can be sorted from high to low according to the future data query probability, and the future data query probability of the Rth data table can be determined as the preset query probability threshold, where R can be a positive integer.

[0192] The embodiments of this application do not limit the query probability prediction model, nor do they limit the input features and training methods of the query probability prediction model.

[0193] Optionally, the query probability prediction model can be used to predict the future data query probability of a data table. It can be based on the historical data query frequency of the data table, specifically in the form of a time series. By analyzing the historical data query frequency time series of the data table, the historical trend of the historical data query frequency can be analyzed, thereby predicting the future data query probability of the data table in the future period.

[0194] Since the data set corresponding to the data table also stores the data from the data table, the probability of future data queries on the data table can be predicted by combining the historical data query frequency of the data set.

[0195] Furthermore, considering data query requests that fail to retrieve data from the dataset but succeed in retrieving data from a table, the data that needs to be added to a preset storage area can be directly determined based on these requests, thereby improving the success rate of data retrieval in the preset storage area. Therefore, information from these data query requests can also be used to predict the future data query probability of a table. Specifically, the distribution of data tables queried by these requests can be used to predict the future data query probability of a table.

[0196] For example, if we identify a number of recent data query requests that failed to retrieve data from the dataset but succeeded in retrieving data from the table, and determine that a significant portion of these requests are for data table 1, then we can conclude that there is a high demand for data table 1 and therefore a higher probability of future data queries for data table 1.

[0197] Therefore, optionally, based on a pre-trained query probability prediction model, the future data query probability is predicted for each of the N data tables. Specifically, this can include: based on the pre-trained query probability prediction model, predicting the future data query probability for each of the N data tables according to at least one of the following: (1) the historical data query frequency of the data set corresponding to the targeted data table; (2) the historical data query frequency of the targeted data table; (3) historical data query requests where no data was found in the data set corresponding to the targeted data table, but data was found in the targeted data table of the target database. This embodiment can combine multiple information to predict the future data query probability of a data table, which can improve the accuracy of the future data query probability of the data table, facilitate the improvement of the success rate of data query in the preset storage area, and improve the efficiency of data query.

[0198] The embodiments of this application do not limit the form of historical data query frequency; specifically, a time series format can be used to facilitate the analysis of trends in historical data query frequency. The embodiments of this application do not limit how the aforementioned historical data query requests are used to predict future data query probabilities; specifically, information about the data tables to be queried in the aforementioned historical data query requests, such as statistical information, can be extracted to predict future data query frequencies. It is understood that in the data tables to be queried in the aforementioned historical data query requests, the frequency of occurrence of the data table is positively correlated with the future data query probability of the data table.

[0199] The embodiments of this application do not limit the specific training method of the query probability prediction model. Optionally, the historical data query frequency of the data table can be used to construct sample features and sample labels for multiple training samples. For example, the data query probability can be determined as a sample label based on the data query frequency within a historical period, and the data query frequency within the period prior to that historical period can be determined as a sample feature. The query probability prediction model can be trained using the constructed training samples. Specifically, determining the data query probability based on the data query frequency can be done by normalizing the data query frequency to obtain the data query probability, and the data query frequency can be positively correlated with the data query probability.

[0200] The embodiments of this application are not limited to the specific method of modifying data in the preset storage area. Optionally, in response to modification of second data in the target database, the modified second data can be synchronized to the second data stored in the preset storage area. Specifically, for the same data (referred to as the second data for convenience) between the target database and the preset storage area, in response to modification of the second data in the target database, the same modification can be performed on the second data in the preset storage area, thereby improving data consistency between the target database and the preset storage area. For example, in conjunction with a distributed transaction in the target database, for a distributed transaction modifying the second data, the sub-transactions can include: the second data in the target database being modified, and the second data in the preset storage area being modified, so that the distributed transaction can be successfully committed after the second data in both the target database and the preset storage area have been modified in the same way.

[0201] In a specific example, the operation logs of the target database can be monitored to capture data table change events in real time. When data is modified, the same data in the preset storage area (cache) can be invalidated immediately, or the same modification can be performed directly on the same data in the preset storage area.

[0202] In one optional embodiment, the data in the preset storage area may have an expiration date, which can be extended according to information such as the frequency of data queries in the preset storage area. By setting an expiration date, the data in the preset storage area can be managed more conveniently.

[0203] When setting the validity period of data in the preset storage area, various criteria can be used to set it, and the embodiments of this application do not limit the specific way to set the validity period. For example, for data with high query frequency, a longer validity period can be set to facilitate successful querying of the data in the preset storage area within the validity period; for data with high modification frequency, a shorter validity period can be set to adapt to the data modification frequency, and real-time data can be obtained from the target database when the data needs to be queried.

[0204] Optionally, the above method may further include: when storing third-party data from the target database in a preset storage area, determining the effective storage duration of the third-party data based on the data query frequency and / or data modification frequency; the determined effective storage duration is positively correlated with the data query frequency; and negatively correlated with the data modification frequency. This embodiment can determine the effective storage duration of data in the preset storage area based on the data query frequency and data modification frequency of the data in the target database, which facilitates the management of data in the preset storage area and improves the accuracy and flexibility of setting the effective storage duration.

[0205] In a specific example, when initializing the preset storage area, the historical data query frequency and historical data modification frequency can be determined based on the historical logs of the target database. This allows for the selection of data to be stored in the preset storage area, along with the required validity period / storage validity duration. For instance, for data with a high historical data query frequency and a low historical data modification frequency, a storage validity duration of 1 hour can be set; for data with a historical data query frequency of 0 and a high historical data modification frequency, a storage validity duration of 1 minute can be set, or the data can be excluded from the preset storage area altogether.

[0206] Based on the above method embodiments, embodiments of this application also provide a data query device. The following will be combined with... Figure 3 The device is described in detail.

[0207] Figure 3 The diagram illustrates a structural block diagram of a data query device according to an embodiment of this application.

[0208] like Figure 3 As shown, the data query device 300 provided in this embodiment includes: a determination module 310, a first query module 320, a feedback module 330, and a second query module 340.

[0209] The determining module 310 is used to determine a target data query request for a target database; the target data query request is used to query first data; the target database contains N data tables, where N is a positive integer; a preset storage area outside the target database is determined; the preset storage area stores N data sets, each data set corresponding to a different data table in the N data tables, and each data set is used to store data in the corresponding data table; the first data table to which the first data belongs is determined. In one embodiment, the determining module 310 can be used to execute step S210 and related steps described above, which will not be repeated here.

[0210] The first query module 320 is used to query the first data in the data set corresponding to the first data table within a preset storage area according to the target data query request. In one embodiment, the first query module 320 can be used to execute the steps S220 and related steps described above, which will not be repeated here.

[0211] The feedback module 330 is used to provide feedback on the first data found in the data set corresponding to the first data table within a preset storage area, in response to the target data query request. In one embodiment, the feedback module 330 can be used to execute the steps S230 and related steps described above, which will not be repeated here.

[0212] The second query module 340 is used to query the first data in the first data table of the target database according to the target data query request when the first data is not found in the data set corresponding to the first data table in the preset storage area. In one embodiment, the second query module 340 can be used to execute the steps S240 and related steps described above, which will not be repeated here.

[0213] According to an embodiment of this application, the second query module 340 is configured to: if first data is found in the first data table of the target database, store the found first data in the data set corresponding to the first data table, and provide feedback on the found first data in response to the target data query request; if first data is not found in the first data table of the target database, provide feedback on the result indicating that the query for first data failed in response to the target data query request.

[0214] According to an embodiment of this application, the first query module 320 is configured to: when it is determined that the first data table belongs to N data tables, query the first data in the data set corresponding to the first data table in a preset storage area according to the target data query request; when it is determined that the first data table does not belong to N data tables, provide feedback on the target data query request to indicate that the query for the first data failed.

[0215] According to an embodiment of this application, a preset storage area is used to store the value range of primary keys in N data tables; the first query module 320 is used to: when it is determined that the primary key value in the target data query request belongs to the value range of the primary key in the first data table, query the first data in the data set corresponding to the first data table in the preset storage area according to the target data query request; when it is determined that the primary key value in the target data query request does not belong to the value range of the primary key in the first data table, provide feedback on the target data query request to indicate that the query for the first data failed.

[0216] According to an embodiment of this application, the above-described apparatus may further include an additional module for: predicting the future data query probability for each of the N data tables based on a pre-trained query probability prediction model; and selecting data from the data tables whose future data query probability is greater than a preset query probability threshold and storing the data in the corresponding data set.

[0217] According to an embodiment of this application, the added module can be specifically used to: based on a pre-trained query probability prediction model, predict the future data query probability for N data tables according to at least one of the following: the historical data query frequency of the data set corresponding to the targeted data table; the historical data query frequency of the targeted data table; and historical data query requests where no data was found in the data set corresponding to the targeted data table, but data was found in the targeted data table of the target database.

[0218] According to an embodiment of this application, the above-mentioned device may further include a filtering module, configured to: generate filtering rules based on the M data query requests when the number M of M data query requests that have not been found in the target database within a preset time period is greater than a preset request number threshold; M is a positive integer; and for any new data query request that conforms to any of the generated filtering rules, provide feedback to indicate that the data query has failed.

[0219] According to an embodiment of this application, the second query module 340 is used to: generate a comprehensive data query request by combining a target data query request and other data query requests for querying other data in the first data table; and query the first data and other data in the first data table of the target database according to the comprehensive data query request; the other data is not stored in the data set corresponding to the first data table.

[0220] According to an embodiment of this application, the above-described apparatus may further include an allocation module, configured to: for any one of the N data tables, determine the upper limit of the storage volume of the data set corresponding to the targeted data table in a preset storage area based on at least one of the following information: the data query frequency of the targeted data table; the data query frequency of the data set corresponding to the targeted data table; wherein the determined upper limit of storage volume is positively correlated with the data query frequency of the targeted data table, and the determined upper limit of storage volume is positively correlated with the data query frequency of the data set corresponding to the targeted data table.

[0221] According to embodiments of this application, the data update method for the preset storage area includes at least one of the following: (1) Data stored in the preset storage area has an expiration date, and data exceeding the expiration date is deleted in the preset storage area; (2) Any one of the N data sets has a storage limit, and for any one of the N data sets, if the proportion of the currently stored data in the corresponding storage limit exceeds a preset proportion threshold, data is selected from the targeted data set for deletion; (3) In the preset storage area, data with a query frequency lower than a preset idle frequency threshold is selected for deletion; (4) Data is selected from the target database and stored in the preset storage area; (5) In response to modification of the second data in the target database, the modified second data is synchronized to the second data stored in the preset storage area. Optionally, the above data update method can be executed by the management module of the preset storage area.

[0222] According to an embodiment of this application, the above-described device may further include an effective duration setting module, used to: determine the effective storage duration of the third data based on the data query frequency and / or data modification frequency of the third data when storing the third data in a preset storage area; the determined effective storage duration is positively correlated with the data query frequency; and the determined effective storage duration is negatively correlated with the data modification frequency.

[0223] According to embodiments of this application, any multiple modules among the determining module 310, the first query module 320, the feedback module 330, the second query module 340, the adding module, the filtering module, the allocation module, the management module, and the valid duration setting module can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the determining module 310, the first query module 320, the feedback module 330, the second query module 340, the adding module, the filtering module, the allocation module, the management module, and the valid duration setting module can be at least partially implemented as hardware circuitry, such as a field-programmable gate array, a programmable logic array, a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit, or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the following modules can be implemented, at least partially, as a computer program module: the determination module 310, the first query module 320, the feedback module 330, the second query module 340, the addition module, the filtering module, the allocation module, the management module, and the effective duration setting module. When the computer program module is run, it can perform the corresponding function.

[0224] The explanation of the above device embodiments can be found in the explanation of other embodiments. Any step in the above method embodiments can be executed by the corresponding module in the above device embodiments.

[0225] Figure 4 A block diagram schematically illustrates an electronic device suitable for implementing a data query method according to an embodiment of this application.

[0226] like Figure 4 As shown, an electronic device 1200 according to an embodiment of this application includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory 1202 or a program loaded from a storage portion 1208 into a random access memory 1203. The processor 1201 may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a dedicated microprocessor. The processor 1201 may also include onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for executing different steps of the method flow according to an embodiment of this application.

[0227] Random access memory 1203 stores various programs and data required for the operation of electronic device 1200. Processor 1201, read-only memory 1202, and random access memory 1203 are interconnected via bus 1204. Processor 1201 executes various steps of the method flow according to embodiments of this application by executing programs in read-only memory 1202 and / or random access memory 1203. It should be noted that the programs may also be stored in one or more memories other than read-only memory 1202 and random access memory 1203. Processor 1201 may also execute various steps of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0228] According to embodiments of this application, the electronic device 1200 may further include an input / output interface 1205, which is also connected to the bus 1204. The electronic device 1200 may also include one or more of the following components connected to the input / output interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube, liquid crystal display, etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card, such as a local area network card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the input / output interface 1205 as needed. A removable medium 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1210 as needed so that computer programs read from it can be installed into the storage section 1208 as needed.

[0229] Embodiments of this application also provide a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0230] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include the read-only memory 1202, and / or random access memory 1203, and / or one or more memories other than read-only memory 1202 and random access memory 1203 described above.

[0231] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this application.

[0232] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1209, and / or installed from the removable medium 1211. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0233] In embodiments of this application, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by processor 1201, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0234] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0235] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0236] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A data query method, characterized in that, The method includes: A target data query request is determined for a target database; the target data query request is used to query first data; the target database contains N data tables, where N is a positive integer; a preset storage area outside the target database is determined; the preset storage area stores N data sets, each data set corresponding to a different data table in the N data tables, and any data set is used to store data in the corresponding data table; the first data table to which the first data belongs is determined; According to the target data query request, query the first data in the data set corresponding to the first data table in the preset storage area; If the first data is found in the data set corresponding to the first data table within the preset storage area, the first data found will be returned in response to the target data query request. If the first data is not found in the data set corresponding to the first data table in the preset storage area, the first data is queried in the first data table of the target database according to the target data query request.

2. The method according to claim 1, characterized in that, The step of querying the first data in the first data table of the target database according to the target data query request includes: If the first data is found in the first data table of the target database, the first data is stored in the data set corresponding to the first data table, and the first data found is returned in response to the target data query request. If the first data is not found in the first data table of the target database, a result indicating that the query for the first data failed is returned for the target data query request.

3. The method according to claim 1, characterized in that, The step of querying the first data in the data set corresponding to the first data table within the preset storage area according to the target data query request includes: If it is determined that the first data table belongs to the N data tables, the first data is queried in the data set corresponding to the first data table in the preset storage area according to the target data query request. If it is determined that the first data table does not belong to the N data tables, a result indicating that the query for the target data failed is fed back for the query request.

4. The method according to claim 1, characterized in that, The preset storage area is used to store the value range of the primary keys in the N data tables; the step of querying the first data in the data set corresponding to the first data table in the preset storage area according to the target data query request includes: If it is determined that the primary key value in the target data query request belongs to the value range of the primary key in the first data table, the first data is queried in the data set corresponding to the first data table in the preset storage area according to the target data query request; If it is determined that the primary key value in the target data query request does not fall within the range of primary key values ​​in the first data table, a result indicating that the query for the first data failed is fed back for the target data query request.

5. The method according to claim 1, characterized in that, The method further includes: Based on a pre-trained query probability prediction model, the probability of future data queries is predicted for each of the N data tables. In data tables where the probability of future data queries is greater than a preset query probability threshold, select data to store in the corresponding data set.

6. The method according to claim 5, characterized in that, The pre-trained query probability prediction model predicts the probability of future data queries for each of the N data tables, including: Based on a pre-trained query probability prediction model, predict the future data query probability for each of the N data tables according to at least one of the following pieces of information: The frequency of historical data queries for the corresponding data set of the targeted data table; The frequency of historical data queries for the target data table; A historical data query request for which no data was found in the data set corresponding to the target data table, but data was found in the target data table of the target database.

7. The method according to claim 1, characterized in that, The method further includes: If the number M of M data query requests that did not find data in the target database within a preset time period is greater than a preset request number threshold, a filtering rule is generated based on the M data query requests; where M is a positive integer. For any new data query request that matches any of the generated filtering rules, a result indicating that the query failed is returned.

8. The method according to claim 1, characterized in that, The step of querying the first data in the first data table of the target database according to the target data query request includes: A comprehensive data query request is generated by combining the target data query request and other data query requests used to query other data in the first data table; Based on the comprehensive data query request, query the first data and the other data in the first data table of the target database; The other data is not stored in the data set corresponding to the first data table.

9. The method according to claim 1, characterized in that, The method further includes: For any of the N data tables, determine the upper limit of the storage capacity of the data set corresponding to the data table in the preset storage area based on at least one of the following information: The frequency of data queries for the target data table; The frequency of data queries for the data set corresponding to the data table; The determined upper limit of storage capacity is positively correlated with the data query frequency of the target data table, and the determined upper limit of storage capacity is positively correlated with the data query frequency of the data set corresponding to the target data table.

10. The method according to claim 1, characterized in that, The data update method for the preset storage area includes at least one of the following: The data stored in the preset storage area has an expiration date. Data that has exceeded its expiration date is deleted from the preset storage area. Each of the N data sets has a corresponding storage limit. For any of the N data sets, if the proportion of the currently stored data in the corresponding storage limit exceeds a preset proportion threshold, data is selected from the targeted data set for deletion. In the preset storage area, data with a data query frequency lower than a preset idle frequency threshold is selected for deletion; Select data from the target database and store it in the preset storage area; In response to modification of the second data in the target database, the modified second data is synchronized to the second data stored in the preset storage area.

11. The method according to claim 1, characterized in that, The method further includes: When storing the third data from the target database in the preset storage area, the effective storage duration of the third data is determined based on the data query frequency and / or data modification frequency of the third data. The determined effective storage duration is positively correlated with the data query frequency; the determined effective storage duration is negatively correlated with the data modification frequency.

12. A data query device, characterized in that, The device includes: The determination module is used to determine a target data query request for a target database; the target data query request is used to query first data; the target database contains N data tables, where N is a positive integer; a preset storage area outside the target database is determined; the preset storage area stores N data sets, different data sets correspond to different data tables in the N data tables, and any data set is used to store data in the corresponding data table; the first data table to which the first data belongs is determined. The first query module is used to query the first data in the data set corresponding to the first data table in the preset storage area according to the target data query request. The feedback module is used to provide feedback on the first data found in the data set corresponding to the first data table in the preset storage area when the first data is found in the target data query request. The second query module is used to query the first data in the first data table of the target database according to the target data query request when the first data is not found in the data set corresponding to the first data table in the preset storage area.

13. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 11.

Images