Data query method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of storage, and particularly provides a data query method and device, electronic equipment and a storage medium. The data query method comprises the following steps: determining an initial prefetch rule based on a data query instruction; the initial prefetch rule is whether to adopt a cache disabling strategy and whether to adopt a prefetch disabling strategy; determining a first comparison result between a current cache data amount and a cache disabling threshold value and a second comparison result between a memory locking data amount and a memory disabling threshold value; determining a target prefetch rule according to the initial prefetch rule, the first comparison result and the second comparison result; and performing a data query operation according to the target prefetch rule. In this way, the target prefetch rule of the data query is dynamically adjusted according to multiple dimensions of the query instruction, the current cache data amount and the memory locking data amount, so that the device memory occupied by prefetching can be reasonably controlled, and the problem of cache pollution can be reduced.

Description

Technical Field

[0001] This application relates to the field of storage technology, specifically to a method, apparatus, electronic device, and storage medium for data retrieval. Background Technology

[0002] In some application data query scenarios, it is often necessary to read and process large amounts of data from a database. Under current technology, to improve the efficiency of data reading and processing, data from each data block within the query range is typically prefetched and loaded into the device memory so that it can be quickly read in subsequent steps.

[0003] Prefetching is a process of loading data in advance. Specifically, during the execution of a query, the data blocks that need to be scanned are determined based on the data index and the query range, and these data blocks are preloaded into the device memory.

[0004] However, during large queries (i.e. queries with large amounts of data), prefetching data may consume too much user memory, resulting in no available user memory for other queries and other operations. Also, when loading prefetched data into the cache, hot data in the cache may be evicted, leading to cache pollution. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, electronic device, and storage medium for data querying, so as to reasonably control the device memory occupied by prefetching and reduce the problem of cache pollution during data querying.

[0006] On the one hand, embodiments of this application provide a data query method, including:

[0007] Based on the data query command, the initial prefetch rules are determined. The initial prefetch rules are whether to adopt a caching-disabled policy and whether to adopt a prefetch-prohibited policy. The caching-disabled policy means prohibiting the writing of query-prefetched data to the cache. The prefetch-prohibited policy means prohibiting the writing of query-prefetched data to device memory. Device memory is divided into cache and user memory.

[0008] Determine the first comparison result between the current cached data volume and the cache-disable threshold, and the second comparison result between the memory-locked data volume and the memory-disable threshold; the memory-locked data volume includes the user memory data volume and the cache-locked data volume;

[0009] Based on the initial prefetch rule, the first comparison result, and the second comparison result, determine the target prefetch rule;

[0010] Perform data query operations according to the target prefetching rules.

[0011] In one implementation, determining the initial prefetching rules based on a data query instruction includes:

[0012] Upon receiving a data query instruction, determine the amount of data to be queried;

[0013] If it is determined that the amount of data to be queried is higher than the query threshold, then the initial prefetch rule is to adopt a caching-disallowing strategy.

[0014] Otherwise, the initial prefetch rule is determined to be an unrestricted policy; the unrestricted policy is to neither use a caching-disabled policy nor a prefetch-disabled policy.

[0015] In one implementation, determining a first comparison result between the current cached data amount and a disabled cache threshold, and a second comparison result between the memory-locked data amount and a disabled memory threshold, includes:

[0016] If it is determined that the current cached data volume is higher than the cache-disabling threshold, then the first comparison result is determined to be the first state; otherwise, it is the second state.

[0017] If the amount of memory-locked data is determined to be higher than the disabled memory threshold, then the second comparison result is determined to be the third state; otherwise, it is the fourth state.

[0018] In one implementation, determining the target prefetch rule based on the initial prefetch rule, the first comparison result, and the second comparison result includes:

[0019] If it is determined that the initial prefetch rule is to adopt an unrestricted strategy, the first comparison result is the second state and the second comparison result is the fourth state, then the initial prefetch rule is determined as the target prefetch rule.

[0020] If the first comparison result is determined to be the first state and the second comparison result is the fourth state, then the target prefetch rule is determined to be to adopt a caching-disabled strategy.

[0021] If the second comparison result is determined to be the third state, then the target prefetching rule is determined to be to adopt the prohibition of prefetching strategy.

[0022] In one embodiment, the method further includes:

[0023] If the threshold update condition is met, obtain the currently unlocked cache and the currently free user memory; the currently unlocked cache is the cache that is not locked; the currently free user memory is the currently free user memory.

[0024] Update the cache disable threshold and memory disable threshold based on the current unlocked cache and the current idle user memory, respectively.

[0025] On one hand, embodiments of this application provide a data query apparatus, including:

[0026] The query unit is used to determine the initial prefetch rules based on the data query instructions. The initial prefetch rules are whether to adopt a caching-disabled policy and whether to adopt a prefetch-prohibited policy. The caching-disabled policy means that the query-prefetched data is prohibited from being written to the cache. The prefetch-prohibited policy means that the query-prefetched data is prohibited from being written to the device memory. The device memory is divided into cache and user memory.

[0027] The comparison unit is used to determine a first comparison result between the current cached data amount and the cache-disable threshold, and a second comparison result between the memory-locked data amount and the memory-disable threshold; the memory-locked data amount includes the user memory data amount and the cache-locked data amount;

[0028] The determining unit is used to determine the target prefetch rule based on the initial prefetch rule, the first comparison result, and the second comparison result;

[0029] The execution unit is used to perform data query operations according to the target prefetching rules.

[0030] In one implementation, the query unit is used for

[0031] Upon receiving a data query instruction, determine the amount of data to be queried;

[0032] If it is determined that the amount of data to be queried is higher than the query threshold, then the initial prefetch rule is to adopt a caching-disallowing strategy.

[0033] Otherwise, the initial prefetch rule is determined to be an unrestricted policy; the unrestricted policy is to neither use a caching-disabled policy nor a prefetch-disabled policy.

[0034] In one embodiment, the comparison unit is used to:

[0035] If it is determined that the current cached data volume is higher than the cache-disabling threshold, then the first comparison result is determined to be the first state; otherwise, it is the second state.

[0036] If the amount of memory-locked data is determined to be higher than the disabled memory threshold, then the second comparison result is determined to be the third state; otherwise, it is the fourth state.

[0037] In one implementation, the determining unit is used to:

[0038] If it is determined that the initial prefetch rule is to adopt an unrestricted strategy, the first comparison result is the second state and the second comparison result is the fourth state, then the initial prefetch rule is determined as the target prefetch rule.

[0039] If the first comparison result is determined to be the first state and the second comparison result is the fourth state, then the target prefetch rule is determined to be to adopt a caching-disabled strategy.

[0040] If the second comparison result is determined to be the third state, then the target prefetching rule is determined to be to adopt the prohibition of prefetching strategy.

[0041] In one embodiment, the execution unit is further configured to:

[0042] If the threshold update condition is met, obtain the currently unlocked cache and the currently free user memory; the currently unlocked cache is the cache that is not locked; the currently free user memory is the currently free user memory.

[0043] Update the cache disable threshold and memory disable threshold based on the current unlocked cache and the current idle user memory, respectively.

[0044] On one hand, embodiments of this application provide an electronic device, including:

[0045] Processor; and

[0046] The memory stores computer instructions that cause the processor to perform the steps of the methods provided in various alternative implementations of any of the above-described data query methods.

[0047] On one hand, embodiments of this application provide a storage medium storing computer instructions for causing a computer to perform the steps of the methods provided in various optional implementations of any of the above-described data query methods.

[0048] The data query method, apparatus, electronic device, and storage medium provided in this application embodiment determine an initial prefetch rule based on a data query instruction. The initial prefetch rule includes whether to adopt a caching disable strategy and whether to adopt a prefetch prohibition strategy. The caching disable strategy prohibits writing the query-prefetched data to the cache. The prefetch prohibition strategy prohibits writing the query-prefetched data to device memory. Device memory is divided into cache and user memory. A first comparison result between the current cached data volume and the caching disable threshold, and a second comparison result between the memory-locked data volume and the memory-locked threshold, are determined. The memory-locked data volume includes user memory data volume and cache-locked data volume. A target prefetch rule is determined based on the initial prefetch rule, the first comparison result, and the second comparison result. The data query operation is then executed according to the target prefetch rule. Thus, during data querying, the target prefetch rule is dynamically adjusted based on the query instruction, the current cached data volume, and the memory-locked data volume, thereby reasonably controlling the device memory occupied by prefetching and reducing cache pollution. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the data query method in the embodiments of this application.

[0051] Figure 2 This is a detailed flowchart of the data query method in the embodiments of this application.

[0052] Figure 3 This is a structural block diagram of the data query device in the embodiments of this application.

[0053] Figure 4 This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0054] The technical solution of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0055] First, some of the terms used in the embodiments of this application will be explained to facilitate understanding by those skilled in the art.

[0056] Terminal devices can be mobile terminals, fixed terminals, or portable terminals, such as mobile phones, sites, units, devices, multimedia computers, multimedia tablets, internet nodes, communicators, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, personal communication system devices, personal navigation devices, personal digital assistants, audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. It is also foreseeable that terminal devices can support any type of user-facing interface (e.g., wearable devices).

[0057] Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms.

[0058] Prefetching is a process of loading data in advance. Specifically, during the execution of a query, the data blocks that need to be scanned are determined based on the data index and the query range, and these data blocks are preloaded into the device memory.

[0059] Holding refers to the phenomenon where the fixed-length block containing data in the cache cannot be reclaimed when the data is referenced. In this embodiment, for ease of explanation, holding the data is referred to as locking.

[0060] The technical concept of this application will be explained below.

[0061] In some application data query scenarios, it is often necessary to read and process large amounts of data from a database. In traditional technology, to improve the efficiency of data reading and processing, data from each data block within the query range in the database is prefetched and loaded into the device memory so that it can be read quickly in subsequent steps.

[0062] However, during large queries (i.e. queries with large amounts of data), prefetching data may consume too much user memory, resulting in no available user memory for other queries and other operations. Also, when loading prefetched data into the cache, hot data in the cache may be evicted, leading to cache pollution.

[0063] For example, OceanBase, in order to fully utilize tenants' device memory, does not limit the upper limit of device memory used for caching; theoretically, the cache can fill a tenant's device memory. When a tenant runs out of free memory, it will attempt to find unheld fixed-length memory blocks in the cache and release them to free up memory space for the tenant. During the execution of large queries, due to the large amount of data that needs to be scanned, a large amount of data will be held in the cache, resulting in no device memory available for other queries and operations of the tenant.

[0064] For example, during the execution of a large query, a large amount of new data will be prefetched and put into the cache. However, if the tenant does not have any spare memory available, the fixed-length blocks that have not been held will be reused to store the new data. However, this may cause the existing hot data in the fixed-length blocks to be replaced by the new data, resulting in a large amount of cache being polluted.

[0065] Based on the deficiencies of the aforementioned related technologies, this application provides a method, apparatus, electronic device, and storage medium for data querying, aiming to reasonably control the device memory occupied by prefetching and reduce cache pollution during data querying.

[0066] This application provides a data query method that can be applied to electronic devices. This application does not limit the type of electronic device, which can be any suitable type of device, such as a terminal device or a server, etc. This application will not elaborate further.

[0067] See Figure 1 The diagram shown is a flowchart of the data query method in an embodiment of this application. The following is a summary of the process. Figure 1 The method is described below, and the specific implementation process is as follows:

[0068] Step 100: Determine the initial prefetching rules based on the data query instructions.

[0069] The initial prefetch rules are whether to use a caching-disabled policy and a prefetch-prohibited policy. The caching-disabled policy means prohibiting the writing of query-prefetched data to the cache; the prefetch-prohibited policy means prohibiting the writing of query-prefetched data to the device memory.

[0070] It's important to note that a cache manages its own memory in fixed-length blocks; data stored in the cache is contained within these blocks. The size of the cache is determined by the amount of data stored within it. The cache's storage space is allocated from the device's memory, which can be divided into cache and user memory. A larger cache requires less user memory, and vice versa.

[0071] As an example, prefetch rules can include the following three methods:

[0072] Rule 1: Use an unrestricted strategy.

[0073] The unlimited policy means that neither the caching disable policy nor the prefetch disable policy is adopted, that is, there are no restrictions on the use of cache and user memory.

[0074] In this way, when the amount of data to be queried is small, or when there is a lot of free device memory, the existing prefetching behavior can be maintained without limiting the amount of cache usage or device memory prefetching.

[0075] Rule 2: Adopt a caching-disabled strategy.

[0076] Thus, rule two only prohibits putting prefetched data into the cache, but it is still possible to send one or more prefetch requests to prefetch data blocks (e.g., data micro-blocks) that will be read later and load the prefetched data into user memory.

[0077] Rule 3: Adopt a no-prefetch strategy.

[0078] Thus, adopting a no-prefetch strategy means that data is not prefetched into the device memory (i.e., it cannot be prefetched into the cache or prefetched into user memory). Only when a query needs to scan a certain data block will the data of the relevant data block be loaded into the device memory synchronously.

[0079] It should be noted that, in the embodiments of this application, the database used by the data query method can be any database, such as OceanBase or MySQL, and there are no restrictions here.

[0080] In one implementation, when performing step 100, the following steps may be adopted:

[0081] S1001: When a data query instruction is received, determine the amount of data to be queried.

[0082] In one implementation, the table to be queried and the table query range are determined according to the data query instruction, and the amount of data to be queried is determined according to the table to be queried and the table query range.

[0083] As an example, based on the data query instruction, if it is determined that all data in the table tabs needs to be queried, then the amount of data in the table tabs is determined as the amount of data to be queried.

[0084] As another example, based on the data query instruction, if it is determined that the first three rows of data in the table tab need to be queried, then the amount of data in the first three rows is determined, and the amount of data to be queried is obtained.

[0085] Furthermore, it is determined that when a data query instruction is received, an unrestricted strategy can be initially used for data prefetching.

[0086] S1002: If it is determined that the amount of data to be queried is higher than the query threshold, then the initial prefetch rule is to adopt a no-caching strategy; otherwise, the initial prefetch rule is to adopt an unrestricted strategy.

[0087] The unrestricted strategy means not using the caching disable strategy and not using the prefetch disable strategy.

[0088] In practical applications, the query threshold can be set according to the actual application scenario. For example, 100M (megabytes) is not limited here.

[0089] Thus, if the amount of data to be queried is higher than the query threshold, it means that the current query is a large query (i.e., a query that involves a large amount of data), and a caching-disallowing strategy should be adopted to prevent the prefetched data from being put into the cache. Otherwise, it means that the current query is not a large query, and an unlimited strategy can be adopted to prefetch data without restriction.

[0090] Step 101: Determine the first comparison result between the current cached data amount and the disable cache threshold, and the second comparison result between the memory locked data amount and the disable memory threshold.

[0091] In one implementation, the process of step 101 may include:

[0092] S1011: If it is determined that the current cached data volume is higher than the cache disable threshold, then the first comparison result is determined to be the first state; otherwise, it is the second state.

[0093] In practical applications, the threshold for disabling caching can be set according to the actual application scenario, and no restrictions are imposed here.

[0094] Since the cache size changes dynamically based on the loaded data, the current cached data amount is the current cache size.

[0095] In this way, if the current cached data volume is higher than the cache disable threshold, it means that the cache memory usage is already high. Therefore, it is prohibited to load prefetched data into the cache. Conversely, if the current cached data volume is lower than the threshold, it is possible to continue loading prefetched data into the cache.

[0096] S1012: If it is determined that the amount of memory-locked data is higher than the memory-disabled threshold, then the second comparison result is determined to be the third state; otherwise, it is the fourth state.

[0097] In practical applications, the threshold for disabling memory can be set according to the actual application scenario, and no restrictions are imposed here.

[0098] It should be noted that device memory is divided into cache and user memory. The storage space for the cache is allocated from the device memory, while the storage space available to users, i.e., user memory, is the storage space outside the cache within the device memory.

[0099] The amount of data locked in memory includes the amount of data in user memory and the amount of data locked in the cache. The amount of data in user memory is the size of the data already stored in user memory. The amount of data locked in the cache refers to the amount of data currently locked in the cache.

[0100] If the amount of data locked in memory exceeds the memory-disabled threshold, it means that too much storage space is occupied in the device's memory. Therefore, the prefetching disallowing policy is executed in subsequent steps to leave more available memory space for the user.

[0101] Step 102: Determine the target prefetch rule based on the initial prefetch rule, the first comparison result, and the second comparison result.

[0102] Specifically, the target prefetching rules are whether to adopt a caching-disabled strategy and whether to adopt a prefetch-prohibited strategy.

[0103] In one implementation, step 102 may include:

[0104] S1021: If it is determined that the initial prefetch rule is an unrestricted strategy, the first comparison result is the second state and the second comparison result is the fourth state, then the initial prefetch rule is determined as the target prefetch rule.

[0105] In this way, if the cache does not exceed the available limit (i.e., the second state) and the memory does not exceed the available limit (i.e., the fourth state of the device), then data can continue to be prefetched without restriction (i.e., an unrestricted strategy is adopted).

[0106] S1022: If the first comparison result is determined to be the first state and the second comparison result is the fourth state, then the target prefetch rule is determined to be to adopt the disabled caching strategy.

[0107] Thus, if only the cache exceeds the available limit (i.e., the first state), while the memory does not exceed the available limit, then regardless of whether the initial prefetch rule adopts an unlimited policy or a disabled caching policy, the disabled caching policy will be adopted to stop prefetching only for the cache.

[0108] S1023: If the second comparison result is determined to be the third state, then the target prefetch rule is determined to be to adopt the prohibition of prefetching strategy.

[0109] Thus, if the device memory exceeds the available limit (i.e., the third state), regardless of whether the initial prefetch rule adopts an unlimited policy or a caching-disabled policy, or whether the cache exceeds the available limit, a prefetching-disabled policy is adopted, and prefetch requests for queries are no longer allowed, thereby stopping all prefetch loading operations.

[0110] Step 103: Perform the data query operation according to the target prefetching rules.

[0111] In one implementation, if the target prefetching rule is determined to be an unrestricted policy or a caching-disabled policy, then the data is prefetched and loaded into the device memory according to the unrestricted policy or the caching-disabled policy, and when a data read instruction is received, the data in the device memory is read based on the data read instruction.

[0112] If the target prefetch rule is determined to be a prefetch prohibition policy, then the prefetch prohibition policy is executed, the prefetch operation ends, and when a data read instruction is received, the corresponding data is read from the database based on the data read instruction.

[0113] In this way, data in the data block can be prefetched and loaded according to the determined target prefetching rules.

[0114] Furthermore, when the threshold update conditions are met, the disable cache threshold and disable memory threshold can also be updated. Then, based on the updated disable cache threshold and disable memory threshold, the target prefetching rule can be redefined, and the data query operation can be performed according to the new target prefetching rule.

[0115] In one implementation, updating the disable cache threshold and disable memory threshold can be achieved using the following steps:

[0116] S1031: If the threshold update condition is met, determine the currently unlocked cache and the currently free user memory.

[0117] Optionally, the threshold update condition can be reaching the threshold update time, determining that there is an idle prefetch slot, determining that a data query instruction has been received, or determining that the latest round of prefetching operation has been completed and there is remaining data to be prefetched.

[0118] In practical applications, the threshold update conditions can be set according to the actual application scenario, and no restrictions are imposed here.

[0119] In one implementation, the following steps can be used to determine the currently unlocked cache and the currently free user memory:

[0120] Update the current cached data size and the memory locked data size. Based on the updated current cached data size and the memory locked data size, and the difference between the current cache size and the locked data size within the current cached data size, determine the current unlocked cache size. Finally, determine the current free user memory size based on the difference between the user memory size and the memory locked data size. In other words, current unlocked cache = current cache size - locked data size within the current cached data size; current free user memory = user memory - memory locked data size.

[0121] Among them, the currently unlocked cache is the cache that is not locked; the currently free user memory is the currently free user memory.

[0122] S1032: Update the cache disable threshold and memory disable threshold based on the current unlocked cache and the current free user memory, respectively.

[0123] In one implementation, the threshold for disabling caching and the threshold for disabling memory are dynamically updated in real time or periodically.

[0124] In one implementation, the following formula can be used to determine the threshold for disabling caching:

[0125] Disable cache threshold = a1(current free user memory + b1 currently unlocked cache);

[0126] In one implementation, the following formula can be used to determine the disabled memory threshold:

[0127] Disable memory threshold = a2(current free user memory + b2 currently unlocked cache);

[0128] Where a1, a2, b1, and b2 are all coefficients, and their values ​​can all be in the range of (0, 1). Since the cost of disabling caching and disabling memory are different, with the cost of disabling caching being less than that of disabling memory, the values ​​of the disable caching threshold and the disable memory threshold are adjusted using the coefficients a1, a2, b1, and b2.

[0129] Once data in user memory is read, the user memory occupied by the read data can be released. Similarly, once data in the locked cache is read, the locked cache can be set to an unlocked cache. In this way, the locked memory in device memory (i.e., the locked cache and the occupied user memory) is dynamically changing. After data is read, the locked memory decreases, allowing the cache disable threshold and memory disable threshold to be updated. Furthermore, based on the updated cache disable threshold and memory disable threshold, the target prefetching rule can be redefined, and data query operations can be performed according to the new target prefetching rule.

[0130] In one application scenario, since the cumulative amount of data placed in the cache does not decrease, enabling a caching-disabled policy will not revert to an unrestricted policy. Furthermore, because the locked memory in device memory changes dynamically, enabling a prefetch-disabled policy can still revert to the previous caching-disabled policy.

[0131] In this embodiment, during data querying, the target prefetching rules are dynamically adjusted based on multiple dimensions, including the query instruction, the current cached data volume, and the memory-locked data volume. This limits the amount of data used for the query. Even if no cache usage limit is set, the system can still sense the user's current memory status and the memory held by the query during prefetching, allowing for real-time dynamic adjustment of prefetching rules and control of prefetching behavior, thus better controlling the device memory used for queries. Furthermore, it can cache as much hot data as possible at the global level, avoiding the problem of hot data not residing in memory. Further, it addresses cache pollution by controlling data storage in the cache, reducing cold data entering the cache at its source, rather than preventing pollution from spreading after cold data enters the cache. This ensures that hot data remains in the cache, thereby reasonably controlling the device memory occupied by prefetching and reducing cache pollution.

[0132] See Figure 2 The diagram shown below is a detailed flowchart of the data query method. The following section will combine... Figure 2 The above data query method will be further illustrated with examples. It should be noted that the initial prefetch rule and the target prefetch rule are the same prefetch rule. Figure 1 In the illustrated embodiment, to facilitate the distinction between prefetch rules configured at different times, the prefetch rules are referred to as initial prefetch rules or target prefetch rules. For the sake of brevity, both initial prefetch rules and target prefetch rules will be uniformly referred to as target prefetch rules. The specific implementation process of this method includes:

[0133] Step 200: When a data query instruction is received, determine that the target prefetching rule is an unrestricted strategy.

[0134] Step 201: Based on the data query instruction, determine whether the amount of data to be queried is higher than the query threshold. If yes, proceed to step 202; otherwise, proceed to step 203.

[0135] Step 202: Update the target prefetch rule to use a no-caching strategy.

[0136] Step 203: Determine the first comparison result between the current cached data amount and the disable cache threshold, and the second comparison result between the memory locked data amount and the disable memory threshold.

[0137] Step 204: Determine whether the first comparison result is the first state. If yes, proceed to step 205; otherwise, proceed to step 206.

[0138] Step 205: Determine the target prefetch rule to adopt a caching-disabled strategy.

[0139] Step 206: Determine whether the second comparison result is the third state. If yes, proceed to step 207; otherwise, proceed to step 208.

[0140] Step 207: Update the target prefetch rule to adopt the prohibition prefetching strategy and stop the prefetching process.

[0141] Step 208: Perform the prefetch data operation according to the target prefetch rules.

[0142] Step 209: If the threshold update condition is met, update the disable cache threshold or disable memory threshold based on the current unlocked cache and the current free user memory, and proceed to step 203.

[0143] Specifically, the steps 200-201 can be found in steps 100-103 above, and will not be repeated here.

[0144] In this embodiment of the application, during data query, the target prefetching rules can be dynamically adjusted in a timely manner according to the query command, the current cached data volume, and the memory locked data volume, thereby reasonably controlling the device memory occupied by prefetching and reducing the problem of cache pollution.

[0145] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0146] Based on the same inventive concept, this application also provides a data query device. Since the principle of the above-described device and apparatus in solving the problem is similar to that of a data query method, the implementation of the above-described device can refer to the implementation of the method, and repeated details will not be described again. This device can be applied to electronic devices. This application does not limit the type of electronic device; it can be any suitable type of device, such as smartphones, tablet computers, etc., which will not be described again in this application.

[0147] See Figure 3 The diagram shown is a structural block diagram of a data query apparatus according to an embodiment of this application. In some embodiments, the data query apparatus exemplified in this application includes:

[0148] The query unit 301 is used to determine the initial prefetching rules based on the data query instructions. The initial prefetching rules are whether to adopt a caching-disabled policy and whether to adopt a prefetch-prohibited policy. The caching-disabled policy means prohibiting the writing of query-prefetched data to the cache. The prefetch-prohibited policy means prohibiting the writing of query-prefetched data to the device memory. The device memory is divided into cache and user memory.

[0149] Comparison unit 302 is used to determine a first comparison result between the current cached data amount and the disabled cache threshold, and a second comparison result between the memory locked data amount and the disabled memory threshold; the memory locked data amount includes the user memory data amount and the cache locked data amount;

[0150] The determining unit 303 is used to determine the target prefetch rule based on the initial prefetch rule, the first comparison result, and the second comparison result;

[0151] Execution unit 304 is used to perform data query operations according to the target prefetching rules.

[0152] In one embodiment, the query unit 301 is used for

[0153] Upon receiving a data query instruction, determine the amount of data to be queried;

[0154] If it is determined that the amount of data to be queried is higher than the query threshold, then the initial prefetch rule is to adopt a caching-disallowing strategy.

[0155] Otherwise, the initial prefetch rule is determined to be an unrestricted policy; the unrestricted policy is to neither use a caching-disabled policy nor a prefetch-disabled policy.

[0156] In one embodiment, the comparison unit 302 is used to:

[0157] If it is determined that the current cached data volume is higher than the cache-disabling threshold, then the first comparison result is determined to be the first state; otherwise, it is the second state.

[0158] If the amount of memory-locked data is determined to be higher than the disabled memory threshold, then the second comparison result is determined to be the third state; otherwise, it is the fourth state.

[0159] In one embodiment, the determining unit 303 is used to:

[0160] If it is determined that the initial prefetch rule is to adopt an unrestricted strategy, the first comparison result is the second state and the second comparison result is the fourth state, then the initial prefetch rule is determined as the target prefetch rule.

[0161] If the first comparison result is determined to be the first state and the second comparison result is the fourth state, then the target prefetch rule is determined to be to adopt a caching-disabled strategy.

[0162] If the second comparison result is determined to be the third state, then the target prefetching rule is determined to be to adopt the prohibition of prefetching strategy.

[0163] In one embodiment, the execution unit 304 is further configured to:

[0164] If the threshold update condition is met, obtain the currently unlocked cache and the currently free user memory; the currently unlocked cache is the cache that is not locked; the currently free user memory is the currently free user memory.

[0165] Update the cache disable threshold and memory disable threshold based on the current unlocked cache and the current idle user memory, respectively.

[0166] The data query method, apparatus, electronic device, and storage medium provided in this application embodiment determine an initial prefetch rule based on a data query instruction. The initial prefetch rule includes whether to adopt a caching disable strategy and whether to adopt a prefetch prohibition strategy. The caching disable strategy prohibits writing the query-prefetched data to the cache. The prefetch prohibition strategy prohibits writing the query-prefetched data to device memory. Device memory is divided into cache and user memory. A first comparison result between the current cached data volume and the caching disable threshold, and a second comparison result between the memory-locked data volume and the memory-locked threshold, are determined. The memory-locked data volume includes user memory data volume and cache-locked data volume. A target prefetch rule is determined based on the initial prefetch rule, the first comparison result, and the second comparison result. The data query operation is then executed according to the target prefetch rule. Thus, during data querying, the target prefetch rule is dynamically adjusted based on the query instruction, the current cached data volume, and the memory-locked data volume, thereby reasonably controlling the device memory occupied by prefetching and reducing cache pollution.

[0167] This application provides an electronic device, including:

[0168] Processor; and

[0169] The memory stores computer instructions that cause the processor to execute the methods of any of the above-described embodiments.

[0170] This application provides a storage medium storing computer instructions for causing a computer to execute the methods of any of the above embodiments.

[0171] Figure 4 A schematic diagram of the structure of an electronic device 4000 is shown. (See reference...) Figure 4 As shown, the electronic device 4000 includes a processor 4010 and a memory 4020, and optionally may also include a power supply 4030, a display unit 4040, and an input unit 4050.

[0172] The processor 4010 is the control center of the electronic device 4000. It connects various components through various interfaces and lines, and performs various functions of the electronic device 4000 by running or executing software programs and / or data stored in the memory 4020.

[0173] In this embodiment, when the processor 4010 calls the computer program stored in the memory 4020, it executes the steps in the above embodiments.

[0174] Optionally, the processor 4010 may include one or more processing units; preferably, the processor 4010 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 4010. In some embodiments, the processor and memory may be implemented on a single chip; in some embodiments, they may also be implemented separately on independent chips.

[0175] The memory 4020 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, various applications, etc.; the data storage area may store data created based on the use of the electronic device 4000, etc. In addition, the memory 4020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device, etc.

[0176] Electronic device 4000 also includes a power supply 4030 (such as a battery) that supplies power to various components. The power supply can be logically connected to processor 4010 through a power management system, thereby enabling the management of charging, discharging, and power consumption.

[0177] The display unit 4040 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device 4000. In this embodiment, it is mainly used to display the display interfaces of various applications in the electronic device 4000, as well as text, images, and other objects displayed on the display interfaces. The display unit 4040 may include a display panel 4041. The display panel 4041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0178] The input unit 4050 can be used to receive information such as numbers or characters input by the user. The input unit 4050 may include a touch panel 4051 and other input devices 4052. The touch panel 4051, also known as a touch screen, can collect touch operations on or near the user (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 4051).

[0179] Specifically, the touch panel 4051 can detect user touch operations and the signals generated by these operations, convert them into touch point coordinates, send them to the processor 4010, and receive and execute commands from the processor 4010. Furthermore, the touch panel 4051 can be implemented using various types of touch technologies, including resistive, capacitive, infrared, and surface acoustic wave. Other input devices 4052 can include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0180] Of course, the touch panel 4051 can cover the display panel 4041. When the touch panel 4051 detects a touch operation on or near it, it transmits the information to the processor 4010 to determine the type of touch event. Subsequently, the processor 4010 provides corresponding visual output on the display panel 4041 according to the type of touch event. Although in Figure 4 In this embodiment, the touch panel 4051 and the display panel 4041 are two separate components to realize the input and output functions of the electronic device 4000. However, in some embodiments, the touch panel 4051 and the display panel 4041 can be integrated to realize the input and output functions of the electronic device 4000.

[0181] The electronic device 4000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity sensor, etc. Of course, depending on the specific application, the electronic device 4000 may also include other components such as a camera. Since these components are not the focus of this application embodiment, therefore... Figure 4 It is not shown in the text and will not be described in detail here.

[0182] Those skilled in the art will understand that Figure 4 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown, or a combination of certain components, or different components.

[0183] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

[0184] Obviously, the above embodiments are merely examples for clear illustration and are not intended to limit the embodiments. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively list all embodiments here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this application.

Claims

1. A method for data querying, the method comprising: Based on the data query command, the initial prefetch rule is determined; the initial prefetch rule is whether to adopt a caching disable strategy and whether to adopt a prefetch prohibition strategy. The caching disable policy refers to prohibiting the writing of query-fetched data to the cache; the prefetch disable policy refers to prohibiting the writing of query-fetched data to device memory; the device memory is divided into cache and user memory; Determine the first comparison result between the current cached data volume and the disable cache threshold, and the second comparison result between the memory locked data volume and the disable memory threshold; The amount of memory-locked data includes the amount of user memory data and the amount of cache-locked data; Based on the initial prefetch rule, the first comparison result, and the second comparison result, a target prefetch rule is determined; the target prefetch rule includes whether to adopt a caching-disabled strategy and whether to adopt a prefetch-disabled strategy. Perform a data query operation according to the target prefetching rules.

2. The method as described in claim 1, wherein determining the initial prefetch rule based on the data query instruction includes: Upon receiving the data query instruction, determine the amount of data to be queried; If it is determined that the amount of data to be queried is higher than the query threshold, then the initial prefetch rule is determined to be a no-caching strategy. Otherwise, the initial prefetch rule is determined to be an unrestricted strategy; the unrestricted strategy is to not use a caching-disabled strategy and not use a prefetch-disabled strategy.

3. The method of claim 2, wherein determining the first comparison result between the current cached data amount and the disabled cache threshold, and the second comparison result between the memory-locked data amount and the disabled memory threshold, comprises: If it is determined that the current cached data volume is higher than the disabled cache threshold, then the first comparison result is determined to be the first state; otherwise, it is the second state. If it is determined that the amount of memory-locked data is higher than the disabled memory threshold, then the second comparison result is determined to be the third state; otherwise, it is the fourth state.

4. The method of claim 3, wherein determining the target prefetch rule based on the initial prefetch rule, the first comparison result, and the second comparison result includes: If it is determined that the initial prefetch rule is an unrestricted strategy, the first comparison result is the second state and the second comparison result is the fourth state, then the initial prefetch rule is determined as the target prefetch rule; If the first comparison result is determined to be in the first state and the second comparison result is in the fourth state, then the target prefetch rule is determined to be to adopt a caching-disabled strategy. If the second comparison result is determined to be the third state, then the target prefetching rule is determined to be a prohibited prefetching strategy.

5. The method according to any one of claims 1-4, further comprising: If the threshold update condition is met, obtain the currently unlocked cache and the currently free user memory; The currently unlocked cache is the cache that is not locked; The currently free user memory refers to the currently free user memory. Update the cache disable threshold and the memory disable threshold based on the currently unlocked cache and the currently idle user memory, respectively.

6. A data query apparatus, the apparatus comprising: The query unit is used to determine the initial prefetching rules based on the data query instructions; the initial prefetching rules are whether to adopt a caching-disabled strategy and whether to adopt a prefetch-disabled strategy. The caching disable policy refers to prohibiting the writing of query-fetched data into the cache; The prefetch prohibition policy means prohibiting the writing of query-prefetched data into device memory; the device memory is divided into cache and user memory; The comparison unit is used to determine a first comparison result between the current cached data amount and the disable cache threshold, and a second comparison result between the memory locked data amount and the disable memory threshold; The amount of memory-locked data includes the amount of user memory data and the amount of cache-locked data; The determining unit is configured to determine a target prefetch rule based on the initial prefetch rule, the first comparison result, and the second comparison result; the target prefetch rule includes whether to adopt a caching-disable strategy and whether to adopt a prefetch-disable strategy. The execution unit is used to perform data query operations according to the target prefetching rules.

7. The apparatus of claim 6, wherein the query unit is used for Upon receiving the data query instruction, determine the amount of data to be queried; If it is determined that the amount of data to be queried is higher than the query threshold, then the initial prefetch rule is determined to be a no-caching strategy. Otherwise, the initial prefetch rule is determined to be an unrestricted strategy; the unrestricted strategy is to not use a caching-disabled strategy and not use a prefetch-disabled strategy.

8. The apparatus of claim 7, wherein the comparison unit is configured to: If it is determined that the current cached data volume is higher than the disabled cache threshold, then the first comparison result is determined to be the first state; otherwise, it is the second state. If it is determined that the amount of memory-locked data is higher than the disabled memory threshold, then the second comparison result is determined to be the third state; otherwise, it is the fourth state.

9. The apparatus of claim 8, wherein the determining unit is configured to: If it is determined that the initial prefetch rule is an unrestricted strategy, the first comparison result is the second state and the second comparison result is the fourth state, then the initial prefetch rule is determined as the target prefetch rule; If the first comparison result is determined to be in the first state and the second comparison result is in the fourth state, then the target prefetch rule is determined to be to adopt a caching-disabled strategy. If the second comparison result is determined to be the third state, then the target prefetching rule is determined to be a prohibited prefetching strategy.

10. The apparatus according to any one of claims 6-9, wherein the execution unit is further configured to: If the threshold update condition is met, then obtain the currently unlocked cache and the currently free user memory; the currently unlocked cache is the cache that is not locked; the currently free user memory is the currently free user memory. Update the cache disable threshold and the memory disable threshold based on the currently unlocked cache and the currently idle user memory, respectively.

11. An electronic device, comprising: processor; as well as A memory storing computer instructions for causing the processor to perform the method according to any one of claims 1-5.

12. A storage medium storing computer instructions for causing a computer to perform the method according to any one of claims 1-5.

Images