Data caching method, apparatus, device, and computer-readable storage medium

Abstract

Embodiments of the present application provide a data caching method, device and equipment and a computer readable storage medium. The method comprises: obtaining target data to be cached and a data identifier of the target data; if the data identifier is update data, writing the target data into a preload cache in a local cache; and if the data identifier is hotspot data, writing the target data into a hotspot cache in the local cache. The data caching method proposed by the present application can cache the target data in the preload cache and the hotspot cache of the local cache. By using two different caching forms, the method can avoid frequently requesting data update from a database, thereby effectively maintaining the stability of the system without increasing the performance loss of the system.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data caching, in particular to a data caching method, device, equipment and computer readable storage medium. BACKGROUND

[0002] Data caching refers to saving data locally to avoid continuously querying data from a database or a server, which can effectively solve the problem of frequent data reading faced by high concurrency access of a system. At present, caching is implemented based on hot spots of user requests, that is, the data query result of the latest user request is written into the cache so that the corresponding data can be directly read from the cache when the user executes the same request next time.

[0003] However, in the scenario of frequent data update in a database, the cache needs to frequently request the database for data update to ensure the effectiveness of the cache data and enable the system to run stably. However, frequent data request of the cache to the database will increase the performance loss of the system. SUMMARY

[0004] Embodiments of the present application provide a data caching method, device, equipment and computer readable storage medium, aiming to solve the technical problem that the existing data caching method needs to increase system performance loss to maintain stable operation of the system in the scenario of frequent data update of the system.

[0005] In one aspect, the present application provides a data caching method, characterized in that it comprises:

[0006] obtaining target data to be cached and a data identifier of the target data;

[0007] if the data identifier is updated data, writing the target data into a preloading cache in a local cache;

[0008] if the data identifier is hot spot data, writing the target data into a hot spot cache in the local cache.

[0009] As an optional embodiment of the present application, before the step of obtaining the target data to be cached, the method further comprises:

[0010] receiving a data query instruction and querying a local cache;

[0011] if the local cache does not exist a query result, sending the data query instruction to a target database;

[0012] receiving a query result returned by the target database and setting the query result returned by the target database as the target data.

[0013] As an optional embodiment of the present application, the local cache query comprises:

[0014] querying a preloaded cache in the local cache;

[0015] if the preloaded cache does not have a query result, querying a hot cache in the local cache.

[0016] As an optional embodiment of the present application, after the step of querying the local cache, the method further comprises:

[0017] querying a remote cache;

[0018] if the remote cache does not have a query result, performing the step of sending the data query instruction to the target database;

[0019] if the remote cache has a query result, writing the query result in the remote cache into the hot cache in the local cache.

[0020] As an optional embodiment of the present application, before the step of obtaining the target data to be cached, the method further comprises:

[0021] generating a data update request according to the cached data in the preloaded cache;

[0022] sending the data update request to the target database;

[0023] receiving an update result returned by the target database, and setting the update result as the target data.

[0024] As an optional embodiment of the present application, before the step of obtaining the target data to be cached, the method further comprises:

[0025] counting data update information of the stored data in a preset database to obtain a data update feature;

[0026] determining preloaded data from the stored data according to the data update feature;

[0027] writing the preloaded data into a preloaded cache in the local cache.

[0028] As an optional embodiment of the present application, before the step of obtaining the target data to be cached, the method further comprises:

[0029] the preloaded cache comprises a full cache area and a non-full cache area;

[0030] the step of writing the preloaded data into the preloaded cache in the local cache comprises:

[0031] The amount of preloaded data is compared with a preset data amount threshold;

[0032] If the amount of data is greater than the data amount threshold, the preloaded data is written to the non-full cache area in the preload cache;

[0033] If the preloaded data is less than or equal to the data volume threshold, then the preloaded data is written into the full cache area of ​​the preload cache.

[0034] On the other hand, embodiments of this application also provide a data caching device, including:

[0035] The cached data acquisition module is used to acquire the target data to be cached, and the data identifier of the target data;

[0036] A preload cache module is used to write the target data into the preload cache in the local cache if the data identifier is updated data;

[0037] The hotspot caching module is used to write the target data into the hotspot cache in the local cache if the data identifier is hot data.

[0038] On the other hand, embodiments of this application also provide a data caching device, which includes a processor, a memory, and a data caching program stored in the memory and executable on the processor. The processor executes the data caching program to implement the steps in the data caching method described above.

[0039] On the other hand, embodiments of this application also provide a computer-readable storage medium storing a data caching program, which is executed by a processor to implement the steps in the data caching method described above.

[0040] The data caching method proposed in this application can cache the target data in a local pre-load cache and a hot spot cache respectively, based on different data identifiers of the target data to be cached. The pre-load cache can effectively cache updated data, while the hot spot cache can cache frequently queried data by users. By using two different caching forms, the cache does not need to frequently request whether the data is updated from the database, thereby effectively maintaining the stability of the system without increasing the system performance loss. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram illustrating an implementation scenario of the data caching method according to an embodiment of this application;

[0043] Figure 2 This is a flowchart illustrating the first embodiment of the data caching method provided in this application.

[0044] Figure 3 This is a flowchart illustrating the second embodiment of the data caching method provided in this application.

[0045] Figure 4 This is a flowchart illustrating the third embodiment of the data caching method provided in this application.

[0046] Figure 5 This is a flowchart illustrating the fourth embodiment of the data caching method provided in this application.

[0047] Figure 6 This is a flowchart illustrating the fifth embodiment of the data caching method provided in this application.

[0048] Figure 7 This is a flowchart illustrating the sixth embodiment of the data caching method provided in this application.

[0049] Figure 8 This is a flowchart illustrating the seventh embodiment of the data caching method provided in this application.

[0050] Figure 9 This is a complete architecture diagram of a data cache provided in the embodiments of this application;

[0051] Figure 10 This is a schematic diagram of an embodiment of the data caching device provided in this application.

[0052] Figure 11 This is a schematic diagram of an embodiment of the data caching device provided in this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of the present invention.

[0054] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to implement and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in the embodiments of this application.

[0055] This application provides a data caching method, apparatus, device, and computer-readable storage medium, which will be described in detail below.

[0056] like Figure 1 As shown, Figure 1 This is a schematic diagram of the implementation scenario of the data caching method in this application embodiment. The data caching scenario in this embodiment mainly includes a preload cache 100, a hot spot cache 200, a target database 300, and a data caching device 400.

[0057] In this embodiment, as a common implementation scenario, the target database 300 is deployed on the server, while the preload cache 100 and hotspot cache 200 are integrated and deployed on the local client. Of course, the preload cache 100 and hotspot cache 200 can also be deployed on a virtual machine, such as the JVM (Java Virtual Machine). This application does not impose any restrictions on this.

[0058] In this embodiment, both the preload cache 100 and the hotspot cache 200 are essentially types of caches, both caching some data from the target database 300. However, their caching methods and the specific data they cache differ. Specifically, the preload cache 100 caches frequently updated data, while the hotspot cache caches data that users frequently need. That is, when a user initiates a data request service such as a query, the system first searches the local caches of the preload cache 100 and the hotspot cache 200. If neither the local cache nor the hotspot cache 200 contains the required data, a further query request is sent to the target database 300. Simultaneously, the results obtained from the query in the target database 300 are cached in the hotspot cache 200, so that subsequent user requests for the same data can find the required data in the hotspot cache 200. Furthermore, for frequently updated data in the database, this data is cached in the preload cache 100, and when this frequently updated data in the database is updated, the preload cache is updated synchronously, thus preventing the hotspot cache 200 from frequently sending data update requests to the database. Specifically, frequently updated data in the database can be pre-determined based on update information within the database. For example, data with an update frequency exceeding a certain threshold can be identified as frequently updated data. After this data is initialized and written to the preload cache 100, subsequent updates to this data in the database will update the data in the preload cache 100 accordingly. Data not written to the preload cache 100 can be considered infrequently updated; even if such data is updated, it will not be written to the preload cache.

[0059] To better understand the difference between pre-loaded cached data and hot-spot cached data, let's take a brief example of pre-loading cached data containing form A. In this case, when form A in the database is updated to form A', the cached form A in the pre-loaded cache will also be updated to A'. In other words, before and after caching, the data cached in the pre-loaded cache is always related to form A. In contrast, the cached data in the hot-spot cache is adjusted based on the actual query requests issued by the user. For example, if form A is pre-cached in the hot-spot cache, when a user issues a query for form B, the corresponding data cannot be found in the hot-spot cache. Therefore, after the database returns form B, form B will be cached in the hot-spot cache. If the cache space is insufficient, form A will be overwritten. In other words, before and after caching, the data cached in the hot-spot cache will be related to form A and form B respectively.

[0060] Furthermore, the data caching device 400 operates the following data caching method: obtaining the target data to be cached and the data identifier of the target data; if the data identifier is updated data, then writing the target data into the preload cache in the local cache; if the data identifier is hot data, then writing the target data into the hot cache in the local cache, so as to realize that different cached data are cached in the corresponding preload cache 100 and hot cache 200 in the local cache. Specifically, the data caching device exists in the form of a plugin, which can be deployed on a local client or on a server. Preferably, the data caching device 400 is integrated with the preload cache 100 and the hot cache 200 and deployed on a local client. The above-mentioned data caching method runs on the data caching device as a program.

[0061] It should be noted that, Figure 1 The schematic diagram of the data caching scenario shown is merely an example. The data caching scenario described in the embodiments of the present invention is intended to more clearly illustrate the technical solutions of the embodiments of the present invention and does not constitute a limitation on the technical solutions provided in the embodiments of the present invention.

[0062] Based on the implementation scenarios of the above data caching methods, an embodiment of the data caching method is proposed.

[0063] like Figure 2 As shown, Figure 2 This is a flowchart illustrating the first embodiment of the data caching method provided in this application. The data caching method in this embodiment includes steps 201-203:

[0064] 201. Obtain the target data to be cached, and the data identifier of the target data.

[0065] In this embodiment, the target data to be cached refers to the data transmitted from the target database to the local client and needs to be cached locally. As can be seen from the aforementioned scenario description, it mainly comes from two sources:

[0066] One scenario is that after a user initiates a data query request, if the corresponding query result cannot be found in the local cache, a query will be requested in the database. After the database returns the corresponding query result, this query result will also be used as the target data to be cached, and needs to be cached synchronously in the local cache.

[0067] Another approach is to update the local cache when database data is updated. To ensure the validity of cached data, the corresponding data in the local cache also needs to be updated. In this case, the database periodically collects updated data that is cached locally and then transmits it to the local cache according to certain rules, thus updating the cached data in the local cache. In other words, data that has been updated within a certain period and is cached locally, transmitted from the database, will also be considered as target data to be cached and needs to be synchronously cached locally.

[0068] Furthermore, to achieve the technical solution of this application, when the database transmits the target data to be cached to the local client, it first marks the target data with different data identifiers. For example, for the first method, after the database obtains the query results, the query results will be marked with the identifier of hot data and packaged and transmitted to the local client to indicate that the target data to be cached this time is data cached based on the user's hot demand. Similarly, for the second method, after statistically obtaining the data cached in the database on the local cache within a preset period that has been updated, this part of the data will be marked with the data identifier of updated data and packaged and transmitted to the local client to indicate that the target data to be cached this time is data cached based on updates.

[0069] For ease of understanding, the following... Figure 3 and Figure 6 The specific implementation methods for obtaining the target data to be cached mentioned above are shown respectively. Please refer to the following sections for details. Figure 3 , Figure 6 And its explanations and descriptions.

[0070] Of course, it should be noted that the above only illustrates two methods for obtaining the target data to be cached. In reality, there are many ways for the database to transmit the target data to the local client, and it is not limited to the two methods mentioned above. However, regardless of the method used, by marking the target data to be cached with a corresponding data identifier, the data caching device on the local client can deploy the target data in the corresponding local cache according to the data identifier marked on the target data after obtaining it.

[0071] 202. If the data identifier is updated data, then the target data is written into the preload cache in the local cache.

[0072] In this embodiment, if the data identifier is "updated data," it indicates that the target data is data updated within a preset period in the database, and the target data needs to be written to the preload cache in the local cache. Specifically, if the target data only contains the updated portion of the data, it needs to be written to the preload cache in the local cache using an incremental update method; if the target data contains the complete updated data, it needs to be written to the preload cache in the local cache using an overwrite method. Specifically, taking the example of form A being cached in the preload cache and form A being updated to form A′ in the database, if the database only uses the updated part of the data, that is, the difference information (A′-A) between form A′ and form A, as the target data, then the target data (A′-A) will be written to the preload cache in the local cache in an incremental update manner. That is, the difference information (A′-A) will be written on the basis of the cached form A, thus obtaining the updated form A′. If the database uses the entire updated data, that is, form A′, as the target data, then the target data A′ will be written to the preload cache in the local cache in an overwrite write manner, thus replacing the cached form A in the local cache with the updated form A′.

[0073] Furthermore, the cached data written to the preload cache needs to be initialized. This means initializing which part of the database data is written to the preload cache so that when this part of the database is updated, the cached data in the preload cache can be updated synchronously. The initialization configuration can be based on user input. Alternatively, as an optional embodiment of this application, the cached data in the preload cache is set based on the data's update characteristics. Specifically, the cached data in the preload cache can be frequently updated data; that is, frequently updated data is cached in the preload cache. In this case, the data cached in the hotspot cache is relatively more stable data, thus eliminating the need to request data updates from the database. For details, please refer to the following sections. Figure 7 And its explanations and descriptions.

[0074] 203. If the data identifier is hot data, then the target data is written into the hot data cache in the local cache.

[0075] In this embodiment of the application, as described above, if the data identifier is hot data, it indicates that the target data is determined based on the user's hot needs. That is, when there are no query results related to the user's query command in the local cache, the corresponding query results in the database are transmitted to the local client as the target data. At this time, the target data needs to be written into the hot data cache in the local cache.

[0076] In this embodiment, it can be understood that both hotspot caching and preloaded caching are forms of local caching, differing only in the cached data and caching strategies. This application addresses this by using a preloaded cache based on database updates, eliminating the need for frequent database updates to be requested from the hotspot cache. Furthermore, considering that existing caching technologies are primarily based on hotspot caching architectures, this preloaded cache ensures that the data caching method proposed in this embodiment can be directly deployed as a plugin on existing caching architectures, offering better scalability.

[0077] The data caching method proposed in this application can cache the target data in a local pre-load cache and a hot-spot cache respectively, based on different data identifiers of the target data to be cached. The pre-load cache can effectively cache updated data, while the hot-spot cache can cache frequently accessed data. By using two different caching forms, the cache does not need to frequently request data updates from the database, thereby effectively maintaining the stability of the system without increasing system performance.

[0078] like Figure 3 As shown, Figure 3 This is a flowchart illustrating a second embodiment of the data caching method provided in this application.

[0079] In this embodiment of the application, an implementation scheme for obtaining target data to be cached based on user data query instructions is proposed, specifically including steps 301 to 303:

[0080] 301. Receive the data query command and check the local cache to determine if the query result exists in the local cache. If yes, return the query result; otherwise, proceed to step 302.

[0081] In this embodiment, a data query instruction can be simply understood as a user request, that is, an instruction entered by the user to request data. It typically includes query fields so that components such as caches and databases can find the corresponding query results based on these fields. For those skilled in the art, the definition of a data query instruction and the specific implementation process of obtaining the corresponding query results based on the data query instruction are common knowledge and will not be elaborated upon here.

[0082] In this embodiment, upon receiving a data query command input by the user, the data caching device on the local client first queries the local cache. Specifically, the local cache includes a preloaded cache and a hotspot cache; that is, the data caching device queries the preloaded cache and the hotspot cache sequentially, and the specific query order can be set based on requirements. If the corresponding content exists in the local cache, meaning the query result corresponding to the data query command can be found in either the preloaded cache or the hotspot cache, the query result is directly returned to the user. Conversely, if the corresponding content does not exist in the local cache, the corresponding data needs to be further retrieved from the database.

[0083] It should be noted that data caching and data querying are parallel and independent processes. That is, although this application states that the target data to be cached is obtained through data querying before data caching, in reality, the data cache is dynamically updated based on the data query results. Furthermore, each data query is also based on the latest data cache. Details will follow. Figure 9 A complete implementation diagram of the data caching architecture is provided, and the process will be fully explained.

[0084] As an optional embodiment of this application, considering that the preloaded cache is updated based on changes in the database and its cached data is more effective, the query is preferentially performed in the preloaded cache. For details on the steps of querying the local cache in this case, please refer to the following sections. Figure 4 And its explanations and descriptions.

[0085] 302, Send the data query instruction to the target database.

[0086] In this embodiment of the application, when there is no query result corresponding to the data query instruction in the local cache, it is necessary to send a data query instruction to the target database so as to perform a query in the target database and obtain the corresponding query result.

[0087] As an optional embodiment of this application, a remote cache can be further set between the local cache and the database. Compared to the local cache, the remote cache can store more data, and compared to the database, the data retrieval speed of the remote cache is faster. Therefore, if the query result is not found in the local cache, a data request will be further sent to the remote cache. See the following for details. Figure 5 And its explanations and descriptions.

[0088] Furthermore, it's important to emphasize that the absence of query results in the local cache can specifically include two scenarios: one is that the query result exists in the database or other caches but is not cached locally; the other is that the query result simply does not exist, meaning it cannot be found in the database or any other cache. If a user repeatedly enters a large number of query commands that do not yield results, multiple database requests will occur, leading to cache penetration.

[0089] However, considering that the core of this invention lies in how to implement data caching, rather than solving problems such as cache penetration caused by abnormal user input, this application embodiment only considers the scenario where the query results of the user's input data query command exist in the database or cache. In fact, cache penetration can be solved through full caching or reverse caching strategies, as follows:

[0090] In layman's terms, full caching can be understood as caching all data in form A in the database. This means that when querying for results related to form A in the full cache, if the query doesn't contain the corresponding result, it indicates that the same result cannot be found in other caches or the database either, since all data in form A is already cached there. In this case, no result can be returned to the user. Reverse caching, on the other hand, associates query commands that also don't contain results in the database with "no result" in the cache. That is, when the query result in the reverse cache is "no result," it's usually indicated by the character "nul." This also signifies that the corresponding result cannot be found in other caches or the database, so no result can be returned to the user. Furthermore, as optional solutions, full caching can be implemented using a HashMap, while reverse caching can be implemented using a Bloom filter.

[0091] To facilitate understanding of the above content, Figure 9 The provided diagram illustrates a complete data caching architecture. The preloading cache employs both full caching and reverse caching strategies. Details regarding full caching and reverse caching will be discussed later. Figure 9 Further explanation will follow.

[0092] 303, Receive the query results returned by the target database, and set the query results returned by the target database as the target data.

[0093] As described above, this embodiment only considers the scenario where the database or cache contains the query results of the user's input data query command. Therefore, when a data query command is sent to the database, the database can obtain the corresponding query results based on the command and return them to the local client. This serves two purposes: firstly, to return the query results to the user, and secondly, to implement local caching of the query results. In this case, the query results returned by the target database can be considered as the target data to be cached.

[0094] Furthermore, when the target database returns the query results, it will mark the query results as hot data. In other words, the data marked as hot data is hot data. According to the description of the relevant content in step 203, the target data, that is, the query results returned by the target database, needs to be written into the hot data cache in the local cache.

[0095] Of course, even if the target database does not contain the corresponding query results, it will not affect the specific implementation of this solution. In this case, the target database only needs to return no results to the local client, and the data caching device will then return no results to the user. Of course, if the local cache adopts a reverse caching strategy, the data caching device will associate "no results" with the data query command in the hot cache.

[0096] like Figure 4 As shown, Figure 4 This is a flowchart illustrating the third embodiment of the data caching method provided in this application.

[0097] In this embodiment of the application, an implementation method for querying the local cache is proposed, specifically including steps 401 to 402:

[0098] 401. Query the preloaded cache in the local cache to determine if the preloaded cache contains a query result. If yes, return the query result; otherwise, proceed to step 402.

[0099] In this embodiment of the application, after receiving the data query instruction, the system first queries the preloaded cache in the local cache. If the query results exist, the system directly returns the query results to the user. The specific query implementation scheme will not be described in detail here.

[0100] In this embodiment of the application, compared with the hot spot cache, the preload cache is obtained based on the updated data of the database. Therefore, its timeliness is better than the data in the hot spot cache, the data is more accurate, but the data coverage is smaller.

[0101] 402, querying the hot cache in the local cache.

[0102] In this embodiment, if the preloaded cache in the local cache does not contain a query result, a further query will be performed in the hot cache. If a query result exists in the hot cache, the query result will be returned to the user. If no query result exists in the hot cache, it means that neither the preloaded cache nor the hot cache contains a query result. In this case, further steps need to be performed, such as sending a data query command to the target database as mentioned above, or performing a query in the remote cache.

[0103] In this application embodiment, a specific implementation scheme for querying the local cache is proposed. Specifically, the preloaded cache in the local cache is queried first, and then the hot cache in the local cache is further queried. Since the preloaded cache is updated based on the data update of the database, that is, its timeliness is better. Therefore, querying the preloaded cache first can further ensure the timeliness of the returned data.

[0104] like Figure 5 As shown, Figure 5 This is a flowchart illustrating the fourth embodiment of the data caching method provided in this application.

[0105] In this embodiment of the application, a further implementation scheme for querying the remote cache is proposed, specifically including steps 501 to 503:

[0106] 501. Query the remote cache to determine if a query result exists in the remote cache. If yes, proceed to step 502; otherwise, proceed to step 503.

[0107] In this embodiment, the remote cache is specifically queried according to the data query instruction, which will not be elaborated here.

[0108] In this embodiment, the remote cache can be simply understood as middleware between the local client and the database. It can form a multi-level cache with the local cache, thereby effectively solving the problem of cached data exceeding the memory capacity of a single node, i.e., the problem of excessively large cached data. Specifically, the remote cache adopts a distributed architecture, such as a Redis cluster, which will not be elaborated further in this application.

[0109] 502, Write the query results in the remote cache to the hotspot cache in the local cache.

[0110] In this embodiment, if the remote cache contains query results, these results need to be returned to the user. Simultaneously, the query results in the remote cache are also written to the hotspot cache in the local cache. Specifically, the writing process can be similar to that of the target database. The remote cache marks the query results with a hotspot identifier and sends them to the local client as target data to be cached. At this point, the data caching device on the local client identifies the identifier of the target data and writes it to the hotspot cache in the local cache.

[0111] 503, Send the data query instruction to the target database.

[0112] In this embodiment of the application, a data query instruction is sent to the target database to request query results only when the remote cache also does not contain query results.

[0113] This application proposes a multi-level caching structure based on local and remote caching, which can effectively increase the amount of cached data in the system, further avoid frequent data requests for target data, and improve the system's concurrency performance.

[0114] like Figure 6 As shown, Figure 6 This is a flowchart illustrating the fifth embodiment of the data caching method provided in this application.

[0115] In this embodiment of the application, another implementation scheme for obtaining the target data to be cached is proposed, specifically including steps 601 to 603:

[0116] 601, Generate a data update request based on the cached data in the preloaded cache.

[0117] In this embodiment of the application, as described above, the cached data marked with the data to be updated needs to be related to the cached data already in the preloaded cache. Therefore, before obtaining the target data, the data caching device on the local client will first obtain a data update request related to the pre-cached data based on the cached data already in the preloaded cache.

[0118] 602, Send the data update request to the target database.

[0119] In this embodiment, sending a data update request to the target database allows the database to retrieve the update status of the corresponding data. Specifically, after the target database obtains the update result associated with the cached data in the preloaded cache based on the data update request, it marks the updated data with a data identifier and then packages and returns it to the local client.

[0120] 603, Receive the update result returned by the target database, and set the update result as the target data.

[0121] In this embodiment of the application, similar to the query results returned by the target database, the updated results returned by the target database can also be used as target data. However, at this time, the data identifier of the target data is updated data, which indicates that the target data is data that has been updated in the database and is related to the cached data in the preload cache. Therefore, in conjunction with the description of the relevant content of step 202, it is necessary to write the target data, that is, the updated results returned by the target database, into the preload cache of the local cache.

[0122] It should be noted that there are many specific implementation schemes for the local client to send data update requests to the target database and receive update results returned by the target database. To further reduce the number of data requests to the database, as an optional embodiment of this application, the above process is implemented based on a message middleware. Specifically, the data caching device on the local client first generates subscription information based on the cached data in the preloaded cache, and then sends it to the message middleware. Subsequently, the target database will count the data that has changed within the period according to a predetermined period, such as daily or every 12 hours, and send it to the message middleware. At this time, the message middleware will further push the corresponding update data to the local client based on the subscription information sent by the data caching device on the local client, thereby facilitating the data caching device to update the preloaded cache in the local cache based on the pushed update data.

[0123] like Figure 7 As shown, Figure 7 This is a flowchart illustrating the sixth embodiment of the data caching method provided in this application.

[0124] This application provides an implementation scheme for writing cache data to a preloaded cache. It should be noted that the writing proposed in this application embodiment can be understood as an initialization writing process. Subsequent updates to the cache data in the preloaded cache are all based on the initialized cache data, specifically including steps 701 to 703:

[0125] 701. Statistical analysis is performed on the data update information of the stored data in the preset database to obtain data update characteristics.

[0126] In this embodiment of the application, as described above, the cached data initialized and written to the preload cache is obtained based on the data's update characteristics. Therefore, it is necessary to first determine which data needs to be initialized and written to the preload cache based on the data's update characteristics. Specifically, update characteristics can be, for example, update frequency, average update data volume, etc. As an optional embodiment of this application, update frequency is selected as the update characteristic.

[0127] In this embodiment, the data update characteristics can be obtained based on the update information of the data already stored in a preset database. Specifically, the preset database can be a complete database, i.e., the aforementioned target database, such as MySQL, or a remote cache, such as a Redis cluster, or a local hotspot cache. It should be noted that while databases store more comprehensive data, they are difficult to statistically analyze update information. In contrast, while hotspot caches may not have as comprehensive a database, they can quickly analyze data update information, facilitating the writing of preloaded caches. Remote caches fall somewhere in between. Therefore, users can determine whether to use the target database, remote cache, or local hotspot cache as the preset database based on their needs, complete the statistical analysis of data update information, and thus define the preloaded cache data.

[0128] 702, determine the preloaded data from the stored data based on the data update characteristics.

[0129] In this embodiment, specifically taking update frequency as a data update characteristic, data with an update frequency higher than a certain value in the pre-stored data in the preset database is usually identified as pre-loaded data. For this part of the data, since it needs frequent updates, a hotspot cache often needs to frequently send update requests to the database. Of course, it is also feasible to determine the pre-loaded data based on other rules, which will not be elaborated upon here.

[0130] 703, Write the preloaded data into the preload cache in the local cache.

[0131] In this embodiment, preloaded data, such as cached data in the aforementioned hotspot cache with an update frequency higher than a certain value, is written into the preloaded cache. This allows the hotspot cache to clear this portion, eliminating the need to send update requests to the database and effectively reducing the number of requests from the hotspot cache to the database. Furthermore, for the cached data stored in the preloaded cache, subsequent updates to the database data can be achieved through message brokers or other information transmission mechanisms.

[0132] As described in step 302 above, to avoid issues such as cache penetration, caching can be performed using either full caching or reverse caching strategies. As an optional embodiment of this application, preloading caching can also be performed using either full caching or reverse caching strategies. Specifically, taking full caching as an example, preloading caching can include a full cache area, where cached data is stored using a full caching strategy. For details on the specific implementation of writing preloaded data into the local cache, please refer to the subsequent sections. Figure 8 And its explanations and descriptions.

[0133] This application proposes a technical solution to initialize cached data in the preloaded cache based on the data update characteristics of cached data in the hot spot cache, which can effectively reduce the preloaded cache and improve the stability of the system.

[0134] like Figure 8 As shown, Figure 8 This is a flowchart illustrating the seventh embodiment of the data caching method provided in this application.

[0135] In this embodiment, the preloaded cache includes both a full cache area and a non-full cache area as an example for illustration. Specifically, it includes steps 801 to 803:

[0136] 801. Compare the amount of preloaded data with a preset data amount threshold to determine whether the amount of data is greater than the data amount threshold. If yes, proceed to step 802; otherwise, proceed to step 803.

[0137] In this embodiment of the application, as described above, full caching can be simply understood as caching all certain data in the database, which requires a large amount of cache space. Therefore, if the amount of data written to the preload cache is too large, it is not suitable to use the full caching strategy, i.e., step 802 needs to be executed; otherwise, the full caching strategy can be considered, specifically, step 803 needs to be executed.

[0138] 802, write the preloaded data into the non-full cache area of ​​the preload cache.

[0139] In this embodiment, a non-full cache strategy is adopted for caching in the non-full cache area of ​​the preloaded cache. For example, for preloaded data with a data volume exceeding the data volume threshold, i.e., a large data volume, a non-full cache strategy can be used to write it, that is, write it to the non-full cache area of ​​the preloaded cache. Specifically, reverse caching or other write strategies can be used in the non-full cache area, which will not be elaborated in this application.

[0140] 803, write the preloaded data into the full cache area of ​​the preload cache.

[0141] In this embodiment, in contrast to step 802 described above, a full caching strategy is adopted for caching in the full cache area of ​​the preloaded cache. Therefore, for preloaded data with a data volume less than the data volume threshold, i.e., a small data volume, a full caching strategy can be used to write it, that is, write it to the full cache area of ​​the preloaded cache, thereby avoiding cache penetration caused by abnormal user input.

[0142] As an optional embodiment of this application, the preload cache may further include other custom cache areas. The write strategy selected in the custom cache area can also be customized based on actual needs. Any write strategy that can be obtained based on the prior art can be set with a corresponding cache area, which will not be elaborated here.

[0143] In this embodiment, a corresponding caching strategy is adopted to write the data to different cache areas of the preload cache based on the size of the data to be written. This can ensure the performance of the preload cache. For example, for preload data with a small amount of data, the full cache strategy is used to write the data to the full cache area. This can reduce the possibility of cache penetration without occupying a lot of cache space, and effectively improve the stability of the system.

[0144] like Figure 9 The diagram shown is a complete architecture diagram of a data cache provided in an embodiment of this application, which is described in detail below.

[0145] In this embodiment, a caching framework employing both local and remote caching is used. Specifically, the local cache is deployed on the JVM, or Java Virtual Machine. The local cache can be divided into pre-loaded cache and hotspot cache, with the hotspot cache implemented based on Caffeine. The pre-loaded cache can be further divided into a full cache implemented based on HashMap, a reverse cache implemented based on Bloom filters, and a custom cache based on user-defined rules. Furthermore, the remote distributed cache uses a Redis cluster, and the target database is a MySQL database.

[0146] Specifically, within the aforementioned caching framework, a pre-loaded cache needs to be pre-initialized and configured, which involves writing data into the pre-loaded cache to achieve preheating. This is based on the aforementioned... Figure 7The proposed solution is determined by the update characteristics of cached data in the hot cache. Cache data with an update frequency higher than a certain threshold in the hot cache is written to the preload cache. During the writing process, the cached data is written to the full cache area, reverse cache area, or other custom cache area in the preload cache, depending on the size of the cache data being written.

[0147] When data needs to be queried, after receiving a data query command, the system first searches the preloaded cache based on the query fields specified in the command. If the data exists, the result is returned directly. If not, there are two possibilities: either the data is not in the preloaded cache but exists in other caches or databases, such as hotspot caches, remote caches, or MySQL; or the queried data simply does not exist, meaning no results are found in other caches or databases. Therefore, to avoid resource consumption, it can first determine whether the search is performed in the full cache area of ​​the preloaded cache. If so, it indicates that the corresponding data is unlikely to exist in other caches and databases, and no results can be returned directly. If not, for example, if the search is performed in the reverse cache area, the principle of a Bloom filter can be used to determine whether other caches or databases might contain the query result. Similarly, if it is unlikely, no results can be returned directly; if it is likely, the hotspot cache query step is initiated. In other words, the query is performed in the hot cache. If the data exists in the hot cache, the result is returned directly. If not, similar to preloading the cache, the hot cache can also be queried based on full caching and reverse caching strategies to determine whether further queries from a remote distributed cache or database are needed. If necessary, a remote cache query is performed, i.e., a query in the Redis cluster. If the data exists, the query result is returned to the local cache to provide the user with the query result. Simultaneously, the query result is written to the hot cache. If the hot cache cleanup mechanism is triggered, such as when the hot cache space reaches its limit, the hot cache is cleaned up according to given rules. If the data does not exist, a request is made to the database, i.e., MySQL. If the data exists, the query result is returned to the remote cache and local cache in turn, and then returned to the user, and written to the remote cache and / or hot cache. If the data does not exist, it indicates that there are no query results for this data query command. Therefore, a mapping relationship between data query commands and no query results can be written in the reverse cache, so that when the user enters the same data query command again, it can be directly determined from the reverse cache that there are no corresponding query results for that data query command.

[0148] Furthermore, this embodiment of the application will further update the cached data of the preloaded cache based on a message subscription and publish mechanism. Specifically, MySQL will publish update information according to certain rules, for example, by statistically analyzing the data changes in the database daily or every 12 hours to obtain update information. If a remote cache exists, it will first update the remote cache based on the update information so that the remote cache can complete the publication of the updated data based on the message middleware. If no remote cache exists, MySQL will directly publish the update information to the message middleware. However, regardless of whether a remote cache exists, after receiving the published update information, the message middleware will return the corresponding update information to the local cache according to the subscription information of the preloaded cache, and use it to update the preloaded cache. Of course, in this process, the preloaded cache will send a subscription request to the message middleware in advance based on the cached data.

[0149] Through the above process, frequently updated data is stored in a preloaded cache and updated periodically using a message subscription and publish mechanism. Meanwhile, data frequently requested by users is cached in a hotspot cache, which can greatly reduce the number of requests to remote cache and database, and improve system performance.

[0150] To better implement the data caching method in the embodiments of this application, a data caching device is also provided in the embodiments of this application, such as... Figure 10 As shown, Figure 10 This is a schematic diagram of an embodiment of a data caching device, specifically including:

[0151] The cached data acquisition module 1001 is used to acquire the target data to be cached and the data identifier of the target data;

[0152] The preload cache module 1002 is used to write the target data into the preload cache in the local cache if the data identifier is updated data;

[0153] The hotspot caching module 1003 is used to write the target data into the hotspot cache in the local cache if the data identifier is hot data.

[0154] In some embodiments of this application, the above-described data caching device further includes:

[0155] The local query module is used to receive data query commands and query the local cache.

[0156] The database query module is used to send the data query instruction to the target database if the local cache does not contain a query result.

[0157] The first cached data determination module is used to receive the query results returned by the target database and set the query results returned by the target database as the target data.

[0158] In some embodiments of this application, the local query module includes:

[0159] The preloaded cache query unit is used to query the preloaded cache in the local cache;

[0160] The hotspot cache query unit is used to query the hotspot cache in the local cache if the preloaded cache does not have a query result.

[0161] In some embodiments of this application, the local query module further includes:

[0162] The remote cache query unit is used to query the remote cache.

[0163] A database query unit is used to send the data query instruction to the target database if the remote cache does not contain a query result;

[0164] A hotspot caching unit is used to write the query result in the remote cache into the hotspot cache in the local cache if the query result is cached in the remote cache.

[0165] In some embodiments of this application, the above-described data caching device further includes:

[0166] The update request module is used to generate data update requests based on cached data in the preloaded cache.

[0167] The request sending module is used to send the data update request to the target database;

[0168] The second cached data determination module is used to receive the update result returned by the target database and set the update result as the target data.

[0169] In some embodiments of this application, the above-described data caching device further includes:

[0170] The statistics module is used to statistically analyze the data update information of the stored data in the preset database and obtain the data update characteristics.

[0171] A preloaded data determination module is used to determine preloaded data from the stored data based on data update characteristics;

[0172] The preload cache writing module is used to write the preloaded data into the preload cache in the local cache.

[0173] In some embodiments of this application, the preloaded cache includes a full cache area and a non-full cache area; the preloaded cache writing module includes:

[0174] The comparison unit is used to compare the amount of the preloaded data with a preset data amount threshold;

[0175] A non-full cache writing unit is used to write the preloaded data into the non-full cache area of ​​the preloaded cache if the data volume is greater than the data volume threshold.

[0176] The full cache write unit is used to write the preloaded data into the full cache area of ​​the preloaded cache if the preloaded data is less than or equal to the data volume threshold.

[0177] This invention also provides a data caching device, such as... Figure 11 As shown, Figure 11 This is a schematic diagram of an embodiment of the data caching device provided in this application.

[0178] The data caching device includes a memory, a processor, and a data caching program stored in the memory and executable on the processor. When the processor executes the data caching program, it implements the steps of the data caching method in any embodiment.

[0179] Specifically, the data caching device may include components such as a processor 1101 with one or more processing cores, a memory 1102 with one or more storage media, a power supply 1103, and an input unit 1104. Those skilled in the art will understand that... Figure 11 The data caching device structure shown does not constitute a limitation on the data caching device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0180] The processor 1101 is the control center of the data cache device. It connects various parts of the data cache device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 1102, and by calling data stored in the memory 1102, it performs various functions of the data cache device and processes data, thereby providing overall monitoring of the data cache device. Optionally, the processor 1101 may include one or more processing cores; preferably, the processor 1101 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1101.

[0181] The memory 1102 can be used to store software programs and modules. The processor 1101 executes various functional applications and data processing by running the software programs and modules stored in the memory 1102. The memory 1102 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of data caching devices, etc. In addition, the memory 1102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1102 may also include a memory controller to provide the processor 1101 with access to the memory 1102.

[0182] The data caching device also includes a power supply 1103 that supplies power to the various components. Preferably, the power supply 1103 can be logically connected to the processor 1101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 1103 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0183] The data buffer device may also include an input unit 1104, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0184] Although not shown, the data caching device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1101 in the data caching device loads the executable files corresponding to the processes of one or more applications into the memory 1102 according to the following instructions, and the processor 1101 runs the applications stored in the memory 1102, thereby implementing the steps in any of the data caching methods provided in the embodiments of the present invention.

[0185] Therefore, embodiments of the present invention provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc. The computer-readable storage medium stores a data caching program, which, when executed by a processor, implements the steps of any of the data caching methods provided in the embodiments of the present invention.

[0186] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0187] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0188] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0189] The data caching method provided by the embodiments of this application has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data caching method, characterized in that, include: Obtain the target data to be cached, and the data identifier of the target data; If the data identifier indicates that data needs to be updated, the target data is written into the preloaded cache in the local cache; the updated data is the data updated in the database within a preset period. If the data identifier indicates hot data, then the target data is written into the hot data cache in the local cache; the hot data is determined based on the user's hot data needs.

2. The data caching method according to claim 1, characterized in that, Before the step of obtaining the target data to be cached, the method further includes: Receive data query commands and query the local cache; If the local cache does not contain the query result, the data query instruction is sent to the target database. Receive the query results returned by the target database, and set the query results returned by the target database as the target data.

3. The data caching method according to claim 2, characterized in that, The query local cache includes: Query the preloaded cache in the local cache; If the preloaded cache does not have a query result, then the hot cache in the local cache is queried.

4. The data caching method according to claim 2, characterized in that, After the step of determining if the local cache does not contain a query result, the method further includes: Query remote cache; If the remote cache does not contain a query result, then the step of sending the data query instruction to the target database is executed; If the query results are cached remotely, then the query results in the remote cache are written into the hotspot cache in the local cache.

5. The data caching method according to claim 1, characterized in that, Before the step of obtaining the target data to be cached, the method further includes: Generate a data update request based on the cached data in the preloaded cache; Send the data update request to the target database; Receive the update result returned by the target database, and set the update result as the target data.

6. The data caching method according to any one of claims 1 to 5, characterized in that, Before the step of obtaining the target data to be cached, the method further includes: The data update information of the stored data in the preset database is statistically analyzed to obtain data update characteristics; Preloaded data is determined from the stored data based on data update characteristics; The preloaded data is written to the preload cache in the local cache.

7. The data caching method according to claim 6, characterized in that, The preloaded cache includes a full cache area and a non-full cache area; The step of writing the preloaded data into the local cache preload cache includes: The amount of preloaded data is compared with a preset data amount threshold; If the amount of data is greater than the data amount threshold, the preloaded data is written to the non-full cache area in the preload cache; If the amount of data is less than or equal to the data amount threshold, then the preloaded data is written into the full cache area of ​​the preload cache.

8. A data caching device, characterized in that, include: The cached data acquisition module is used to acquire the target data to be cached, and the data identifier of the target data; A preload cache module is used to write the target data into the preload cache in the local cache if the data identifier indicates that the data is to be updated; the updated data is the data updated in the database within a preset period. A hotspot caching module is used to write the target data into the hotspot cache in the local cache if the data identifier indicates hotspot data; the hotspot data is determined based on the user's hotspot needs.

9. A data caching device, characterized in that, The data caching device includes a processor, a memory, and a data caching program stored in the memory and executable on the processor, wherein the processor executes the data caching program to implement the steps of the data caching method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a data caching program, which is executed by a processor to implement the steps of the data caching method according to any one of claims 1 to 7.

Images