A data caching method and apparatus, electronic device, and storage medium

Abstract

The application discloses a data caching method and device, electronic equipment and a computer readable storage medium. The method is applied to a caching system, the caching system comprises a first cache and a second cache, the first cache and the second cache are connected with a database, and the first cache is a Java virtual machine cache. The method comprises the following steps: loading target data from the second cache by the first cache every preset time interval; wherein the target data is data with a change frequency lower than a first preset value, a reading frequency higher than a second preset value and a data volume smaller than a third preset value; if the target data does not exist in the second cache, requesting a distributed lock from the database by the first cache, so as to load the target data in the database into the second cache. The target data with low change frequency, high reading frequency and small data volume is stored and operated in a pure memory level by the Java virtual machine cache, the network overhead is reduced, the caching performance is improved, and thus the caching demand in a complex scene is met.

Description

Technical Field

[0001] This application relates to the field of storage technology, and more specifically, to a data caching method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Distributed computing can be understood as the deployment of services across different locations and data centers. Redis (Remote Dictionary Server) caching, as a superior distributed caching service, has been widely used. In related technologies, Redis is often used as the sole cache. However, complex scenarios involve intricate business logic and high performance requirements for the caching system. Furthermore, relying on Redis incurs network overhead, creating a performance bottleneck. Therefore, a caching system that only includes Redis cannot meet the caching needs of complex scenarios.

[0003] Therefore, how to improve the performance of caching systems to meet the caching needs in complex scenarios is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a data caching method, apparatus, electronic device, and computer-readable storage medium, which improves the performance of the caching system and thus meets the caching needs in complex scenarios.

[0005] To achieve the above objectives, this application provides a data caching method applied to a caching system, the caching system including a first-level cache and a second-level cache, both of which are connected to a database, the first-level cache being specifically a Java Virtual Machine cache;

[0006] The method includes:

[0007] The target data is loaded from the second-level cache at preset time intervals through the first-level cache; wherein the target data is data whose change frequency is lower than a first preset value, whose read frequency is higher than a second preset value, and whose data volume is less than a third preset value;

[0008] If the target data is not present in the second-level cache, a distributed lock is requested from the database through the first-level cache to load the target data from the database into the second-level cache.

[0009] The first-level cache is used to cache the target data, and the second-level cache is used to cache other data besides the target data.

[0010] The Java Virtual Machine cache includes an index part and a data part. The data part includes multiple partitions, and the data structure of each partition is a Map structure.

[0011] The Java Virtual Machine cache includes an index portion and a data portion, wherein the data portion is stored using a serialized storage method.

[0012] The secondary cache includes a Redis cache or a Memcached cache.

[0013] The Java Virtual Machine cache connects to the database through a distributed application coordination service.

[0014] This also includes:

[0015] The Java Virtual Machine cache listens for events in the distributed application coordination service and performs corresponding operations based on the listened-for events; the events include update events and session events, the update events are events that update target data in the database, and the session events are communication connection events.

[0016] Wherein, if the event is specifically the update event, then the step of performing the corresponding operation based on the monitored event includes:

[0017] The Java Virtual Machine cache listens for notification messages generated by the distributed application coordination service; wherein, the notification message includes version information of the updated target data;

[0018] The latest version of the target data is retrieved from the database based on the version information in the notification message.

[0019] The step of retrieving the latest version of the target data from the database based on the version information in the notification message includes:

[0020] Retrieve the latest version of the target data from the database;

[0021] Determine whether the version information of the target data being pulled is consistent with the version information in the notification message;

[0022] If not, the process will re-enter the step of pulling the latest version of the target data from the database until the version information of the pulled target data matches the version information in the notification message.

[0023] Wherein, if the event is specifically a session event, then performing the corresponding operation based on the listened-to event includes:

[0024] Generate alarm information, enter safe mode, or take the service corresponding to the Java Virtual Machine cache offline.

[0025] This also includes:

[0026] If a business request is received, it is determined whether the business data corresponding to the business request exists in the cache system;

[0027] If so, the business data in the cache system will be returned directly;

[0028] If not, a distributed lock is requested from the database. If the request is successful, the business request is sent to the database to query the business data corresponding to the business request.

[0029] Before requesting the distributed lock from the database, the process also includes:

[0030] Set the returned data to empty;

[0031] Accordingly, querying the database for the business data corresponding to the business request includes:

[0032] If the business data corresponding to the business request is found in the database, the cache system and the returned data are updated based on the business data, the returned data is returned, and the distributed lock is released.

[0033] If the business data corresponding to the business request is not found in the database, the returned data is returned directly, and the distributed lock is released.

[0034] To achieve the above objectives, this application provides a data caching device applied to a caching system. The caching system includes a first-level cache and a second-level cache, both connected to a database. Specifically, the first-level cache is a Java Virtual Machine cache. The first-level cache is used to cache target data whose change frequency is lower than a first preset value, whose read frequency is higher than a second preset value, and whose data volume is less than a third preset value. The second-level cache is used to cache other data besides the target data. The device includes:

[0035] The first loading module is used to load the target data from the second-level cache at preset time intervals using the first-level cache;

[0036] The second loading module is used to request a distributed lock from the database through the first-level cache when the target data is not present in the second-level cache, so as to load the target data in the database into the second-level cache.

[0037] To achieve the above objectives, this application provides an electronic device, comprising:

[0038] Memory, used to store computer programs;

[0039] A processor for implementing the data caching method described above when executing the computer program.

[0040] To achieve the above objectives, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data caching method described above.

[0041] As can be seen from the above scheme, the caching method provided in this application is applied to a caching system, which includes a first-level cache and a second-level cache. Both the first-level cache and the second-level cache are connected to a database. The first-level cache is specifically a Java Virtual Machine cache. The method includes: loading target data from the second-level cache through the first-level cache at preset time intervals; wherein, the target data is data whose change frequency is lower than a first preset value, whose read frequency is higher than a second preset value, and whose data volume is less than a third preset value; if the target data does not exist in the second-level cache, then requesting a distributed lock from the database through the first-level cache to load the target data from the database into the second-level cache.

[0042] In this application, the Java Virtual Machine (JVM) cache is used as the first-level cache, and an externally dependent caching service is used as the second-level cache. Target data that is frequently changed, frequently accessed, and small in volume is stored and manipulated purely in memory through the JVM cache, thus avoiding the performance bottleneck caused by the network consumption of the externally dependent second-level cache. Therefore, the caching method provided in this application reduces network overhead and improves caching performance, thereby meeting the caching needs in complex scenarios. This application also discloses a caching device, electronic device, and computer-readable storage medium that can achieve the same technical effects.

[0043] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the following detailed description to explain this disclosure, but do not constitute a limitation of this disclosure. In the drawings:

[0045] Figure 1 This is a flowchart illustrating a data caching method according to an exemplary embodiment;

[0046] Figure 2 This is a structural diagram illustrating a data caching system according to an exemplary embodiment;

[0047] Figure 3 This is a structural diagram of a Java Virtual Machine cache according to an exemplary embodiment;

[0048] Figure 4 A flowchart illustrating another data caching method according to an exemplary embodiment;

[0049] Figure 5 A flowchart illustrating yet another data caching method according to an exemplary embodiment;

[0050] Figure 6 This is a structural diagram illustrating a data caching device according to an exemplary embodiment;

[0051] Figure 7 This is a structural diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, in the embodiments of this application, "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0053] This application discloses a data caching method that improves the performance of the caching system, thereby meeting the caching needs in complex scenarios.

[0054] See Figure 1 A flowchart illustrating a data caching method according to an exemplary embodiment, such as... Figure 1 As shown, it includes:

[0055] S101: Load target data from the second-level cache at preset time intervals using the first-level cache; wherein, the target data is data whose change frequency is lower than a first preset value, whose read frequency is higher than a second preset value, and whose data volume is less than a third preset value;

[0056] In this embodiment, the execution entity is a caching system, which is as follows: Figure 2 As shown, it includes a first-level cache 101 and a second-level cache 102, both of which are connected to the database 103. The first-level cache 101 is specifically a Java Virtual Machine cache. The first-level cache 101 is used to cache target data whose change frequency is lower than a first preset value, whose read frequency is higher than a second preset value, and whose data volume is less than a third preset value. The second-level cache 102 is used to cache other data besides the target data.

[0057] In this embodiment, the caching system includes a multi-level cache, namely a first-level cache 101 and a second-level cache 102. The first-level cache resides within the Java Virtual Machine (JVM), i.e., the JVM cache. The JVM is a specification for computing devices; it is a virtual computer implemented by simulating various computer functions on a real computer. The second-level cache may include Redis cache, Memcached cache, etc., without specific limitations here. Redis is an open-source, ANSI C-written, network-enabled, in-memory or persistent log-structured key-value database that provides APIs (Application Programming Interface) in multiple languages.

[0058] The Java Virtual Machine (JVM) cache 101 is used to cache target data whose change frequency is lower than a first preset value, whose read frequency is higher than a second preset value, and whose data volume is less than a third preset value. The first, second, and third preset values ​​are not specifically limited here and can be set according to the actual business scenario. Of course, the data types cached in the JVM cache can also be set according to the actual business scenario. The principle is to cache data with low change frequency, high read frequency, and small data volume in the JVM cache. For example, in a user authentication system, the purpose of an external business request to the authentication system is to query whether a user's authentication information exists. If it exists, it needs to return sensitive information such as the user's name, ID card number, and mobile phone number. To ensure data interaction security, it is necessary to clearly define the business caller's exclusive channel number and key for authentication and identity recognition. First, the channel number and key will not change after generation. Second, each business request needs to be authenticated first before the business data can be requested. Therefore, every business request requires the reading of authentication data, meaning that the reading of authentication data is relatively frequent. Furthermore, authentication data typically only includes the channel number and key. Each business requester corresponds to one set of authentication data, i.e., one channel number and one key. The number of business requesters is controllable, meaning the storage size of authentication data is very small. Therefore, the authentication data used for authentication conforms to the principles of low-frequency changes, high-frequency reads, and small data volume, and can be cached in the Java Virtual Machine cache.

[0059] In practical implementation, Java Virtual Machine caching 101 can include HashMap, thread-safe ConcurrentHashMap, Caffeine, etc., without specific limitations. All three are key-value stores, meaning they all include an index (key) and a data part (value). Their specific implementation details differ. The HashMap and ConcurrentHashMap provided by the JDK (Java Development Kit) are simple to operate and implement, while Caffeine is a well-encapsulated framework natively designed for local caching, offering many caching features.

[0060] Understandably, because the Java Virtual Machine cache 101 uses a key-value format for storage, it can only support single-dimensional data storage of the index portion (key), making it suitable for storing one-to-one mapping data. For example, storing a one-to-one relationship between "name" and "ID number" would be done in the format: <name, ID number>. Therefore, for complex data storage and retrieval scenarios, the data portion (value) can be processed.

[0061] As a feasible implementation method, the data portion is stored using serialization. Serialization is a flexible storage method; if the data format is very complex, it can be stored using a specific text format. For example, multi-dimensional data can be reduced to one-dimensional data using a specified format for storage.

[0062] As another feasible implementation, the data portion includes multiple partitions, each partition having a Map structure. In specific implementations, the introduction of regions enables the Java Virtual Machine cache to support multi-dimensional business logic. For example... Figure 3As shown, each Java Virtual Machine cache 101 (labeled LocalCache in the diagram) implementation internally holds a visibility Map. <Region,LocalCache<Object,Object> Each Region is a singleton, meaning it is initialized only once. The Java Virtual Machine cache can be simply understood as two nested Map data structures. Data access is based on Region partitions, which is equivalent to adding the concept of partitions to the one-to-one mapping relationship. In the example of the mapping relationship between "name" and "ID number" above, if it is necessary to store the identity information of each student in class A and class B, two partitions, Region A and Region B, can be created. Region A is used to store the <name, ID number> of each student in class A, and Region B is used to store the <name, ID number> of each student in class B. It can be seen that this implementation optimizes the data structure of the Java Virtual Machine cache, uses a multi-dimensional hash table for key-value data storage, reduces the performance consumption caused by serialization, supports data query matching with O(1) time complexity, and further improves performance compared to linear data matching.

[0063] Furthermore, the second-level cache 102 is used to store other data besides the target data, i.e., data with a huge volume of business data, uncontrollable changes, and high-frequency access. For the user authentication system, the data cached in the second-level cache 102 may include channel values, authorization codes, business authorizations, authorization fields, de-identified fields, and business information, etc. The specific data types and storage data structures are shown in Table 1:

[0064] Table 1

[0065]

[0066]

[0067] It should be noted that during the initial startup phase of the caching system service, target data with a change frequency lower than a first preset value, a read frequency higher than a second preset value, and a data volume less than a third preset value needs to be loaded into the Java Virtual Machine cache. In specific implementation, the Java Virtual Machine cache checks whether the target data exists in the second-level cache every preset time interval, i.e., it performs a polling operation on the second-level cache for the target data. If it exists, the target data in the second-level cache is directly loaded into the Java Virtual Machine cache.

[0068] S102: If the target data does not exist in the second-level cache, a distributed lock is requested from the database through the first-level cache to load the target data from the database into the second-level cache.

[0069] In practice, if the target data is not found in the second-level cache, the Java Virtual Machine (JVM) cache requests the database to fetch the target data. Understandably, to avoid frequent data requests and retrievals from multiple JVM caches in the JVM cluster simultaneously, this embodiment uses a distributed lock for control. Once a JVM cache successfully requests the distributed lock, the database loads the target data into the second-level cache; that is, the second-level cache fetches the target data from the database. Only one data retrieval operation is initiated at a time. If the distributed lock is not successfully requested, it means that other JVM caches have successfully requested the distributed lock, and the second-level cache is fetching the target data from the database. The JVM caches that did not successfully request the distributed lock wait for the next asynchronous polling to load the target data. In other words, the second-level cache acts as an intermediary during the JVM cache's loading of target data.

[0070] Therefore, this embodiment controls the loading of target data through a distributed lock. At any given time, only one Java Virtual Machine cache that acquires the distributed lock is stored in the distributed JVM cluster. The target data in the database is loaded into the second-level cache, meaning that the target data in the database is only loaded once. Multiple Java Virtual Machine caches in the JVM cluster load the target data from the second-level cache, thus improving the loading efficiency of the target data.

[0071] The caching method provided in this application is applied to a caching system that includes multi-level caching. Specifically, the Java Virtual Machine (JVM) cache is used as the first-level cache, and externally dependent caching services are used as the second-level cache. Target data that is frequently changed, frequently accessed, and has a small data volume is stored and manipulated purely in memory through the JVM cache, avoiding the performance bottleneck caused by the network consumption of the externally dependent second-level cache. Therefore, the caching method provided in this application reduces network overhead, improves caching performance, and thus meets the caching needs in complex scenarios.

[0072] It should be noted that data updates in the second-level cache can be triggered directly by business methods, while data updates in the Java Virtual Machine cache need to be coordinated through a distributed application coordination service because the JVM cluster is a distributed cluster.

[0073] Based on the above embodiments, as a preferred implementation, the Java Virtual Machine cache 101 is connected to the database 103 through a distributed application coordination service 104. In a specific implementation, the JVM can be deployed in a distributed manner, and the JVM cluster includes multiple Java Virtual Machine caches 101. The distributed application coordination service 104 performs global scheduling of the multiple Java Virtual Machine caches 101 in the distributed JVM cluster, coordinating secondary caches to implement a multi-level caching scheme. The distributed application coordination service 104 here may include ZooKeeper, which is a distributed, open-source distributed application coordination service that provides consistency services for distributed applications.

[0074] Accordingly, the caching method provided in the above embodiments further includes: listening to events of the distributed application coordination service through the Java Virtual Machine cache, and performing corresponding operations based on the listened events; the events include update events and session events, the update event being an event that updates target data in the database, and the session event being a communication connection event.

[0075] In practice, each Java Virtual Machine cache in the JVM cluster performs corresponding operations by listening to events from the distributed application coordination service. These events include update events, which drive the update logic of the target data in the database and can include data changes, data expiration, etc. Session events are used to drive the change logic of communication connections, such as interrupting the connection.

[0076] If the aforementioned event is specifically an update event, then the step of performing the corresponding operation based on the monitored event includes: the Java Virtual Machine cache listens for the notification message generated by the distributed application coordination service; wherein, the notification message includes version information of the updated target data; and the latest version of the target data is retrieved from the database based on the version information in the notification message.

[0077] It's understandable that the CAP theorem is a fundamental principle in distributed development, namely, the three properties C (Consistency), A (Availability), and P (Partition Tolerance). Theoretically, only two of these properties can be satisfied simultaneously. This embodiment employs an AP design, meaning it doesn't prioritize data consistency but prioritizes service availability. In other words, data can be temporarily inaccurate, but the service must always be available. In practice, data consistency in the Java Virtual Machine cache relies on a distributed application coordination service. By monitoring the path, data changes trigger path changes, generating update events. The distributed application coordination service encapsulates the updated data version number into a notification message. Upon receiving this notification message, the Java Virtual Machine cache retrieves the latest version of the target data from the database, ensuring data consistency.

[0078] Specifically, the step of retrieving the latest version of the target data from the database based on the version information in the notification message includes: retrieving the latest version of the target data from the database; determining whether the version information of the retrieved target data is consistent with the version information in the notification message; if not, re-entering the step of retrieving the latest version of the target data from the database until the version information of the retrieved target data is consistent with the version information in the notification message. In a specific implementation, the Java Virtual Machine cache continuously listens for notification messages from the distributed application coordination service and simultaneously retrieves the latest version of the target data from the database. The specific retrieval process is similar to the target data loading process described in the previous embodiment, and also requires a secondary cache for relay, which will not be elaborated here. The Java Virtual Machine cache compares whether the version information of the retrieved target data is consistent with the version information in the latest received notification message. If they are consistent, the data is consistent; if not, it needs to be retrieved again. For example, target data A is updated to B, and the version number of data B is 1, generating notification message a. When the Java Virtual Machine (JVM) cache hears notification message a, it fetches data from the database. At this time, data B is updated to C, and the version number of data C is 2, generating notification message b. At this time, the version number 2 of the data fetched by the JVM cache is inconsistent with the version number 1 in notification message a, so it continues to fetch data. At this time, the JVM cache receives notification message b, and the version number of the data fetched from the database is 2, which is consistent with the version number 2 in notification message a, and the fetching is complete.

[0079] It should be noted that when the target data in the database is updated, during the process of the Java Virtual Machine (JVM) cache fetching the updated data, the data in the JVM cache is dirty data. Data in multiple JVM caches are updated successively and eventually become consistent with the database. Even if the data is inconsistent during the update process, the JVM cache service is still available.

[0080] If the above event is specifically a session event, then the steps of performing the corresponding operation based on the listened event include: generating alarm information, entering safe mode, or controlling the Java Virtual Machine cache service to go offline.

[0081] In practical implementation, the Distributed Application Coordination Service (DOCSS) is the sole communication channel between the Java Virtual Machine (JVM) cache and the database. If a network or middleware failure occurs, communication may be impossible, and data cannot be modified. To address this extreme situation, one feasible implementation is to start a periodically polling daemon thread to monitor the data in the JVM cache. This daemon thread performs a polling task, verifying the data in the JVM cache at preset time intervals, and generating an alarm message if an anomaly is detected. Another feasible implementation is to generate an alarm message, enter safe mode, or take the corresponding JVM cache service offline if a communication failure occurs between the JVM cache and the DOCSS. Alarm messages can be pushed via third-party services such as telephone, SMS, or mobile apps. Entering safe mode degrades the service corresponding to the JVM cache.

[0082] As can be seen, this implementation method uses a distributed application coordination service to perform distributed scheduling and management of a distributed JVM cluster. Data can be loaded and hot-updated instantly without service downtime or restart, thus achieving data updates that have no impact on services and are imperceptible to business operations, thereby improving the maintainability of the system.

[0083] This application discloses another caching method, specifically:

[0084] See Figure 4 A flowchart illustrating another caching method according to an exemplary embodiment, such as... Figure 4 As shown, it includes:

[0085] S201: If a business request is received, determine whether the business data corresponding to the business request exists in the cache system; if yes, proceed to S202; if no, proceed to S203.

[0086] S202: Directly return the business data from the cache system;

[0087] S203: Request a distributed lock from the database. After the request is successful, send the business request to the database to query the business data corresponding to the business request in the database.

[0088] In this embodiment, when a business request arrives, the system first checks if the corresponding business data exists in the cache system. If it exists, it returns the data directly. If it does not exist, the system requests data from the database. It is understood that to prevent multiple business requests from arriving at the database simultaneously, a race condition to acquire a distributed lock is required. Business requests that successfully acquire the distributed lock can then smoothly reach the database to retrieve the data.

[0089] This application discloses a caching method, which, compared to the previous embodiment, further explains and optimizes the technical solution. Specifically:

[0090] See Figure 5 A flowchart illustrating another caching method according to an exemplary embodiment, such as... Figure 5 As shown, it includes:

[0091] S301: If a service request is received, determine whether the service data corresponding to the service request exists in the cache system; if yes, proceed to S302; if no, proceed to S303.

[0092] S302: Directly return the business data from the cache system;

[0093] S303: Set the returned data to null;

[0094] S304: Request a distributed lock from the database. If the request is successful, send the business request to the database.

[0095] S305: Determine whether the business data corresponding to the business request is found in the database; if yes, proceed to S306; if no, proceed to S307.

[0096] S306: Update the cache system and the returned data based on the business data, and proceed to S307;

[0097] S307: Return the returned data and release the distributed lock.

[0098] In this embodiment, if the business data corresponding to the business request does not exist in the caching system, null-value caching is performed to prevent cache penetration, i.e., the returned data is set to empty. If the business data is found in the database, the cache and returned data are immediately updated, the returned data is returned, and the distributed lock is released. If the data is not found, the returned data is returned directly, and the distributed lock is released.

[0099] It's important to note that if a business method involves changes to business data, aspect-oriented programming (AOP) is required to ensure that the cache is cleared immediately, guaranteeing consistency between the cache and the database. Specifically, after updating the database, the system determines whether to remove cached data based on the returned results. If the database update is successful, the cached data is removed immediately; otherwise, it is not removed.

[0100] The following describes a data caching device provided in an embodiment of this application. The data caching device described below and the data caching method described above can be referred to each other.

[0101] See Figure 6 A structural diagram of a data caching device according to an exemplary embodiment is shown, such as... Figure 6 As shown, it includes:

[0102] The first loading module 601 is used to load the target data from the second-level cache through the first-level cache at preset time intervals;

[0103] The second loading module 602 is used to request a distributed lock from the database through the first-level cache when the target data does not exist in the second-level cache, so as to load the target data in the database into the second-level cache.

[0104] The caching device provided in this application embodiment is applied to a caching system including multi-level caching. Specifically, the Java Virtual Machine cache is used as the first-level cache, and the externally dependent caching service is used as the second-level cache. Target data that is frequently changed, frequently accessed, and has a small data volume is stored and manipulated purely in memory through the Java Virtual Machine cache, avoiding the performance bottleneck caused by the network consumption of the externally dependent second-level cache. Therefore, the caching device provided in this application embodiment reduces network overhead and improves caching performance, thereby meeting the caching needs in complex scenarios.

[0105] Based on the above embodiments, as a preferred implementation, the first-level cache is used to cache the target data, and the second-level cache is used to cache other data besides the target data.

[0106] Based on the above embodiments, as a preferred embodiment, the Java Virtual Machine cache includes an index part and a data part, the data part includes multiple partitions, and the data structure of each partition is a Map structure.

[0107] Based on the above embodiments, as a preferred implementation, the Java Virtual Machine cache includes an index portion and a data portion, wherein the data portion is stored in a serialized storage manner.

[0108] Based on the above embodiments, as a preferred implementation, the secondary cache includes a Redis cache or a Memcached cache.

[0109] Based on the above embodiments, as a preferred implementation, the Java Virtual Machine cache connects to the database through a distributed application coordination service.

[0110] Based on the above embodiments, as a preferred embodiment, it further includes:

[0111] A listening module is used to listen for events of the distributed application coordination service through the Java Virtual Machine cache; the events include update events and session events, the update events are events that update target data in the database, and the session events are communication connection events;

[0112] The execution module is used to perform corresponding operations based on the events that are listened to.

[0113] Based on the above embodiments, as a preferred implementation, if the event is specifically the update event, then the execution module includes:

[0114] A listening unit is configured to listen for notification messages generated by the distributed application coordination service through the Java Virtual Machine cache; wherein the notification message includes version information of the updated target data;

[0115] The pull unit is used to pull the latest version of the target data from the database based on the version information in the notification message.

[0116] Based on the above embodiments, as a preferred embodiment, the pull unit includes:

[0117] The pull subunit is used to pull the latest version of the target data from the database;

[0118] The judgment subunit is used to determine whether the version information of the target data being pulled is consistent with the version information in the notification message; if not, the workflow of the pull subunit is restarted until the version information of the target data being pulled is consistent with the version information in the notification message.

[0119] Based on the above embodiments, as a preferred implementation, if the event is specifically the session event, then the execution module is specifically a module that generates alarm information, enters safe mode, or controls the service corresponding to the Java Virtual Machine cache to go offline.

[0120] Based on the above embodiments, as a preferred embodiment, it further includes:

[0121] The judgment module is used to determine whether the business data corresponding to the business request exists in the cache system when a business request is received; if yes, the return module's workflow is started; if no, the query module's workflow is started.

[0122] The return module is used to directly return the business data from the cache system;

[0123] The query module is used to request a distributed lock from the database. After the request is successful, the business request is sent to the database to query the business data corresponding to the business request.

[0124] Based on the above embodiments, as a preferred implementation, before requesting the distributed lock from the database, the method further includes:

[0125] The setting module is used to set the returned data to empty when the business data corresponding to the business request does not exist in the caching system;

[0126] Accordingly, the query module includes:

[0127] The first return unit is used to update the cache system and the return data based on the business data corresponding to the business request when the business data corresponding to the business request is found in the database, return the return data, and release the distributed lock;

[0128] The second return unit is used to directly return the returned data and release the distributed lock when the business data corresponding to the business request is not found in the database.

[0129] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0130] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 7 This is a structural diagram of an electronic device according to an exemplary embodiment, such as... Figure 7 As shown, the electronic device includes:

[0131] Communication interface 1 enables information exchange with other devices, such as network devices;

[0132] Processor 2 is connected to communication interface 1 to enable information interaction with other devices and, when running a computer program, executes the data caching method provided by one or more of the above-mentioned technical solutions. The computer program is stored on memory 3.

[0133] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general will label all buses as Bus System 4.

[0134] The memory 3 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0135] It is understood that memory 3 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 2 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0136] The methods disclosed in the embodiments of this application can be applied to processor 2, or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 2 or by instructions in the form of software. The processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0137] When processor 2 executes the program, it implements the corresponding processes in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0138] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 3 that stores a computer program, which can be executed by a processor 2 to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0139] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0140] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0141] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data caching method, characterized by, The application is applied to a cache system, the cache system comprises a first cache and a second cache, the first cache and the second cache are connected with a database, the first cache is a Java virtual machine cache, and the second cache is an externally dependent cache service; the first cache is used for caching target data, and the second cache is used for caching other data except the target data, the target data is data with a change frequency lower than a first preset value, a reading frequency higher than a second preset value and a data volume smaller than a third preset value; The method comprises: loading the target data from the second cache by the first cache every preset time period; if the target data does not exist in the second cache, requesting a distributed lock from the database by the first cache to load the target data in the database into the second cache.

2. The data caching method of claim 1, wherein, The Java virtual machine cache comprises an index part and a data part, and the data part comprises a plurality of partitions, and the data structure of each partition is a Map structure.

3. The data caching method of claim 1, wherein, The Java virtual machine cache comprises an index part and a data part, and the storage mode of the data part is a serialization storage mode.

4. The data caching method of claim 1, wherein, The second cache comprises a Redis cache or a Memcached cache.

5. The data caching method of claim 1, wherein, The Java virtual machine cache is connected with the database through a distributed application coordination service.

6. The data caching method of claim 5, wherein, Further comprising: listening to events of the distributed application coordination service by the Java virtual machine cache, and performing corresponding operations according to the listened events; The events comprise update events and session events, the update events are events of updating target data in the database, and the session events are communication connection events.

7. The data caching method of claim 6, wherein, If the events are specifically the update events, the performing corresponding operations according to the listened events comprises: listening to a notification message generated by the distributed application coordination service by the Java virtual machine cache; wherein the notification message comprises version information of updated target data; pulling the latest version of target data from the database based on the version information in the notification message.

8. The data caching method of claim 7, wherein, The pulling the latest version of target data from the database based on the version information in the notification message comprises: pulling the latest version of target data from the database; judging whether the version information of the pulled target data is consistent with the version information in the notification message; if not, re-entering the step of pulling the latest version of target data from the database until the version information of the pulled target data is consistent with the version information in the notification message.

9. The data caching method of claim 6, wherein, If the events are specifically the session events, the performing corresponding operations according to the listened events comprises: generating alarm information, or entering a security mode, or controlling a corresponding service of the Java virtual machine cache to be offline.

10. The data caching method of claim 1, wherein, Further comprising: if a business request is received, judging whether business data corresponding to the business request exists in the cache system; if yes, directly returning the business data in the cache system; If not, a distributed lock is requested from the database, and after the request is successful, the service request is sent to the database to query the service data corresponding to the service request in the database.

11. The data caching method of claim 10, wherein, Before requesting the distributed lock from the database, the method further includes: setting the return data as null; Correspondingly, the querying the service data corresponding to the service request in the database includes: If the service data corresponding to the service request is queried in the database, the cache system and the return data are updated based on the service data, the return data is returned, and the distributed lock is released; If the service data corresponding to the service request is not queried in the database, the return data is directly returned, and the distributed lock is released.

12. A data cache apparatus, characterized by The application is applied to a cache system, the cache system includes a first-level cache and a second-level cache, the first-level cache and the second-level cache are connected with a database, the first-level cache is a Java virtual machine cache, and the second-level cache is an externally dependent cache service; the first-level cache is used for caching target data, and the second-level cache is used for caching other data except the target data; the target data is data with a change frequency lower than a first preset value, a reading frequency higher than a second preset value, and a data volume smaller than a third preset value; The device includes: a first loading module configured to load the target data from the second-level cache through the first-level cache every preset time period; a second loading module configured to request a distributed lock from the database through the first-level cache when the target data does not exist in the second-level cache, so as to load the target data in the database into the second-level cache.

13. An electronic device, comprising: The device includes: a memory configured to store a computer program; a processor configured to execute the computer program to implement the steps of the data caching method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the data caching method according to any one of claims 1 to 11.

Images