Distributed caching method, system and equipment based on database and JVM (Java Virtual Machine)

Abstract

The invention relates to the field of data caching, and discloses a distributed caching method, system and device based on a database and a JVM. A host information table in a database is obtained, and nodes are configured according to the host information table, so that a distributed cluster is constructed; any node in the distributed cluster is used as a Me host, and the remaining nodes in the distributed cluster are used as Bro hosts; in response to a request instruction of any cache object, the corresponding cache object in the cache of the Me host is obtained for response according to the request instruction, and the request instruction comprises at least one of a deletion request, an updating request and an obtaining request; according to the response result, the request instruction is sent to the cache service of the Bro host according to the user identifier, distributed cache processing is achieved, no time consumed by network communication and deserialization exists, and at least 10-25 ms time delay is saved; through the distributed cache, the access speed of the data is improved, and the expansibility and fault tolerance of the system are enhanced.

Description

Technical Field

[0001] The present application relates to the field of data caching, and in particular to a distributed caching method, system and device based on a database and a JVM. Background Art

[0002] Caching technology is a commonly used technology in the computer field. It is a method to optimize data storage and access speed. It uses high-speed storage media to store frequently accessed data to reduce the number of accesses to slow storage media (such as disks or databases). The overall performance and response speed of the system can be improved through caching.

[0003] In the related technologies, the mainstream cache technology is mainly Redis (Remote Dictionary Server), and a few overseas companies use JBoss (open source application server) Cache (i.e., cache). However, when the above cache solution is used in the industrial Internet or within the enterprise, it involves the use of cache middleware, which inevitably increases the number of IO accesses. At the same time, it also involves serialization and deserialization operations, which greatly reduces the cache reading efficiency. In the case of large amounts of data, it will not only increase the time consumption, but also reduce the reliability of the cache system. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present application discloses a distributed caching method, system and device based on a database and a JVM, which are used to solve the technical problems of low caching efficiency and low reliability in the above-mentioned distributed caching solution.

[0005] The first aspect of the present application discloses a distributed caching method based on a database and a JVM, comprising: obtaining a host information table in a database, and configuring each node according to the configuration information in the host information table to build a distributed cluster, wherein each node is configured with a JVM as a running platform for executing Java programs, and the host information table includes the user identification, address, port and status of each node; any one of the nodes in the distributed cluster is used as a Me host, and the remaining nodes in the distributed cluster are used as Bro hosts; in response to a request instruction of any cache object, first obtain the corresponding cache object in the Me host cache according to the request instruction for response, wherein the request instruction includes at least one of a deletion request, an update request and a retrieval request; according to the response result, send the request instruction to the cache service of the Bro host to implement distributed cache processing, wherein the cache object is stored in the local cache using a Map structure.

[0006] In certain embodiments of the first aspect of the present application, if the request instruction is the deletion request carrying a Key value to be deleted, it also includes: parsing the deletion request to determine the first Key value; searching in the local cache according to the first Key value, and verifying after determining the first Key value, if the verification passes, deleting the Value value in the cache associated with the local first Key value; obtaining the Bro host corresponding to the Me host in the host cluster service; sending a first synchronous request or / and a first asynchronous request carrying the first cache name to be deleted and the first Key value to each Bro host; if the first asynchronous request is received, feeding back a deletion asynchronous response to the Me host, and adding the first asynchronous request to the request queue until the first asynchronous request is executed, after finding the corresponding cache according to the first cache name, after the first Key value is verified, deleting the Value value in the cache associated with the local first cache name; if the first synchronous request is received, after finding the corresponding cache according to the first cache name in the first synchronous request, after the first Key value is verified, deleting the Value value in the cache associated with the local first cache name, and synchronously returning the deletion result to the Me host.

[0007] In certain embodiments of the first aspect of the present application, if the request instruction is the update request, it also includes: parsing the update request to obtain a second Key value and a second Value value; putting the second Value value into the key value corresponding to the second Key value locally, and obtaining the Bro host corresponding to the Me host in the host cluster service; sending a second cache name to be updated, a second Key value and a JSON file to each Bro host for serialization processing, and compressing them into a second synchronous request and / or a second asynchronous request; if the second asynchronous request is received, feeding back an updated asynchronous response to the Me host, and adding the second asynchronous request to the request queue until the second asynchronous request is executed, and the compressed second asynchronous request is processed. The row is deserialized and restored into a JSON object. After the corresponding cache is found according to the second cache name, and after the second Key value is verified, the JSON object is passed into the key value in the cache corresponding to the second cache name and the second Key value to update the corresponding second Value value; if the second synchronization request is received, the JSON file compressed in the second synchronization request is deserialized and restored into a JSON object. After the corresponding cache is found according to the second cache name, and after the second Key value is verified, the JSON object is passed into the key value in the cache corresponding to the second cache name and the second Key value to update the corresponding second Value value, and the update result is synchronously returned to the Me host.

[0008] In certain embodiments of the first aspect of the present application, if the request instruction is the acquisition request, it also includes: parsing the cache acquisition request to obtain a third Key value; querying whether the third Key value exists in the cache, and if the third Key value exists, returning an existence result; if the third Key value does not exist, locking the cached data in the cache, and then querying whether the third Key value exists in the cache; if the third Key value exists, releasing the lock and returning an existence result; if the third Key value still does not exist, executing the Lambda expression method to obtain data from the cache, and if the execution fails, releasing the lock; if the execution succeeds, putting the obtained data into the cache; if asynchronous caching is performed, asynchronously obtaining the Bro host, releasing the lock, and notifying the Bro host of the update operation through an asynchronous request; if synchronous caching is performed, synchronously obtaining the Bro host, notifying the Bro host of the update operation through a synchronous request, and releasing the lock; if the Bro host list is empty, it is determined that there is no valid Bro host.

[0009] In certain embodiments of the first aspect of the present application, it also includes: pre-setting a scheduled scanning task, monitoring the cache by executing the task; if the cache is detected to be expired, removing the cache; if it is determined that the current cache is expired when accessing the cache, deleting the expired cache and determining that the current acquisition of the cache has failed.

[0010] In certain embodiments of the first aspect of the present application, it also includes: if it is monitored that the cache exceeds the cache space of the cache, scanning the data items in the cache to determine the number of times each data item is accessed within a preset time period; sorting the number of accesses corresponding to each data item to determine the data item with the least number of accesses; removing the data item with the least number of accesses according to a least recently used mechanism to control the cache size of the cache space.

[0011] In certain embodiments of the first aspect of the present application, distributed cache processing is implemented, and also includes: creating a cache instance based on a preset cache capacity and the number of objects to be cached, the cache instance being divided into a small queue and a large queue, the small queue being used to store objects whose access frequency is lower than an access threshold, and the large queue being used to store objects whose access frequency is equal to and / or greater than the access threshold; writing the object to be cached to the cache instance, first performing a write operation to the small queue to complete the distributed cache of the Bro host; wherein, when the preset cache capacity is insufficient, the cache Value value is preferentially eliminated from the small queue.

[0012] In certain embodiments of the first aspect of the present application, the object to be cached is written to the cache instance, and the method also includes: calling a read method to query the corresponding value according to the key of the object to be cached. If the key exists, the corresponding value is returned and stored in the Map structure, and the access frequency of the object to be cached is increased; if the key does not exist, a null value is returned.

[0013] The second aspect of the present application discloses a distributed cache system based on a database and a JVM, including: a cluster construction module, which is used to obtain a host information table in a database, and configure each node according to the configuration information in the host information table to build a distributed cluster, wherein each node is configured with a JVM as a running platform for executing Java programs, and the host information table includes the user identification, address, port and status of each node; a primary and secondary configuration module, which is used to use any of the nodes in the distributed cluster as a Me host, and the remaining nodes in the distributed cluster as Bro hosts; a first cache module, which is used to respond to a request instruction of any cache object, and first obtain the corresponding cache object in the Me host cache according to the request instruction for response, wherein the request instruction includes at least one of a deletion request, an update request and an acquisition request; a second cache module, which is used to send the request instruction to the cache service of the Bro host according to the response result to implement distributed cache processing, wherein the cache object is stored in the local cache using a Map structure.

[0014] The third aspect of the present application discloses an electronic device, comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above method.

[0015] Beneficial effects of the present application: The present application obtains the host information table in the database, and configures each node according to the configuration information in the host information table, constructs a distributed cluster, and ensures the compatibility and consistency between the nodes in the cluster; any of the nodes in the distributed cluster is used as the Me host, and the remaining nodes in the distributed cluster are used as the Bro host, so that the flexible node management and load balancing capabilities are also improved; in response to the request instruction of any cache object, the corresponding cache object in the Me host cache is first obtained according to the request instruction to respond, wherein the request instruction includes at least one of a delete request, an update request and an acquisition request; according to the response result, the request instruction is sent to the cache service of the Bro host to implement distributed cache processing. On the one hand, the use of the Map structure for storage optimizes memory usage and access speed; on the other hand, the consistency and availability of data in the cluster are ensured. Compared with the traditional cache middleware, the present application does not have any network communication and deserialization time consumption, and at least saves 10-25ms of delay; the distributed cache of the present application not only improves the access speed of data, but also enhances the scalability and fault tolerance of the system, and improves the overall performance and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is an application environment diagram of an implementation of a distributed caching method based on a database and a JVM in an embodiment of the present application;

[0017] Figure 2 It is a flowchart of a distributed caching method based on a database and a JVM in an embodiment of the present application;

[0018] Figure 3 This is a cache deletion flow chart in a distributed cache method based on a database and a JVM in an embodiment of the present application;

[0019] Figure 4 This is a cache deletion and update flow chart in a distributed cache method based on a database and a JVM in an embodiment of the present application;

[0020] Figure 5 This is a cache acquisition flow chart in a distributed cache method based on a database and a JVM in an embodiment of the present application;

[0021] Figure 6 It is a structural diagram of a distributed cache system based on a database and a JVM in an embodiment of the present application;

[0022] Figure 7 It is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and sub-samples in the embodiments can be combined with each other without conflict.

[0024] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0025] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0026] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0027] Unless otherwise stated, the term "plurality" means two or more.

[0028] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0029] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0030] Terminology explanation:

[0031] Hash Map is an implementation of the Map interface based on a hash table in Java. The stored content is a key-value pair mapping, and each key can be mapped to at most one value.

[0032] Lambda expressions are a concise way to represent callable objects (such as functions) and are used where a function object is needed, but you don't want to explicitly define a complete function.

[0033] JSON, JavaScript Object Notation is a lightweight data exchange format.

[0034] JVM is the abbreviation of Java Virtual Machine. JVM is a specification for computing devices. It is a fictional computer that is implemented by simulating various computer functions on an actual computer. It can be understood as a part of the memory cache.

[0035] Combination Figure 1 As shown, an implementation environment diagram of a distributed caching method based on database and JVM provided in an embodiment of the present disclosure includes a user end and a server end, wherein the user end and the server end are connected to a network and communicate with each other. In this application, the user end is the front end and the server end is the back end.

[0036] It should be understood that the distributed caching method based on database and JVM is applied to the terminal. In some embodiments, the terminal can be at least one of a computer device, a desktop device, a smart phone, and a tablet device; the server can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms; the software can be an application program based on the distributed caching method of database and JVM, etc., but is not limited to the above forms.

[0037] Combination Figure 2 As shown, the embodiment of the present disclosure provides a flow chart of a distributed caching method based on a database and a JVM, including:

[0038] Step S201, obtaining a host information table in a database, and configuring each node according to the configuration information in the host information table to build a distributed cluster, wherein each node is configured with a JVM as a running platform for executing Java programs, and the host information table includes a user ID, address, port, and status of each node;

[0039] Among them, the host information table read includes the user ID of each node (used to distinguish nodes managed by different users), address (IP address or domain name), port (port number for node communication) and status (such as online, offline, etc.). Each node is configured through the above configuration information, such as installing JVM (Java Virtual Machine) as the operating platform for executing Java programs, ensuring that each node has the basic environment and dependencies required to run a distributed cluster.

[0040] A distributed Me host cluster is constructed through a host information table. Exemplarily, a database is used to construct a Me host cluster, and the core table is the host information table.

[0041] For example, the main fields of the host information table are:

[0042]

[0043] When each Me host starts, it compares its own Me host ID, IP, port and the information in the host information table. If there is a same IP+port, the Me host ID is overwritten, otherwise, a new Me host information is inserted.

[0044] In this embodiment, the Host Instance Service mainly provides Me host cluster related services, which has two main objects: me (Me host) and bro (Bro host). Me is used to store its own Me host information, and bro is a read-write consistency list used to store other Me host information. In order to ensure that the information of the Me host is updated in a timely and effective manner, the following interfaces and methods are provided:

[0045] Hello interface: for other Me hosts to call. After being triggered, it will detect whether the new Me host exists in the host information table. If it exists, it will be added to the bro list. At the same time, it will return the ok information, indicating that the machine is running normally.

[0046] Offline interface: Before each application goes offline, it will delete its own information in the host information table and notify the offline interface of the Me host member in bro. The notified Me host will remove the information of the invalid Me host in bro.

[0047] RefreshHost method: Starts as a daemon thread immediately after the application starts, and runs again at a certain interval (for example, 60 seconds). This method will read the Me host information and access its Hello interface. If it returns normally, it will be saved to bro and the number of offline times will be reset to 0. If it fails, the number of offline times will be increased by 1. If the number of offline times of this machine is 0, and the number of offline times of the Me host is greater than a certain value (for example, 3), then remove the Me host.

[0048] Step S202: any node in the distributed cluster is used as a Me host, and the remaining nodes in the distributed cluster are used as Bro hosts;

[0049] It should be noted that in a distributed cluster, any node is selected as the Me host (also called the master node), and the selection of the Me host is determined based on a certain strategy (such as polling, random, priority, etc.). After the Me host is determined, the remaining nodes in the distributed cluster are marked as Bro hosts (also called slave nodes), where the Bro host plays a supporting and backup role in the distributed cluster.

[0050] Step S203, in response to a request instruction of any cache object, first obtain the corresponding cache object in the Me host cache according to the request instruction for response, wherein the request instruction includes at least one of a delete request, an update request and an obtain request;

[0051] Among them, the Me host receives request instructions from any object, including but not limited to deletion requests, update requests or acquisition requests, etc. The request instructions are transmitted through a certain communication protocol (such as HTTP, GRPC, etc.). According to the request instructions, the corresponding cache object is obtained from the cache or database. The cache object can be any type of data, such as strings, numbers, objects, arrays, etc.

[0052] For example, the cache object is stored in the local cache of the Me host using the JVM internal Map structure (such as HashMap, ConcurrentHashMap, etc.). The local cache provides the ability to quickly access data and reduces the delay in database access.

[0053] Step S204: according to the response result, the request instruction is sent to the cache service of the Bro host to implement distributed cache processing, wherein the cache object is stored in the local cache using a Map structure.

[0054] For example, to query, obtain, or delete a cache object, if the response result is completed on the Me host and the instruction has been fully responded to, then other Bro hosts do not need to be accessed. On the contrary, if the response result of the Me host cannot be fully realized, then the Bro hosts in the cluster need to execute the same request instruction.

[0055] It should be noted that the distributed cache of the present application is to cache objects (data) in a distributed manner in a cluster, distributing the data to multiple nodes, each node storing part of the cached data, and using a certain algorithm (such as a consistent hashing algorithm) to achieve distributed storage and access of data. When a node needs to access data, it first checks whether the required data exists in its own cache. If not, it sends a request to other nodes to obtain the data.

[0056] In this embodiment, when processing request instructions and storing caches, it is necessary to ensure the security and consistency of concurrent access. This can be achieved using technologies such as lock mechanisms, transaction management, or distributed locks; in a distributed cluster, nodes may fail due to failures, so compatible fault-tolerant processing mechanisms are required, such as node failure detection, automatic recovery, and data retry. In order to improve the performance of distributed caches, technologies such as cache preheating, cache invalidation strategies (such as LRU, LFU, etc.), data compression, and serialization can be used. When transmitting data in a distributed cluster, it is necessary to ensure the security of the data, and encryption technology (such as SSL / TLS) can be used to protect the data transmission process. In the above manner, an efficient and reliable distributed cache system is constructed to meet the needs of large-scale data access and storage.

[0057] In some embodiments, the cache service is mainly provided by CacheService. Services that need to use the cache need to obtain the Cache instance from CacheService and set its key parameter definitions:

[0058]

[0059] Cache features:

[0060] (1) Cache size control: When the cache size exceeds the upper limit, the cache with the least recent use is removed according to the principle of least recently used (LRU mechanism).

[0061] (2) Expiration strategy: 1) Use scheduled task scanning, and remove the cache when an expired cache is found. 2) When accessing the cache, check its expiration time. If it exceeds the deadline, remove the cache and fail to obtain the cache this time.

[0062] (3) Delete interface: Receive the cache name and key value transmitted by other Me hosts, find the corresponding Cache and delete the key value.

[0063] (4) Update interface: Receive the cache name, key value and compressed serialized JSON message transmitted by other Me hosts, find the cache corresponding to this machine, deserialize the compressed JSON according to the cache type, and put the deserialized Java object result into the corresponding key value.

[0064] (5) Cache acquisition process (Lambda expression method): Synchronization is used as an example to demonstrate common cache usage scenarios in programs. Here, the cache acquisition request contains a KEY and a Lambda expression method that obtains the downstream (such as a database or other remote request) based on the KEY when the cache is empty. The illustration shows how to use some JVM mechanisms to ensure effective cache updates and eventual consistency.

[0065] In some embodiments, see Figure 3 , a cache deletion flow chart in a distributed cache method based on a database and a JVM in an embodiment of the present application, which includes deletion steps under synchronous and asynchronous operations. Here, a deletion request refers to deleting a certain KEY in an application.

[0066] If the request instruction is the deletion request carrying a key value to be deleted, parsing the deletion request to determine a first key value;

[0067] Search the local cache according to the first Key value, and verify after the first Key value is determined. If the verification passes, delete the Value value in the local cache associated with the first Key value;

[0068] Get the Bro host corresponding to the Me host in the host cluster service;

[0069] Sending a first synchronous request or / and a first asynchronous request carrying a first cache name to be deleted and a first Key value to each of the Bro hosts;

[0070] If the first asynchronous request is received, a deletion asynchronous response is fed back to the Me host, and the first asynchronous request is added to the request queue until the first asynchronous request is executed. After the corresponding cache is found according to the first cache name, after the first Key value is verified, the Value value in the cache associated with the local first cache name is deleted;

[0071] If the first synchronization request is received, after finding the corresponding cache according to the first cache name in the first synchronization request, after the first Key value is verified, the Value value in the local cache associated with the first cache name is deleted, and the deletion result is synchronously returned to the Me host.

[0072] Specifically, by parsing the deletion request, the first Key value, i.e., KEY, is determined. The first Key value is used to search in the local cache to determine whether there is a cache object corresponding to the Key value. If the corresponding cache object is found, further verification is performed (for example, checking permissions, timestamps, etc.). If the verification passes, the Value value associated with the first Key value in the local cache is deleted. The host cluster service is queried to obtain the information of all Bro hosts (i.e., slave nodes) corresponding to the current Me host. According to the system configuration, a deletion request carrying the first cache name to be deleted and the first Key value is sent to each Bro host. For example, it can be a synchronous request (waiting for the Bro host to respond before continuing to execute) or an asynchronous request (not waiting for a response, directly adding to the request queue). If an asynchronous request is received, a deletion asynchronous response is immediately fed back to the Me host, indicating that the request has been received. The asynchronous request is added to the request queue and waits for subsequent execution. When the asynchronous request is executed, the corresponding cache is searched according to the first cache name, the first Key value is verified, and then the associated Value value is deleted. If a synchronous request is received, the corresponding cache is searched according to the first cache name immediately. After verifying the first Key value, the associated Value value is deleted. The deletion result is synchronously returned to the Me host, indicating that the deletion operation has been successfully completed.

[0073] Through the above method, an efficient and reliable distributed cache deletion process is built to ensure data consistency and security.

[0074] In some embodiments, see Figure 4 , is a cache deletion and update flow chart in a distributed cache method based on a database and a JVM in an embodiment of the present application, wherein the update steps of synchronous and asynchronous operations are shown. The request here refers to setting KEY and VALUE in the application, and the expiration time can also be set additionally, as described in detail as follows:

[0075] If the request instruction is the update request, parse the update request to obtain a second Key value and a second Value value;

[0076] Put the second Value value into the key value corresponding to the second Key value in the local computer, and obtain the Bro host corresponding to the Me host in the host cluster service;

[0077] Sending the second cache name to be updated, the second Key value and the JSON file to each Bro host for serialization processing, and compressing them into a second synchronous request and / or a second asynchronous request;

[0078] If the second asynchronous request is received, an updated asynchronous response is fed back to the Me host, and the second asynchronous request is added to the request queue until the second asynchronous request is executed, the compressed second asynchronous request is deserialized and restored to a JSON object, and after the corresponding cache is found according to the second cache name, after the second Key value is verified, the JSON object is passed into the key value in the cache corresponding to the second cache name and the second Key value to update the corresponding second Value value;

[0079] If the second synchronization request is received, the compressed JSON file in the second synchronization request is deserialized and restored to a JSON object. After the corresponding cache is found according to the second cache name, after the second Key value is verified, the JSON object is passed to the key value in the cache corresponding to the second cache name and the second Key value to update the corresponding second Value value, and the update result is synchronously returned to the Me host.

[0080] Through the above method, an efficient and reliable distributed cache update process is built to ensure the consistency and security of data updates.

[0081] In some embodiments, see Figure 5 , is a cache acquisition flow chart in a distributed cache method based on a database and JVM in an embodiment of the present application; wherein the cache acquisition request includes a KEY, and a Lambda expression method for acquiring a downstream (such as a database or other remote request) according to the KEY when the cache is empty, as described in detail as follows:

[0082] If the request instruction is the acquisition request, it also includes:

[0083] Parsing the cache acquisition request to obtain a third Key value;

[0084] Query whether the third Key value exists in the cache, and if so, return a result of existence;

[0085] If the third Key value does not exist, the cached data in the cache is locked, and then the cache is queried to see if the third Key value exists; if the third Key value exists, the lock is released and a result indicating that the third Key value exists is returned;

[0086] If the third Key value still does not exist, execute the Lambda expression method to obtain data from the cache, and if the execution fails, release the lock;

[0087] If the execution is successful, the acquired data is put into the cache;

[0088] If asynchronous caching is performed, the Bro host is obtained asynchronously, the lock is released, and the Bro host is notified of the update operation through an asynchronous request;

[0089] If synchronous caching is performed, the Bro host is obtained synchronously, the Bro host is notified of the update operation through a synchronous request, and the lock is released;

[0090] If the Bro host list is empty, it is determined that there is no valid Bro host.

[0091] Through the above method, an efficient and reliable distributed cache acquisition process is constructed to ensure the consistency and security of data acquisition.

[0092] In some embodiments, the distributed cache method based on database and JVM further includes:

[0093] Pre-setting a scheduled scanning task, and monitoring the cache by executing the task;

[0094] If it is detected that the cache has expired, the cache is removed; if it is determined that the current cache has expired when the cache is accessed, the expired cache is deleted, and it is determined that the current acquisition of the cache has failed.

[0095] Through the above method, by regularly scanning and immediately deleting expired cache, it is possible to ensure that the data in the cache is always up to date, thereby improving the accuracy of the data. Among them, by regularly cleaning up expired cache, it is possible to reduce unnecessary storage overhead and optimize memory usage, thereby improving system performance; by avoiding the use of expired data, it is possible to reduce errors and anomalies caused by data inconsistency, thereby enhancing the stability of the system. In addition, by pre-setting scheduled scanning tasks and monitoring and processing the cache, the data accuracy, performance and stability of the cache system are effectively improved.

[0096] In some embodiments, the distributed cache method based on database and JVM further includes:

[0097] If it is monitored that the cache exceeds the cache space of the cache, the data items in the cache are scanned to determine the number of accesses of each of the data items within a preset time period;

[0098] Sorting the access times corresponding to the data items to determine the data item with the least access times;

[0099] The data item with the least number of accesses is removed according to a least recently used mechanism to control the cache size of the cache space.

[0100] By removing infrequently used data items in the above manner, it is possible to ensure that the most valuable data is retained in the cache, thereby improving the efficiency of cache usage. Among them, by continuously monitoring and dynamically managing the cache space, it is possible to ensure that the cache size is always kept within the preset limit, avoiding the problem of cache overflow. By optimizing cache usage and avoiding cache overflow, unnecessary memory access and disk I / O operations can also be reduced, thereby improving overall system performance.

[0101] In summary, by monitoring cache space, scanning data items, sorting and removing infrequently used data items, the cache size is effectively controlled and the cache usage efficiency is optimized, thereby improving the overall performance of the system.

[0102] In some embodiments, distributed cache processing is implemented, and further includes: creating a cache instance based on a preset cache capacity and the number of objects to be cached, the cache instance being divided into a small queue and a large queue, the small queue being used to store objects whose access frequency is lower than an access threshold, and the large queue being used to store objects whose access frequency is equal to and / or greater than the access threshold; writing the object to be cached to the cache instance, and first performing a write operation on the small queue to complete the distributed cache of the Bro host; wherein, when the preset cache capacity is insufficient, cache Value values ​​are preferentially eliminated from the small queue.

[0103] Specifically, dividing the cache instances into small queues and large queues, and allocating objects to different queues according to access frequency can more effectively manage cache resources. For example, objects with lower access frequency are stored in small queues, while objects with higher access frequency are stored in large queues. This helps the system locate and access high-frequency objects more quickly, thereby improving the cache hit rate and access efficiency.

[0104] When writing an object to be cached to a cache instance, it first attempts to write to the small queue. This strategy simplifies the writing process. If the access frequency of the object subsequently increases above the threshold, it is migrated from the small queue to the large queue. This dynamic adjustment mechanism ensures the effective use of cache resources.

[0105] Through the above method, the distributed cache between Bro hosts is realized in each node, and different Bro hosts can share the same cache resources, thereby improving the availability and fault tolerance of the cache.

[0106] When a Bro host needs to access an object, it first tries to find it from its own small queue. If it cannot find it, it then tries to find it from other Bro hosts' small queues or large queues. This distributed search mechanism helps reduce cache misses.

[0107] When the preset cache capacity is insufficient, cached Values ​​are eliminated from the small queue first. This strategy is based on an assumption: objects in the small queue are accessed less frequently, so their elimination has a relatively small impact on the data. By eliminating objects in the small queue first, more high-frequency objects can be retained in the cache, thereby improving the cache hit rate and overall performance.

[0108] On the basis of the above embodiment, the object to be cached is written to the cache instance, and it also includes: calling a read method, querying the corresponding value according to the key of the object to be cached, if the key exists, returning the corresponding value stored in the Map structure and increasing the access frequency of the object to be cached; if the key does not exist, returning a null value.

[0109] Through the above methods, the key-based fast query mechanism can efficiently obtain data in the cache; for existing keys, the system directly returns the corresponding value and updates the access frequency without additional data processing steps; by tracking the access frequency, it is easier to implement and optimize the cache strategy to improve the cache hit rate and overall performance. In short, through a simple key-value pair query and access frequency update mechanism, efficient and concise cache management is achieved, which helps to improve the overall performance of the system and user experience.

[0110] See also Figure 6 , is a schematic diagram of the structure of a distributed cache system based on a database and a JVM in an embodiment of the present application, including:

[0111] The cluster building module 601 is used to obtain the host information table in the database, and configure each node according to the configuration information in the host information table to build a distributed cluster, wherein each node is configured with a JVM as a running platform for executing Java programs, and the host information table includes the user ID, address, port and status of each node;

[0112] A primary and secondary configuration module 602, configured to set any of the nodes in the distributed cluster as a Me host and the remaining nodes in the distributed cluster as Bro hosts;

[0113] The first cache module 603 is used to respond to a request instruction of any cache object, and first obtain the corresponding cache object in the Me host cache according to the request instruction to respond, wherein the request instruction includes at least one of a delete request, an update request and an acquisition request;

[0114] The second cache module 604 is used to send the request instruction to the cache service of the Bro host according to the response result to implement distributed cache processing, wherein the cache object is stored in the local cache using a Map structure.

[0115] It should be noted that the distributed cache system based on database and JVM in this application corresponds one-to-one to the distributed cache method based on database and JVM. The corresponding technical details and technical solutions refer to the above-mentioned distributed cache method based on database and JVM, and will not be repeated here.

[0116] Through the above method, the distributed cache system based on the database and JVM obtains the host information table in the database, and configures each node according to the configuration information in the host information table to build a distributed cluster, thereby ensuring the compatibility and consistency between the nodes in the cluster; any of the nodes in the distributed cluster is used as the Me host, and the remaining nodes in the distributed cluster are used as the Bro host, so that the flexible node management and load balancing capabilities are also improved; in response to the request instruction of any cache object, the corresponding cache object in the Me host cache is first obtained according to the request instruction to respond, wherein the request instruction includes at least one of a delete request, an update request and an acquisition request; according to the response result, the request instruction is sent to the cache service of the Bro host to realize distributed cache processing. On the one hand, the Map structure inside the JVM is used for storage, which optimizes memory usage and access speed; on the other hand, the consistency and availability of data in the cluster are ensured. Compared with the traditional cache middleware, this application does not have any network communication and deserialization time consumption, saving at least 10-25ms of delay; the distributed cache of this application not only improves the access speed of data, but also enhances the scalability and fault tolerance of the system, and improves the overall performance and efficiency.

[0117] In other embodiments, the present disclosure also provides an electronic device, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above method.

[0118] Figure 7 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 7 The computer system 700 of the electronic device shown is only an example and should not bring any limitation to the functions and application scope of the embodiments of the present application.

[0119] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 to the random access memory (RAM) 703, such as executing the method in the above embodiment. In the random access memory 703, various programs and data required for system operation are also stored. The central processing unit 701, the read-only memory 702 and the random access memory 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0120] The following components are connected to the input / output interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read therefrom is installed into the storage section 708 as needed.

[0121] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 709, and / or installed from a removable medium 711. When the computer program is executed by the central processing unit 701, various functions defined in the method of the present application are executed.

[0122] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. Parts and subsamples of some embodiments may be included in or replace parts and subsamples of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of a stated sub-sample, whole, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other sub-samples, wholes, steps, operations, elements, components and / or groups of these. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0123] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the methods, devices and units described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0124] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some sub-samples can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0125] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the method, system and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based method for performing the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A distributed caching method based on database and JVM, characterized in that: include: Obtaining a host information table in a database, and configuring each node according to the configuration information in the host information table to build a distributed cluster, wherein each node is configured with a JVM as a running platform for executing Java programs, and the host information table includes a user ID, address, port, and status of each node; The node in the distributed cluster that receives the request instruction is used as the Me host, and the remaining nodes in the distributed cluster that do not receive the request instruction are used as Bro hosts; In response to the request instruction of any cache object, first obtain the corresponding cache object in the Me host cache according to the request instruction to respond, wherein the request instruction includes at least one of a delete request, an update request and an obtain request; According to the response result, the request instruction is sent to the cache service of the Bro host according to the user identifier to implement distributed cache processing, wherein the cache object is stored in the local cache using a Map structure.

2. The method according to claim 1, characterized in that If the request instruction is the deletion request carrying a key value to be deleted, it also includes: Parsing the deletion request to determine the first Key value; Search the local cache according to the first Key value, and verify after the first Key value is determined. If the verification passes, delete the Value value in the local cache associated with the first Key value; Get the Bro host corresponding to the Me host in the host cluster service; Sending a first synchronous request or / and a first asynchronous request carrying a first cache name to be deleted and a first Key value to each of the Bro hosts; If the first asynchronous request is received, a deletion asynchronous response is fed back to the Me host, and the first asynchronous request is added to the request queue until the first asynchronous request is executed. After the corresponding cache is found according to the first cache name, after the first Key value is verified, the Value value in the cache associated with the local first cache name is deleted; If the first synchronization request is received, after finding the corresponding cache according to the first cache name in the first synchronization request, after the first Key value is verified, the Value value in the local cache associated with the first cache name is deleted, and the deletion result is synchronously returned to the Me host.

3. The method according to claim 1, characterized in that If the request instruction is the update request, it also includes: Parsing the update request to obtain a second Key value and a second Value value; Put the second Value value into the key value corresponding to the second Key value in the local computer, and obtain the Bro host corresponding to the Me host in the host cluster service; Sending the second cache name to be updated, the second Key value and the JSON file to each Bro host for serialization processing, and compressing them into a second synchronous request and / or a second asynchronous request; If the second asynchronous request is received, an updated asynchronous response is fed back to the Me host, and the second asynchronous request is added to the request queue until the second asynchronous request is executed, the compressed second asynchronous request is deserialized and restored to a JSON object, and after the corresponding cache is found according to the second cache name, after the second Key value is verified, the JSON object is passed into the key value in the cache corresponding to the second cache name and the second Key value to update the corresponding second Value value; If the second synchronization request is received, the compressed JSON file in the second synchronization request is deserialized and restored to a JSON object. After the corresponding cache is found according to the second cache name, after the second Key value is verified, the JSON object is passed to the key value in the cache corresponding to the second cache name and the second Key value to update the corresponding second Value value, and the update result is synchronously returned to the Me host.

4. The method according to claim 1, characterized in that If the request instruction is the acquisition request, it also includes: Parsing the cache acquisition request to obtain a third Key value; Query whether the third Key value exists in the cache, and if so, return a result of existence; If the third Key value does not exist, the cached data in the cache is locked, and then the cache is queried to see if the third Key value exists; if the third Key value exists, the lock is released and a result indicating that the third Key value exists is returned; If the third Key value still does not exist, execute the Lambda expression method to obtain data from the cache, and if the execution fails, release the lock; If the execution is successful, the acquired data is put into the cache; If asynchronous caching is performed, the Bro host is obtained asynchronously, the lock is released, and the Bro host is notified of the update operation through an asynchronous request; If synchronous caching is performed, the Bro host is obtained synchronously, the Bro host is notified of the update operation through a synchronous request, and the lock is released; If the Bro host list is empty, it is determined that there is no valid Bro host.

5. The method according to any one of claims 1 to 4, characterized in that: Also includes: Pre-setting a scheduled scanning task, and monitoring the cache by executing the task; If it is detected that the cache has expired, remove the cache; If it is determined that the current cache is expired when the cache is accessed, the expired cache is deleted, and it is determined that the current acquisition of the cache fails.

6. The method according to any one of claims 1 to 4, characterized in that: Also includes: If it is monitored that the cache exceeds the cache space of the cache, the data items in the cache are scanned to determine the number of accesses of each of the data items within a preset time period; Sorting the access times corresponding to the data items to determine the data item with the least access times; The data item with the least number of accesses is removed according to a least recently used mechanism to control the cache size of the cache space.

7. The method according to claim 1, characterized in that Implementing distributed cache processing also includes: creating a cache instance based on a preset cache capacity and the number of objects to be cached, the cache instance being divided into a small queue and a large queue, the small queue being used to store objects whose access frequency is lower than an access threshold, and the large queue being used to store objects whose access frequency is equal to and / or greater than the access threshold; writing the object to be cached to the cache instance, first performing a write operation to the small queue to complete the distributed cache of the Bro host; wherein, when the preset cache capacity is insufficient, the cache Value value is preferentially eliminated from the small queue.

8. The method according to claim 7, characterized in that Writing the object to be cached to the cache instance also includes: calling a read method to query the corresponding value according to the key of the object to be cached. If the key exists, the corresponding value is returned and stored in the Map structure and the access frequency of the object to be cached is increased; if the key does not exist, a null value is returned.

9. A distributed cache system based on database and JVM, characterized in that: include: A cluster building module, used to obtain a host information table in a database, and configure each node according to the configuration information in the host information table to build a distributed cluster, wherein each node is configured with a JVM as a running platform for executing Java programs, and the host information table includes a user ID, address, port and status of each node; A primary and secondary configuration module, used to set any of the nodes in the distributed cluster as a Me host and the remaining nodes in the distributed cluster as Bro hosts; A first cache module, configured to respond to a request instruction of any cache object, and first obtain a corresponding cache object in the Me host cache according to the request instruction for response, wherein the request instruction includes at least one of a delete request, an update request, and an obtain request; The second cache module is used to send the request instruction to the cache service of the Bro host according to the response result and the user identifier to implement distributed cache processing, wherein the cache object is stored in the local cache using a Map structure.

10. An electronic device, characterized in that: include: Processor and memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method according to claims 1 to 8.

Images