Cache data processing method and device and storage medium

Abstract

The invention provides a cache data processing method and device and a storage medium, and the method comprises the steps: a data operation interceptor obtains a database access statement in real time, and if the execution of the database access statement causes the database change, generates a cache failure event according to the database access statement, the cache failure event is sent to the transaction aware event scheduler, the transaction aware event scheduler monitors a database transaction corresponding to a database access statement in real time, sends the cache failure event to the event broadcaster, and sends transaction execution information to the event broadcaster according to an execution result of the database transaction; and the event broadcaster temporarily stores the cache failure event and determines whether to send the cache failure event to the event processor according to the transaction execution information, and the event processor performs cache data updating according to the cache failure event after receiving the cache failure event. According to the method and the device, decoupling of the business logic and the cache mechanism can be realized, and the consistency of the database data and the cache data is ensured.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a cache data processing method, device, and storage medium. Background Art

[0002] The data caching mechanism effectively improves data access speed and system performance by storing data copies in memory or disk. It is currently widely used in database access optimization scenarios.

[0003] In the existing technology, when introducing a caching mechanism into an application, it is generally necessary to modify the application's business logic to a certain extent. In particular, when introducing a distributed cache, it is necessary to handle the distributed characteristics of the cache and integrate the distributed cache client library into the business logic.

[0004] However, when data caching mechanisms are applied to complex business scenarios, they can be challenging to modify business logic, increasing maintenance costs and complexity. Furthermore, the caching mechanism may not be able to cover complex business logic, leading to low cache hit rates or inconsistent cached data. Furthermore, due to limitations in the deployment environment and other objective factors, some applications may not be able to incorporate distributed caching, making it difficult to ensure cache consistency across services. Summary of the Invention

[0005] The purpose of this application is to provide a cache data processing method, device and storage medium to address the deficiencies in the above-mentioned prior art, so as to solve the problems in the prior art that it is difficult to ensure the consistency of cache data and database data and that the cache mechanism is difficult to apply when the application business logic is complex.

[0006] To achieve the above objectives, the technical solutions adopted in this application are as follows: In a first aspect, the present application provides a cache data processing method, which is applied to a cache data processing system, wherein the cache data processing system includes: a data operation interceptor, a transaction-aware event scheduler, an event broadcaster, and an event processor, and the method includes: The data operation interceptor obtains database access statements in real time. If the execution of the database access statement causes a database change, a cache invalidation event is generated according to the database access statement, and the cache invalidation event is sent to the transaction-aware event scheduler. The cache invalidation event records the database operation information corresponding to the database access statement. The transaction-aware event scheduler monitors the database transaction corresponding to the database access statement in real time, sends the cache invalidation event to the event broadcaster, and sends transaction execution information to the event broadcaster based on the execution result of the database transaction, wherein the transaction execution information is used to indicate the execution result of the database transaction; The event broadcaster temporarily stores the cache invalidation event and determines whether to send the cache invalidation event to the event processor according to the transaction execution information; If the event processor receives the cache invalidation event, it updates the cache data according to the cache invalidation event.

[0007] Optionally, generating a cache invalidation event according to the database access statement includes: Extracting information from the database access statement to obtain database operation information in the database access statement, wherein the database operation information includes at least: event type, table name, change data, and service identifier; The database operation information is encapsulated as the cache invalidation event according to a preset format.

[0008] Optionally, the real-time monitoring of execution information of the database transaction corresponding to the database access statement includes: Get the value of the target attribute in the transaction operation interface in real time; If the value of the target attribute is a valid value, the database transaction corresponding to the database access statement is monitored in real time.

[0009] Optionally, the sending transaction execution information to the event broadcaster according to the execution result of the database transaction includes: If the execution result of the database transaction is submitted, sending first transaction execution information to the event broadcaster, where the first transaction execution information is used to indicate that the transaction has been submitted; If the execution result of the database transaction is rolled back, second transaction execution information is sent to the event broadcaster, where the second transaction execution information is used to indicate that the transaction has been rolled back.

[0010] Optionally, determining whether to send the cache invalidation event to the event processor according to the transaction execution information includes: If the transaction execution information is the first transaction execution information, sending the cache invalidation event to the event processor; If the transaction execution information is the second transaction execution information, the cache invalidation event is cleared.

[0011] Optionally, the broadcast mode of the event broadcaster includes: a local mode and a distributed mode. In the local mode, the event broadcaster communicates with the event processor in a one-to-one correspondence. In the distributed mode, the event broadcaster is connected to a message middleware, multiple event processors are connected to the message middleware, and subscribe to messages from the message middleware. The sending the cache invalidation event to the event processor includes: If the broadcast mode is local mode, sending the cache invalidation event to the event handler of the corresponding connection; If the broadcast mode is a distributed mode, the cache invalidation event is stored in the message queue of the message middleware, and when the message in the message middleware is updated, the cache invalidation event is subscribed to from the message queue of the message middleware.

[0012] Optionally, updating cache data according to the cache invalidation event includes: Extracting the database operation information from the cache invalidation event; The cache is updated according to the database operation information so that the data in the cache is consistent with the data in the database.

[0013] Optionally, updating the cache according to the database operation information includes: If the cache is a multi-level cache, a target cache is determined according to the database operation information, and the target cache is updated according to the database operation information.

[0014] Optionally, updating the cache according to the database operation information includes: Determine the target cache and target cache update strategy corresponding to the database operation information according to the database operation information and a preset cache naming mapping table; The target cache is updated according to the target cache update policy.

[0015] In a second aspect, an embodiment of the present application further provides an electronic device comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the cache data processing method as described in any one of the first aspects.

[0016] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cache data processing method as described in any one of the first aspects are executed.

[0017] The beneficial effects of the present application are: by monitoring database access statements at the service level and sensing database transactions, the database access statements that can cause database updates are determined after the database transactions are completed, and the data in the cache is modified according to the determined database access statements, thereby ensuring the consistency of the database and cache data. Compared with the prior art, the present application does not require modifications to the business logic of the monolithic application, thereby achieving the decoupling of the business logic and the cache mechanism. In addition, the present application combines SQL statement monitoring and transaction perception to enable the cache mechanism to have transaction perception capabilities, and then makes corresponding modifications to the data in the cache only when the transaction is successfully committed, further ensuring data consistency. By updating the cached data through the event broadcaster and event handler, even for distributed caches or cross-service caches, the effective forwarding of database access statements can be achieved, thereby ensuring data consistency in distributed caches and cross-service caches.

[0018] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of an application scenario of a cache data processing system provided by an embodiment of the present application is shown; Figure 2 A flow chart of a cache data processing method provided by an embodiment of the present application is shown; Figure 3 A flow chart of generating a cache invalidation event provided by an embodiment of the present application is shown; Figure 4 A flowchart of monitoring database transactions provided by an embodiment of the present application is shown; Figure 5 A flowchart of sending transaction execution information provided by an embodiment of the present application is shown; Figure 6 A flowchart of receiving a cache invalidation event provided by an embodiment of the present application is shown; Figure 7 A flowchart of updating cache data provided by an embodiment of the present application is shown; Figure 8A flowchart of another method for updating cache data provided by an embodiment of the present application is shown; Figure 9 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0022] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0023] Data caching mechanisms can be used to improve database query efficiency and system responsiveness. Common caching architectures include local caching and distributed caching. For example, MyBatis (a Java-based persistence framework), a database access framework, features a plugin mechanism that allows developers to intercept SQL (Structured Query Language) before and after execution, implementing various extended features.

[0024] However, current monolithic applications have complex business logic and frequent multi-table join queries, making existing caching strategies difficult to fully cover all scenarios. Consequently, these strategies are ineffective. Because caching mechanisms are closely tied to business logic code and lack automated cache management mechanisms, improving the effectiveness of caching strategies in complex monolithic applications inevitably requires reworking the application's business logic, resulting in lengthy and difficult-to-maintain code.

[0025] On the other hand, distributed caching has high requirements for the network environment. Most current distributed caching architectures use network isolation technology, which can affect normal cache communication and data synchronization, leading to data inconsistencies. Some cross-service applications cannot access distributed caching services due to deployment environment or technology stack limitations, making cross-service cache consistency difficult to ensure.

[0026] In addition, the existing caching mechanism cannot perceive the commit or rollback status of database transactions. The caching mechanism is not coordinated enough with the database. The caching mechanism cannot automatically update or invalidate the cache based on the status of the transaction. Developers need to handle it manually, which also increases the difficulty of development and the complexity of system maintenance.

[0027] Based on this, the present application proposes a cache data processing method, which monitors the database access statements at the SQL service level and perceives the database transactions, thereby determining the database access statements that can cause the database to be updated after the database transaction is completed, and modifying the data in the cache according to the determined database access statements, thereby ensuring the consistency of the database and cache data. Compared with the existing technology, the present application monitors the data changes involved in SQL at the SQL service level, and does not need to modify the business logic of the single application, thereby realizing the decoupling of the business logic and the cache mechanism. In addition, the present application combines SQL statement monitoring and transaction perception to enable the cache mechanism to have transaction perception capabilities, and then only makes corresponding modifications to the data in the cache when the transaction is successfully submitted, further ensuring the consistency of the data.

[0028] The cache data processing method provided in this application updates the cache data through an event broadcaster and an event processor. Even for distributed caches or cross-service caches, it can achieve effective forwarding of database access statements, thereby ensuring data consistency in distributed caches and cross-service caches.

[0029] The cache data processing method of this application can be applied to Figure 1 In the cache data processing system shown. Figure 1 The cache data processing system is connected to the database and application. Users can initiate data modification operations on the database in applications installed on the electronic device. The cache data system can monitor the user's modification operations on the database, execute the data modification operations to modify the data in the database, and simultaneously modify the corresponding data in the cache. For example, the cache data processing system can be MyBatis. In addition to its own cache mechanism, the cache data processing system can also be integrated with other external caches.

[0030] Reference Figure 1 The cache data processing system includes: a data operation interceptor, a transaction-aware event scheduler, an event broadcaster, and an event processor. The data operation interceptor is used to obtain user-initiated database access statements, and the transaction-aware event scheduler is used to perceive the database transactions corresponding to the database access statements. When the database transaction is completed, it prompts the event broadcaster to send cache invalidation events to the event processor for data update.

[0031] In local mode, where both the application and cache are deployed locally, the data operation interceptor, transaction-aware event scheduler, event broadcaster, and event processor can all be deployed locally. In distributed mode, the cache of the cache data processing system can be a distributed architecture. In this case, the application, data operation interceptor, transaction-aware event scheduler, and event broadcaster can be deployed on the same device, and the event processor can be deployed in each distributed cache.

[0032] Figure 2 This is a flow chart of a cache data processing method provided by an embodiment of the present application, which can be applied to Figure 1 In the cache data processing system shown, refer to Figure 2 , the method comprising: S201. The data operation interceptor obtains database access statements in real time. If the execution of the database access statement causes database changes, a cache invalidation event is generated based on the database access statement, and the cache invalidation event is sent to the transaction-aware event scheduler. The cache invalidation event records the database operation information corresponding to the database access statement.

[0033] Optionally, the user generates a database access statement through an interface operation. Before executing the database access statement, the data operation interceptor can obtain the database access statement in real time. Specifically, the data access statement can be, for example, an SQL statement. For ease of description, the following description uses SQL as an example of the database access statement. The data operation interceptor can be in the same thread as the main program that performs the database modification operation, and the data operation interceptor is executed before the main program, forming a sequential execution relationship with the main program.

[0034] The data operation interceptor monitors all database access operations within an application in real time and captures the SQL statements executed by these operations. Based on these captured SQL statements, the data operation interceptor analyzes whether the execution of these statements will cause database data changes. For example, INSERT, UPDATE, and DELETE statements often cause database changes, and cache invalidation events can be generated based on these database access statements.

[0035] Alternatively, a cache invalidation event may be an event that may cause cache invalidation. It should be understood that data changes may cause cache invalidation, so a cache invalidation event may be generated based on database access statements that may cause data changes. As a possible implementation, information may be extracted from the database access statements and the extracted information may be encapsulated as a cache invalidation event.

[0036] Database operation information includes: event type, operation object, changed data, and business identifier. The event type can indicate the type of SQL statement operation, such as INSERT, UPDATE, and DELETE. The operation object can indicate the table name operated on by the SQL statement and the location of the changed data. The changed data can indicate the data before and after the SQL statement is executed. The business identifier can indicate the operator who issued the SQL statement or the source module.

[0037] After generating a cache invalidation event, the data operation interceptor can send the cache invalidation event to the transaction-aware event scheduler. The transaction-aware event scheduler can perceive the database transaction and determine whether the database transaction corresponding to the cache invalidation event is successfully executed, so as to manage and schedule the cache invalidation event and ensure that the cache invalidation event can be processed in the correct order and at the appropriate time.

[0038] It is worth noting that the above-mentioned S201 step is to obtain SQL statements at the SQL service level, which will not cause any changes to the internal business logic of the application, nor will it have any impact on the execution process of the SQL statements. Therefore, the business logic and the acquisition of cache invalidation events are decoupled, and user operations that may cause database changes can be obtained in a timely manner without changing the business logic of the application.

[0039] S202. The transaction-aware event scheduler monitors the database transactions corresponding to the database access statements in real time, sends cache invalidation events to the event broadcaster, and sends transaction execution information to the event broadcaster based on the execution results of the database transactions. The transaction execution information is used to indicate the execution results of the database transactions.

[0040] Optionally, the transaction-aware event scheduler can monitor database transactions corresponding to database access statements in real time, such as tracking the start, execution, and final commit or rollback status of database transactions, as well as reclaiming and closing JDBC connections.

[0041] For example, a transaction-aware event scheduler can be a static proxy class for JDBC (Java Database Connectivity), performing proxy operations within the database connection lifecycle to ensure thread safety. As a proxy for the Connection interface, the transaction-aware event scheduler can intercept and extend method calls on the Connection interface, such as transaction commit, rollback, and connection close. In a multi-threaded environment, multiple threads can access the transaction-aware event scheduler simultaneously. The transaction-aware event scheduler's lifecycle is synchronized with the opening and closing of the Connection interface. That is, when the Connection interface is opened, the transaction-aware event scheduler is created; when the Connection interface is closed, the transaction-aware event scheduler is destroyed accordingly. When the connection is closed, the transaction-aware event scheduler can perform cleanup tasks, such as disconnecting the database connection and releasing database resources.

[0042] In one possible implementation, after the data operation interceptor generates a cache invalidation event and sends it to the transaction-aware event scheduler, the transaction-aware event scheduler can temporarily retain the cache invalidation event and monitor the database transaction corresponding to the database access statement in real time until the corresponding database transaction is completed.

[0043] After the database transaction is executed, the transaction-aware event scheduler can send transaction execution information to the event broadcaster based on the execution result of the database transaction.

[0044] Optionally, the transaction execution information is used to indicate the final status of the database transaction. For example, if the transaction is successfully committed, the transaction execution information will indicate "committed successfully"; if the transaction is rolled back, the transaction execution information will indicate "rolled back".

[0045] It should be noted that, in the embodiment of the present application, by monitoring the database transactions corresponding to the database access statements and sending transaction execution information to the event broadcaster after the database transaction is completed, it can be ensured that the cached data is modified after the database transaction is completed, thereby avoiding cached data modification errors caused by database transaction rollback, and thus ensuring the consistency of cached data and database data.

[0046] S203: The event broadcaster temporarily stores the cache invalidation event and determines whether to send the cache invalidation event to the event processor according to the transaction execution information.

[0047] In one possible implementation, the transaction-aware event scheduler does not save cache invalidation events. Upon receiving a cache invalidation event, the transaction-aware event scheduler can immediately send the cache invalidation event to the event broadcaster, which then temporarily stores the cache invalidation event. The transaction-aware event scheduler monitors the execution results of the database transaction corresponding to the cache invalidation event and sends transaction execution information to the event broadcaster based on the execution results of the database transaction. At this point, if the database transaction is successfully committed, the event broadcaster sends the cache invalidation event to the event handler. If the database transaction is rolled back, the event broadcaster does not send the cache invalidation event to the event handler.

[0048] In another possible implementation, the transaction-aware event scheduler can temporarily store cache invalidation events and monitor the execution results of the database transactions corresponding to the cache invalidation events in real time. If the database transaction is successfully committed, the transaction-aware event scheduler sends the cache invalidation event to the event broadcaster, which then sends the cache invalidation event to the event handler. If the database transaction is rolled back, the transaction-aware event scheduler does not send the cache invalidation event to the event broadcaster.

[0049] Optionally, the event broadcaster can send cache invalidation events one by one and synchronously. When the number of cache invalidation events is large or the cache invalidation events are abnormal, the cache invalidation events can also be sent to the event handler asynchronously in a delayed manner.

[0050] In local mode, both the event broadcaster and the event handler are deployed locally. In this case, the event broadcaster can send cache invalidation events directly to the event handler.

[0051] In distributed mode, event processors may be deployed in each distributed cache or cross-module cache. At this time, the event broadcaster can store cache invalidation events in the message queue, and each event processor subscribes to the message queue to obtain updated cache invalidation events.

[0052] S204: If the event processor receives a cache invalidation event, it updates the cache data according to the cache invalidation event.

[0053] Optionally, when there are multiple distributed caches, an event processor can be deployed in each distributed cache. The event processor can receive cache invalidation events sent by the event broadcaster and update or clear the cache data according to the cache invalidation event to ensure that the cache is consistent with the database data.

[0054] If there is a multi-level cache strategy, after receiving the cache invalidation event, the event handler can first determine the target cache module corresponding to the cache invalidation event and update or clear the cache data of the target cache module.

[0055] In an embodiment of the present application, by monitoring database access statements at the service level and sensing database transactions, the database access statements that can cause database updates are determined after the database transactions are completed, and the data in the cache is modified according to the determined database access statements, thereby ensuring the consistency of the database and cache data. Compared with the prior art, the present application does not require modifications to the business logic of the monolithic application, thereby achieving the decoupling of the business logic and the cache mechanism. In addition, the present application combines SQL statement monitoring and transaction perception to enable the cache mechanism to have transaction perception capabilities, and then makes corresponding modifications to the data in the cache only when the transaction is successfully submitted, further ensuring data consistency. By updating the cached data through the event broadcaster and event handler, even for distributed caches or cross-service caches, the effective forwarding of database access statements can be achieved, thereby ensuring data consistency in distributed caches and cross-service caches.

[0056] Next, we will explain the steps of generating cache invalidation events based on database access statements. Figure 3 As shown, the above step S201 includes: S301. Extract information from database access statements to obtain database operation information in the database access statements. The database operation information at least includes: event type, table name, changed data, and business identifier.

[0057] Optionally, the data operation interceptor can parse the database access statement to obtain the table name, event type, changed data, business identifier, etc. The event type can be the operation type of the database access statement. By parsing the condition field of the database access statement, the table name, changed column name, and changed data can be obtained. The business identifier can be used to indicate the operator who initiated the database access statement, the source module, etc.

[0058] S302: Encapsulate the database operation information into a cache invalidation event according to a preset format.

[0059] Optionally, the cache invalidation event may have a preset format. By filling the database operation information into the corresponding position according to the preset format and instantiating it, a structured cache invalidation event may be generated.

[0060] In a possible implementation, during the process of encapsulating database operation information, users can modify the content and structure of cache invalidation events to adapt to the actual needs of different businesses.

[0061] The following is a further explanation of the database transactions corresponding to the above real-time monitoring database access statements. Figure 4 As shown, the above step S202 includes: S401. Obtain the value of the target attribute in the transaction operation interface in real time.

[0062] Optionally, the transaction operation interface is used to execute database access statements and manage the transactions corresponding to the database access statements. The target attribute is used to indicate the number of database transactions currently associated with the transaction operation interface. For example, using MyBatis as an example, the transaction operation interface can be the BaseExecutor in MyBatis, and the target attribute can be the transaction attribute of the BaseExecutor.

[0063] S402: If the value of the target attribute is a valid value, the database transaction corresponding to the database access statement is monitored in real time.

[0064] The value of the target attribute is a valid value, and may be a preset value. For example, when executing a database transaction in MyBatis, the transaction attribute is assigned a preset valid value. Therefore, when monitoring the database transaction corresponding to the database access statement, the transaction attribute value of the BaseExecutor can be read. If the transaction attribute contains a valid value, it indicates that a database transaction exists.

[0065] It is worth noting that users can set the auto-commit mode or transaction commit mode. In the auto-commit mode, the database access statement does not generate the corresponding database transaction. At this time, the value of the target attribute is an invalid value. The transaction-aware event scheduler can send the cache invalidation event directly to the event broadcaster, and the event broadcaster directly publishes the cache invalidation event.

[0066] In transaction commit mode, database access statements that cause database data changes will generate corresponding database transactions. At this time, the value of the target attribute is a valid value, and the transaction-aware event scheduler can monitor the database transactions.

[0067] The following is a further explanation of sending transaction execution information to the event broadcaster based on the execution result of the database transaction. Figure 5 As shown, the above step S202 includes: S501. If the execution result of the database transaction is submitted, first transaction execution information is sent to the event broadcaster, where the first transaction execution information is used to indicate that the transaction has been submitted.

[0068] Optionally, the transaction-aware event scheduler may receive a transaction callback event, where the transaction callback event is used to indicate that the database transaction has been successfully committed. When the transaction-aware event scheduler receives the transaction callback event indicating successful commit, it may send the first transaction execution information to the event broadcaster.

[0069] After the event broadcaster receives the first transaction execution information, it can determine that the database transaction has been successfully submitted, that is, the data in the database has been modified according to the database access statement. Therefore, the event broadcaster can send the cache invalidation event to the event processor, and the event processor will modify the data in the cache.

[0070] S502: If the execution result of the database transaction is rolled back, second transaction execution information is sent to the event broadcaster, where the second transaction execution information is used to indicate that the transaction has been rolled back.

[0071] Optionally, if a transaction rolls back, a transaction callback event will be triggered. In this case, the transaction callback event is used to indicate that the database transaction is rolled back. When the transaction-aware event scheduler receives the transaction callback event indicating a rollback, it can send second transaction execution information to the event broadcaster.

[0072] It should be noted that the transaction rollback indicates that the transaction was not successfully executed, that is, the data in the database was not modified. Therefore, after the event broadcaster receives the second transaction execution information, it may not send the cache invalidation event to the event handler.

[0073] The following is the above-mentioned determination of whether to send a cache invalidation event to the event processor based on transaction execution information, including: If the transaction execution information is the first transaction execution information, a cache invalidation event is sent to the event processor.

[0074] Optionally, if the transaction execution information is the first transaction execution information, it indicates that the database transaction is successfully executed. At this time, the event broadcaster can send a cache invalidation event to the event processor, and the event processor updates the data in the cache according to the cache invalidation event.

[0075] If the transaction execution information is the second transaction execution information, the cache invalidation event is cleared.

[0076] Optionally, if the transaction execution information is the second transaction execution information, it indicates that the database transaction is rolled back, that is, the database transaction is canceled or not executed. At this time, the event broadcaster can clear all temporarily stored cache invalidation events.

[0077] In an embodiment of the present application, by perceiving database transactions and sending transaction execution information to the event broadcaster after the database transaction is completed, it is possible to ensure that changes to cached data occur after the database transaction is completed, thereby ensuring the consistency of database data and cached data.

[0078] Optionally, the broadcast mode of the event broadcaster includes: a local mode, in which the event broadcaster communicates with the event handler in a one-to-one correspondence.

[0079] In another possible implementation, the broadcast mode of the event broadcaster includes: a distributed mode, in which the event broadcaster is connected to the message middleware, multiple event handlers are connected to the message middleware, and subscribe to messages from the message middleware.

[0080] In local mode, the event handler saves the registration information in the event broadcaster, such as Figure 6 As shown, the steps of sending the cache invalidation event to the event handler include: S601: If the broadcast mode is the local mode, a cache invalidation event is sent to an event handler of the corresponding connection.

[0081] Optionally, the event handler of the corresponding connection may be an event handler that stores registration information in the event broadcaster. After receiving the first transaction execution information, the event broadcaster may send a cache invalidation event to the registered event handler.

[0082] S602: If the broadcast mode is the distributed mode, the cache invalidation event is stored in the message queue of the message middleware, and when the message in the message middleware is updated, the cache invalidation event is subscribed from the message queue of the message middleware.

[0083] Optionally, the event broadcaster can store the cache expiration event in a message queue. When there are message queues with multiple topics in the message middleware, the event broadcaster can determine the message queue corresponding to the cache expiration event based on the content or business identifier in the cache expiration event, and store the cache expiration event in the corresponding message queue.

[0084] Optionally, the message middleware may include message queues of multiple topics. For example, according to business needs, the data of the distributed cache may be stored in message queues of different topics respectively. Each distributed cache subscribes to its corresponding message queue. When the subscribed message queue is updated, the event handler of the distributed cache may obtain the updated cache invalidation event from the message queue, or the message queue may push the updated cache invalidation event to the event handler that has subscribed to the message queue.

[0085] In an embodiment of the present application, when the cache mechanism is cross-module or distributed, by setting up a message middleware, it can be ensured that all event processors can receive updated cache invalidation events in a timely manner. Even if the distributed cache adopts network isolation technology, the cache invalidation event can be synchronized by subscribing to the message queue, thereby ensuring data consistency across service caches and distributed caches.

[0086] The following is a further explanation of the cache data update based on the cache invalidation event. Figure 7 As shown, the above step S204 includes: S701. Extract database operation information from a cache invalidation event.

[0087] S702: Update the cache according to the database operation information to make the data in the cache consistent with the data in the database.

[0088] Optionally, the event processor can parse the cache invalidation event, extract the database operation information, and match the database operation information with the local cache naming strategy to determine the cache data corresponding to the database operation information in the local cache, and determine the corresponding cache invalidation logic based on the database operation information, thereby realizing the update of the cache data.

[0089] When the cache is a multi-level cache, the steps of updating the cache according to the database operation information include: If the cache is a multi-level cache, the target cache is determined according to the database operation information, and the target cache is updated according to the database operation information.

[0090] Optionally, the target cache may be the cache where the cache data corresponding to the data operation information is located. By matching the database operation information with a preset cache naming mapping table, the target cache may be determined, and the cache data in the target cache may be updated according to the database operation information.

[0091] like Figure 8 As shown, the above steps of updating the target cache according to the database operation information specifically include: S801: Determine a target cache corresponding to the database operation information and a target cache update strategy based on the database operation information and a preset cache naming mapping table.

[0092] S802: Update the target cache according to the target cache update policy.

[0093] The cache naming mapping table stores the correspondence between the local cache and the content in the database. For example, the corresponding cache data in the target cache corresponding to the changed data in the local cache can be determined by matching the table name and the changed column name in the database operation information in the cache naming mapping table.

[0094] As a possible implementation, the cache naming mapping table may also store cache update policies available for each cache data. After receiving database operation information, the cache update policy corresponding to the database operation information may be used as the target cache update policy.

[0095] Exemplarily, the database operation information includes an event type. When the event type is a delete operation, the target cache update strategy may be to delete the specified record in the target cache; when the event type is an update operation, the target cache update strategy may be to query the database and update the data in the target cache; when the event type is to insert data, the target cache update strategy may be not to process the data; when the event type is an update operation and the business identifier is a preset identifier, the target cache update strategy may be to mark the cached data expired and wait for lazy loading to refresh.

[0096] It is worth noting that the correspondence between database operation information and cache update strategy can be determined based on actual business needs. This application only provides a possible example and should not be limited to this.

[0097] Based on the same inventive concept, a cache data processing system corresponding to the cache data processing method is also provided in the embodiment of the present application. Since the principle of solving the problem by the system in the embodiment of the present application is similar to the above-mentioned cache data processing method in the embodiment of the present application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be repeated.

[0098] Reference Figure 1 The schematic diagram of the cache data processing system shown in FIG. The cache data processing system includes: a data operation interceptor, a transaction-aware event scheduler, an event broadcaster, and an event processor.

[0099] The data operation interceptor is used to obtain database access statements in real time. If the execution of a database access statement causes a database change, a cache invalidation event is generated based on the database access statement and sent to the transaction-aware event scheduler. The cache invalidation event records the database operation information corresponding to the database access statement. The transaction-aware event scheduler is used to monitor database transactions corresponding to database access statements in real time, send cache invalidation events to the event broadcaster, and send transaction execution information to the event broadcaster based on the execution results of the database transactions. The transaction execution information is used to indicate the execution results of the database transactions. The event broadcaster is used to temporarily store cache invalidation events and determine whether to send cache invalidation events to the event processor based on transaction execution information; The event handler is used to update cache data according to the cache invalidation event when receiving the cache invalidation event.

[0100] Optionally, the data operation interceptor is specifically used to: Extracting information from database access statements to obtain database operation information in the database access statements. The database operation information includes at least: event type, table name, changed data, and business identifier; Encapsulate database operation information as cache invalidation events according to the preset format.

[0101] Optionally, the transaction-aware event scheduler is specifically used to: Get the value of the target attribute in the transaction operation interface in real time; If the value of the target attribute is a valid value, the database transaction corresponding to the database access statement is monitored in real time.

[0102] Optionally, the transaction-aware event scheduler is specifically used to: If the execution result of the database transaction is submitted, first transaction execution information is sent to the event broadcaster, where the first transaction execution information is used to indicate that the transaction has been submitted; If the execution result of the database transaction is rolled back, second transaction execution information is sent to the event broadcaster, where the second transaction execution information is used to indicate that the transaction has been rolled back.

[0103] Optionally, the event broadcaster is specifically used to: If the transaction execution information is the first transaction execution information, sending a cache invalidation event to the event processor; If the transaction execution information is the second transaction execution information, the cache invalidation event is cleared.

[0104] Optionally, the broadcast mode of the event broadcaster includes: local mode and distributed mode. In the local mode, the event broadcaster communicates with the event processor one-to-one. In the distributed mode, the event broadcaster is connected to the message middleware, multiple event processors are connected to the message middleware, and subscribe to messages from the message middleware. Optionally, the event broadcaster is specifically used to: If the broadcast mode is local mode, the cache invalidation event is sent to the event handler of the corresponding connection.

[0105] If the broadcast mode is distributed mode, the cache invalidation event is stored in the message queue of the message middleware, and when the message in the message middleware is updated, the cache invalidation event is subscribed from the message queue of the message middleware.

[0106] Optionally, the event handler is specifically used to: Extract database operation information from cache invalidation events; Update the cache according to the database operation information to make the data in the cache consistent with the data in the database.

[0107] Optionally, the event handler is specifically used to: If the cache is a multi-level cache, the target cache is determined according to the database operation information, and the target cache is updated according to the database operation information.

[0108] Optionally, the event handler is specifically used to: Determine the target cache and target cache update strategy corresponding to the database operation information based on the database operation information and the preset cache naming mapping table; Update the target cache according to the target cache update policy.

[0109] The embodiment of the present application monitors the database access statements at the service level and perceives the database transactions, thereby determining the database access statements that can cause the database to be updated after the database transaction is completed, and modifying the data in the cache according to the determined database access statements, thereby ensuring the consistency of the database and cache data. Compared with the prior art, the present application does not need to modify the business logic of the single application, and realizes the decoupling of the business logic and the cache mechanism. In addition, the present application combines SQL statement monitoring and transaction perception to enable the cache mechanism to have transaction perception capabilities, and then makes corresponding modifications to the data in the cache only when the transaction is successfully submitted, further ensuring data consistency. By updating the cached data through the event broadcaster and event handler, even for distributed caches or cross-service caches, the effective forwarding of database access statements can be achieved, thereby ensuring the data consistency in the distributed cache and cross-service cache.

[0110] Figure 9 A structural schematic diagram of an electronic device provided in an embodiment of the present application is shown, including: a processor 901, a storage medium 902 and a bus 903. The storage medium 902 stores machine-readable instructions executable by the processor 901. When the electronic device runs the cache data processing method as in the embodiment, the processor 901 communicates with the storage medium 902 through the bus 903, and the processor 901 executes the machine-readable instructions and the preamble of the method item of the processor 901 to perform the steps in the above-mentioned cache data processing method.

[0111] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor executes the steps in the above-mentioned cache data processing method.

[0112] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute the methods described in other embodiments. For the specific execution method steps and principles, please refer to the description of the embodiment and will not be repeated here.

[0113] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0114] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0115] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0116] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0117] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0118] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present application, or make equivalent replacements for some of the technical features thereof. However, these modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A cache data processing method, characterized in that: Applied to a cache data processing system, the cache data processing system includes: a data operation interceptor, a transaction-aware event scheduler, an event broadcaster, and an event processor, the method includes: The data operation interceptor obtains database access statements in real time. If the execution of the database access statement causes a database change, a cache invalidation event is generated according to the database access statement, and the cache invalidation event is sent to the transaction-aware event scheduler. The cache invalidation event records the database operation information corresponding to the database access statement. The transaction-aware event scheduler monitors the database transaction corresponding to the database access statement in real time, sends the cache invalidation event to the event broadcaster, and sends transaction execution information to the event broadcaster based on the execution result of the database transaction, wherein the transaction execution information is used to indicate the execution result of the database transaction; The event broadcaster temporarily stores the cache invalidation event and determines whether to send the cache invalidation event to the event processor according to the transaction execution information; If the event processor receives the cache invalidation event, it updates the cache data according to the cache invalidation event.

2. The method according to claim 1, characterized in that Generating a cache invalidation event according to the database access statement includes: Extracting information from the database access statement to obtain database operation information in the database access statement, wherein the database operation information includes at least: event type, table name, change data, and service identifier; The database operation information is encapsulated as the cache invalidation event according to a preset format.

3. The method according to claim 1, characterized in that The real-time monitoring of the execution information of the database transaction corresponding to the database access statement includes: Get the value of the target attribute in the transaction operation interface in real time; If the value of the target attribute is a valid value, the database transaction corresponding to the database access statement is monitored in real time.

4. The method according to claim 1, wherein The sending transaction execution information to the event broadcaster according to the execution result of the database transaction includes: If the execution result of the database transaction is submitted, sending first transaction execution information to the event broadcaster, where the first transaction execution information is used to indicate that the transaction has been submitted; If the execution result of the database transaction is rolled back, second transaction execution information is sent to the event broadcaster, where the second transaction execution information is used to indicate that the transaction has been rolled back.

5. The method according to claim 4, characterized in that The determining, according to the transaction execution information, whether to send the cache invalidation event to the event processor comprises: If the transaction execution information is the first transaction execution information, sending the cache invalidation event to the event processor; If the transaction execution information is the second transaction execution information, the cache invalidation event is cleared.

6. The method according to claim 4, characterized in that The broadcast modes of the event broadcaster include: local mode and distributed mode. In the local mode, the event broadcaster communicates with the event processor in a one-to-one correspondence. In the distributed mode, the event broadcaster is connected to a message middleware, multiple event processors are connected to the message middleware, and subscribe to messages from the message middleware. The sending the cache invalidation event to the event processor includes: If the broadcast mode is local mode, sending the cache invalidation event to the event handler of the corresponding connection; If the broadcast mode is a distributed mode, the cache invalidation event is stored in the message queue of the message middleware, and when the message in the message middleware is updated, the cache invalidation event is subscribed to from the message queue of the message middleware.

7. The method according to claim 1, characterized in that The updating of cache data according to the cache invalidation event includes: Extracting the database operation information from the cache invalidation event; The cache is updated according to the database operation information so that the data in the cache is consistent with the data in the database.

8. The method according to claim 7, characterized in that Updating the cache according to the database operation information includes: If the cache is a multi-level cache, a target cache is determined according to the database operation information, and the target cache is updated according to the database operation information.

9. The method according to claim 8, characterized in that Updating the target cache according to the database operation information includes: Determine the target cache and target cache update strategy corresponding to the database operation information according to the database operation information and a preset cache naming mapping table; The target cache is updated according to the target cache update policy.

10. An electronic device, characterized in that: include: A processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the cache data processing method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the cache data processing method according to any one of claims 1 to 9 are executed.

Images