Cache data processing method, device and storage medium

Abstract

The application provides a cache data processing method and device and a storage medium, and the method comprises the following steps: a data operation interceptor acquires a database access statement in real time; if the execution of the database access statement causes a database change, a cache invalidation event is generated according to the database access statement, and the cache invalidation event is sent to a transaction-aware event scheduler; the transaction-aware event scheduler monitors a database transaction corresponding to the database access statement in real time, sends the cache invalidation event to an event broadcaster, and sends transaction execution information to the event broadcaster according to the execution result of the database transaction; the event broadcaster temporarily stores the cache invalidation event, and determines whether to send the cache invalidation event to an event processor according to the transaction execution information; after the event processor receives the cache invalidation event, the cache data is updated according to the cache invalidation event. The application can realize the decoupling of business logic and the cache mechanism, and ensure the consistency of database data and cache data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, in particular, to a cache data processing method, device and storage medium. BACKGROUND

[0002] The data cache mechanism effectively improves the data access speed and system performance by storing data copies in memory or disk, and is currently widely used in database access optimization scenarios.

[0003] In the prior art, when the cache mechanism is introduced into an application, the business logic of the application needs to be modified to a certain extent, especially when a distributed cache is introduced, the distributed characteristics of the cache need to be handled, and the client library of the distributed cache needs to be integrated into the business logic.

[0004] However, when the data cache mechanism is applied to complex business scenarios, on the one hand, the difficulty of modifying the business logic is high, which increases the maintenance cost and modification complexity, on the other hand, the cache mechanism may not be able to cover complex business logic, which may result in low cache hit rate or inconsistent cache data. Moreover, due to the limitation of objective factors such as deployment environment, some applications may not be able to introduce a distributed cache, which may also result in difficulty in guaranteeing the cache consistency across services. SUMMARY

[0005] The present application aims to solve the problems of difficulty in guaranteeing the consistency of cache data and database data in the prior art, and difficulty in applying the cache mechanism when the application business logic is complex.

[0006] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows:

[0007] In a first aspect, the present application provides a cache data processing method applied to a cache data processing system, the cache data processing system comprising a data operation interceptor, a transaction-aware event scheduler, an event broadcaster and an event processor, and the method comprising:

[0008] The data operation interceptor acquires a database access statement in real time, and if the execution of the database access statement causes a database change, generates a cache invalidation event according to the database access statement, and sends the cache invalidation event to the transaction-aware event scheduler, wherein the cache invalidation event records the database operation information corresponding to the database access statement;

[0009] The transaction-aware event scheduler monitors a database transaction corresponding to the database access statement in real time, sends the cache invalidation event to the event broadcaster, and sends transaction execution information to the event broadcaster according to an execution result of the database transaction, the transaction execution information being used to indicate the execution result of the database transaction.

[0010] The event broadcaster temporarily stores the cache invalidation event, and determines whether to send the cache invalidation event to the event processor according to the transaction execution information.

[0011] If the event processor receives the cache invalidation event, cache data is updated according to the cache invalidation event.

[0012] Optionally, the cache invalidation event is generated according to the database access statement, and the method comprises:

[0013] Information extraction is performed on the database access statement to obtain database operation information in the database access statement, the database operation information at least comprising: event type, table name, changed data, and business identifier;

[0014] The database operation information is encapsulated into the cache invalidation event in a preset format.

[0015] Optionally, the execution information of the database transaction corresponding to the database access statement is monitored in real time, and the method comprises:

[0016] The value of a target attribute in a transaction operation interface is acquired in real time;

[0017] If the value of the target attribute is a valid value, the database transaction corresponding to the database access statement is monitored in real time.

[0018] Optionally, the transaction execution information is sent to the event broadcaster according to the execution result of the database transaction, and the method comprises:

[0019] If the execution result of the database transaction is committed, first transaction execution information is sent to the event broadcaster, the first transaction execution information being used to indicate that the transaction is committed.

[0020] If the execution result of the database transaction is rolled back, second transaction execution information is sent to the event broadcaster, the second transaction execution information being used to indicate that the transaction is rolled back.

[0021] Optionally, whether to send the cache invalidation event to the event processor is determined according to the transaction execution information, and the method comprises:

[0022] if the transaction execution information is the first transaction execution information, sending the cache invalidation event to the event processor;

[0023] if the transaction execution information is the second transaction execution information, emptying the cache invalidation event.

[0024] Optionally, the broadcast mode of the event broadcaster comprises a local mode and a distributed mode, in the local mode, the event broadcaster communicates with the event processor one by one, in the distributed mode, the event broadcaster is connected to a message middleware, a plurality of event processors are connected to the message middleware, and the message middleware is subscribed to messages.

[0025] The sending of the cache invalidation event to the event processor comprises:

[0026] if the broadcast mode is the local mode, the cache invalidation event is sent to the corresponding connected event processor;

[0027] if the broadcast mode is the distributed mode, the cache invalidation event is stored in a message queue in the message middleware, and the cache invalidation event is obtained from the message queue in the message middleware when a message in the message middleware is updated.

[0028] Optionally, the cache data updating according to the cache invalidation event comprises:

[0029] the database operation information is extracted from the cache invalidation event;

[0030] the cache is updated according to the database operation information, so that the data in the cache is consistent with the data in the database.

[0031] Optionally, the cache updating according to the database operation information comprises:

[0032] if the cache is a multi-level cache, a target cache is determined according to the database operation information, and the target cache is updated according to the database operation information.

[0033] Optionally, the cache updating according to the database operation information comprises:

[0034] a target cache and a target cache updating strategy corresponding to the database operation information are determined according to the database operation information and a preset cache naming mapping table;

[0035] the target cache is updated according to the target cache updating strategy.

[0036] In a second aspect, the embodiments of the present application further provide an electronic device, comprising a processor, a storage medium and a bus, the storage medium stores machine readable instructions executable by the processor, when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine readable instructions to perform the steps of the cache data processing method according to any one of the first aspect.

[0037] In a third aspect, the embodiments of the present application further provide a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is run by a processor, the steps of the cache data processing method according to any one of the first aspect are executed.

[0038] The beneficial effects of the present application are: by monitoring the database access statements at the service level and perceiving the database transactions, the database access statements that can cause database updates are determined after the completion of the database transactions, and the data in the cache is modified according to the determined database access statements, thereby ensuring the consistency of the database and the cache data. Compared with the prior art, the present application does not need to modify the business logic of the monomer application, and realizes the decoupling of the business logic and the cache mechanism. Moreover, by combining SQL statement monitoring and transaction perception, the cache mechanism has transaction perception capability, and then the data in the cache is modified accordingly when the transaction is successfully submitted, further ensuring the consistency of the data. Through the event broadcaster and the event processor, the cache data is updated, even for distributed cache or cross-service cache, the effective forwarding of the database access statements can be realized, thereby ensuring the consistency of the data in the distributed cache and the cross-service cache.

[0039] In order to make the above objectives, characteristics and advantages of the present application more apparent and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0041] Figure 1 An application scenario schematic diagram of a cache data processing system provided by the embodiments of the present application is shown;

[0042] Figure 2 A flowchart of a cache data processing method provided by the embodiments of the present application is shown;

[0043] Figure 3 A flowchart of generating a cache invalidation event is shown;

[0044] Figure 4 A flowchart of monitoring a database transaction is shown;

[0045] Figure 5 A flowchart of sending transaction execution information is shown;

[0046] Figure 6 A flowchart of receiving a cache invalidation event is shown;

[0047] Figure 7 A flowchart of updating cache data is shown;

[0048] Figure 8 A flowchart of updating cache data is shown;

[0049] Figure 9 A structural schematic diagram of an electronic device is shown. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings for the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0051] It should be noted that the term “comprising” will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0052] Data caching mechanism can be used to improve database query efficiency and system response performance. Common cache architectures include local cache and distributed cache. For example, MyBatis (a Java-based persistence layer framework) as a framework for database access, its plug-in mechanism allows developers to intercept before and after SQL (Structured Query Language, structured query language) execution to achieve various extension functions.

[0053] However, the business logic of the current monomer application is complex, the multi-table association query is frequent, and the existing cache strategy is difficult to fully cover all scenarios, so the application effect of the cache strategy is not good. Since the cache mechanism is closely related to the business logic code, there is a lack of automatic cache management mechanism, if you want to improve the application effect of the cache strategy in the complex monomer application, you have to inevitably modify the business logic of the application, so it will lead to long and difficult to maintain code.

[0054] On the other hand, distributed cache has high requirements for network environment, and most of the current distributed cache architectures use network isolation technology, which will also affect the normal communication and data synchronization of the cache, so it will cause the problem of data inconsistency. Some cross-service applications cannot access distributed cache services due to deployment environment or technology stack restrictions, so it is difficult to guarantee the consistency of cross-service cache.

[0055] In addition, the existing cache mechanism cannot perceive the commit or rollback state of the database transaction, and the coordination between the cache mechanism and the database is insufficient. The cache mechanism cannot automatically update or invalidate the cache according to the state of the transaction, which requires the developer to handle it manually, which also increases the development difficulty and the complexity of system maintenance.

[0056] Based on this, the application provides a cache data processing method, which listens to the database access statements at the SQL service level and perceives the database transaction, so as to determine the database access statements that can cause database update after the completion of the database transaction, and modify the data in the cache according to the determined database access statements, so as to guarantee the consistency of the database and the cache data. Compared with the prior art, the application listens to the data changes involved in SQL at the SQL service level, without modifying the business logic of the monomer application, realizing the decoupling of business logic and cache mechanism. And, by combining SQL statement listening and transaction perception, the cache mechanism has transaction perception capability, so that the data in the cache is modified accordingly only when the transaction is successfully submitted, further guaranteeing the consistency of the data.

[0057] The cache data processing method provided by the application updates the cache data through the event broadcaster and the event processor, so that the database access statements can be effectively forwarded even for distributed cache or cross-service cache, thereby guaranteeing the consistency of the data in the distributed cache and the cross-service cache.

[0058] The cache data processing method of the application can be applied to Figure 1 The cache data processing system shown in the figure. Refer to Figure 1The cache data processing system is communicatively connected with a database and an application, a user can initiate a data modification operation on the database in an application installed on an electronic device, the cache data processing system can listen to the modification operation of the user on the database, execute the data modification operation to modify the data in the database, and synchronize the corresponding data in the cache. For example, the cache data processing system can be MyBatis, and the cache data processing system can be integrated with other external caches in addition to the self-provided cache mechanism.

[0059] Referring to Figure 1 The cache data processing system includes a data operation interceptor, a transaction-aware event scheduler, an event broadcaster, and an event processor. The data operation interceptor is configured to obtain an access statement initiated by a user on a database, the transaction-aware event scheduler is configured to perceive a database transaction corresponding to the access statement on the database, and prompt the event broadcaster to send a cache invalidation event to the event processor for data updating when the database transaction is completed.

[0060] In the local mode, that is, when the application and the cache are both deployed locally, the data operation interceptor, the transaction-aware event scheduler, the event broadcaster, and the event processor can all be deployed locally. In the distributed mode, the cache of the cache data processing system can be a distributed architecture, at this time, the application, the data operation interceptor, the transaction-aware event scheduler, and the event broadcaster can be deployed on the same device, and the event processor can be deployed in each distributed cache.

[0061] Figure 2 is a flowchart of a cache data processing method provided by an embodiment of the present application. The method can be applied to Figure 1 as shown in the cache data processing system, referring to Figure 2 The method comprises the following steps.

[0062] In S201, the data operation interceptor obtains a database access statement in real time. If the execution of the database access statement causes a change in the database, a cache invalidation event is generated according to the database access statement, and the cache invalidation event is sent to the transaction-aware event scheduler. The cache invalidation event records database operation information corresponding to the database access statement.

[0063] Optionally, the user generates the database access statement by operating on an interface. The data operation interceptor can obtain the database access statement in real time before executing the database access statement. The data access statement can be a SQL statement, for example. For ease of description, the following description takes the database access statement as a SQL statement. The data operation interceptor can be in the same thread as the main program that executes the modification operation on the database, and the data operation interceptor is executed before the main program, which is a sequential execution relationship.

[0064] The data operation interceptor can monitor all database access operations in the application in real time, and obtain the SQL statements executed by the operations in real time. For the obtained SQL statements, the data operation interceptor can analyze whether the execution of the statements will cause data changes of the database. For example, the INSERT, UPDATE and DELETE statements usually cause database changes, and at this time, the cache invalidation events can be generated according to the database access statements.

[0065] Optionally, the cache invalidation event can be an event that can cause cache invalidation. It should be understood that data changes can cause cache invalidation, and therefore the cache invalidation events can be generated according to the database access statements that can cause data changes. As a possible implementation, information extraction can be performed on the database access statements, and the extracted information can be encapsulated as a cache invalidation event.

[0066] The database operation information includes an event type, an operation object, changed data, a business identifier, and the like. The event type can indicate the operation type of the SQL statement, such as INSERT, UPDATE and DELETE. The operation object can indicate the table name and the changed data position of the SQL statement operation. The changed data can indicate the data before and after the execution of the SQL statement. The business identifier can indicate the operation personnel or the source module that issues the SQL statement.

[0067] After the data operation interceptor generates the cache invalidation event, the cache invalidation event can be sent to the transaction-aware event scheduler. The transaction-aware event scheduler can perceive the database transaction, and determine whether the database transaction corresponding to the cache invalidation event is successfully executed, so as to manage and schedule the cache invalidation event, and ensure that the cache invalidation event can be processed in the correct order and at the appropriate time.

[0068] It should be noted that the above S201 step is to obtain the SQL statement at the SQL service level, and does not change the internal business logic of the application, nor does it affect the execution process of the SQL statement. Therefore, the decoupling of the business logic and the cache invalidation event acquisition is achieved, and thus the user operation that can cause database changes can be obtained in time without changing the business logic of the application.

[0069] S202, the transaction-aware event scheduler monitors the database transaction corresponding to the database access statement in real time, sends the cache invalidation event to the event broadcaster, and sends transaction execution information to the event broadcaster according to the execution result of the database transaction. The transaction execution information is used to indicate the execution result of the database transaction.

[0070] Optionally, the transaction-aware event dispatcher can monitor the database transaction corresponding to the database access statement in real time. For example, the transaction-aware event dispatcher can track the start, execution process, and final commit or rollback state of the database transaction, and can perform connection recycling and closing for JDBC.

[0071] For example, the transaction-aware event dispatcher can be a static proxy class of JDBC (Java Database Connectivity), which can perform proxy operations during the life cycle of the database connection, thereby ensuring thread safety. As a proxy of the Connection interface, the transaction-aware event dispatcher can intercept and extend the method calls of the Connection interface, such as transaction commit, rollback, connection closing, and the like. In a multi-threaded environment, multiple threads can access the transaction-aware event dispatcher simultaneously, and the life cycle of the transaction-aware event dispatcher is synchronized with the opening and closing of the Connection interface, i.e., the transaction-aware event dispatcher is created when the Connection interface is opened, and the transaction-aware event dispatcher is destroyed when the Connection interface is closed. When the connection is closed, the transaction-aware event dispatcher can perform cleaning work, such as disconnecting the database connection and releasing database resources.

[0072] In one possible implementation, after the data operation interceptor generates the cache invalidation event and sends it to the transaction-aware event dispatcher, the transaction-aware event dispatcher can temporarily retain the cache invalidation event and monitor the database transaction corresponding to the database access statement in real time until the corresponding database transaction is completed.

[0073] After the database transaction is executed, the transaction-aware event dispatcher can send transaction execution information to the event broadcaster according to the execution result of the database transaction.

[0074] Optionally, the transaction execution information is used to indicate the final state of the database transaction. For example, if the transaction is successfully committed, the transaction execution information indicates "commit success"; if the transaction is rolled back, the transaction execution information indicates "rollback".

[0075] It should be noted that, in the embodiments of the present application, the database transaction corresponding to the database access statement is monitored, and the transaction execution information is sent to the event broadcaster after the database transaction is executed, which can ensure that the modification of the cache data is performed after the database transaction is completed, thereby avoiding the modification error of the cache data caused by the rollback of the database transaction, and ensuring the consistency of the cache data and the database data.

[0076] S203、The event broadcaster temporarily stores the cache invalidation event, and determines whether to send the cache invalidation event to the event processor according to the transaction execution information.

[0077] In a possible implementation, the transaction-aware event scheduler does not save the cache invalidation event. After receiving the cache invalidation event, the transaction-aware event scheduler can immediately send the cache invalidation event to the event broadcaster, and the event broadcaster temporarily stores the cache invalidation event. The transaction-aware event scheduler monitors the execution result of the database transaction corresponding to the cache invalidation event, and sends the transaction execution information to the event broadcaster according to the execution result of the database transaction. At this time, if the database transaction is successfully committed, the event broadcaster sends the cache invalidation event to the event processor. If the database transaction is rolled back, the event broadcaster does not send the cache invalidation event to the event processor.

[0078] In another possible implementation, the transaction-aware event scheduler can temporarily store the cache invalidation event and monitor the execution result of the database transaction corresponding to the cache invalidation event in real time. If the execution result of the database transaction is successful commitment, the transaction-aware event scheduler sends the cache invalidation event to the event broadcaster, and the event broadcaster sends the cache invalidation event to the event processor. If the execution result of the database transaction is rollback, the transaction-aware event scheduler does not send the cache invalidation event to the event broadcaster.

[0079] Optionally, the event broadcaster can send the cache invalidation event one by one and synchronously. When the number of cache invalidation events is large or the cache invalidation events are abnormal, the event broadcaster can also send the cache invalidation events to the event processor asynchronously.

[0080] In the local mode, the event broadcaster and the event processor are both deployed locally. At this time, the event broadcaster can directly send the cache invalidation event to the event processor.

[0081] In the distributed mode, the event processors can be deployed in each distributed cache or cross-module cache respectively. At this time, the event broadcaster can store the cache invalidation event in the message queue, and each event processor subscribes to the message queue to obtain the updated cache invalidation event.

[0082] S204, if the event processor receives the cache invalidation event, the cache data is updated according to the cache invalidation event.

[0083] Optionally, when there are multiple distributed caches, one event processor can be deployed in each distributed cache. The event processor can receive the cache invalidation event sent by the event broadcaster, and update or clear the cache data according to the cache invalidation event, so as to ensure that the cache is consistent with the database data.

[0084] If there is a multi-level cache strategy, after receiving the cache invalidation event, the event processor can first determine the target cache module corresponding to the cache invalidation event, and update or clear the cache data of the target cache module.

[0085] In this embodiment, by listening to database access statements at the service layer and being aware of database transactions, the system identifies database access statements that can trigger database updates after a transaction is completed. The system then modifies the cached data based on these identified access statements, ensuring consistency between the database and cached data. Compared to existing technologies, this application does not require modification of the business logic of a monolithic application, achieving decoupling between business logic and the caching mechanism. Furthermore, by combining SQL statement listening and transaction awareness, this application endows the caching mechanism with transaction awareness capabilities, allowing for modifications to the cached data only upon successful transaction commit, further guaranteeing data consistency. Updating cached data through event broadcasters and event handlers ensures effective forwarding of database access statements, even for distributed or cross-service caches, thereby guaranteeing data consistency in both distributed and cross-service caches.

[0086] The following explains the steps for generating cache invalidation events based on database access statements, such as... Figure 3 As shown, the above step S201 includes:

[0087] S301. Extract information from the database access statement to obtain the database operation information in the database access statement. The database operation information includes at least: event type, table name, changed data, and business identifier.

[0088] Optionally, the data manipulation interceptor can parse the database access statement to obtain the table name, event type, changed data, business identifier, etc. The event type can be the operation type of the database access statement. By parsing the condition fields of the database access statement, the table name, the changed column name, and the changed data can be obtained. The business identifier can be used to indicate the operator who initiated the database access statement, the source module, etc.

[0089] S302. Encapsulate the database operation information into a cache invalidation event according to the preset format.

[0090] Optionally, cache invalidation events can have a preset format. By filling database operation information into the corresponding positions according to the preset format and instantiating it, structured cache invalidation events can be generated.

[0091] In one possible implementation, during the encapsulation of database operation information, users can modify the content and structure of cache invalidation events to adapt to the actual needs of different businesses.

[0092] The following is a further explanation of the database transactions corresponding to the database access statements mentioned above in real-time monitoring, such as...Figure 4 As shown in the above S202 step includes:

[0093] S401, real-time acquisition of the value of the target attribute in the transaction operation interface.

[0094] Optionally, the transaction operation interface is used to execute the database access statement and manage the transaction corresponding to the database access statement. The target attribute is used to indicate the number of database transactions currently associated with the transaction operation interface. For example, in MyBatis, the transaction operation interface can be BaseExecutor in MyBatis, and the target attribute can be the transaction attribute of BaseExecutor.

[0095] S402, if the value of the target attribute is a valid value, real-time monitoring of the database transaction corresponding to the database access statement.

[0096] Wherein, the value of the target attribute is a valid value, which can be a preset value of the target attribute. For example, in MyBatis, the transaction attribute is assigned a preset valid value when executing a database transaction, so when monitoring the database transaction corresponding to the database access statement, the value of the transaction attribute of BaseExecutor can be read. If there is a valid value in the transaction attribute, it means that there is a database transaction.

[0097] It is worth noting that the user can set an automatic commit mode or a transaction commit mode. In the automatic commit mode, the database access statement does not generate a corresponding database transaction, and the value of the target attribute is invalid at this time. The cache invalidation event can be directly sent to the event broadcaster by the transaction-aware event scheduler, and the cache invalidation event can be directly published by the event broadcaster.

[0098] In the transaction commit mode, the database access statement that causes the database data to change will generate a corresponding database transaction, and the value of the target attribute is valid at this time. The transaction-aware event scheduler can monitor the database transaction.

[0099] The following is a further description of the above sending transaction execution information to the event broadcaster according to the execution result of the database transaction, as shown in Figure 5 As shown in the above S202 step includes:

[0100] S501, if the execution result of the database transaction is committed, the first transaction execution information is sent to the event broadcaster, and the first transaction execution information is used to indicate that the transaction has been committed.

[0101] Optionally, the transaction-aware event dispatcher can receive a transaction callback event, the transaction callback event being used to indicate that the database transaction has been successfully committed. When the transaction-aware event dispatcher receives the transaction callback event indicating the successful commit, the transaction-aware event dispatcher can send the first transaction execution information to the event broadcaster.

[0102] After the event broadcaster receives the first transaction execution information, the event broadcaster can determine that the database transaction has been successfully committed, i.e., the data in the database has been modified according to the database access statement, and thus the event broadcaster can send the cache invalidation event to the event processor for modifying the data in the cache.

[0103] S502, if the execution result of the database transaction is rolled back, the second transaction execution information is sent to the event broadcaster, the second transaction execution information being used to indicate that the transaction has been rolled back.

[0104] Optionally, if the transaction is rolled back, a transaction callback event is triggered, and at this time the transaction callback event is used to indicate that the database transaction is rolled back. When the transaction-aware event dispatcher receives the transaction callback event indicating the rollback, the transaction-aware event dispatcher can send the second transaction execution information to the event broadcaster.

[0105] It should be noted that the transaction is rolled back, which means that the transaction is not successfully executed, i.e., the data in the database is not modified. Therefore, after the event broadcaster receives the second transaction execution information, the event broadcaster can not send the cache invalidation event to the event processor.

[0106] The following is a determination of whether to send the cache invalidation event to the event processor according to the transaction execution information, including:

[0107] If the transaction execution information is the first transaction execution information, the cache invalidation event is sent to the event processor.

[0108] Optionally, if the transaction execution information is the first transaction execution information, it means that the database transaction is successfully executed. At this time, the event broadcaster can send the cache invalidation event to the event processor, and the event processor can update the data in the cache according to the cache invalidation event.

[0109] If the transaction execution information is the second transaction execution information, the cache invalidation event is emptied.

[0110] Optionally, if the transaction execution information is the second transaction execution information, it means that the database transaction is rolled back, i.e., the database transaction is cancelled or not executed. At this time, the event broadcaster can empty all the cache invalidation events temporarily stored.

[0111] In the embodiments of the present application, the database transaction is perceived, and the transaction execution information is sent to the event broadcaster after the database transaction is executed, so that the change of the cache data can be ensured to occur after the execution of the database transaction, and the consistency between the database data and the cache data is ensured.

[0112] Optionally, the broadcast mode of the event broadcaster includes a local mode, in which the event broadcaster communicates with the event processor one by one.

[0113] In another possible implementation, the broadcast mode of the event broadcaster includes a distributed mode, in which the event broadcaster is connected to a message middleware, and a plurality of event processors are connected to the message middleware and subscribe to messages from the message middleware.

[0114] In the local mode, the event processor saves the registration information in the event broadcaster, as shown in Figure 6 The step of sending the cache invalidation event to the event processor includes:

[0115] S601, if the broadcast mode is the local mode, the cache invalidation event is sent to the corresponding connected event processor.

[0116] Optionally, the corresponding connected event processor can be the event processor whose registration information is saved in the event broadcaster. After the event broadcaster receives the first transaction execution information, the cache invalidation event can be sent to the registered event processor.

[0117] S602, if the broadcast mode is the distributed mode, the cache invalidation event is stored in the message queue in the message middleware, and the cache invalidation event is obtained from the message queue in the message middleware when the message in the message middleware is updated.

[0118] Optionally, the event broadcaster can store the cache invalidation event in the message queue. When there are a plurality of message queues of different topics in the message middleware, the event broadcaster can determine the message queue corresponding to the cache invalidation event according to the content or the business identifier in the cache invalidation event, and store the cache invalidation event in the corresponding message queue.

[0119] Optionally, the message middleware can include a plurality of message queues of different topics. For example, the data of the distributed cache can be stored in different message queues of different topics according to business requirements, and each distributed cache subscribes to the corresponding message queue. When the subscribed message queue is updated, the event processor of the distributed cache can obtain the updated cache invalidation event from the message queue, or the message queue can push the updated cache invalidation event to the event processor that subscribes to the message queue.

[0120] In the embodiments of the present application, when the cache mechanism is cross-module or distributed, the message middleware is set to ensure that all event processors can timely receive updated cache invalidation events. Even if the distributed cache adopts network isolation technology, the synchronization of cache invalidation events can be realized through subscribing to the message queue, thereby ensuring the data consistency of cross-service cache and distributed cache.

[0121] The following is a further description of the above cache data updating according to the cache invalidation event. As shown in Figure 7 The S204 step includes:

[0122] S701, extracting database operation information from the cache invalidation event.

[0123] S702, updating the cache according to the database operation information to make the data in the cache consistent with the data in the database.

[0124] Optionally, the event processor can parse the cache invalidation event, extract the database operation information, and match the database operation information with the local cache naming strategy to determine the corresponding cache data of the database operation information in the local cache, and determine the corresponding cache invalidation logic according to the database operation information, thereby realizing the updating of the cache data.

[0125] When the cache is a multi-level cache, the above step of updating the cache according to the database operation information includes:

[0126] If the cache is a multi-level cache, the target cache is determined according to the database operation information, and the target cache is updated according to the database operation information.

[0127] Optionally, the target cache can be the cache where the cache data corresponding to the database operation information is located. By matching the database operation information with the preset cache naming mapping table, the target cache can be determined, and the cache data in the target cache can be updated according to the database operation information.

[0128] As shown in Figure 8 The above step of updating the target cache according to the database operation information specifically includes:

[0129] S801, determining the target cache corresponding to the database operation information and the target cache update strategy according to the database operation information and the preset cache naming mapping table.

[0130] S802, updating the target cache according to the target cache update strategy.

[0131] The cache naming mapping table stores the correspondence between the local cache and the content in the database. For example, the corresponding cache data of the changed data in the local cache can be determined by matching the table name and the changed column name in the database operation information in the cache naming mapping table.

[0132] As a possible implementation, the cache naming mapping table can also save the cache update strategy available for each cache data. After receiving the database operation information, the cache update strategy corresponding to the database operation information can be used as the target cache update strategy.

[0133] For example, the database operation information includes an event type. When the event type is a deletion operation, the target cache update strategy can be to delete the specified record in the target cache. When the event type is an update operation, the target cache update strategy can be to query the database and update the data in the target cache. When the event type is an insertion operation, the target cache update strategy can be to not perform data processing. When the event type is an update operation and the business identifier is a preset identifier, the target cache update strategy can be to mark the cache data as expired and wait for lazy loading refresh.

[0134] It should be noted that the correspondence between the database operation information and the cache update strategy can be determined based on actual business needs. The present application only gives one possible example, which should not be limited.

[0135] Based on the same inventive concept, the present application also provides a cache data processing system corresponding to the cache data processing method. Since the system in the present application has a similar problem-solving principle to the above-mentioned cache data processing method of the present application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be repeated.

[0136] Referring to Figure 1 the structural schematic diagram of the cache data processing system. The cache data processing system comprises a data operation interceptor, a transaction-aware event scheduler, an event broadcaster, and an event processor.

[0137] The data operation interceptor is configured to acquire a database access statement in real time. If the execution of the database access statement causes a database change, a cache invalidation event is generated according to the database access statement, and the cache invalidation event is sent to the transaction-aware event scheduler. The cache invalidation event records the database operation information corresponding to the database access statement.

[0138] The transaction-aware event scheduler is configured to monitor the database transaction corresponding to the database access statement in real time, send the cache invalidation event to the event broadcaster, and send transaction execution information to the event broadcaster according to the execution result of the database transaction. The transaction execution information is used to indicate the execution result of the database transaction.

[0139] The event broadcaster is configured to temporarily store the cache invalidation event, and determine whether to send the cache invalidation event to the event processor according to transaction execution information;

[0140] The event processor is configured to update cache data according to the cache invalidation event when the cache invalidation event is received.

[0141] Optionally, the data operation interceptor is specifically configured to:

[0142] extract information from the database access statement to obtain database operation information in the database access statement, the database operation information at least including: an event type, a table name, changed data, and a business identifier;

[0143] encapsulate the database operation information into the cache invalidation event according to a preset format.

[0144] Optionally, the transaction-aware event scheduler is specifically configured to:

[0145] acquire a value of a target attribute in the transaction operation interface in real time;

[0146] if the value of the target attribute is a valid value, monitor a database transaction corresponding to the database access statement in real time.

[0147] Optionally, the transaction-aware event scheduler is specifically configured to:

[0148] if an execution result of the database transaction is committed, send first transaction execution information to the event broadcaster, the first transaction execution information being used to indicate that the transaction is committed;

[0149] if the execution result of the database transaction is rolled back, send second transaction execution information to the event broadcaster, the second transaction execution information being used to indicate that the transaction is rolled back.

[0150] Optionally, the event broadcaster is specifically configured to:

[0151] if the transaction execution information is the first transaction execution information, send the cache invalidation event to the event processor;

[0152] if the transaction execution information is the second transaction execution information, clear the cache invalidation event.

[0153] Optionally, a broadcast mode of the event broadcaster includes: a local mode and a distributed mode, in the local mode, the event broadcaster and the event processor correspond to each other to communicate, in the distributed mode, the event broadcaster is connected to a message middleware, a plurality of event processors are connected to the message middleware, and the plurality of event processors subscribe to messages from the message middleware;

[0154] Optionally, the event broadcaster is specifically configured to:

[0155] If the broadcast mode is the local mode, the cache invalidation event is sent to an event handler corresponding to the connection.

[0156] If the broadcast mode is the distributed mode, the cache invalidation event is stored in a message queue in a message middleware, and the cache invalidation event is subscribed from the message queue in the message middleware when a message in the message middleware is updated.

[0157] Optionally, the event handler is specifically configured to:

[0158] extract database operation information from the cache invalidation event;

[0159] update the cache according to the database operation information, so that the data in the cache is consistent with the data in the database.

[0160] Optionally, the event handler is specifically configured to:

[0161] If the cache is a multi-level cache, the target cache is determined according to the database operation information, and the target cache is updated according to the database operation information.

[0162] Optionally, the event handler is specifically configured to:

[0163] determine the target cache and the target cache update strategy corresponding to the database operation information according to the database operation information and a preset cache naming mapping table;

[0164] update the target cache according to the target cache update strategy.

[0165] Embodiments of the present application monitor database access statements at the service level, and perceive database transactions, so as to determine database access statements that can cause database updates after the completion of the database transactions, and modify the data in the cache according to the determined database access statements, thereby ensuring the consistency of the database and the cache data. Compared with the prior art, the present application does not need to modify the business logic of the monomer application, and realizes the decoupling of the business logic and the cache mechanism. Moreover, the present application combines SQL statement monitoring and transaction perception, so that the cache mechanism has transaction perception capability, and then the data in the cache is modified accordingly when the transaction is successfully submitted, thereby further ensuring the consistency of the data. Through the update of the cache data by the event broadcaster and the event handler, even for distributed cache or cross-service cache, the effective forwarding of the database access statements can be realized, thereby ensuring the consistency of the data in the distributed cache and the cross-service cache.

[0166] Figure 9A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown, which includes a processor 901, a storage medium 902, and a bus 903. The storage medium 902 stores machine readable instructions executable by the processor 901. When the electronic device runs the cache data processing method as in the embodiments, the processor 901 and the storage medium 902 communicate through the bus 903. The processor 901 executes the machine readable instructions, and the processor 901 performs the preamble of the method to execute the steps in the cache data processing method.

[0167] The present application also provides a computer readable storage medium, which stores a computer program. When the computer program is run by a processor, the processor executes the steps in the cache data processing method.

[0168] In the embodiments of the present application, when the computer program is run by the processor, the computer program can also execute other machine readable instructions to perform other methods as described in the embodiments. For the specific method steps and principles, refer to the description of the embodiments, which will not be described in detail here.

[0169] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, and can be electrical, mechanical or other forms.

[0170] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the present embodiment.

[0171] In addition, each functional unit in the embodiments provided in the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0172] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0173] It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third" and the like are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0174] Finally, it should be noted that the above-described embodiments are only specific implementations of the present application, which are used to illustrate the technical solutions of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present application, or make equivalent replacements to some technical features. These modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of processing data in a cache, the method comprising: The application is applied to a cache data processing system, and the cache data processing system comprises a data operation interceptor, a transaction-aware event scheduler, an event broadcaster and an event processor, and the method comprises the following steps: The data operation interceptor acquires a database access statement in real time, generates a cache invalidation event according to the database access statement if execution of the database access statement causes a database change, and sends the cache invalidation event to the transaction-aware event scheduler, wherein database operation information corresponding to the database access statement is recorded in the cache invalidation event; The transaction-aware event scheduler monitors a database transaction corresponding to the database access statement in real time, sends the cache invalidation event to the event broadcaster, and sends transaction execution information to the event broadcaster according to an execution result of the database transaction, wherein the transaction execution information is used for indicating the execution result of the database transaction; The event broadcaster temporarily stores the cache invalidation event, and determines whether to send the cache invalidation event to the event processor according to the transaction execution information; If the event processor receives the cache invalidation event, the event processor updates cache data according to the cache invalidation event; The broadcast mode of the event broadcaster comprises a local mode and a distributed mode, in the local mode, the event broadcaster and the event processor are in one-to-one correspondence communication, and in the distributed mode, the event broadcaster is connected to a message middleware, a plurality of event processors are connected to the message middleware, and the message middleware is subscribed to a message; The process of sending the cache invalidation event to the event processor comprises the following steps: If the broadcast mode is the local mode, the cache invalidation event is sent to the corresponding connected event processor; If the broadcast mode is the distributed mode, the cache invalidation event is stored in a message queue in the message middleware, and the cache invalidation event is obtained from the message queue in the message middleware when a message in the message middleware is updated.

2. The method of claim 1, wherein, The cache invalidation event is generated according to the database access statement, which comprises the following steps: Information extraction is performed on the database access statement to obtain database operation information in the database access statement, wherein the database operation information at least comprises an event type, a table name, changed data and a business identifier; The database operation information is encapsulated into the cache invalidation event according to a preset format.

3. The method of claim 1, wherein, The execution information of the database transaction corresponding to the database access statement is monitored in real time, which comprises the following steps: The value of a target attribute in a transaction operation interface is acquired in real time; If the value of the target attribute is a valid value, the database transaction corresponding to the database access statement is monitored in real time.

4. The method of claim 1, wherein, The transaction execution information is sent to the event broadcaster according to the execution result of the database transaction, which comprises the following steps: If the execution result of the database transaction is committed, first transaction execution information is sent to the event broadcaster, wherein the first transaction execution information is used for indicating that the transaction is committed. If the execution result of the database transaction is rolled back, the second transaction execution information is sent to the event broadcaster, the second transaction execution information being used to indicate that the transaction is rolled back.

5. The method of claim 4, wherein, The determining whether to send the cache invalidation event to the event processor according to the transaction execution information comprises: If the transaction execution information is the first transaction execution information, the cache invalidation event is sent to the event processor; If the transaction execution information is the second transaction execution information, the cache invalidation event is emptied.

6. The method of claim 1, wherein, The updating cache data according to the cache invalidation event comprises: The database operation information is extracted from the cache invalidation event; The cache is updated according to the database operation information, so that the data in the cache is consistent with the data in the database.

7. The method of claim 6, wherein, The updating cache according to the database operation information comprises: If the cache is a multi-level cache, the target cache is determined according to the database operation information, and the target cache is updated according to the database operation information.

8. The method of claim 7, wherein, The updating the target cache according to the database operation information comprises: The target cache and the target cache update strategy corresponding to the database operation information are determined according to the database operation information and a preset cache naming mapping table; The target cache is updated according to the target cache update strategy.

9. An electronic device, comprising: It comprises: A processor, a storage medium and a bus, the storage medium stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the storage medium communicate through the bus, the processor executes the machine readable instructions to execute the steps of the cache data processing method in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, the computer program is executed by the processor to execute the steps of the cache data processing method in any one of claims 1 to 8.

Images