Cache data processing method and device, computer equipment and storage medium
The cache data processing method addresses inconsistent data issues by monitoring and updating cache data based on target logs, ensuring data consistency and enhancing system performance.
Patent Information
- Application Number
- CN202411108736.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-12-06
AI Technical Summary
Inconsistent data between business system databases and cache databases due to unsynchronized data changes across multiple systems sharing a database table, leading to performance issues.
A cache data processing method that monitors cache configuration information, processes target logs to determine and update cache data accordingly, ensuring data consistency by generating cache keys and expiration times, and maintaining a mapping relationship in memory.
Ensures data consistency between business system databases and cache databases by directly processing cache data without going through the business system, addressing inconsistencies and improving system performance.
Smart Images

Figure CN119088833A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of database technology, and in particular to a cache data processing method, device, computer equipment and storage medium. Background Art
[0002] Using cache in Web application development can reduce the pressure on database access, thereby improving system performance, response speed and stability. The common way to use cache is to process the data in the cache after a business system changes the data in the table. However, there are two problems with this method of use: 1. In the scenario where multiple business systems share a database table, since the data change operations on the table are not shared between the systems, after a system changes the data in the table, it will not be updated synchronously to other systems, resulting in inconsistency between the cache and the latest data in the database, causing serious impact on the business. 2. Changing the table data directly through the database client without going through the business system interface will also cause inconsistency between the cache and the latest data in the database.
[0003] Therefore, how to ensure the consistency of data in the business system database and the data in the cache database has become a problem that needs to be solved urgently. Summary of the invention
[0004] In view of this, in order to solve the above technical problems or part of the technical problems, the embodiments of the present invention provide a cache data processing method, apparatus, computer equipment and storage medium.
[0005] In a first aspect, an embodiment of the present invention provides a cache data processing method, including:
[0006] Monitoring cache configuration information of each cache method in the business system from a target processing platform for storage, the target processing platform being used to receive the cache configuration information sent by the business system;
[0007] When receiving a target log sent by a first database corresponding to the business system, searching for target cache configuration information from the plurality of cache configuration information according to the target log, the target log representing the processing operation of the business system on the data in the first database;
[0008] Determine target cache data from a cache database according to the target cache configuration information;
[0009] The target cache data is processed according to the target log.
[0010] In a possible implementation, the cache configuration information is generated in the following manner:
[0011] Obtain the unique identifier, cache expiration time, queried database name, and queried table name of each cache method in the business system;
[0012] Generate a cache key using the unique identifier and the project number of the cache method;
[0013] The cache key, the cache expiration time, the database name and the table name are used as the cache configuration information.
[0014] In a possible implementation, the slave target processing platform monitors cache configuration information of each cache method in the business system for storage, including:
[0015] Generate a mapping relationship by using the database name and the table name in each monitored cache configuration information as a key and the cache configuration information as a value;
[0016] The mapping relationship is stored in the memory.
[0017] In a possible implementation, searching for target cache configuration information from the plurality of cache configuration information according to the target log includes:
[0018] Obtain the target database name and target table name in the target log;
[0019] Determining the target cache configuration information from the mapping relationship according to the target database name and the target table name;
[0020] The determining the target cache data from the cache database according to the target cache configuration information includes:
[0021] Obtaining a first cache key in the target cache configuration information;
[0022] The target cache data corresponding to the first cache key is determined from the cache database, and each cache key corresponds to the cache data in the cache database one by one.
[0023] In a possible implementation, the processing the target cache data according to the target log includes:
[0024] When the operation information contained in the target log is a deletion operation or an update operation, obtaining a cache identifier of the target cache data;
[0025] Obtaining a database identifier of the original data from the target log;
[0026] When the cache identifier and the database identifier are inconsistent, determining that the operation information is a deletion operation;
[0027] Deleting the target cache data in the cache database according to the deletion operation;
[0028] Or, when the cache identifier and the database identifier are consistent, it is determined that the operation information is an update operation;
[0029] Obtaining updated data from the target log;
[0030] The target cache data is updated to the updated data according to the update operation.
[0031] In a possible implementation, the method further includes:
[0032] When the operation information contained in the target log is an insert operation, obtaining insert data from the target log;
[0033] Inserting the insert data into the cache database;
[0034] Set the cache expiration time of the inserted data.
[0035] In a possible implementation, the method further includes:
[0036] When any target cache method is executed, obtaining a second cache key in the target cache method;
[0037] Query the cache data corresponding to the second cache key in the cache database;
[0038] When the query result is empty, obtaining target data corresponding to the target cache method from the first database;
[0039] Saving the target data in the cache database, wherein the target data corresponds to the second cache key;
[0040] When the query result is not empty, the data corresponding to the query result is used as the return value of the target cache method.
[0041] In a second aspect, an embodiment of the present invention provides a cache data processing device, including:
[0042] A monitoring module, used to monitor cache configuration information of each cache method in the business system from a target processing platform for storage, the target processing platform being used to receive the cache configuration information sent by the business system;
[0043] a search module, configured to search for target cache configuration information from the plurality of cache configuration information according to the target log when receiving a target log sent by the first database corresponding to the business system, wherein the target log represents a processing operation of the business system on the data in the first database;
[0044] A determination module, configured to determine target cache data from a cache database according to the target cache configuration information;
[0045] A processing module is used to process the target cache data according to the target log.
[0046] In a third aspect, an embodiment of the present invention provides a computer device, comprising: a processor and a memory, wherein the processor is used to execute a cache data processing program stored in the memory to implement the cache data processing method described in any one of the first aspects above.
[0047] In a fourth aspect, an embodiment of the present invention provides a storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the cache data processing method described in any one of the first aspects above.
[0048] The cache data processing scheme provided by the embodiment of the present invention monitors the cache configuration information of each cache method in the business system from the target processing platform for storage, and the target processing platform is used to receive the cache configuration information sent by the business system; when receiving the target log sent by the first database corresponding to the business system, the target cache configuration information is searched from the multiple cache configuration information according to the target log, and the target log represents the processing operation of the business system on the data in the first database; the target cache data is determined from the cache database according to the target cache configuration information; and the target cache data is processed according to the target log. Therefore, the cache configuration information in the business system can be obtained and stored in advance, and the target log of the database of the business system can be received. The data in the cache database can be processed according to the cache configuration information and the target log, so that when multiple business systems share the cache database table, the data in the cache database can be directly processed without going through the business system, thereby ensuring the consistency of the data in the database of the business system and the data in the cache database. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 A schematic diagram of a flow chart of a cache data processing method provided by an embodiment of the present invention;
[0050] Figure 2 A schematic flow chart of another cache data processing method provided by an embodiment of the present invention;
[0051] Figure 3 A schematic diagram of the structure of a cache data processing device provided by an embodiment of the present invention;
[0052] Figure 4A schematic diagram of the structure of a computer device provided in an embodiment of the present invention;
[0053] Figure 5 A schematic diagram of a process for obtaining cache configuration information provided by an embodiment of the present invention;
[0054] Figure 6 A flowchart of a method for generating a mapping relationship provided by an embodiment of the present invention;
[0055] Figure 7 A schematic flow chart of another cache data processing method provided by an embodiment of the present invention;
[0056] Figure 8 A timing diagram of a cache data processing method provided by an embodiment of the present invention;
[0057] Fig. 9 A schematic flow chart of another cache data processing method provided by an embodiment of the present invention;
[0058] Fig.10 A schematic structural diagram of a cache data processing system provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0060] To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present invention.
[0061] Figure 1 A flow chart of a cache data processing method provided by an embodiment of the present invention is as follows: Figure 1 As shown, the method specifically includes:
[0062] S11. The target processing platform monitors cache configuration information of each cache method in the business system for storage, and the target processing platform is used to receive the cache configuration information sent by the business system.
[0063] The cache data processing method provided by the embodiment of the present invention is applied to a computer device, which may include but is not limited to: a server, a desktop computer, a central processing unit, etc. The execution subject is a cache listener (for example, a binlog cache listener) pre-built in the computer device, specifically by pre-acquiring and storing the cache configuration information in the business system, and receiving the target log of the database of the business system, and processing the data in the cache database according to the cache configuration information and the target log, wherein the cache listener can be a Java application, which is used to monitor the cache configuration information sent by the business system and generate a mapping relationship from the queried database name, the queried table name to the cache configuration information list. At the same time, the binlog log of the MySQL database corresponding to the business system will be monitored and three SQLs will be parsed: INSERT, UPDATE and DELETE SQL, and then the cache will be maintained according to the SQL type. Among them, most cache databases store data in the form of key-value pairs and provide support for multiple data structures. Open source implementations include Redis, Memcached, etc.
[0064] In this embodiment, the business system may include: a business system for processing vehicle data, for example, a TSP business system or a VHR business system. The target processing platform may be a pre-built distributed stream processing platform for obtaining cache configuration information sent by the business system. For example, the target processing platform may include: Apache Kafka, which is a message system based on the publish-subscribe model. Kafka has the characteristics of high scalability, high throughput and persistence. The cache configuration information includes but is not limited to: cache key (also known as cache key, used to find the corresponding data in the cache database), cache expiration time, queried database name, queried table name, etc. The cache configuration information is constructed and sent to the target processing platform when the business system is started. The cache listener will subscribe to the information of the target processing platform. When the cache configuration information is listened to, it may cache the configuration information and store it.
[0065] S12: When receiving a target log sent by the first database corresponding to the business system, searching for target cache configuration information from the plurality of cache configuration information according to the target log, wherein the target log represents the processing operation of the business system on the data in the first database.
[0066] In this embodiment, the first database may be a database used by the business system to process business data. For example, the first database may be MySQL, which is a relational database management system that supports multiple users, multiple threads, and multiple tables. It is widely used in the development of Web applications and has good compatibility on various operating systems. The target log may be a binlog log, for example, a MySQL binlog log: a log file used by the MySQL database server to record changes to the database, which contains detailed information on all change operations performed in the database, such as the contents of SQL such as INSERT (insert operation), UPDATE (update operation), and DELETE (delete operation). The binlog log records these changed SQL statements, enabling users to trace and recover operations in the database.
[0067] When the first database receives a data processing operation from the business system, it will record it in the target log and send the target log to the cache listener. The target log may include but is not limited to: the database name, table name, data before change, data after change, cache key, etc. corresponding to the processing operation of the business system. The cache listener listens to the first database, and when the target log sent by the first database is received, the database name and table name contained in the target log are obtained, and the target cache configuration information is searched from the multiple cache configuration information stored in the cache listener. The database name and table name contained in the target cache configuration information are consistent with those in the target log.
[0068] S13: Determine target cache data from a cache database according to the target cache configuration information.
[0069] In this embodiment, when the determined target cache configuration information is one, the cache key in the target cache information can be directly obtained, each cache key can correspond to a unique cache data in the cache database, and the cache data corresponding to the cache key in the target cache information is determined to be the target cache data. When the determined target cache configuration information is multiple, the cache data corresponding to the cache key in each target cache information can be determined to be the target cache data, and multiple target cache data can be obtained, or the cache key contained in the target log can be obtained, and the cache data corresponding to the cache key contained in the target log in the cache database can be determined to be the target cache data.
[0070] S14: Process the target cache data according to the target log.
[0071] In this embodiment, the types of processing operations contained in the target log may include but are not limited to: deletion operations, update operations, and insertion operations, etc. Processing the target cache data may include: deleting the target cache data, updating the cache data, and inserting data into the cache database according to the insertion operation.
[0072] The cache data processing method provided by the embodiment of the present invention monitors the cache configuration information of each cache method in the business system from the target processing platform for storage, and the target processing platform is used to receive the cache configuration information sent by the business system; when receiving the target log sent by the first database corresponding to the business system, the target cache configuration information is searched from the multiple cache configuration information according to the target log, and the target log represents the processing operation of the business system on the data in the first database; the target cache data is determined from the cache database according to the target cache configuration information; and the target cache data is processed according to the target log. Therefore, the cache configuration information in the business system can be obtained and stored in advance, and the target log of the database of the business system can be received. The data in the cache database can be processed according to the cache configuration information and the target log, so that when multiple business systems share the cache database table, the data in the cache database can be directly processed without going through the business system, thereby ensuring the consistency of the data in the database of the business system and the data in the cache database.
[0073] Figure 2 A flow chart of another cache data processing method provided by an embodiment of the present invention is as follows: Figure 2 As shown, the method specifically includes:
[0074] S21. Monitor cache configuration information of each cache method in the business system from the target processing platform.
[0075] In this embodiment, the target processing platform receives the cache configuration information sent by the business system, and the cache listener listens to the target processing platform to obtain the cache configuration information, wherein the cache configuration information is generated in the following manner: obtaining the unique identifier of each cache method in the business system, the cache expiration time, the queried database name, and the queried table name; generating a cache key using the unique identifier and the project number of the cache method; and using the cache key, database name, and table name as the cache configuration information.
[0076] Specifically, Figure 5The figure shows a flow chart of a method for obtaining cache configuration information provided by an embodiment of the present invention. The cache configuration information is generated in the business system. When the business system is started, all cache methods in the code are scanned, and the @BinlogCache annotation on the cache method is obtained, and the attributes on the annotation are further obtained, including: the unique identifier of each cache method, the database name for querying the cache expiration time, and the table name for querying. Then, a cache key (cache key) is generated based on the pre-set project number and the unique identifier. The project number needs to be unique between each business system, and the unique identifier needs to be unique in the business system code. The cache key, cache expiration time, the database name for querying, and the table name for querying are used to construct a cache configuration information ( Figure 5 After all cache methods are processed, all cache configuration information is sent to the Kafka server (target processing platform) through the Kafka client, so that the cache listener listens to the Kafka server to obtain the cache configuration information.
[0077] Among them, @BinlogCache is a custom Java annotation. When the annotated Java method is executed, it will first query the data in the cache database through the cache key. If the query result is not empty, it will directly return the data in the cache; if the query result is empty, it will execute SQL to query the MySQL database and save the query result in the cache database using the cache key. It can contain 4 attributes: cache expiration time, unique identifier of the cache method, query database name, and query table name. The cache method is a Java language method annotated with @BinlogCache.
[0078] S22. Using the database name and table name in each monitored cache configuration information as a key and the cache configuration information as a value to generate a mapping relationship; and storing the mapping relationship in memory.
[0079] In this embodiment, if Figure 6 FIG. 1 is a flow chart of a method for generating a mapping relationship provided by an embodiment of the present invention. After the cache listener application is started, once the Kafka message sent by the business system is listened to, the list of cache configuration information therein is parsed ( Figure 6 Cache key configuration object list), and then loop processing, extract from the cache configuration information: the queried database name, the queried table name as the key, the cache configuration information as the value, save in a mapping relationship, save the mapping relationship to the memory, where the value of the mapping relationship can be multiple cache configuration information.
[0080] S23. When receiving a target log sent by the first database corresponding to the business system, obtain the target database name and the target table name in the target log; and determine the target cache configuration information from a mapping relationship according to the target database name and the target table name.
[0081] In this embodiment, when the business system processes the data of the first database, the first database generates a target log (for example, a binlog log), and the cache listener listens to the first database. When the target log is received, the target database name and target table name contained in the target log are extracted, and a key consistent with the target database name and the target table name is searched from the mapping relationship, and the cache configuration information corresponding to the found key is used as the target cache configuration information.
[0082] S24, obtaining a first cache key in the target cache configuration information; determining target cache data corresponding to the first cache key from the cache database, wherein each cache key corresponds one-to-one to cache data in the cache database.
[0083] In this embodiment, the target cache configuration information may be one or more, so the first cache key contained in each target cache configuration information is obtained, or the target log may contain information of the cache key currently required. When multiple target cache configuration information are determined, a cache key that is consistent with the cache key contained in the target log is determined from the multiple target cache configuration information as the first cache key.
[0084] Since each cache key corresponds in advance to a segment of cache data in the cache database, the cache data corresponding to the first cache key is determined from the cache database as the target cache data.
[0085] S25. When the operation information contained in the target log is a delete operation or an update operation, obtain the cache identifier of the target cache data; obtain the database identifier of the original data from the target log; when the cache identifier and the database identifier are inconsistent, determine that the operation information is a delete operation; and delete the target cache data in the cache database according to the delete operation.
[0086] In this embodiment, the SQL statement in the target log is obtained to determine whether it is an insert operation. If it is not an insert operation, it is determined that the operation information may be a delete operation or an update operation. At this time, further judgment is required to obtain the cache identifier of the target cache data. The cache identifier is the value cacheId of the id field of the cache data, and the database identifier of the original data before the change contained in the target log is obtained. The database identifier is the value sqlId of the id field in the data before the change. It is determined whether the database identifier and the cache identifier are consistent. If they are inconsistent, it is determined that the operation information is a delete operation, and the target cache data is deleted from the cache database.
[0087] In a possible implementation, when the cache identifier and the database identifier are consistent, the operation information is determined to be an update operation; updated data is obtained from the target log; and the target cache data is updated to the updated data according to the update operation.
[0088] S26. When the operation information contained in the target log is an insert operation, obtain the insert data from the target log; insert the insert data into the cache database; and set a cache expiration time for the insert data.
[0089] In this embodiment, the SQL statement in the target log is obtained, and it is determined whether the operation information is an insert operation. If it is an insert operation, the insert data is obtained from the target log, the insert data is inserted into the cache database, and the insert data is set to correspond to the first cache key. The expiration time of the insert data is set according to the cache expiration time included in the target log or the target cache configuration information.
[0090] As an example, Figure 7 The flowchart of another cache data processing method provided by an embodiment of the present invention is shown. After the binlog cache listener application is started, once the MySQL binlog log (target log) is listened to, the SQL statement and the database name, table name, data before change (original data), and data after change of the operation are parsed, and the value sqlId (database identifier) of the id field in the data before change is obtained. The database name and table name are used to find the corresponding cache key configuration object (target cache configuration information) from the mapping relationship, and the cache key (first cache key), cache expiration time expireTime, etc. in the cache configuration key object are obtained.
[0091] Determine whether the SQL type is INSERT SQL (insert operation). If so, insert the changed cache data into the cache database and save it under this cache key (the first cache key), and use the expireTime attribute to set the cache expiration time.
[0092] Determine whether the SQL type is UPDATE SQL (update operation) or DELETE SQL (delete operation). If so, query the corresponding cache data in the cache database through the cache key (first cache key), and obtain the value cacheId (cache identifier) of the id field of the cache data, and determine whether cacheId is equal to sqlId. If so, further determine whether the SQL type is UPDATE SQL. If so, use the changed data to update the cache data in the cache database; otherwise, if the SQL type is DELETE SQL, delete the cache data in the cache database.
[0093] As another example, Figure 8 The figure shows a timing diagram of a cache data processing method provided by an embodiment of the present invention. Taking the business system of a vehicle as an example, the business system pre-registers cache configuration information to the binlog cache listener service. When the business system changes vehicle data for the database, the binlog cache listener listens to and processes the MySQL binlog log (target log) of the database, queries the cache data related to the target log from the cache system (cache database) through the cache configuration information, and changes the cache data in the cache system according to the operation information in the target log.
[0094] In one possible implementation, when any target cache method is executed, the second cache key in the target cache method is obtained; the cache data corresponding to the second cache key is queried in the cache database; when the query result is empty, the target data corresponding to the target cache method is obtained from the first database; the target data is saved in the cache database, and the target data corresponds to the second cache key; when the query result is not empty, the data corresponding to the query result is used as the return value of the target cache method.
[0095] Specifically, Fig. 9 The figure shows a flow chart of another cache data processing method provided by an embodiment of the present invention. Before the target cache method is executed, the cache method is intercepted, the @BinlogCache annotation of the cache method is obtained, and then the cache expiration time expireTime of the annotation and the method unique identifier methodId are obtained. The cache key is generated using the preset project number and the unique identifier as the second cache key. First, the cache database is queried using the cache key (second cache key). If the query result is empty, the code in the cache method is executed, that is, SQL is used to query the first database, and then the SQL query result is used as the target data, the target data is saved in the cache database, and corresponds to the cache key (second cache key), and then the SQL query result is used as the return value of the method.
[0096] If the query result obtained by querying the cache database through the cache key (the second cache key) is not empty, the query value of the query result is used as the return value of the target cache method.
[0097] The cache data processing method provided by the embodiment of the present invention monitors the cache configuration information of each cache method in the business system from the target processing platform. The database name and table name in each monitored cache configuration information are used as the key, and the cache configuration information is used as the value to generate a mapping relationship; the mapping relationship is stored in the memory. When the target log sent by the first database corresponding to the business system is received, the target database name and target table name in the target log are obtained; the target cache configuration information is determined from the mapping relationship according to the target database name and the target table name. The first cache key in the target cache configuration information is obtained; the target cache data corresponding to the first cache key is determined from the cache database. When the operation information contained in the target log is a delete operation or an update operation, the cache identifier of the target cache data is obtained; the database identifier of the original data is obtained from the target log; when the cache identifier and the database identifier are inconsistent, it is determined that the operation information is a delete operation; and the target cache data in the cache database is deleted according to the delete operation. When the operation information contained in the target log is an insert operation, the insert data is obtained from the target log; the insert data is inserted into the cache database; and the cache expiration time of the insert data is set. Therefore, by pre-acquiring and storing the cache configuration information in the business system and receiving the target log of the database of the business system, the data in the cache database can be processed according to the cache configuration information and the target log, so that when multiple business systems share the cache database table, the data in the cache database can be directly processed without going through the business system, ensuring the consistency of the data in the database of the business system and the data in the cache database. This solves the problem of inconsistent data between the cache and the database caused by multiple business systems sharing the database table or directly changing the table data through the database client.
[0098] like Fig.10 The figure shows a structural diagram of a cache data processing system provided by an embodiment of the present invention, wherein the business system includes: a TSP business system and a VHR business system, the target processing platform is a Kafka system, the cache listener provides a binlog cache listener service, the first database is a database, and the cache database is a database in the cache system.
[0099] Taking the vehicle data in the database as an example, after the TSP business system and the VHR business system are started, they will send all their cache key configurations (cache configuration information) to the Kafka system. After the binlog cache listener subscribes to the messages of the Kafka system, it will save these cache key configurations in memory. After the TSP business system, the VHR business system or the database client changes the vehicle data in the database, the database system will push the binlog log (target log) of the vehicle data to the binlog cache listener. At this time, the binlog cache listener changes the corresponding vehicle cache data through the cache key configuration in memory.
[0100] Figure 3 A schematic diagram of the structure of a cache data processing device provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the device specifically includes:
[0101] A monitoring module 31, used for monitoring cache configuration information of each cache method in the business system from a target processing platform for storage, the target processing platform being used for receiving the cache configuration information sent by the business system;
[0102] A search module 32, configured to search for target cache configuration information from the plurality of cache configuration information according to the target log when receiving a target log sent by the first database corresponding to the business system, wherein the target log represents a processing operation of the business system on the data in the first database;
[0103] A determination module 33, configured to determine target cache data from a cache database according to the target cache configuration information;
[0104] The processing module 34 is used to process the target cache data according to the target log.
[0105] In a possible implementation manner, the monitoring module is specifically configured to use the database name and the table name in each monitored cache configuration information as a key and the cache configuration information as a value to generate a mapping relationship;
[0106] The mapping relationship is stored in the memory.
[0107] In a possible implementation manner, the search module is specifically used to obtain the target database name and the target table name in the target log;
[0108] Determining the target cache configuration information from the mapping relationship according to the target database name and the target table name;
[0109] The determination module is specifically used to obtain the first cache key in the target cache configuration information;
[0110] The target cache data corresponding to the first cache key is determined from the cache database, and each cache key corresponds to the cache data in the cache database one by one.
[0111] In a possible implementation manner, the processing module is specifically configured to obtain a cache identifier of the target cache data when the operation information contained in the target log is a deletion operation or an update operation;
[0112] Obtaining a database identifier of the original data from the target log;
[0113] When the cache identifier and the database identifier are inconsistent, determining that the operation information is a deletion operation;
[0114] Deleting the target cache data in the cache database according to the deletion operation;
[0115] Or, when the cache identifier and the database identifier are consistent, it is determined that the operation information is an update operation;
[0116] Obtaining updated data from the target log;
[0117] The target cache data is updated to the updated data according to the update operation.
[0118] In a possible implementation manner, the processing module is further configured to obtain insertion data from the target log when the operation information contained in the target log is an insert operation;
[0119] Inserting the insert data into the cache database;
[0120] Set the cache expiration time of the inserted data.
[0121] In a possible implementation manner, the processing module is further configured to obtain a second cache key in any target cache method when the target cache method is executed;
[0122] Query the cache data corresponding to the second cache key in the cache database;
[0123] When the query result is empty, obtaining target data corresponding to the target cache method from the first database;
[0124] Saving the target data in the cache database, wherein the target data corresponds to the second cache key;
[0125] When the query result is not empty, the data corresponding to the query result is used as the return value of the target cache method.
[0126] The device provided in this embodiment can be as follows Figure 3 The device shown in FIG. 1 may perform the following steps: Figure 1-2 All steps of the cache data processing method are implemented Figure 1-2 For details, please refer to the technical effect of the cache data processing method shown in Figure 1-2 For the sake of brevity, the relevant description is not repeated here.
[0127] Figure 4 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention, Figure 4The computer device 400 shown includes: at least one processor 401, a memory 402, at least one network interface 404 and other user interfaces 403. The various components in the computer device 400 are coupled together via a bus system 405. It is understood that the bus system 405 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 405 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 405 is not shown in FIG. Figure 4 Various buses are labeled as bus system 405 .
[0128] The user interface 403 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).
[0129] It can be understood that the memory 402 in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 402 described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0130] In some implementations, the memory 402 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 4021 and application programs 4022 .
[0131] The operating system 4021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 4022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 4022.
[0132] In the embodiment of the present invention, by calling the program or instruction stored in the memory 402, specifically, the program or instruction stored in the application 4022, the processor 401 is used to execute the method steps provided by each method embodiment, for example, including:
[0133] Monitoring cache configuration information of each cache method in the business system from a target processing platform for storage, the target processing platform being used to receive the cache configuration information sent by the business system;
[0134] When receiving a target log sent by a first database corresponding to the business system, searching for target cache configuration information from the plurality of cache configuration information according to the target log, the target log representing the processing operation of the business system on the data in the first database;
[0135] Determine target cache data from a cache database according to the target cache configuration information;
[0136] The target cache data is processed according to the target log.
[0137] The method disclosed in the above embodiment of the present invention can be applied to the processor 401, or implemented by the processor 401. The processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 401 or the instruction in the form of software. The above processor 401 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 402, and the processor 401 reads the information in the memory 402 and completes the steps of the above method in combination with its hardware.
[0138] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.
[0139] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0140] The computer device provided in this embodiment may be as follows Figure 4 The computer device shown in , can execute Figure 1-2 All steps of the cache data processing method are implemented Figure 1-2 For details, please refer to the technical effect of the cache data processing method shown in Figure 1-2 For the sake of brevity, the relevant description is not repeated here.
[0141] The embodiment of the present invention also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.
[0142] When one or more programs in the storage medium can be executed by one or more processors, the cache data processing method executed on the device side can be implemented.
[0143] The processor is used to execute the cache data processing program stored in the memory to implement the following steps of the cache data processing method executed on the device side:
[0144] Monitoring cache configuration information of each cache method in the business system from a target processing platform for storage, the target processing platform being used to receive the cache configuration information sent by the business system;
[0145] When receiving a target log sent by a first database corresponding to the business system, searching for target cache configuration information from the plurality of cache configuration information according to the target log, the target log representing the processing operation of the business system on the data in the first database;
[0146] Determine target cache data from a cache database according to the target cache configuration information;
[0147] The target cache data is processed according to the target log.
[0148] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0149] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0150] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A cache data processing method, characterized in that: include: Monitoring cache configuration information of each cache method in the business system from a target processing platform for storage, the target processing platform being used to receive the cache configuration information sent by the business system; When receiving a target log sent by a first database corresponding to the business system, searching for target cache configuration information from the plurality of cache configuration information according to the target log, the target log representing the processing operation of the business system on the data in the first database; Determine target cache data from a cache database according to the target cache configuration information; The target cache data is processed according to the target log.
2. The method according to claim 1, characterized in that The cache configuration information is generated in the following way: Obtain the unique identifier, cache expiration time, queried database name, and queried table name of each cache method in the business system; Generate a cache key using the unique identifier and the project number of the cache method; The cache key, the cache expiration time, the database name and the table name are used as the cache configuration information.
3. The method according to claim 2, characterized in that The slave target processing platform monitors cache configuration information of each cache method in the business system for storage, including: Generate a mapping relationship by using the database name and the table name in each monitored cache configuration information as a key and the cache configuration information as a value; The mapping relationship is stored in the memory.
4. The method according to claim 3, characterized in that: The step of searching the target cache configuration information from the plurality of cache configuration information according to the target log includes: Obtain the target database name and target table name in the target log; Determining the target cache configuration information from the mapping relationship according to the target database name and the target table name; The determining the target cache data from the cache database according to the target cache configuration information includes: Obtaining a first cache key in the target cache configuration information; The target cache data corresponding to the first cache key is determined from the cache database, and each cache key corresponds to the cache data in the cache database one by one.
5. The method according to claim 4, characterized in that The processing of the target cache data according to the target log includes: When the operation information contained in the target log is a deletion operation or an update operation, obtaining a cache identifier of the target cache data; Obtaining a database identifier of the original data from the target log; When the cache identifier and the database identifier are inconsistent, determining that the operation information is a deletion operation; Deleting the target cache data in the cache database according to the deletion operation; Or, when the cache identifier and the database identifier are consistent, it is determined that the operation information is an update operation; Obtaining updated data from the target log; The target cache data is updated to the updated data according to the update operation.
6. The method according to claim 5, characterized in that The method further comprises: When the operation information contained in the target log is an insert operation, obtaining insert data from the target log; Inserting the insert data into the cache database; Set the cache expiration time of the inserted data.
7. The method according to claim 1, characterized in that The method further comprises: When any target cache method is executed, obtaining a second cache key in the target cache method; Query the cache data corresponding to the second cache key in the cache database; When the query result is empty, obtaining target data corresponding to the target cache method from the first database; Saving the target data in the cache database, wherein the target data corresponds to the second cache key; When the query result is not empty, the data corresponding to the query result is used as the return value of the target cache method.
8. A cache data processing device, characterized in that: include: A monitoring module, used to monitor cache configuration information of each cache method in the business system from a target processing platform for storage, the target processing platform being used to receive the cache configuration information sent by the business system; a search module, configured to search for target cache configuration information from the plurality of cache configuration information according to the target log when receiving a target log sent by the first database corresponding to the business system, wherein the target log represents a processing operation of the business system on the data in the first database; A determination module, configured to determine target cache data from a cache database according to the target cache configuration information; A processing module is used to process the target cache data according to the target log.
9. A computer device, characterized in that: include: A processor and a memory, wherein the processor is used to execute a cache data processing program stored in the memory to implement the cache data processing method according to any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the cache data processing method according to any one of claims 1 to 7.
