A method and system for managing data caching and batch synchronization based on multi-level caching

Abstract

This invention belongs to the field of data caching and batch synchronization technology, and discloses a method and system for data caching and batch synchronization of a management system based on multi-level caching. The method includes: caching pre-acquired data using a caching strategy based on a pre-built caching architecture; extracting operation data from the cached data; and performing local operation processing and operation type judgment by combining a transaction engine and an operation classifier; acquiring the storage status data of the caching architecture in real time; optimizing the synchronization strategy using a decision tree algorithm; and performing batch synchronization processing by combining a chained version number mechanism; acquiring the running status data of the caching architecture; constructing a caching performance index system using the running status data; and performing data storage alarm processing by combining an autoregressive integral moving average model and association rules. This invention significantly optimizes the response speed of high-frequency operations and effectively reduces the load on backend servers, achieving efficient utilization of hardware resources.

Description

Technical Field

[0001] This invention relates to the field of data caching and batch synchronization technology, and in particular to a method and system for managing data caching and batch synchronization based on multi-level caching. Background Technology

[0002] Currently known management system data caching and batch synchronization technologies can be categorized into the following four types:

[0003] 1) Pure front-end caching solutions, primarily relying on local storage technologies such as LocalStorage and IndexedDB, are commonly used for static page caching. This solution only supports data read operations and cannot handle data addition, deletion, or modification needs. For data synchronization, a full fetch strategy is typically used when the page refreshes, common in news and information applications. However, this method has poor real-time performance, with data update intervals measured in minutes, and lacks transaction guarantees. Due to the lack of version control mechanisms, data writing follows the principle of later writes overwriting earlier writes, resulting in a data conflict probability exceeding 10% when multiple devices operate simultaneously. 2) Back-end centralized caching solutions build a single-level cache architecture from the database to Redis, widely used in e-commerce flash sales, order centers, and other systems. However, the front-end still needs to frequently initiate requests, with network overhead accounting for up to 80%. Data synchronization is achieved through timed invalidation combined with proactive updates via message queues, but in high-concurrency scenarios, the cache invalidation strategy is prone to "cache breakdown" issues. Furthermore, this solution lacks local data autonomy; if the network is interrupted, the system will be completely unusable in edge computing scenarios. 3) Distributed cache synchronization solutions, exemplified by Redis Cluster with Sentinel mode, can support terabyte-level data storage. Asynchronous replication mechanisms enable throughput of up to 100,000 QPS, suitable for scenarios with high eventual consistency requirements, such as financial transfers. However, this solution has significant limitations: cross-shard transaction processing logic is complex, and implementing ACID properties is costly; data replication latency between master and slave nodes can reach up to 50ms, potentially leading to the reading of stale data. In transaction scenarios with strong consistency requirements, additional technical means are needed to ensure data accuracy. 4) The current optimal hybrid architecture solution integrates client-side and server-side caching, typically a three-tier architecture combining PWA with Redis and a database, performing well in mobile office apps. However, the caching strategy configuration for this architecture is complex, requiring manual handling of version conflicts. Data synchronization comprehensively employs polling, long polling, and WebSocket technologies, commonly used in instant messaging systems. However, in environments with network fluctuations (such as 2G networks), the synchronization failure rate can reach as high as 30%. Furthermore, its offline capabilities only support static resource caching, failing to meet the offline usage requirements of dynamic data operations (such as form submissions).

[0004] From the above four solutions, we can summarize the following three problems in traditional solutions, ranging from performance bottlenecks to architectural innovation: 1) The computing power contradiction in digital transformation: As the enterprise digitalization process accelerates, the contradiction between the computing power demand of business systems and the performance bottlenecks of traditional architectures becomes increasingly prominent. With the deep evolution of enterprise ERP systems and industrial MES, high-frequency business scenarios such as real-time order processing in e-commerce and monitoring of industrial production lines have raised the system response performance requirements to sub-second or even millisecond levels. In traditional architectures, front-end requests need to reach the database directly through network links. When the server's query per second (QPS) approaches the threshold, the response latency will increase exponentially. Taking financial transaction scenarios as an example, a delay of 100 milliseconds can lead to economic losses of tens of millions of yuan. In addition, edge computing scenarios spurred by 5G and IoT technologies (such as intelligent inspection terminals and mobile medical devices) require systems to support temporary storage and offline processing of data lost due to network outages in weak network environments (such as underground mines and remote areas), and to automatically complete synchronization after the network is restored, further exacerbating the computing power pressure. 2) The technical bottleneck of traditional architectures: Traditional technical architectures have exposed significant limitations in dealing with modern business needs. While single-level caching solutions (such as Redis) can alleviate some pressure, the front end still needs to frequently initiate network requests, causing a large number of invalid requests to penetrate the caching layer and directly access the database, easily exhausting the relational database connection pool resources. When the Redis cluster fails, all requests will instantly flood the database, triggering a system-wide service avalanche. Furthermore, in a front-end / back-end separated architecture, the data transmission path "front-end → API gateway → service layer → database" has significant flaws. The front end lacks local data autonomy, and the system becomes paralyzed in offline environments. The lack of an effective version coordination mechanism between multi-level caches results in a data conflict rate as high as 3%-5%, severely impacting data consistency and business continuity. 3) The need for technological evolution: traditional scaling solutions handle peak traffic by horizontally expanding server nodes, but resource waste is prominent. Data shows that the annual growth rate of single-machine operation and maintenance costs reaches 20%, while the average CPU utilization only remains at 30%-40%. In cloud service architectures, database call costs account for more than 60% of the total system cost. This imbalance between hardware cost and efficiency urgently requires more optimized technical solutions to achieve a balance between performance improvement and cost control.

[0005] In summary, the pain points of traditional management systems include: performance bottlenecks, resource waste, offline unavailability, poor scalability, inability to support dynamic operations, lack of collaborative architecture with the front end, and high data conflict rates. Performance bottlenecks include high server load due to real-time requests, with response latency reaching 300-500ms; resource waste due to repeated database requests for high-frequency operations (such as list queries), consuming significant computing resources; offline unavailability due to complete system paralysis during network interruptions, impacting business continuity; poor scalability due to the high cost of horizontal server scaling as data volume increases; inability to support dynamic operations due to front-end caching only storing static data, unable to support dynamic operations (such as adding, deleting, and modifying data); lack of collaborative architecture with the front end due to back-end caching (Redis only serving as a database acceleration layer, not forming a collaborative architecture with the front end); and high data conflict rates due to the lack of complete transaction consistency guarantees in offline solutions, resulting in a high data conflict rate (3%-5%).

[0006] Therefore, how to provide a system and method for managing data caching and batch synchronization based on multi-level caching is an urgent problem to be solved. Summary of the Invention

[0007] This invention provides a method and system for managing system data caching and batch synchronization based on multi-level caching, in order to solve the above-mentioned technical problems in the prior art.

[0008] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or to describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0009] According to a first aspect of the present invention, a method for managing system data caching and batch synchronization based on multi-level caching is provided.

[0010] In one embodiment, a method for managing system data caching and batch synchronization based on multi-level caching includes:

[0011] Based on a pre-built caching architecture, caching strategies are used to cache pre-acquired data. Operation data is then extracted from the cached data, and local operation processing and operation type determination are performed based on the operation data in conjunction with the transaction engine and operation classifier.

[0012] The system acquires real-time storage status data of the cache architecture, optimizes the preset synchronization strategy based on the storage status data and operation type using a decision tree algorithm, and performs batch synchronization processing on the data after local operation processing based on the optimized synchronization strategy and a chain version number mechanism.

[0013] Obtain the runtime status data of the cache architecture, and use the runtime status data to construct a cache performance index system. Based on the cache performance index system, combine the autoregressive integral moving average model and association rules to detect storage anomalies, and handle data storage alarms based on the storage anomaly detection results.

[0014] According to a second aspect of the present invention, a system for managing data caching and batch synchronization based on multi-level caching is provided.

[0015] In one embodiment, a system for managing data caching and batch synchronization based on multi-level caching includes: a cache processing module, a synchronization processing module, and a storage alarm module;

[0016] The cache processing module is used to cache the pre-acquired data based on a pre-built cache architecture and caching strategy, and extract operation data based on the cached data. The operation type of the operation data is determined by the transaction engine and operation classifier.

[0017] The synchronization processing module is used to obtain the storage status data of the cache architecture in real time, optimize the preset synchronization strategy based on the storage status data and operation type using a decision tree algorithm, and perform batch synchronization processing on the cached data based on the optimized synchronization strategy and a chain version number mechanism.

[0018] The storage alarm module is used to obtain the running status data of the cache architecture, and use the running status data to build a cache performance index system. Based on the cache performance index system, it combines an autoregressive integral moving average model and association rules to detect storage anomalies, and performs data storage alarm processing based on the storage anomaly detection results.

[0019] According to a third aspect of the present invention, a computer device is provided.

[0020] In some embodiments, the computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the above-described method for managing system data caching and batch synchronization based on multi-level caching.

[0021] According to a fourth aspect of the present invention, a computer-readable storage medium is provided.

[0022] In one embodiment, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the above-described method for managing system data caching and batch synchronization based on multi-level caching.

[0023] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0024] 1. This invention significantly optimizes the response speed of high-frequency operations through front-end local caching and transaction logs, achieving a response level similar to local applications; it greatly improves the efficiency of complex transaction processing through an efficient transaction processing mechanism, effectively reducing the load on back-end servers; at the same time, it optimizes the key-value storage database caching strategy, significantly improving the cache hit rate, reducing server CPU resource consumption, and achieving efficient utilization of hardware resources; furthermore, through deep optimization of the caching architecture and resource integration, it significantly reduces the number of back-end servers deployed, significantly reduces network bandwidth consumption and data center energy consumption, optimizes the PUE value, and effectively controls operating costs.

[0025] 2. This invention further ensures offline operation processing through transaction logs, supports long-term offline operations, and ensures data consistency. After network recovery, data synchronization is automatically performed without manual intervention, ensuring business continuity. It enhances adaptability in offline and weak network environments, significantly improves data transmission reliability and multi-terminal data synchronization efficiency, greatly reduces business interruption time, and provides users with a smooth and stable user experience. At the same time, through the efficient scalability of key-value storage databases, it significantly reduces horizontal scaling costs compared to traditional database scaling. It also supports flexible multi-cloud deployment architectures and further optimizes network resource usage through front-end local caching, reducing cloud service costs. Furthermore, it effectively solves distributed data conflict problems by combining chained version number mechanisms and transaction log mechanisms.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0028] Figure 1 This is a flowchart illustrating a multi-level caching-based system data caching and batch synchronization method according to an exemplary embodiment;

[0029] Figure 2 This is a schematic diagram of the system structure according to an exemplary embodiment;

[0030] Figure 3 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment;

[0031] Figure 4 This is a diagram illustrating the end-to-end technical architecture of a multi-level caching-based system data caching and batch synchronization method in practical applications, according to an exemplary embodiment.

[0032] Figure 5This is a cache layering design diagram of a cache architecture in a multi-level cache-based system data caching and batch synchronization method according to an exemplary embodiment;

[0033] Figure 6 This is a data synchronization flow diagram of the cache architecture in a multi-level cache-based system data caching and batch synchronization method according to an exemplary embodiment;

[0034] Figure 7 This is a deployment diagram of the data acquisition components in a practical application of a multi-level caching-based system data caching and batch synchronization method according to an exemplary embodiment.

[0035] Figure 8 This is a dynamic synchronization decision tree logic diagram of a multi-level caching-based system data caching and batch synchronization method in practical application, as illustrated in an exemplary embodiment.

[0036] Figure 9 This is a data caching and batch synchronization architecture diagram of a multi-level caching-based management system data caching and batch synchronization method, illustrated according to an exemplary embodiment.

[0037] Figure label:

[0038] 201. Cache processing module; 202. Synchronization processing module; 203. Storage alarm module. Detailed Implementation

[0039] Figure 1 An embodiment of the multi-level caching-based system data caching and batch synchronization method of the present invention is shown.

[0040] In this optional embodiment, the method for managing system data caching and batch synchronization based on multi-level caching includes:

[0041] Step S101: Based on the pre-built cache architecture, cache strategy is used to cache the pre-acquired data. Operation data is extracted from the cached data, and local operation processing and operation type determination are performed based on the operation data in combination with the transaction engine and operation classifier.

[0042] Step S102: Obtain the storage status data of the cache architecture in real time, optimize the synchronization strategy of the preset synchronization strategy using the decision tree algorithm according to the storage status data and operation type, and perform batch synchronization processing on the data after local operation processing based on the optimized synchronization strategy and the chain version number mechanism.

[0043] Step S103: Obtain the running status data of the cache architecture, and use the running status data to construct a cache performance index system. Based on the cache performance index system, combine the autoregressive integral moving average model and association rules to perform storage anomaly detection, and perform data storage alarm processing based on the storage anomaly detection results.

[0044] In this optional embodiment, the process of caching pre-acquired data based on a pre-built caching architecture and caching strategy, extracting operation data from the cached data, and performing local operation processing and operation type determination based on the operation data in conjunction with a transaction engine and operation classifier includes: caching the pre-acquired data sequentially to the database of the caching architecture, the key-value storage database, and the front-end cache based on the pre-built caching architecture and caching strategy; extracting operation data from the front-end cached data, and performing local operation processing and operation type determination based on the operation data in conjunction with a transaction engine and operation classifier to obtain local transaction logs and operation types.

[0045] In this optional embodiment, based on a pre-built caching architecture and caching strategy, the pre-acquired data is sequentially cached in the database of the caching architecture, the key-value storage database, and the front-end cache. This includes: storing the pre-acquired data in the database of the caching architecture according to the pre-built caching architecture, and replicating the data in the database to the standby database through master-slave replication; synchronously storing the stored data in the standby database to the key-value storage database of the caching architecture based on change data capture, and using push real-time changes to perform front-end real-time caching of the data shards stored in the key-value storage database in the caching architecture.

[0046] In this optional embodiment, the process of extracting operation data based on the front-end cached data, and performing local operation processing and operation type determination based on the operation data in conjunction with the transaction engine and operation classifier to obtain local transaction logs and operation types includes: extracting operation data based on the front-end cached data, performing local operation processing based on the operation data, recording the local operation processing through the transaction engine to obtain local transaction logs; and determining the operation type of the operation data using the operation classifier based on the local transaction logs and pre-set operation classification rules.

[0047] In this optional embodiment, the real-time acquisition of storage status data of the cache architecture, the optimization of the preset synchronization strategy using a decision tree algorithm based on the storage status data and operation type, and the batch synchronization processing of locally processed data based on the optimized synchronization strategy combined with a chain version number mechanism include: real-time acquisition of storage status data of the cache architecture, and calculation of the synchronization optimization period using a linear weighting method based on the storage status data and operation type; extraction of operation feature data based on the storage status data and operation type, and optimal selection of the preset synchronization strategy using a decision tree algorithm combined with the operation feature data to obtain the optimal synchronization strategy; integration of the synchronization optimization period and the optimal synchronization strategy to obtain the optimized synchronization strategy, and batch synchronization processing of locally processed data based on the optimized synchronization strategy combined with a chain version number mechanism.

[0048] In this optional embodiment, the operation feature data includes: operation type feature data, data activity feature data, network quality feature data, data sensitivity feature data, operation data volume feature data, and synchronization time consumption feature data.

[0049] In this optional embodiment, batch synchronization processing of data processed locally based on the optimized synchronization strategy combined with the chained version number mechanism includes: batch synchronizing and storing the data processed locally to the key-value storage database based on the optimized synchronization strategy combined with the chained version number mechanism; and persistently storing the batch synchronized data in the key-value storage database to the database using the optimized synchronization strategy.

[0050] In this optional embodiment, obtaining the runtime status data of the cache architecture and constructing a cache performance indicator system using the runtime status data, performing storage anomaly detection based on the cache performance indicator system combined with an autoregressive integral moving average model and association rules, and performing data storage alarm processing based on the storage anomaly detection results include: obtaining the runtime status data of the cache architecture and constructing a cache performance indicator system using the runtime status data; obtaining the normal fluctuation range of the cache performance indicators based on the cache performance indicator system combined with an autoregressive integral moving average model, obtaining the alarm threshold of the cache performance indicators based on the normal fluctuation range of the cache performance indicators; performing storage anomaly detection on the cache architecture using association rules combined with the alarm threshold of the cache performance indicators, and performing data storage alarm processing based on the storage anomaly detection results.

[0051] In this optional embodiment, the cache performance indicator system includes: cache performance indicators, transaction processing indicators, synchronization operation indicators, and resource utilization indicators; cache performance indicators include: key-value storage database hit rate, memory fragmentation rate, number of connections, front-end read / write time, and cache hit rate; transaction processing indicators include: transaction success rate, average time, rollback rate, and offline transaction backlog; synchronization operation indicators include: synchronization cycle deviation, data transmission volume, number of conflict resolution attempts, and push real-time changes to connection status; resource utilization indicators include: server or memory utilization, disk utilization, network bandwidth, and packet loss rate.

[0052] Figure 2 An embodiment of the multi-level caching-based management system data caching and batch synchronization system of the present invention is shown.

[0053] In this optional embodiment, the system data caching and batch synchronization system based on multi-level caching includes: a cache processing module 201, a synchronization processing module 202, and a storage alarm module 203.

[0054] The cache processing module 201 is used to cache the pre-acquired data based on a pre-built cache architecture and a cache strategy, and extract operation data based on the cached data. The operation type of the operation data is determined by the transaction engine and the operation classifier.

[0055] The synchronization processing module 202 is used to obtain the storage status data of the cache architecture in real time, optimize the preset synchronization strategy according to the storage status data and operation type using the decision tree algorithm, and perform batch synchronization processing on the cached data based on the optimized synchronization strategy and the chain version number mechanism.

[0056] The storage alarm module 203 is used to obtain the running status data of the cache architecture, and use the running status data to construct a cache performance index system. Based on the cache performance index system, it combines an autoregressive integral moving average model and association rules to detect storage anomalies, and performs data storage alarm processing based on the storage anomaly detection results.

[0057] It should be noted that the following explanations are provided for the keywords, abbreviations, and key terms used in this invention: ERP system stands for Enterprise ERP Management System. Industrial MES refers to Manufacturing Execution System (MES). QPS stands for Queries-per-second, which is the amount of request data processed per unit of computing resources per unit of time (to generate inference results). Redis (Remote Dictionary Server) is an open-source, network-enabled, in-memory or persistent log-structured key-value database written in ANSI C. RedisCluster is a distributed solution for Redis, designed to address the limitations of single-machine Redis capacity and single point of failure. By distributing data across multiple nodes, it achieves distributed storage, retrieval, and failover, thereby building a highly available and scalable Redis cluster system. MySQL is a relational database management system. LocalStorage is part of the Web Storage API, allowing developers to store key-value pairs in users' browsers. IndexedDB is a method for storing data on the browser side. WebSocket (i.e., push real-time changes) is a protocol for full-duplex communication over a single TCP connection. JWT (JSON Web Token) is a JSON-formatted token used for authentication and information exchange in distributed web applications. JWT protects documents from unauthorized access. AES-256, or Advanced Encryption Standard, is the longest and strongest encryption level offered. TLS 1.3, or Transport Layer Security (TLS), and its predecessor, Secure Sockets Layer (SSL), are security protocols designed to provide security and data integrity for internet communications. TLS 1.3 is the latest version of the TLS encryption protocol.

[0058] It should be noted that the technological breakthrough of this invention is a multi-level caching collaborative architecture (i.e., caching architecture). The core differences between this invention and pure front-end caching solutions, back-end centralized caching solutions, distributed caching synchronization solutions, and the current optimal hybrid architecture solutions are shown in Table 1.

[0059] Table 1. Key differences between this invention and existing solutions.

[0060] ;

[0061] The essence of this invention is to address the triple challenges of "explosive data growth + increased real-time requirements + widespread adoption of edge computing." By reconstructing the caching hierarchy, innovating the synchronization mechanism, and strengthening offline capabilities, it fills the technical gaps in existing solutions regarding high performance, strong consistency, and offline autonomy, representing the evolution of management system architecture from "server-centric" to "edge-cloud collaboration." The innovative technical principles of this invention include: a three-level caching collaborative architecture, a chained version number mechanism, a dynamic synchronization decision tree, and atomicity guarantees for transaction logs. The three-level caching collaborative architecture breaks the traditional unidirectional request model of "front-end → server," overcoming the performance bottleneck of traditional front-end / back-end separation architectures. It establishes a new model of "database → Redis → front-end local cache → user operation," forming a closed-loop data flow and achieving "front-end priority processing + asynchronous batch synchronization." This allows 90% of high-frequency operations to be completed locally, resolving the contradiction between real-time performance and offline capabilities in traditional solutions. The chained version number mechanism, where each data object maintains a version chain, resolves multi-level cache conflicts by accurately recording the operation order and type, enabling cross-level cache operation order tracing. The conflict resolution efficiency is 40% higher than traditional timestamp schemes. Dynamic synchronization decision trees, based on multi-dimensional adaptive adjustment of synchronization strategies according to operation characteristics such as operation type, data activity, and network status, calculate the optimal synchronization cycle in real time, balancing real-time performance and resource consumption, thus improving bandwidth utilization by 3 times in industrial scenarios. Transaction log atomicity guarantee means that offline operations are encapsulated as transactions, and automatically retried after network recovery.

[0062] It should be further explained that the end-to-end technical architecture from the terminal to the backend, such as Figure 4As shown in the diagram, this architecture illustrates the complete process from the user terminal to the data layer, as well as the integration with the monitoring system. At the application layer, the user terminal communicates with the load balancer via HTTPS, which distributes requests to the front-end application server. Simultaneously, the front-end application server maintains a real-time connection with the load balancer via WebSocket. The service layer receives REST API requests from the front-end application server. These requests are first processed by the back-end API gateway, then routed to the microservice cluster via service discovery. The microservice cluster processes the business logic and returns the results to the change capture service. The change capture service then writes the data to the master database of the database cluster and simultaneously to the Redis master node. The microservice cluster writes to the Redis cluster via the Redis protocol and synchronizes data changes using CDC (ChangeData Capture) technology. At the data layer, the Redis cluster uses a master-slave replication model, resulting in Redis slave node 1 and Redis slave node 2. The Redis master node handles write operations, while multiple Redis slave nodes handle read operations to improve read performance and data redundancy. The database cluster also uses a master-slave replication model, resulting in slave database 1 and slave database 2, ensuring data consistency and high availability.The entire system also integrates a monitoring system to monitor performance, detect errors, and collect metrics at each level to ensure stable operation. The diagram shows the deployment architecture, employing a multi-layered decoupled design. From user access to data storage, it is divided into five layers, with each component working collaboratively through standardized protocols. The user access layer is the entry point for user requests. User terminals communicate with the backend via HTTPS. The load balancer uses Nginx / Apache to distribute traffic and offload SSL certificates, supporting 100,000 requests per second. Together, they distribute user requests to different servers, achieving data entry and distribution. The application service layer carries the business logic. The front-end application server deploys Vue / React front-end applications, maintaining a long connection with the Redis cluster via WebSocket to receive real-time data change notifications. The backend API gateway is based on Spring Cloud. Gateway / APISIX implements request routing, permission verification, and traffic control, providing unified management of microservice access. The microservice cluster uses Kubernetes to deploy Java / Python microservices, each responsible for a specific function, automatically registering routes to achieve functional business decomposition. The data caching layer provides high-performance read / write acceleration. A Redis master-slave cluster uses a 3-master, 6-slave architecture, acting as a temporary storage container for frequently accessed data. The master node handles write operations, and the slave nodes handle read requests, with automatic failover in sentinel mode. The core data persistence layer provides persistent storage. The database cluster uses a MySQL cluster (master-slave replication) + distributed database to store all data. The master database handles write operations, and the slave databases handle queries. The monitoring and operations layer provides end-to-end observability, acting as a monitoring center to track the health status of each link, proactively detect faults, and provide timely operational responses. Through layered decoupling, protocol standardization, and component redundancy, a high-performance, highly available, and highly secure three-tiered caching architecture is achieved. This architecture is like building a "highway + intelligent logistics network," offering fast user access, easy business expansion, and stable fault recovery, making it particularly suitable for supporting large-scale online businesses. The three core designs include: caching + database, microservice decomposition, and end-to-end monitoring. The system employs a dual-layer "caching + database" storage approach to separate high-frequency and low-frequency data. High-frequency data is stored in Redis (temporary storage), while low-frequency data is stored in the database (main repository). "Microservice decoupling" decouples business logic, allowing each service to be developed, deployed, and scaled independently. "End-to-end monitoring" enables early fault detection, monitoring every step from user requests to the database. This combination of caching acceleration, microservice decoupling, and end-to-end monitoring supports high-concurrency business, making the management system faster, more stable, and more scalable. The three-tiered caching architecture—comprising a database, a Redis cache cluster, and a front-end local cache—breaks the traditional one-way request model between front-end and back-end. The database → Redis → front-end local cache form a closed data flow loop, with each caching layer fulfilling a different responsibility. The database layer serves as the core for persistent data storage.The Redis caching cluster layer handles high-speed data reading and temporary storage. The front-end local cache handles localized user operations, reducing reliance on the backend. These three layers interact and synchronize data through specific protocols and strategies, achieving "front-end priority processing + asynchronous batch synchronization," completing 90% of high-frequency operations (such as list queries and form editing) locally, significantly reducing the load on the backend server. The caching layer design is as follows. Figure 5 As shown, the database layer replicates from the master to the slave database. The slave database stores changes captured by CDC in the Redis cache layer. The Redis cache layer performs sharding, resulting in data shard 1, data shard 2, and data shard 3. The data shards are pushed to the front end for real-time caching via WebSocket. The front end persists local transaction logs through IndexedD. The data caching and batch synchronization process is mainly divided into data storage, cache synchronization, transaction processing, conflict resolution, synchronization strategy engine, and monitoring. Each process has a clear division of labor and works closely together to ensure efficient operation and data consistency of data caching and batch synchronization. The data storage process manages the database, Redis cache, and front-end local cache, providing basic operation interfaces for data storage, reading, updating, and deletion, while maintaining data metadata such as data version and storage location. The cache synchronization process realizes data synchronization between various levels of cache, including initial synchronization from the database to the Redis cache, initial loading from the Redis cache to the front-end local cache, and real-time and periodic synchronization between the front-end local cache and the Redis cache. The data synchronization flow is as follows: Figure 6 As shown, the database CDC captures changes and synchronizes them to the Redis cache. The Redis cache initializes and loads the data into the front-end local cache. Local operations (insert, delete, modify) are performed in the front-end local cache, and the operations (including chained version numbers) are recorded. A local transaction log is generated, and the local transaction log is periodically batch-synchronized into the Redis cache. The Redis cache is persisted to the database on a regular basis. Changes to other nodes in the database are synchronized to the Redis cache. The Redis cache pushes real-time changes (WebSocket) to the front-end local cache.

[0063] The transaction processing process encapsulates user add, delete, and modify operations into transactions, ensuring atomicity, consistency, isolation, and durability (ACID properties). In offline scenarios, transaction logs are recorded, and transactions are automatically committed or rolled back after network recovery. The conflict resolution process uses a chained version number mechanism to detect and resolve data conflicts between different levels of cache, ensuring data consistency. The synchronization strategy engine dynamically calculates the optimal synchronization cycle and method based on factors such as data activity, network quality, and sensitivity level, achieving efficient data synchronization. The monitoring process monitors the system's operational status in real time, collecting and analyzing data such as performance metrics, transaction processing status, and synchronization operation records from each level of cache to provide a basis for system optimization and troubleshooting. Through a complete "collection-processing-storage-analysis" chain, early fault prediction, performance optimization, and cost control are achieved, constructing a closed-loop management system from metric collection to decision support. Figure 7 As shown in the diagram, this is the deployment diagram of the data collection components. Client-side event tracking collects JS performance data through a front-end collector and sends the data to the data aggregator via the HTTP protocol. Redis collects information via the INFO command, which is processed by the Redis exporter and transmitted to the data aggregator via the gRPC protocol. The application server interacts with the Java agent through the JMX interface, and the Java agent then sends the data to the data aggregator via the TCP protocol. The slow query logs of the database are also sent to the data aggregator after being parsed by the SQL monitor. The entire process ensures that data from the client, Redis, application server, and database can be effectively collected and aggregated for further analysis and monitoring. Data collection is performed from each module of the deployment architecture, and the data collection metric classification (i.e., the cache performance metric system) is shown in Table 2 below.

[0064] Table 2. Classification of data collection indicators

[0065] ;

[0066] After cleaning, aggregating, and hierarchically storing the indicator data, a visual dashboard is generated through analysis. If an indicator exceeds an alarm threshold, the operation and maintenance unit is notified to enhance fault tolerance and security. The chained version number mechanism in this invention maintains a version chain for each data object. Each node in the version chain represents a data operation, using the format "v{version number}-{operation type identifier}", such as "v1-c" for the first creation operation, "v3-u" for the third update operation, and "v7-d" for the seventh deletion operation. This method accurately records the operation history and sequence of the data. During data synchronization and conflict detection, the version chain is used for operation sequence tracing and conflict resolution. Compared to traditional timestamp schemes, this mechanism improves conflict resolution efficiency by 40%, effectively ensuring data consistency across multi-level caches. The dynamic synchronization decision tree, i.e., the synchronization strategy engine module, integrates operations research and machine learning techniques to calculate the optimal synchronization cycle in real time using the following formula: ;in, f The data change frequency is calculated by counting the number of times data is added, deleted, or modified within a unit of time. q The network quality score ranges from 0 to 1 and can be comprehensively evaluated using indicators such as network latency and packet loss rate. s The data sensitivity coefficient is set according to the importance and confidentiality of the data; for example, 0.1 for ordinary data and 0.9 for sensitive data. α , β , γ These are weighting coefficients, which can be adjusted according to the actual business scenario. Simultaneously, based on the decision tree algorithm and multi-dimensional features (i.e., operation feature data) such as operation type, data activity, and network status (as shown in Table 3 below), the optimal synchronization method (real-time synchronization, periodic synchronization, or offline temporary storage) is automatically selected.

[0067] Table 3 Operational characteristic data and their quantification

[0068] ;

[0069] This can be understood as a "smart data synchronization manager," which determines step-by-step whether to "process urgently," "process on a schedule," or "store offline and process when the network is available." The value of this process is to make data synchronization "smart and flexible": that is, urgent operations are handled immediately, non-urgent operations are processed on a scheduled basis, data is stored when there is no network, and conflicts can be automatically resolved, ultimately ensuring that the data is both accurate and fast. Figure 8As shown, this diagram is a dynamic synchronization decision tree logic diagram. The diagram determines the synchronization process based on the real-time requirements of the operation request. If the real-time requirement is high, real-time synchronization is initiated. If not, the data activity level is assessed. If the data activity is high-frequency, a short-cycle synchronization is set to 1 minute; if the data activity is medium-frequency, a medium-cycle synchronization is set to 5 minutes; and if the data activity is low-frequency, a long-cycle synchronization is set to 30 minutes. A pre-synchronization check is then performed to check network availability. If available, batch synchronization to Redis is initiated. If not, local transaction logs are generated, and batch synchronization to Redis is resumed over the network. Finally, Redis processes the data to determine if there are conflicts. If conflicts are found, they are resolved. After conflict resolution, a chain-based version number merging process is performed until successful synchronization. Otherwise, synchronization is successful. The transaction log ensures atomicity by encapsulating user operations on the front end into transactions and recording them in the transaction log. The transaction log contains information such as transaction ID, operation type (create, update, delete), operation object (table name, record ID), operation data, and version number. In offline scenarios, transactions are temporarily stored in the front-end local cache, ensuring atomicity even during network interruptions. Once the network is restored, the transaction processing automatically commits the transactions based on the transaction log. During commit, the transaction operations are first synchronized to the Redis cache cluster, and then asynchronously synchronized to the database. If an error occurs during synchronization, the transaction processing module performs a rollback operation based on the error type, ensuring data consistency and integrity. For example, when committing a transaction containing multiple operations, if one operation fails, the entire transaction is rolled back, avoiding data inconsistencies caused by partial successes. The front-end local caching technology combines LocalStorage and IndexedDB. LocalStorage stores small amounts of frequently accessed data, such as user configuration information and recent access records. Its advantages include simple operation, data stored in key-value pairs, and direct access via JavaScript. IndexedDB stores large amounts of structured data, such as business form data and list data. It supports transaction operations, index queries, and has a large data storage capacity, meeting the data storage needs of complex business scenarios. During data storage, sensitive data is encrypted using AES-256 to ensure data security during local storage. Simultaneously, data indexes are created to improve data query efficiency. For example, for order data, indexes can be created based on fields such as order ID, user ID, and order time to quickly retrieve order data based on specific conditions. Redis cache cluster configuration and optimization utilize a Cluster mode, achieving horizontal data scaling through sharding to support terabyte-level data storage.Each shard consists of one master node and multiple slave nodes. The master node handles write operations and some read operations, while slave nodes are used for data backup and to distribute read load, improving system availability and performance. To improve cache hit rate, the following optimization strategies are adopted: reasonable key-value design, cache eviction policy, and cache preheating. Reasonable key-value design means following naming conventions and grouping related data keys for easy management and querying. For example, for user-related data, keys are named in the format "user:{user_id}:{data_type}". Cache eviction policy involves selecting appropriate eviction policies based on business needs, such as LRU (Least Recently Used) and LFU (Least Frequently Used), ensuring that hot data is always stored in the cache. Cache preheating involves loading hot data into the Redis cache before storage starts, reducing latency during initial access. Data synchronization between the database and Redis cache uses a combination of Change Data Capture (CDC) technology and message queues. CDC technology monitors database change operations (such as insert, update, and delete) in real time, extracts the changed data, and sends it to a message queue. The Redis cache cluster subscribes to the message queue, retrieves the changed data from the queue, and performs corresponding processing based on data type and operation type, such as updating cached data and deleting expired cache. To ensure the accuracy and consistency of data synchronization, a transaction message mechanism is introduced. When a transaction operation is performed in the database, a transaction message is simultaneously sent to the message queue. The message queue only sends the message to the Redis cache cluster after confirming that the database transaction has been successfully committed. If the database transaction is rolled back, the message queue automatically deletes the corresponding transaction message to avoid inconsistencies between the Redis cache and the database. Data interaction between the frontend and the Redis cache is conducted via WebSocket and HTTP protocols. For operations with high real-time requirements (such as real-time message push and status updates), a long connection is established using the WebSocket protocol to achieve real-time bidirectional data transmission. For non-real-time operations (such as batch data synchronization and query requests), the HTTP protocol is used for interaction. Simultaneously, JWT (JSON Web Token) is used for authentication and authorization to ensure that only legitimate frontend applications can access Redis cached data, guaranteeing data security. Performance optimization measures include: improving cache hit rate, reducing the number of network requests, asynchronous processing, and batch operations; security mechanisms include: data encryption, authentication and authorization, and monitoring and alerting. Improving cache hit rate means increasing the hit rate of Redis cache and front-end local cache through reasonable caching strategy design and data prefetching mechanism.For example, for frequently accessed data, a proactive caching update strategy is adopted, updating caches at all levels promptly when data changes; for infrequently accessed data, lazy loading is used to reduce cache resource consumption. The number of network requests is reduced, meaning the front-end local cache handles user operation requests as much as possible, only interacting with the back-end when necessary. For repetitive query operations, data is retrieved directly from the local cache, avoiding frequent requests to the back-end. Asynchronous processing and batch operations are implemented, processing time-consuming operations (such as data synchronization and complex calculations) asynchronously to avoid blocking the user interface. Batch operations are also used, merging multiple small requests into a single large request to reduce network overhead. For example, during data synchronization, multiple operations within a certain timeframe are merged into a single batch request and sent to the Redis cache cluster. Data encryption is used, encrypting data during transmission and sensitive data stored in the local cache, Redis cache, and database. The transport layer uses TLS 1.3 encryption to ensure data security during network transmission; the storage layer uses high-strength encryption algorithms such as AES-256 to encrypt data and prevent data leakage. Identity authentication and authorization employ multi-factor authentication to ensure only authorized users can access the system. Simultaneously, a role-based access control (RBAC) model provides fine-grained permission management, allowing users with different roles to access data and functions only within their authorized scope. Monitoring and alerting involve recording and auditing all operations, including user login, data access, and data modification. Log analysis promptly identifies potential security risks and abnormal behaviors, enabling appropriate action. A top-down analytical model architecture is established, extracting features from historical data, training a baseline model, and using the trained model for real-time prediction. Deviation calculations are then performed, followed by root cause analysis and optimization recommendations. An ARIMA (Autoregressive Integral Moving Average) model is used to establish normal fluctuation ranges for each indicator. Association rule mining is employed to pinpoint the root cause of problems, such as discovering a strong correlation between "Insufficient Redis memory" and "Increased synchronization failure rate."

[0070] Together, we complete multi-dimensional analysis and anomaly detection. The deployment architecture employs a distributed architecture, supporting multi-node and multi-region deployment. The database uses a master-slave replication architecture, with the master database handling write operations and the slave databases handling read operations and data backup, improving system availability and performance. The Redis cache cluster uses a Cluster mode, distributed across multiple nodes, achieving data sharding and load balancing. The front-end application is deployed on edge nodes or user terminals, accelerating the loading of static resources through a CDN (Content Delivery Network) to improve user access speed. During deployment, considering the needs of different business scenarios, flexible configuration options are provided, such as cache size, synchronization cycle, and concurrent connection count, which users can adjust according to their actual needs. Operation and maintenance management involves establishing a comprehensive operation and maintenance management system, including monitoring and alerting, log analysis, performance tuning, and troubleshooting. The monitoring system collects performance metrics (such as CPU utilization, memory utilization, network bandwidth, cache hit rate, etc.) of various levels of cache, database, and servers in real time. When metrics exceed thresholds, timely alert information (such as SMS, email, WeChat notifications, etc.) is sent so that operation and maintenance personnel can handle them promptly. The log analysis system centrally manages and analyzes various logs generated by the system (such as operation logs, error logs, and synchronization logs). Through log mining, it identifies potential system problems and performance bottlenecks. Based on monitoring data and log analysis results, operations and maintenance personnel perform performance tuning on data caching and batch synchronization, such as adjusting caching strategies, optimizing database query statements, and adjusting server resource allocation. Simultaneously, a fault diagnosis process and knowledge base are established to improve fault handling efficiency and ensure stable system operation.

[0071] like Figure 9As shown in the diagram, this is an architecture diagram. After data changes occur at the data source layer, they are synchronized to the Redis cache cluster in real time through the cache synchronization layer. Simultaneously, the conflict resolution module performs version comparison to handle potential conflicts. The front-end local cache retrieves data from the Redis cache cluster upon initialization and tracks data during user operations through monitoring. It is then categorized and processed by the transaction engine, and dynamically adjusted through the synchronization strategy engine. The categorization and dynamic adjustment results are passed to the operation classifier for real-time, periodic, and offline operations. Real-time operations are prioritized for synchronization to the Redis cache cluster via a WebSocket channel. Periodic operations are triggered periodically to the Redis cache cluster via a batch synchronization queue. Offline operations are synchronized to the Redis cache cluster via local transaction logs after network recovery. This architecture diagram can be understood as a "command flow for data flow and collaboration between different cache layers." Its core problem is how to efficiently synchronize data between the data source, Redis, and the front-end local cache, while handling conflicts and monitoring status. Table 4 below breaks down the role and collaboration logic of each process layer by layer. The entire architecture consists of five layers, from "data source" to "front-end user operation," with each layer responsible for different aspects.

[0072] Table 4 Core Layers and Roles

[0073] ;

[0074] Hierarchical Collaboration Process (Taking "User Modifying Data" as an Example): Assuming a user modifies data on the front end, the entire process is as follows: A. Front-end Local Cache → Transaction Engine → Operation Classifier. When a user modifies data on the front-end page, the data is first stored in the front-end local cache (like a draft box, ensuring offline operation). The transaction engine records this operation (generating an operation log to ensure traceability). The operation classifier determines the operation type: "Real-time operation" (e.g., transfer, order submission) → uses the WebSocket channel (prioritizes synchronization, ensuring second-level effectiveness); "Periodic operation" (e.g., batch modification of product information) → enters the batch synchronization queue (triggered periodically, e.g., once every 5 minutes); "Offline operation" (modification when the network is poor) → stores in the local transaction log (synchronized after network recovery). B. Synchronization Strategy Engine → Redis Cache Cluster. The synchronization strategy engine determines when and how to synchronize to Redis based on operation type and data activity (dynamically adjusting the synchronization cycle): Real-time operations → immediately synchronize to Redis via WebSocket; Periodic operations → batch synchronize to Redis after the scheduled time; Offline operations → after network recovery, retrieve data from the local transaction log and batch synchronize to Redis. C. Redis Cache Cluster → Cache Synchronization Layer + Conflict Resolution. After Redis receives data synchronized from the front end, the cache synchronization layer will synchronize back to the data source (such as the database) to ensure consistency between the data source and cached data. If data in Redis conflicts with data in the data source or other endpoints (e.g., simultaneously modifying the same record), the conflict resolution process uses "version comparison" (e.g., chained version numbers) to determine whose modification is "newer," resolves the conflict, and then merges the data. D. Monitoring → Full-Process Tracking. Monitoring tracks the entire data flow: recording who modified the data and when (transaction engine logs); monitoring whether Redis synchronization is successful and whether conflict resolution is reasonable; providing data to operations personnel for troubleshooting and optimizing synchronization strategies. The core of this architecture is to enable intelligent data flow between different levels of caching: front-end local caching ensures smooth user operation (even offline); Redis cluster accelerates data read / write and reduces data source pressure; classification synchronization + strategy engine balances "real-time performance" and "resource consumption"; conflict resolution + monitoring and alerting ensure data consistency and allow for rapid problem localization. It can be understood as a "data logistics system," where the data source is the factory, Redis is the logistics center, the front-end local cache is the express delivery station, the operation classifier is the courier (determining urgent / slow packages), the synchronization strategy engine is the logistics scheduler (determining when to dispatch), conflict resolution is customer service (handling delivery disputes), and monitoring is logistics tracking (keeping track of package location). The beneficial effects of this invention include improved core performance, optimized resource costs, offline capability, enhanced user experience, upgraded scalability, guaranteed data reliability, and forward-looking technological ecosystem.Core performance enhancements include: significantly optimizing the response speed of high-frequency operations through front-end local caching and transaction log preprocessing technologies, achieving near-native application response levels; greatly improving the efficiency of complex transaction processing through efficient transaction processing mechanisms, effectively reducing backend server load; and optimizing Redis caching strategies to significantly increase cache hit rate, reduce server CPU resource consumption, and achieve efficient utilization of hardware resources. Resource cost optimization involves significantly reducing the number of backend servers deployed through deep architectural optimization and resource integration, significantly reducing network bandwidth consumption and data center energy consumption, optimizing PUE values, and effectively controlling operating costs. Offline capability is achieved by building a complete offline operation system that supports long-term offline operations and ensures data consistency. After network recovery, the system automatically performs data synchronization without manual intervention, ensuring business continuity. Enhanced user experience is achieved by strengthening the system's adaptability in offline and weak network environments, significantly improving data transmission reliability and multi-terminal data synchronization efficiency, greatly reducing business interruption time, and providing users with a smooth and stable user experience. Scalability upgrades leverage the efficient scalability of Redis clusters, significantly reducing horizontal scaling costs compared to traditional database scaling. Simultaneously, it supports flexible multi-cloud deployment architecture and further optimizes network resource utilization through front-end local caching, reducing cloud service costs. Data reliability is guaranteed by employing AES-256 high-strength encryption to protect front-end data storage and prevent data tampering. A chained version number and transaction log mechanism effectively resolves distributed data conflict issues. During data transmission, the TLS 1.3 protocol is used to ensure secure and reliable data transmission. The system boasts a forward-looking technological ecosystem, with a highly open and compatible architecture that seamlessly integrates with cutting-edge technologies such as edge computing, AI stream computing, and metaverse, providing solid underlying technical support for innovative applications such as 5G+Industrial Internet, real-time decision analysis, and high-concurrency virtual interaction.

[0075] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores static and dynamic information data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the above-described embodiment of the multi-level cache-based system data caching and batch synchronization method.

[0076] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0077] In addition, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described embodiments of the management system data caching and batch synchronization method based on multi-level caching.

[0078] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described embodiments of the system data caching and batch synchronization method based on multi-level caching.

[0079] This invention is not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this invention is limited only by the appended claims.

Claims

1. A method for managing system data caching and batch synchronization based on multi-level caching, characterized in that, include: Based on the pre-built caching architecture, the pre-acquired data is stored in the database of the caching architecture, and the data in the database is copied to the standby database through master-slave replication; Based on change data capture, the storage data of the standby database is synchronized and stored in the key-value storage database of the caching architecture, and the data shards stored in the key-value storage database are cached in real time at the front end of the caching architecture by pushing real-time changes. Based on the data cached at the front end, operation data is extracted, and local operation processing and operation type determination are performed based on the operation data in combination with the transaction engine and operation classifier to obtain local transaction logs and operation types. The system acquires real-time storage status data of the caching architecture and calculates the synchronization optimization cycle using a linear weighting method based on the storage status data and operation type. Operation feature data is extracted based on storage state data and operation type, and the optimal synchronization strategy is obtained by using a decision tree algorithm in combination with the operation feature data. The synchronization optimization cycle and the optimal synchronization strategy are integrated to obtain the optimized synchronization strategy. Based on the optimized synchronization strategy and the chain version number mechanism, the data after local operation processing is batch synchronized. The operational feature data includes: operational type feature data, data activity feature data, network quality feature data, data sensitivity feature data, operational data volume feature data, and synchronization time feature data; Obtain the runtime status data of the cache architecture, and use the runtime status data to construct a cache performance index system. Based on the cache performance index system, combine the autoregressive integral moving average model and association rules to detect storage anomalies, and handle data storage alarms based on the storage anomaly detection results.

2. The method for managing system data caching and batch synchronization based on multi-level caching according to claim 1, characterized in that, The process of extracting operation data based on front-end cached data, and then performing local operation processing and operation type determination based on the operation data in conjunction with the transaction engine and operation classifier to obtain local transaction logs and operation types includes: Extract operation data based on the data cached on the front end, perform local operation processing based on the operation data, and record the local operation processing through the transaction engine to obtain the local transaction log; Based on the local transaction log and pre-defined operation classification rules, the operation classifier is used to determine the operation type of the operation data.

3. The method for managing system data caching and batch synchronization based on multi-level caching according to claim 1, characterized in that, The batch synchronization of data processed locally based on the optimized synchronization strategy combined with the chained version number mechanism includes: Based on the optimized synchronization strategy and the chained version number mechanism, the data processed locally is batch synchronized and stored to the key-value storage database. The optimized synchronization strategy is used to persist the data that has been synchronized in batches in the key-value store database to the database.

4. The method for managing system data caching and batch synchronization based on multi-level caching according to claim 1, characterized in that, The process of acquiring the runtime status data of the cache architecture, constructing a cache performance indicator system using the runtime status data, detecting storage anomalies based on the cache performance indicator system combined with an autoregressive integral moving average model and association rules, and handling data storage alarms based on the storage anomaly detection results includes: Obtain runtime status data of the caching architecture and use the runtime status data to build a caching performance indicator system; The normal fluctuation range of cache performance indicators is obtained by combining the cache performance indicator system with the autoregressive integral moving average model, and the alarm threshold of cache performance indicators is obtained based on the normal fluctuation range of cache performance indicators. The cache architecture is used to detect storage anomalies by combining association rules with alarm thresholds of cache performance metrics, and data storage alarms are processed based on the storage anomaly detection results.

5. The method for managing system data caching and batch synchronization based on multi-level caching according to claim 4, characterized in that, The cache performance index system includes: cache performance index, transaction processing index, synchronization operation index, and resource utilization index; The cache performance metrics include: key-value store database hit rate, memory fragmentation rate, number of connections, front-end read / write time, and cache hit rate; The transaction processing metrics include: transaction success rate, average execution time, rollback rate, and offline transaction backlog. The synchronization operation metrics include: synchronization cycle deviation, data transmission volume, number of conflict resolution attempts, and push of real-time changes to connection status. The resource utilization metrics include: server or memory utilization, disk utilization, network bandwidth, and packet loss rate.

6. A system for managing system data caching and batch synchronization based on multi-level caching, characterized in that, The management system data caching and batch synchronization system based on multi-level caching includes: a cache processing module, a synchronization processing module, and a storage alarm module; The cache processing module is used to store pre-acquired data into the database of the cache architecture according to the pre-built cache architecture, and to copy the data from the database to the standby database through master-slave replication; based on change data capture, it synchronously stores the storage data of the standby database into the key-value storage database of the cache architecture, and uses push real-time changes to perform front-end real-time caching of the data shards stored in the key-value storage database for the cache architecture; it extracts operation data based on the front-end cached data, and performs local operation processing and operation type judgment based on the operation data in combination with the transaction engine and operation classifier to obtain local transaction logs and operation types; The synchronization processing module is used to acquire the storage status data of the cache architecture in real time, and calculate the synchronization optimization cycle using a linear weighted method based on the storage status data and operation type; extract operation feature data based on the storage status data and operation type, and use a decision tree algorithm combined with the operation feature data to optimally filter the preset synchronization strategy to obtain the optimal synchronization strategy; integrate the synchronization optimization cycle and the optimal synchronization strategy to obtain the optimized synchronization strategy, and perform batch synchronization processing on the locally processed data based on the optimized synchronization strategy combined with a chain version number mechanism; the operation feature data includes: operation type feature data, data activity feature data, network quality feature data, data sensitivity feature data, operation data volume feature data, and synchronization time feature data; The storage alarm module is used to acquire the running status data of the cache architecture, construct a cache performance index system using the running status data, perform storage anomaly detection based on the cache performance index system combined with an autoregressive integral moving average model and association rules, and perform data storage alarm processing based on the storage anomaly detection results.

Images