Redis cache management method and device, computer equipment and storage medium
By encapsulating database write operations as signed domain events and generating random challenge values in Redis cache management, and combining multinomial commitment and conflict-free replication algorithms, the consistency and privacy security issues in Redis cache management are solved. This achieves efficient data-governance closed-loop management and dynamic load balancing, improving system performance and reliability.
Patent Information
- Application Number
- CN202511109079.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-08
- Publication Date
- 2025-11-18
- Estimated Expiration
- 2045-08-08
AI Technical Summary
Existing Redis cache management suffers from issues such as unprovable consistency, slow governance response, weak privacy compliance, lengthy write paths, dirty writes, and cache penetration, making it particularly difficult to meet massive write demands in high-concurrency and hotspot migration scenarios.
By encapsulating database write operations as signed domain events, generating random challenge values and calculating multinomial commitments, and employing a conflict-free copy data type merging algorithm and a federated learning framework for closed-loop management of data governance and verification, we achieve link-level atomic consistency, constant-time verification of log integrity, lock-free concurrent writing, and dynamic load balancing of hot keys.
It achieves link-level atomic consistency in high-concurrency environments, avoids binary log replay latency, ensures data integrity and privacy, provides auditable predictive prefetching and dynamic load balancing, and reduces read latency and write conflicts.
Smart Images

Figure CN120973862A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of database synchronization and distributed cache, and particularly relates to a Redis cache management method and device, a computer device and a storage medium. BACKGROUND
[0002] Redis, as a high-throughput and low-latency key-value database, has become an important component for read-write acceleration and traffic peak shaving in Internet services. With the continuous growth of online transactions, real-time recommendations, and log analysis, the capacity and concurrency of single-node Redis often cannot meet the needs of massive writes and hot migration, and the industry generally adopts a three-layer architecture of "database + message queue + cache" to maintain consistency.
[0003] However, existing technologies rely on binary log parsing or timed incremental pulling: changes are pushed to the cache by parsing the binary logs of the master database; due to the coupling of log format and database version, upgrading causes the parser to fail, and the delay is easily affected by network jitter; applications write to the database and cache simultaneously or refresh through invalid signals; the write path is long and the transaction is fragmented, which is prone to dirty writes and cache penetration; passive monitoring methods are relied on to count QPS, and then the shards or hot key routes are manually adjusted; governance lag leads to single-point overload; some systems try to use centralized prediction models to warm up keys, but the training set holds complete access logs, which poses a risk of data leakage, and the model update lacks a public and verifiable process; the above factors combined result in unprovable consistency, slow governance response, and weak privacy compliance. SUMMARY
[0004] To address the many problems existing in the above-mentioned prior art, the present application provides a Redis cache management method, device, computer device and storage medium, which encapsulates database write operations as domain events with signatures and generates a random challenge value; event streams are sequentially aggregated into event blocks and written to a data lake, and the root commitment is calculated through a polynomial commitment; after verifying the root commitment, the cache node replays the events and writes them to Redis using the conflict-free replicated data type merge algorithm, and then migrates shards and refreshes the random challenge value according to real-time key heat; subsequently, a federated learning framework is used to aggregate gradient updates to the prediction model in a homomorphic encryption and differential privacy manner, generate a summary chain, and form a data-governance-verification closed loop.
[0005] A Redis cache management method, comprising: intercepting database write operations at the object-relational mapping layer, extracting entity identifiers, operation types, changed fields, data schema versions, and hybrid logical clock timestamps, encapsulating them as domain events with digital signatures, generating a random challenge value, and submitting the domain events and corresponding database transactions to an event bus through a distributed transaction; aggregating the domain events in the received order into event blocks to write a consistent data log, computing an integrity commitment for the event blocks and generating a root commitment, writing the root commitment and a random challenge value to log metadata, and copying the event blocks to decentralized storage; verifying the event log based on the root commitment and the random challenge value at the cache node, consuming the event blocks after, merging the converged state map using a conflict-free replicated data type algorithm to write to Redis, and performing shard migration and random challenge value refreshing according to real-time key hotness; According to the key hotness and the root commitment, a distributed model is trained to predict a pre-fetch key set and preheat in the cache replica; the local gradient is homomorphically encrypted and added with differential privacy noise to participate in secure aggregation to update the distributed model parameters, the updated model hash and the root commitment are generated to generate an abstract, which is written to the distributed ledger through a verifiable proof mechanism, and the abstract is synchronized to the data log closed loop.
[0006] Preferably, the domain events are aggregated into an event block every preset number in the order of continuous submission, and the starting submission number and the ending submission number are recorded for each event block before writing to the consistent data log.
[0007] Preferably, the integrity commitment is obtained by polynomial mapping of all domain event hash values in an event block to obtain a block-level commitment, and then all block-level commitments are polynomial aggregated in the order of event block index to generate the root commitment.
[0008] Preferably, the conflict-free replicated data type merging algorithm covers the older modification with the newer modification with a hybrid logical clock timestamp when there is concurrent modification of the same key, and maintains the union for the fields that are not covered.
[0009] Preferably, the key hotness is obtained by summing the key access frequency and the key access latency multiplied by the weight, respectively, and the shard migration is triggered when the key hotness reaches a threshold, and a new random challenge value is calculated from the updated key hotness vector through a secure hash function.
[0010] Preferably, the local training gradient is first homomorphically encrypted, then added with noise that satisfies the differential privacy constraint, and then the encrypted gradient is uploaded to a secure aggregation service for ciphertext summation and decryption to obtain the global gradient to update the distributed model parameters.
[0011] Preferably, the distributed model training is based on a federated learning framework, and after updating the sequence prediction network locally at each cache node, the parameter aggregation is completed through the secure aggregation service, and after the parameter aggregation is completed, the abstract generated by the model hash and the root commitment is written to the distributed ledger.
[0012] A Redis cache management device for implementing the Redis cache management method, comprising: An event generation module is configured to intercept database write operations at an object-relational mapping layer, extract entity identifiers, operation types, changed fields, data schema versions, and hybrid logical clock timestamps, package the extracted information into digitally signed domain events, generate random challenge values, and submit the domain events and corresponding database transactions to an event bus through a distributed transaction; A log construction module is configured to aggregate the domain events into event blocks in a received order, write the event blocks into a consistent data log, calculate integrity commitments for the event blocks, generate root commitments, write the root commitments and random challenge values into log metadata, and replicate the event blocks to a decentralized storage; A projection governance module is configured to verify the integrity of the event log based on the root commitments and random challenge values at a cache node, sequentially consume the event blocks after verification, generate a converged state map by using a conflict-free replicated data type merge algorithm, write the converged state map into Redis, perform shard migration and refresh random challenge values according to real-time key hotness, and the like. A prefetch verification module is configured to train a distributed model to predict a prefetch key set according to key hotness and root commitments, preheat cache replicas, participate in secure aggregation to update distributed model parameters after homomorphic encryption of local gradients and addition of differential privacy noise, generate a digest by hashing the updated model and the root commitments, write the digest into a distributed ledger through a verifiable proof mechanism, and synchronize the digest to a data log to close the integrity, governance, and prefetching cycle.
[0013] A computer device includes a processor and a storage medium. The storage medium is configured to store instructions; and The processor is configured to operate according to the instructions to perform steps of the method of Redis cache management. The storage medium is configured to store instructions; and The processor is configured to operate according to the instructions to perform steps of the method of Redis cache management.
[0014] A storage medium having a computer program stored thereon, the program being executed by a processor to implement steps of the method of Redis cache management.
[0015] Compared with the prior art, the advantages and beneficial effects of the present application are that: Through event double writing and hybrid logic clock serialization, link-level atomic consistency is realized, and binary log replay delay is fundamentally avoided; through polynomial commitment and random challenge binding, log integrity constant-time verification is realized, solving the problems of low efficiency and easy playback of traditional hash chains; through the conflict-free replicated data type merging algorithm, the final convergence of lock-free concurrent writing is realized, avoiding concurrent update write conflicts; through the hotness vector driven shard migration, dynamic load balancing of hot keys is realized, replacing manual monitoring and lagging adjustment; through federated learning encryption aggregation and zero-knowledge proof, privacy-safe and auditable prediction prefetching is realized, filling the gap of centralized models that are easy to leak data and difficult to prove legitimacy; through the digest write-back log metadata, the three-chain closure of data, models and governance state is realized, and the risk of version drift is eliminated. BRIEF DESCRIPTION OF DRAWINGS
[0016] Figure 1 A flowchart of the method of the present application is shown; Figure 2 A schematic diagram of the federated learning secure aggregation of the present application is shown; Figure 3 A structure block diagram of the device of the present application is shown. DETAILED DESCRIPTION
[0017] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to one of ordinary skill in the art that one or more embodiments can be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques have been omitted to avoid unnecessarily obscuring the concept of the present disclosure.
[0018] The terms used herein are merely used to describe specific embodiments and are not intended to limit the present disclosure. The terms "include", "comprise" and the like used herein indicate the presence of the described features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.
[0019] All terms used herein (including technical and scientific terms) have meanings commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted to have meanings consistent with the context of the present specification, and should not be interpreted in an idealized or overly formal manner.
[0020] As Figure 1 shown, a Redis cache management method includes: Intercepting database write operations at the object-relational mapping layer, extracting entity identifier, operation type, change field, data schema version and hybrid logical clock timestamp, packaging as digitally signed domain event, generating random challenge value, submitting the domain event and corresponding database transaction to the event bus through distributed transaction; In the present application, first intercept the insert, update or delete operation of the database at the object-relational mapping layer, this step is the source of the whole synchronization link, and is also the prerequisite for ensuring the consistency of the subsequent Redis cache and the source data. After interception, the system immediately constructs a domain event in memory, which consists of five fields: entity identifier for uniquely pointing to the modified data row, operation type for explicitly indicating the semantics of this change, change field set for recording the affected column and its new value, data schema version for reflecting the current table structure version, and hybrid logical clock timestamp for containing physical time and logical sequence number for cross-node sorting. In order to ensure the tamper resistance of the event in the transmission and storage process, the system uses the elliptic curve digital signature algorithm to sign the event to obtain the signature value .
[0021] In order to quickly verify the event order in the subsequent steps of the link and avoid external replay, the system uses the signature value and hybrid logical clock timestamp to derive a random challenge value , the calculation method is as follows:
[0022] Where, represents the secure hash algorithm, is the digital signature value, is the hybrid logical clock timestamp. The random challenge value only depends on the event itself and can be independently recalculated at any node, which is used for subsequent integrity commitment verification.
[0023] After completing the signature and random challenge value generation, the system binds the domain event and the original database transaction through the distributed transaction mechanism. The specific implementation is to start a one-phase local transaction at the application layer, and at the same time, to prewrite a to-be-submitted message to the event bus; when the database write operation is confirmed, submit the two operations uniformly, and if any step fails, roll back the whole. This design realizes the atomic consistency of "source and event", and avoids the problem of sequence misalignment or more writing and less deleting in the traditional binary log parsing scheme under high concurrency.
[0024] The intercepted write operation instead of passive reading of the log can inject more context information at the business code level, such as request source, tenant identification, and the like, to provide sufficient metadata for subsequent accurate projection. Secondly, the hybrid logical clock timestamp as a serial number can ensure the total order of events without global clock synchronization, so that the conflict-free replicated data type merging algorithm can converge in a monotonic manner, reducing conflict rollback. Finally, the digital signature and the random challenge value jointly construct a verifiable chain, which, together with the integrity commitment in the second step, can verify in constant time whether the event has been tampered with at the cache node, thereby preventing malicious replay or man-in-the-middle modification.
[0025] In embodiment 1, when performing an email field update in a user table, the entity identification is the primary key ID 1024, the operation type is update, the changed field set only contains “email = new address”, the data mode version is the current table structure hash, and the hybrid logical clock timestamp record is the physical time plus the local logical count. The interceptor assembles five fields into a JSON serialization to calculate the signature value , calculates the random challenge value according to the formula , and submits it to the event bus together with the database write operation. The event bus assigns a continuous serial number immediately after receiving the message, and enters the subsequent event block aggregation process. If a second node attempts to update the same row concurrently, its logical count will definitely increase, so the timestamp will be larger, and finally the later event will overwrite the earlier event when merging, ensuring that the latest value is stored in the cache.
[0026] Through the above design, the application eliminates log parsing delay, cross-node clock drift and message tampering risk at the source, and lays a reliable foundation for subsequent integrity commitment, convergence state mapping generation and pre-fetching model training.
[0027] The domain events are aggregated in the receiving order to write an event block into a consistent data log, an integrity commitment is calculated for the event block and a root commitment is generated, the root commitment and a random challenge value are written into log metadata, and the event block is copied to a decentralized storage; In the application, a system constructs a consistent data log according to the receiving order of domain events. The implementation process includes four links: event block division, integrity commitment calculation, root commitment generation and decentralized replication. First, the event stream uses a globally increasing serial number as a reference, and every several events are aggregated into an event block, and the original order is kept unchanged in the event block. This can not only reduce the disk overhead of a single write, but also provide boundaries for subsequent batch verification. When the event block is written into the consistent data log, a data lake format supporting transaction semantics is used to ensure that the log can obtain an atomic view when loaded on any node, avoiding the half-written state of traditional stream processing between checkpoints.
[0028] To prevent log from being maliciously tampered, the present application calculates an integrity commitment based on polynomial mapping for each event block. All events within an event block are first converted into numerical values of consistent length by secure hashing, and then sequentially taken as polynomial coefficients. The system selects a random number in a pre-agreed finite field As a challenge point, the polynomial is evaluated at this point to obtain the block-level commitment. This process can compress the entire block of events in one mapping, without the need for item-by-item checking, reducing the amount of calculation and supporting constant-time verification compared to the row-by-row hash chain method.
[0029] All block-level commitments are combined again according to the block sequence number to generate a root commitment. The root commitment is a single numerical value, which can be used to verify the entire log at any node, and can also be written into the log metadata for automatic checking by downstream nodes. Its calculation formula is:
[0030] wherein represents the root commitment, represents the commitment of the event block, is a random challenge value, is the prime modulus of the finite field. This formula ensures that even if only one event block is tampered, the root commitment will be inconsistent, so that the cache node is rejected.
[0031] After generating the root commitment, the system writes the root commitment and the random challenge value into the log metadata together. The random challenge value is derived from the random quantity in the first step and can be independently reproduced at any node without additional synchronization, providing an entropy source for cross-node verification.
[0032] To improve disaster recovery and audit capabilities, each event block file is synchronously replicated to a decentralized storage. The decentralized storage uses a content addressing mechanism, and after uploading the block file, a unique content identifier is obtained, which is written back to the log metadata. This brings two effects: first, even if the main data lake is unavailable due to hardware or software failure, the system can still reconstruct the event block at any node through the content identifier; second, content addressing naturally prevents tampering, and any modification will cause the identifier to change, which can be detected when it does not match the log metadata.
[0033] In the peak write scenario, 20000 domain events are generated within 1 minute, and the system divides them into 19 event blocks of 1024 events each to write into the data lake. The block-level commitment is calculated for each event block separately, and then the root commitment is obtained through a formula. After the root commitment is written into the log metadata together with a random challenge value, the decentralized storage replication is triggered synchronously. Assuming that the 3rd event block has a bit flip during transmission, the block-level commitment recalculated after the flip does not match the original record, causing the root commitment verification to fail. The cache node immediately rejects the batch of data to prevent incorrect writing to Redis. The system then re-pulls the complete block file from the decentralized storage and recalculates the commitment through the content identifier, and finally recovers the consistent data log.
[0034] Preferably, the domain events are aggregated into an event block in the order of continuous submission every preset number, and the starting submission number and the ending submission number are recorded for each event block before writing into the consistent data log.
[0035] In the second step of the present application, the system receives the domain event stream generated in real time by the first step, and merges the domain events into event blocks according to the continuous submission order. The event block size can be selected as a fixed number of entries according to the writing amount and verification overhead. For example, in a high-concurrency scenario, the system can set 1024 events to form an event block; in a low-concurrency scenario, the number of entries can be dynamically reduced to limit the delay. Before writing into the consistent data log, the system records the first and last submission numbers for each event block, which can quickly locate the missing or tampered events when searching for fault points later. The sequence number information is also written into the log metadata, ensuring that any node loading the log can directly infer the block range without scanning each entry.
[0036] The consistent data log adopts a data lake format that supports transaction semantics. When a event block file is written, the system immediately submits the transaction metadata to ensure that the log is either completely readable or completely rolled back logically. This design ensures that subsequent cache nodes do not read the half-written state even if they restart during log writing.
[0037] After the event block is written, the system calculates the integrity commitment for the event block. The commitment calculation uses a polynomial mapping method, taking the secure hash values of all events in the block as polynomial coefficients, and evaluating them at a random challenge point in a pre-set finite field . Assuming that the event block contains events, and the event hash is in order , the block-level commitment is calculated as follows:
[0038] is the secure hash value of the th event, is a random challenge point, is a prime modulus. After the calculation is completed, the system re-executes the same polynomial aggregation on all block-level commitments in the order of event block index to obtain the root commitment . The root commitment provides single-point verification capability for the entire log. Any node only needs to recalculate the block-level commitment and aggregate to compare whether the root commitment is consistent. Once the single-block data is tampered with, the root commitment changes, and the cache node can refuse to continue consumption.
[0039] Writing the root commitment and the random challenge value into the log metadata can play a dual role. First, the random challenge value is derived by using a digital signature and a hybrid logical clock timestamp, which is unique in the entire link. After the root commitment is bound to the random challenge value, it can prevent attackers from bypassing verification by replaying old log files. Second, the log metadata can directly obtain the verification parameters when loaded downstream, avoiding the delay caused by network synchronization challenge points.
[0040] To ensure consistency of disaster recovery and audit, the system copies each event block file to a decentralized storage platform. Decentralized storage uses a content addressing mechanism, and a unique content identifier is obtained after file upload. Any modification will change the identifier. The system writes this identifier back to the metadata and periodically compares the metadata record with the remote identifier in the background task. If an inconsistency is found, an alarm is issued and the system is rolled back from the last consistent copy. This mechanism allows the log to be automatically rebuilt on other nodes even if the main data lake is damaged or the network is isolated.
[0041] Batch writing of event blocks reduces the number of disk I / Os and improves throughput. Moreover, batch integrity commitment only needs constant-level operation within the block size range, and its complexity is much smaller than that of row-by-row hash chains. The polynomial commitment scheme has batch verifiable characteristics, so that the cache node only needs constant time when verifying the complete log. Writing the root commitment to the metadata allows any subsequent step to complete the entire chain consistency check in milliseconds. Decentralized replication provides distributed redundancy for the log, reducing the risk of cache disconnection caused by single-point failure.
[0042] Example 3: 20,000 events generated within 1 minute are divided into 19 event blocks and written to the data lake, with each block file size of about 6 megabytes. After the system calculates the commitment of each block, it aggregates the root commitment and writes it to the metadata. Then the block files are uploaded to the distributed storage through content addressing, returning 19 content identifiers. If the third event block file is damaged due to disk failure, the cache node can locate the inconsistent block index when verifying the root commitment and refuse to load. The background task detects the mismatch of the identifier and automatically retrieves the complete file from the decentralized node, recalculates the commitment, and restores the entire log chain. Through this process, the present application ensures that the Redis cache can maintain a consistent data view with the source database when facing hardware failure, network tampering, or replay attacks.
[0043] Preferably, the integrity commitment is obtained by polynomial mapping of all field event hash values in an event block to obtain a block-level commitment, and then polynomial aggregation of all block-level commitments in the order of event block indexes to generate the root commitment.
[0044] In the log construction phase of the present application, the system needs to create a verifiable and replayable persistence link for field events under the premise of ensuring high throughput. To this end, the system uses event blocks combined with polynomial commitments to store and batch verify event streams.
[0045] In principle, field events enter the cache write queue with a monotonically increasing commit sequence number. The system aggregates continuous field events into event blocks according to a preset number. For example, when the preset number is set to 1024, events with sequence numbers from zero to 1023 constitute the first level event block. From the first item to the last item, each event block generates the first and last commit sequence numbers before being written to the consistent data log. The first and last sequence numbers provide two types of capabilities for the downstream. First, quickly distinguish block boundaries to avoid scanning the full log file item by item during subsequent verification. Second, when network jitter causes event omission or repeated writing, the first and last sequence numbers can quickly locate the abnormal range.
[0046] After completing the event block division, the system calculates the block-level integrity commitment for each event block. The integrity commitment uses a polynomial mapping mechanism, the core idea of which is to treat the event hash sequence as the coefficients of a polynomial, and the block-level commitment can be obtained by evaluating at a random challenge point. Assuming that the th event block contains events, and the event hash sequence is denoted as . Select a random number as the challenge point, and obtain the block-level commitment in the finite field according to the formula . represents the commitment value of the th event block, represents the secure hash value of the th event in the th event block, represents the random challenge point, represents the modulus of the finite field. This formula only needs to scan once to aggregate the data volume of the entire block, and the time complexity is linearly related to the number of events. Compared with the traditional row-by-row hash chain, the verification process of polynomial mapping can perform constant time checking on the entire block, significantly reducing the log review cost.
[0047] After generating all block-level commitments, the system performs a polynomial aggregation again on the block-level commitments in the order of event block indexes to obtain the root commitment . The root commitment formula is wherein represents the total number of event blocks, and other letters have the same meaning as the previous formula. The root commitment not only represents the single-point check value of the complete log file, but also can be bound with a random challenge value and written into the log metadata. This has two meanings. First, the root commitment can verify the integrity of the entire log in constant time at the cache node. Once any event block is maliciously modified or lost, it will cause the formula inequality to be established, and the cache node will immediately refuse to consume. Second, writing the root commitment and the random challenge value into the same metadata entry, combined with the random challenge value derived in the first step, can prevent attackers from replaying old files to the system, because the challenge value will be updated in the new running cycle, causing the old root commitment to fail the verification.
[0048] Data security is not only reflected in the calculation commitment, but also in the storage path. The system synchronously replicates the event block file to the decentralized storage platform. Decentralized storage uses a content addressing mechanism, and each file gets a unique content identifier after uploading. The identifier is one-to-one mapped with the block-level commitment. Once the block file is modified, the content identifier changes, which does not match the metadata record, so that the background audit task can discover abnormalities and trigger automatic rollback. The replication process uses an asynchronous pipeline, and the main write process does not have to wait for remote confirmation to continue generating new event blocks, thereby maintaining high write throughput.
[0049] By aggregating domain events into event blocks and writing them to a data lake that supports transaction semantics, the system can ensure that the log is complete or rolled back after the node crashes or the network flashes, eliminating the risk of half-written. Second, the double-layer structure of block-level commitment and root commitment allows the cache node to verify the authenticity of millions of events by verifying only ten numbers when starting, significantly reducing the replay pressure. Third, decentralized replication constitutes a disaster recovery channel. Even if the main data lake is damaged, the log can be reconstructed at other nodes using the content identifier, ensuring that the Redis cache has a stable data source.
[0050] Example 4, assuming that 2048 domain events are received within one minute. The system sets that every 1024 events constitute an event block, so two event blocks with block indexes 0 and 1 are generated, and the first and last sequence number ranges are recorded in the metadata. After calculating the block-level commitment for each block event, a fixed challenge point The root commitment is generated. The root commitment and the random challenge value are written into the log metadata at the same time. Then the two event block files are uploaded to the decentralized storage, returning content identifiers A and B, respectively. If damage occurs in block index 1, the commitment is recalculated for the block and found to be inconsistent, so the root commitment verification fails immediately, and the cache node refuses to load block 1 data. The background process pulls back the complete file from the decentralized storage according to the content identifier B, recalculate the commitment to restore consistency, and pass the verification again, so that the Redis projection link can continue to run without manual intervention.
[0051] In summary, the application introduces a low-cost and high-reliability data integrity baseline for Redis cache by combining event block aggregation with polynomial commitment. The scheme can efficiently operate in a high-concurrency environment without complex consensus protocols and provides cross-region disaster recovery capabilities, providing a trusted data source for subsequent adaptive projection and distributed model prefetching stages.
[0052] In the cache node, the event log is verified based on the root commitment and the random challenge value, the event block is consumed after consumption, the conflict-free replicated data type merging algorithm is used to generate the converged state mapping and write to Redis, and the shard migration and random challenge value refreshing are performed according to the real-time key heat; The application first receives the event block and the root commitment from the data lake in the cache node. The node independently recalculates the block-level commitment for each event block with a random challenge value as input, and then performs polynomial aggregation to obtain the local root commitment in block index order. If the local root commitment is consistent with the log metadata record, it proves that the log has not been tampered with during transmission and storage, and the node enters the consumption stage; if it is inconsistent, it is immediately rejected and requests the decentralized storage to pull the original block file again, thereby ensuring that the data written to Redis subsequently is trusted. After verification, the node starts to sequentially traverse the event block. During traversal, each domain event is processed by the conflict-free replicated data type merging algorithm. The algorithm uses an observer deletion mapping structure to save the key field version, and uses a hybrid logical clock timestamp to determine the coverage relationship: the newer timestamp covers the older version, and the non-conflict field is written in set. After merging, the converged state mapping is obtained, and the mapping is written to Redis through a batch pipeline to ensure that the read side can read a consistent snapshot at a time.
[0053] The node starts the key heat probe during writing, and records the delay and frequency of each key access in the kernel layer. The system accumulates the key access frequency and average delay in a predefined time window, and calculates the heat value. The heat value formula is as follows:
[0054] Among them, the key heat value, the access frequency in the window, the average access delay, and is the weight coefficient. The node scans the heat value set every fixed time, and once the heat value of a key is greater than the threshold, it is considered as a hot spot. The hot spot key is moved to the Redis node with lower load through the distributed cache shard migration tool. After successful migration, the node recalculates the latest heat vector and generates a new random challenge value using a secure hash function and writes it to the log metadata. The new random challenge value will participate in the root commitment calculation when the next batch of event blocks is verified, forming a continuous and uninterrupted verifiable chain.
[0055] The cache node can detect log integrity in milliseconds, and will not cause dirty writing due to tampering data; the conflict-free replicated data type merging algorithm naturally tolerates concurrent writing, ensuring final convergence; hotness-driven shard migration dynamically balances hot keys, preventing single-node overload; random challenge value refresh feeds hot scheduling information to the verification chain, forming a closed loop of data integrity and governance logic.
[0056] In the high concurrency mode, the system receives 5 event blocks every 10 seconds. The node writes to Redis after verifying that the root commitment is consistent. In one window calculation, the access frequency of key ID 30 is 800 times, the average delay is 0.5 milliseconds, and the weight is set as . Substituting the formula gives . The threshold is set to 900, so the migration is triggered. After the migration is completed, the node hashes the updated hotness vector into a new random challenge value and appends it to the metadata. When the next batch of event blocks arrives, the new random challenge value participates in the polynomial mapping, ensuring that the root commitment is synchronized with the data governance. After this process, the Redis cluster read delay is reduced by about 35% in the peak stage, while the integrity verification is correct, meeting the dual requirements of high consistency and high performance.
[0057] Preferably, the conflict-free replicated data type merging algorithm covers the older modification with the newer modification in the mixed logical clock timestamp when there is concurrent modification in the same key, and maintains the union for the fields that are not covered.
[0058] The application uses a conflict-free replicated data type merging algorithm on the cache node side to solve the cache inconsistency problem caused by multiple concurrent writes. The algorithm designs a line-level merging rule based on the observer deletion mapping structure, which can ensure final convergence under lock-free conditions. The core idea is: regarding each column of the same key as an independent element that can be merged, adding a mixed logical clock timestamp to the element, and maintaining a deletion observer set for the entire row. When updates from multiple event blocks conflict in the same row, the algorithm performs optimization coverage based on the timestamp; if the column does not have a coverage relationship, the values are kept and exist in parallel, which is equivalent to performing a union on the column set.
[0059] The mixed logical clock timestamp is composed of a physical time component and a logical count component. The physical time provides a causal relationship approximate ordering, and the logical count distinguishes the writing order when the physical time is equal. Suppose that two updates come from nodes A and B, and they act on the same path key. Node A gives the timestamp , and node B gives the timestamp . When the algorithm compares the order, it first compares the physical time; if , the update of node A is newer; if , compare the logical count, Then node A is still newer; only if both dimensions are not greater than the other is it considered older. The judgment logic can be expressed as:
[0060] where the symbol represents "timestamp newer than"; represents physical time, represents logical count.
[0061] The merging operation is performed in two steps. In the first step, the algorithm scans the mapping structure and compares the conflicting columns; if the column version comes from two different timestamps, the newer version is retained. In the second step, the algorithm checks the deletion observer set. When the deletion operation enters, the column is not immediately physically deleted, but a deletion mark is inserted into the set, and the timestamp of the deletion initiator is recorded. Any subsequent update, if its timestamp is later than the latest deletion timestamp in the set and the modified column has been marked for deletion, is written as a new version; otherwise, the deletion takes effect and the old value is removed from the set, ensuring monotonic consistency.
[0062] In order to avoid potentially expensive lock operations on the complete mapping, the present application splits the mapping structure into Redis hash buckets at the key granularity at the implementation level, and the column value storage format within the bucket is (payload, timestamp). The deletion observer set is mapped to a special key space, and the key name is formed by appending a prefix to the original key name, such as del#user table#ID=1024. In this way, both the local atomicity of the same key can be guaranteed using Redis single-element atomic write, and the blocking caused by locking the entire hash table in high-concurrency scenarios can be avoided.
[0063] The converged state mapping after merging is written in batch. In the implementation, the node writes changes into the local pipeline in a fixed batch size. The first stage of the pipeline writes column values using MULTI and HMSET, and the second stage sends key names to the notification channel using PUBLISH. The notification channel updates the local routing cache at the subscription end, so that read requests can hit new values in real time.
[0064] Another innovative point of the algorithm is to integrate heat-driven shard migration and random challenge value refreshing into the merging process. When the node collects key access indicators through kernel probes, it calculates the key heat value in real time. When the heat value reaches a threshold, the node triggers a cluster shard migration command to migrate the hot key to a node with lower load. After the migration is complete, the node generates a random challenge value from the latest heat vector using a secure hash algorithm and writes it into the data log metadata. The new random challenge value directly affects the commitment calculation of the next batch of event blocks, forming a closed loop between governance actions and integrity verification, thereby preventing replaying old logs from causing the governance state to deviate from the data state.
[0065] Embodiment 6: The key "order: 1234" is updated at node A and node B respectively. Node A modifies the column "status" from "paid" to "shipped", timestamp . Node B writes the column "logistics number" near the timestamp . The algorithm receives the two changes and compares the timestamps: the physical time is the same but the logical count is different, node A is newer, so the "status" column adopts the value of node A, and the "logistics number" column is directly inserted. In the final converged state map, the "order: 1234" key contains the data of the "shipped" and "logistics number" columns. High-frequency access makes the hot value exceed the threshold, and the node migrates the key to node C through CLUSTER MIGRATE. After the migration is completed, the hot vector hash gets a new random challenge value , which is written into the metadata for the next round of commitment verification, without human intervention throughout the process.
[0066] The design ensures eventual consistency while realizing lock-free high-speed writing. In online e-commerce scene testing, under 50,000 requests per second per table, the merging algorithm and sharding migration are still completed in sub-second, the Redis read delay 99th percentile is reduced by 25%, and there is no single and dirty write. The application achieves significant results in cache consistency and performance.
[0067] Preferably, the key hotness is obtained by multiplying the key access frequency and the key access latency by the weight and then summing, when the key hotness reaches the threshold, the sharding migration is triggered, and the new random challenge value is calculated by the security hash function with the updated key hotness vector.
[0068] After the cache node completes the converged state map writing to Redis, the application uses the key hotness driven sharding migration mechanism to continuously optimize read-write load. The key hotness comes from two observable indicators: access frequency and average access latency within the window. The node intercepts each GET, SET, DEL command through the kernel layer probe, records the key name and nanosecond-level timestamp, and aggregates in user mode with a fixed time window. Let the access frequency of a certain key in the statistical window be , and the average access latency be . In order to measure "busy" and "slow" in the same dimension, the system assigns weights and to the two indicators (both are configured by the operation and maintenance side during deployment, and once set, they are not adjusted at runtime). The key hotness value is calculated as follows:
[0069] The key hotness value is , the access frequency is , the average latency is , and is a constant weight. It can be seen that the more frequent the access or the higher the delay, the greater the heat. The node traverses the key heat table at the end of each window and selects the key whose heat reaches a threshold value . The threshold value can be determined according to the cluster size and the expected delay target, and is generally the ninetieth percentile of the key heat distribution in the past period of time. When the key is marked as a hot spot, the node triggers a distributed cache shard migration process. The migration command moves the hot key slot to a node with lower load according to the Redis cluster protocol. After the migration is completed, the new node starts to receive read and write requests for the key, and the original node only retains the bare pointer for client redirection. At the same time when the migration is completed, the node updates the key heat vector . In order to maintain the continuity of the integrity chain, the present application calculates a new random challenge value locally on the node through a secure hash function:
[0070] wherein is the last round random challenge value, is the serialized bytes of the latest heat vector. The new random challenge value is written into the data log metadata together with the root commitment, and can be perceived when the next batch of event blocks is verified. The ingenuity of the present application is that the governance operation (i.e. migration) itself updates the random challenge value, and the new random challenge value becomes the input of the subsequent integrity verification link, forming a three-ring closed loop of data-governance-verification.
[0071] On the principle level, heat calculation only relies on local statistics and does not require global locks or centralized coordination. Shard migration is implemented through Redis cluster native commands and can be completed online while keeping the key available. The design of deriving a random challenge value from a heat vector makes the governance operation part of the integrity chain, avoiding the problem of "data consistency but inconsistent governance state" in traditional cache systems.
[0072] Hotspot balancing, through dynamic migration, disperses high-concurrency keys to different nodes, reducing single-node delay jitter. Chain verification, the new random challenge value makes the next round of root commitment verification cover the log range after governance, and if an attacker replays the old log before migration, it will result in verification failure. Operation and maintenance is simplified, and there is no need for external monitoring to push migration signals. The cache node's endogenous heat probe can close-loop governance.
[0073] Example 7: In the Double Eleven seconds killing scene, the access frequency of the same user inventory key in the middle layer soared. In the statistical window of 30 seconds, the access times , the average delay milliseconds, , . Substituting the formula gives:
[0074] threshold value Therefore, the migration is triggered. The node calculates a new random challenge value and completes the event metadata update. The monitoring data shows that the CPU usage of the node where the key is located after migration is reduced from 85% to 55%, and the read 99th percentile delay is reduced from 4.8 ms to 2.1 ms. The subsequent event block verification proves that the migration process does not damage the integrity chain.
[0075] In summary, the application uses the weight heat formula to quickly identify hot keys, dynamically balances the load through the native migration mechanism, and refreshes the random challenge value after governance to continue the verifiable link, balancing performance and security, providing a consistent and low-latency Redis cache solution for high-concurrency business.
[0076] According to the key heat and root commitment, a distributed model is used to train and predict the pre-fetch key set and preheat the cache copy; the local gradient is homomorphically encrypted and added with differential privacy noise to participate in secure aggregation to update the distributed model parameters, the updated model hash and the root commitment are used to generate a digest, which is written into the distributed ledger through a verifiable proof mechanism, and the digest is synchronized to the data log closed loop.
[0077] After completing the convergence state mapping and shard migration, the application introduces a federated prefetch mechanism, which learns the access sequence locally in each cache node through distributed model training, and loads the predicted key set into the slave node cache in advance, thereby significantly reducing the first read latency; At the same time, encrypted aggregation and zero-knowledge proof are used to ensure that the model update process and the final result are verifiable. This step includes five consecutive links: input extraction, local training, ciphertext aggregation, model verification, and data closed loop.
[0078] First, after the cache node completes the heat evaluation, the latest key heat vector and the root commitment of the previous step are used as model inputs. The key heat vector represents the usage trend of each key in the future time window in terms of semantics, which is used as the context of the sequence prediction network; the root commitment is used as an integrity label to ensure that the model generated subsequently can be bound with the previous log version, preventing the contradiction of "training with correct data and verifying with old data".
[0079] The local training stage uses a sequence-to-sequence structure, the encoder uses a lightweight time convolution to capture short-term key access patterns, and the decoder uses a gated recurrent network to predict the key labels that need to be pre-fetched in the future window. The training samples come from the access sequences of each node in the recent several windows, avoiding cross-tenant data leakage. The local network can generate a gradient tensor after each round of backpropagation. In order to share learning results without exposing gradient content, the system homomorphically encrypts the gradient. The encryption library uses an integer homomorphism scheme to quantize the gradient tensor and encode it as a polynomial coefficient, and then performs a modular exponential operation with the public key to obtain the ciphertext. The ciphertext supports addition operations, so the server side can complete the sample summation without decryption.
[0080] Only encryption is not enough to resist replay attacks and manifold attacks, the present application introduces differential privacy noise after encryption. The noise obeys Gaussian distribution, and the standard deviation depends on the gradient Sensitivity and privacy budget calculation, but for the sake of brevity, the formula is not expanded in the specification. The node adds quantized noise to the gradient ciphertext in the ciphertext domain, so that the server side cannot recover the real gradient of the individual node.
[0081] After uploading the ciphertext, the aggregation server performs homomorphic addition on the ciphertext from all nodes to obtain the global gradient ciphertext, and decrypts in a secure hardware environment, restores the quantized value coefficient by using the private key corresponding to the decryption key. The server then updates the distributed model parameters and issues new weights. At this time, the present application needs to combine the new model hash with the previous log version root commitment to generate a digest, to ensure that the model is bound to the data version. The digest calculation directly uses the secure hash function:
[0082] wherein is the digest, is the hash value of the updated model weight, is the last round root commitment. The digest then enters the zero-knowledge proof circuit, and the circuit constraints include two: first, the new model comes from a legal aggregation process, that is, the input gradient is homomorphically encrypted and differential privacy noise is injected; second, the data version corresponds to the model version one by one, that is, the hash splicing value must match the root commitment of the log record. After the proof is generated, the system writes the proof and the digest as a transaction into the distributed ledger. Any node of the ledger can call the public verification key to check whether the proof is correct without knowing the specific model parameters and gradient content.
[0083] To close the governance and verification cycle, the system will write the digest back to the consistency data log metadata after successfully writing the chain. When the next event block arrives, the new digest will be used to participate in the root commitment comparison. If an attacker replays the old model or changes the model hash, it will result in a verification failure.
[0084] The direct effect of introducing the pre-fetch model is to improve the key first read hit rate. Because the predicted key set has been preheated from the node, the client read operation only needs to access from memory without going to the master node. The introduction of encryption and differential privacy makes it impossible for tenants to spy on each other's model data, while the server cannot analyze the gradient of a single node. Zero-knowledge proof provides strong verifiability for operation and audit, proving that the model is indeed trained by legal data and that the training process meets the privacy budget.
[0085] Example 8: Assuming each of the two cache nodes collects 100,000 key sequences, local training for one round obtains a gradient tensor with 64 dimensions. After quantization, the maximum value is not more than 2000, and the ciphertext size generated after encoding using the homomorphic encryption public key of the node is about 128 bytes. Each ciphertext is added with Gaussian noise with a variance of four to meet the privacy budget . The server sums up the ciphertext in the ciphertext domain to obtain the global gradient ciphertext, decrypts and restores the gradient, applies the average gradient to the model weight, and calculates the model hash as h1. h1 is spliced with the root commitment C0 of the previous round, and the digest d1 is obtained by hashing. The server calls the proof circuit to generate a proof and writes it to the ledger. The blockchain transaction contains the fields . The client verifies after pulling the new block, which indicates that the model is legitimate. Subsequently, within 1 minute, the prediction model is locally inferred on the node, and the top 128 most likely access keys are selected to pre-MGET from the node memory. The statistical data shows that the first read hit rate is improved from 85 to 93, and the average read delay is reduced from 1 millisecond to 0.6 milliseconds. When the next batch of logs arrives, the node writes d1 to the metadata, and d1 is read for constructing a new root commitment comparison when verifying the new event block. The entire cycle maintains self-consistency and a closed loop. This example verifies the comprehensive advantages of the present application in the dimensions of performance, privacy, and verifiability.
[0086] Preferably, as shown in Figure 2 , homomorphic encryption is first performed on the local training gradient, then noise satisfying the differential privacy constraint is added, and then the encrypted gradient is uploaded to the secure aggregation service for ciphertext summation and decryption to obtain the global gradient to update the distributed model parameters.
[0087] In the distributed model updating link of the present application, each cache node first performs forward and backward propagation according to the local key access sequence to obtain a gradient tensor. The gradient tensor maintains the same dimension as the model weight, but is easily leaked in the case of unprotected business access patterns. Therefore, the present application uses a homomorphic encryption scheme to encrypt the gradient. The specific implementation is as follows: after quantizing the floating-point gradient to the integer domain, the public key is used to perform modular exponentiation to obtain the ciphertext . Wherein represents the local quantized gradient vector, represents the encryption operation. Since the selected encryption algorithm supports the additive homomorphic property, there is , which ensures that the server can complete the summation in the ciphertext domain without decryption.
[0088] After encryption, noise needs to be added to the ciphertext to meet the differential privacy constraint. The node first encrypts the noise with the same quantization and encryption, and the noise distribution is a Gaussian distribution with a mean of 0 and a standard deviation , According to the privacy budget With gradient sensitivity calculation. Encryption noise With gradient ciphertext, get perturbed ciphertext:
[0089] Wherein The ciphertext uploaded by the first The node. All nodes upload their respective ciphertexts to a secure aggregation service. The aggregation service does not know the private key and can only homomorphically sum the ciphertexts:
[0090] According to the homomorphic property, at this time:
[0091] The secure aggregation service is in a trusted hardware environment and holds the private key . Inside the trusted execution environment, the service decrypts To get:
[0092] Wherein The global gradient after adding noise. Since the noise and sensitivity are determined in the offline stage, it can be theoretically proven that each round of training satisfies Differential privacy. The service then updates the distributed model parameters according to the learning rate. After updating the model weights, a new hash value Is calculated and concatenated with the root commitment Of the previous round to generate an abstract:
[0093] To ensure that any third party can verify the legitimacy of the model, the invention adds two constraints in the zero-knowledge proof circuit: one, the input gradient ciphertext must come from homomorphic encryption and contain noise, and the Decrypted by the server is consistent with the public random seed. Two, the concatenated hash value must correspond to the root commitment before updating, i.e. The hash result is . The circuit produces a proof After running. The system writes And the abstract As a transaction to the distributed ledger. Since the ledger only accepts transactions with verifiable content, if the proof does not match the public verification key, the transaction will be rejected.
[0094] After the chain is written successfully, the node receives the new weights and preheats the prediction key set in the background. The key set comes from the probability sorting of the model's top output, and the top Keys are selected to be pulled into the slave node memory in a pipeline manner. The preheating operation does not change the main writing process, but only improves the hit rate.
[0095] The last step is the closed loop synchronization. The node writes the digest into the consistent data log metadata, replacing the old digest. The next batch of event blocks reads the new digest when generating the block-level commitment, participating in the root commitment comparison, thus incorporating the model update status into the integrity check chain. If an attacker attempts to roll back the model file, although they can get the old weights, the hash value does not match the latest root commitment, and the downstream node will detect a failure in the commitment verification phase.
[0096] This design brings multiple effects at the Redis cache level. First, pre-fetching the model reduces the first access delay and improves the read throughput in high-concurrency scenarios; second, homomorphic encryption and differential privacy ensure that the gradient between tenants is invisible and the single-node behavior is anonymized; third, zero-knowledge proof shields the private key decryption process in the trusted execution environment, and external users can verify the model legitimacy without trusting the server. Finally, the digest written into the data log and the root commitment form a strong binding, making the model update and event chain advance synchronously, preventing data and model version drift.
[0097] Embodiment 9: There are 8 cache nodes, each with a sampling window length of 10 seconds and a quantized gradient vector dimension of 64. The node uses an integer homomorphic encryption library with a public key modulus of 4096 bits and a ciphertext size of about 128 bytes. In order to meet , the system selects . After aggregation, the global gradient is decrypted, and the learning rate is 0.01 to update the weights. The model hash and root commitment are concatenated to produce the digest valued1. The size of the zero-knowledge proof is 20 kilobytes, and the write chain enters the Ethereum second layer network, with a confirmation time of 2 seconds. Subsequently, the node uses the new model to predict the top 128 keys of the heat, and preheats through the pipeline batch MGET method. Online stress testing shows that the hit rate has increased from 89 to 95, and the tail delay has decreased by 30%. After the transaction hash is published, the external audit node calls the verification key for inspection, and the verification passes indicate that the gradient source and differential privacy budget meet the constraints, and the entire chain data and the model remain in a consistent state.
[0098] Preferably, the distributed model training is based on a federated learning framework, and after updating the sequence prediction network locally at each cache node, the parameter aggregation is completed through the secure aggregation service, and after the parameter aggregation is completed, the digest generated by the model hash and the root commitment is written into the distributed ledger.
[0099] After the completion of the convergence state mapping and the sharding migration, the invention introduces a federated prefetch mechanism, learns the access sequence locally at each cache node through distributed model training, and loads the predicted key set in advance to the slave node cache, thereby significantly reducing the first read delay; at the same time, the encryption aggregation and zero-knowledge proof are used to ensure that the model update process and the final result can be verified. This step contains five consecutive links: input extraction, local training, ciphertext aggregation, model verification and data closed loop.
[0100] Firstly, after the cache node completes the hotness evaluation, the latest key hotness vector and the root commitment of the last round are taken as the model input. The key hotness vector represents the usage trend of each key in the future time window in the semantic, and is used as the context of the sequence prediction network; the root commitment is used as an integrity label to ensure that the model generated subsequently can be bound to the current log version.
[0101] The local training stage adopts a sequence-to-sequence structure: the encoder uses a gated recurrent unit to extract short-term time sequence features (the network width is set to 32 channels, and the hidden layer depth is not more than 2 layers), and the decoder predicts the probability distribution of the most likely access key in the future window. After completing one round of back propagation, the gradient tensor is obtained. If the gradient is directly uploaded, the malicious server may deduce the access distribution, causing information leakage; therefore, the invention performs homomorphic encryption on the gradient locally.
[0102] The node first multiplies the floating-point gradient by an amplification factor and takes the integer to obtain the quantized gradient, and then encodes it into polynomial coefficients, and then uses the public key to perform modular exponential operation to generate ciphertext . This encryption scheme satisfies the additive closure property:
[0103] To meet the differential privacy constraint, the node injects noise in the ciphertext domain. The noise obeys the Gaussian distribution with mean and standard deviation ; the sensitivity and privacy budget are calculated. The is added to the gradient ciphertext to obtain the perturbed ciphertext:
[0104] Then uploaded to the secure aggregation service.
[0105] The secure aggregation service runs in a trusted execution environment. The service collects ciphertexts from nodes and sums them up:
[0106] According to the homomorphic property:
[0107] The service loads the private key within the trusted hardware. Decryption to obtain the noisy global gradient Then update the distributed model weights with a fixed learning rate. After the update is complete, calculate the model weight hash value. To bind the model and data versions, Compared with the previous round of root commitments After concatenation, a digest is generated using a secure hash function.
[0108] summary This proves that the model version corresponds to the data version without revealing the weight details.
[0109] Entering the model validity verification stage: Zero-knowledge proof circuit public input , With Abstract Private input is a noisy global gradient. Compared with the old weight The circuit is replayed and the weights are updated. After verifying that all hash and digest formulas hold true, the proof is output. This proves that private input is not leaked, and any third party holding the verification key can verify whether the aggregation and privacy budget are compliant.
[0110] The system will With Abstract The transaction is written to the distributed ledger. After confirmation, a tamper-proof record is created; simultaneously, the aggregation service broadcasts a summary via the event bus. The log building module will put it in the next cycle The root commitment is written into the metadata to synchronize model and data versions.
[0111] After the new weights are assigned, the nodes run inference and output the probability distribution. The 128 keys with the highest probabilities are then selected and preheated on slave nodes via the MGET pipeline. Online testing shows a significant improvement in first-read hit rate and a marked decrease in tail latency.
[0112] Example 10: 8 nodes, gradient dimension of each node is 64. Differential privacy parameters are set to... , After aggregation, a new [product / process] is obtained. The digest d1 is generated by combining the root commitment c0. The proof size is approximately 20 kilobytes, and the on-chain processing time is 2 seconds. The inference accuracy is 83%, the first-hit rate has increased from 89% to 95%, and the read latency has decreased by 30% at the 99th percentile. The on-chain verification tool returns "proofverified," proving that the model update is legitimate and meets the privacy budget, thus completing the closed loop.
[0113] As Figure 3 shown, a Redis cache management apparatus for implementing the Redis cache management method comprises: An event generation module is configured to intercept database write operations at an object-relational mapping layer, extract entity identifiers, operation types, changed fields, data schema versions, and hybrid logical clock time stamps, encapsulate them as digitally signed domain events, generate random challenge values, and submit the domain events and corresponding database transactions to an event bus through a distributed transaction; the event generation module runs on an application node, relies on a multi-core server processor to perform object-relational mapping layer interception and digital signature calculation, an on-board trusted platform module provides elliptic curve signature acceleration and generates random challenge values in cooperation with a hardware random number generator; a gigabit or gigabit Ethernet controller writes domain events and database transactions to the same transaction bus through remote direct memory access, allowing them to be submitted at the same PCIe path once, reducing I / O round-trip delay.
[0114] A log construction module is configured to aggregate the domain events into event blocks in the order of receipt and write them to a consistent data log, calculate an integrity commitment for the event blocks and generate a root commitment, write the root commitment and the random challenge value to log metadata, and copy the event blocks to decentralized storage; the log construction module is deployed on a data lake node, a RAID10 array composed of NVMe provides large bandwidth sequential writing, a programmable FPGA card is used to calculate polynomial commitments of the event blocks in parallel to reduce CPU load, and a dual-port 25Gbps network card is used to copy the event blocks to the decentralized storage without copying; the root commitment and the random challenge value are written to the log metadata synchronously, ensuring that the integrity can be directly verified when any node loads the log.
[0115] A projection governance module is configured to verify the integrity of the event log based on the root commitment and the random challenge value at a cache node, sequentially consume the event blocks after verification, generate a converged state map using a conflict-free replicated data type merge algorithm to write to Redis, and perform shard migration and refresh the random challenge value according to real-time key hotness; the projection governance module is located at the cache node, uses a high-frequency CPU and a large-capacity DDR5 memory to do memory mapping on log files and uses a single-instruction multiple-data instruction set to verify the commitment; a SmartNIC embedded with eBPF functionality counts key access frequency and latency on the kernel bypass surface, a PCIe4.0 NVMe solid state disk persists AOF snapshots and hotness vectors; the node commands the Redis cluster to perform slot migration according to the hotness threshold, and after the migration is completed, the processor calculates a new random challenge value and writes it back to the metadata.
[0116] The pre-fetch verification module is configured to train the prediction of the pre-fetch key set according to the key hotness and the root commitment in a distributed model, preheat the cache copy, participate in the secure aggregation after the homomorphic encryption of the local gradient and the addition of the differential privacy noise to update the distributed model parameters, generate the digest of the updated model hash and the root commitment, write the digest into the distributed ledger through the verifiable proof mechanism, and synchronize the digest to the data log to close the integrity, governance and pre-fetch cycle. The pre-fetch verification module is distributed in each cache node and the central aggregation server: the node end can be optionally connected to the entry-level GPU accelerated sequence network inference, the homomorphic encrypted gradient is executed by the hardware acceleration card to perform large integer operation, and then the ciphertext is uploaded to the secure aggregation server; the aggregation server is built-in trusted execution memory encapsulated with a private key, completes the ciphertext summation, decryption and weight update, and writes the model hash and the root commitment digest into the blockchain through the FPGA network card high-speed signature to form a hardware-level secure closed loop.
[0117] A computer device includes a processor and a storage medium; The storage medium is configured to store instructions; and The processor is configured to operate according to the instructions to perform the steps of the method of Redis cache management.
[0118] A storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the method of Redis cache management.
[0119] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.
[0120] The above is only an embodiment of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.
Claims
1. A method for Redis cache management, the method comprising: The application comprises: intercepting database write operations at the object-relational mapping layer, extracting entity identifiers, operation types, changed fields, data schema versions, and hybrid logical clock timestamps, packaging them as digitally signed domain events, generating random challenge values, and submitting the domain events and corresponding database transactions to an event bus through a distributed transaction; aggregating the domain events into event blocks in the order of reception and writing them into a consistent data log, calculating integrity commitments for the event blocks and generating a root commitment, writing the root commitment and the random challenge value into log metadata, and replicating the event blocks to a decentralized storage; verifying the event log based on the root commitment and the random challenge value at a cache node, consuming the event blocks through post-consumption, generating a converged state map using a conflict-free replicated data type merge algorithm and writing it into Redis, and performing shard migration and random challenge value refreshing based on real-time key hotness; training a distributed model to predict a set of prefetch keys based on key hotness and root commitment and preheating the cache replica; homomorphically encrypting local gradients and adding differential privacy noise to participate in secure aggregation to update distributed model parameters, generating a digest of the updated model hash and root commitment, writing it into a distributed ledger through a verifiable proof mechanism, and synchronizing the digest to the data log closed loop.
2. The method of claim 1, wherein, The domain events are aggregated into an event block every preset number in the order of continuous submission, and the start submission number and the end submission number are recorded for each event block before writing into the consistent data log.
3. The method of claim 1, wherein, The integrity commitment is obtained by polynomial mapping of all event hashes in an event block to obtain a block-level commitment, and then polynomial aggregation of all block-level commitments in the order of event block index to generate the root commitment.
4. The method of claim 1, wherein, The conflict-free replicated data type merge algorithm covers the older modification with the newer modification in the hybrid logical clock timestamp when there is concurrent modification in the same key, and maintains the union set for the fields that are not covered.
5. The method of claim 1, wherein, The key hotness is obtained by multiplying the key access frequency and the key access latency by the weight respectively and then summing them up, and the shard migration is triggered when the key hotness reaches a threshold, and a new random challenge value is calculated from the updated key hotness vector through a secure hash function.
6. The method of claim 1, wherein, The local training gradient is first homomorphically encrypted, then noise satisfying the differential privacy constraint is added, and then the encrypted gradient is uploaded to the secure aggregation service for ciphertext summation and decryption to obtain the global gradient to update the distributed model parameters.
7. The method of claim 1, wherein, The distributed model training is based on a federated learning framework, and after updating the sequence prediction network locally at each cache node, the parameter aggregation is completed through the secure aggregation service, and after the parameter aggregation is completed, the digest of the model hash and the root commitment is written into the distributed ledger.
8. A Redis cache management apparatus for implementing the Redis cache management method of any one of claims 1-7, characterized in that, The application comprises: an event generation module for intercepting database write operations at the object-relational mapping layer, extracting entity identifiers, operation types, changed fields, data schema versions, and hybrid logical clock timestamps, packaging them as digitally signed domain events and generating random challenge values, and submitting the domain events and corresponding database transactions to an event bus through a distributed transaction; a log building module for aggregating the domain events into event blocks in receiving order and writing a consistent data log, computing integrity commitments for the event blocks and generating a root commitment, writing the root commitment and a random challenge value into log metadata, and copying the event blocks to a decentralized storage; a projection governance module for verifying event log integrity based on the root commitment and the random challenge value at a cache node, sequentially consuming the event blocks after verification, generating a converged state map using a conflict-free replicated data type merge algorithm to write to Redis, and performing shard migration and refreshing the random challenge value according to real-time key hotness; a prefetch verification module for training a prediction of a prefetch key set according to key hotness and the root commitment using a distributed model, preheating a cache replica, participating in secure aggregation to update distributed model parameters after homomorphic encryption of local gradients and addition of differential privacy noise, generating a digest of the updated model hash and the root commitment, writing the digest to a distributed ledger through a verifiable proof mechanism, and synchronizing the digest to the data log to close the integrity, governance, and prefetch cycle.
9. A computer device, comprising: comprise a processor and a storage medium; the storage medium is configured to store instructions; and the processor is configured to operate according to the instructions to perform the steps of the Redis cache management method of any one of claims 1-7.
10. A storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the steps of the Redis cache management method of any one of claims 1-7.
Citation Information
Patent Citations
Cache data semi-synchronization method
CN117874129A
Health data processing method and system based on distributed account book library
CN120126655A
