Incremental CRC Update for Constant-Time Packet Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The calculation of cyclic redundancy checks (CRC) for data packets can become a bottleneck due to linear asymptotic complexity and requires access to the packet payload, leading to CPU cache congestion and increased latency.
Innovation Solution
An incremental CRC update method that performs CRC calculations in constant time by utilizing the existing CRC value in the packet and performing XOR operations on header and payload size-related values, without relying on the actual payload data, thus avoiding cache overload.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional CRC calculation is performed on the entire packet payload, then accurate error detection is achieved, but the calculation complexity increases linearly with packet size and CPU cache becomes congested
Solution Approach 1:
The patent segments the packet into header and payload portions, calculating CRC independently for each segment. The header CRC is calculated once and reused for all packets with identical headers, while only the payload CRC needs to be recalculated for each packet. This segmentation reduces the overall calculation complexity from linear O(n) to constant O(1) for the header portion.
Solution Approach 2:
The patent performs preliminary CRC calculation on the packet header before processing the payload. This preliminary header CRC is stored and reused for subsequent packets with the same header, eliminating the need to recalculate it repeatedly. This preliminary action significantly reduces the cumulative CRC calculation burden in high-throughput scenarios.
2Measurement precision
If the entire packet payload is loaded to CPU cache for CRC calculation, then complete CRC accuracy is achieved, but cache congestion increases and latency is introduced
Solution Approach 1:
The patent extracts the header portion from the complete packet and processes it separately from the payload. By taking out the header CRC calculation and handling it independently with precomputation, the method eliminates the need to load the entire packet into CPU cache, thereby reducing cache congestion and associated latency while maintaining CRC accuracy through separate payload processing.
3Productivity
If multiple matrix multiplications are performed for incremental CRC update as described by Mark Adler, then logarithmic time complexity is achieved, but extra CPU cache usage is required
Solution Approach 1:
The patent uses simple XOR operations and lookup tables instead of complex matrix multiplications for incremental CRC updates. This approach uses inexpensive, lightweight computational operations that can be performed with minimal CPU cache usage, replacing the more resource-intensive matrix multiplication method while achieving comparable or better performance for the specific use case of header reuse.
Data Source
AI summary
Performing a constant time cyclic redundancy check (CRC) over an entire packet to obtain a constant time CRC value. A first CRC is performed on an original header of the packet and a second CRC is performed on a modified header of the packet. The size of the payload of the packet is obtained. An XOR operation is performed on the results of the first and second CRC to calculate a third result. An intermediate CRC value is obtained by performing a CRC on a number of zero values corresponding to the size of the payload using the third result as an initial value. The intermediate CRC value may be employed with other packets having a same size and same header as the packet. The constant time CRC value is obtained by performing an XOR operation on the intermediate CRC value and the original CRC value contained in the packet.