A method and apparatus for enhancing lossless compression of video
Patent Information
- Application Number
- CN202611110957.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-24
- Publication Date
- 2026-10-30
AI Technical Summary
一旦因视频压缩系统故障导致视频中断或丢失，进而可能造成关键时间节点行为链断裂，影响案件侦破效率
本发明的增强视频无损压缩方法及装置，其通过在发送端缓存带有唯一标识的原始数据帧，并将缓存释放时机延迟至接收到下一数据分组的起始帧时执行，使得在第二节点发生故障时，缓存队列中始终保留有当前数据分组完整的原始数据帧序列。当检测到异常时，系统能够从当前数据分组的起始帧开始回退输出原始数据帧，替代残缺的处理后数据帧。这种基于数据分组边界的延迟清理与故障回退协同机制，一方面利用原始数据帧填补了故障期间的数据空缺，保证了数据流的连续性；另一方面，通过回退至分组起始帧进行输出，确保了输出数据符合编码依赖结构，避免了因帧间参考缺失导致的解码花屏现象，从而实现了高可靠性的无损容错传输。
Smart Images
Figure CN123020804A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data processing technology, specifically relating to a method and apparatus for enhancing lossless video compression. Background Technology
[0002] With the surge in the number of video acquisition terminals in scenarios such as security monitoring, mobile live streaming, and industrial inspection, and the widespread adoption of ultra-high-definition video applications, traditional transmission networks and storage infrastructure face severe challenges. On the one hand, high-resolution video streams, such as 4K / 8K, lead to an exponential increase in transmission bandwidth demand, resulting in frequent network congestion and latency issues. On the other hand, the long-term retention requirements for massive amounts of video data keep storage costs high, especially given legal requirements that surveillance videos in key areas must be retained for more than 90 days, further escalating the cost of storing massive amounts of raw video data. To address the bottlenecks in transmission bandwidth and storage resources across industries, mature video compression system solutions are widely adopted. These solutions utilize intelligent compression algorithms based on inter-frame prediction / transform coding to achieve relatively high compression ratios while maintaining the integrity of critical information, effectively controlling bitstream expansion while optimizing storage TCO.
[0003] After introducing a video compression system, common deployment schemes include front-end deployment and side-mounted deployment. The video compression system is located in the middle of the link between the camera, video platform, and storage system, and its reliability has a significant impact on video forwarding and storage. Stable and uninterrupted video streaming is of particular practical importance, especially in judicial evidence collection and public safety incident investigations, where surveillance video is crucial electronic evidence with legal validity. If video compression system failure leads to video interruption or loss, it may cause a break in the chain of actions at critical time points, affecting the efficiency of case investigation. Therefore, very high reliability requirements are placed on the video compression system, and video interruption due to video compression system failure must be avoided as much as possible. Summary of the Invention
[0004] In view of one or more of the above-mentioned defects or improvement needs of the prior art, the present invention provides an enhanced lossless video compression method and apparatus, which are effective.
[0005] To achieve the above objectives, the present invention provides an enhanced lossless video compression method, comprising the following steps: Obtain the original data frame, assign a unique identifier to the original data frame, and store the original data frame with the unique identifier into the cache queue; The original data frame is sent to the second node for processing, and the processed data frame with a corresponding unique identifier is received from the second node. In response to receiving the processed data frame, the state of the corresponding original data frame in the buffer queue is updated, and the processed data frame is output, wherein the buffer release of the original data frame is delayed until the start frame belonging to the next data packet is received; In response to the detection of an anomaly in the second node, the start frame and subsequent raw data frames of the current data packet are retrieved from the cache queue and output.
[0006] As a further preferred embodiment of the present invention, the buffer release of the original data frame is delayed until the start frame belonging to the next data packet is received, including the following steps: Parse the header information of the received data frame and extract the frame type identifier; When the frame type identifier indicates that the current data frame is the starting frame of the next data packet, the cache release of all original data frames belonging to the previous data packet in the cache queue is triggered.
[0007] As a further preferred embodiment of the present invention, the data group is a group of images (GOP), the starting frame is a key frame (I-frame), and the non-starting frames in the data group are prediction frames (P-frames) or B-frames. The process of parsing the header information of the received data frame and extracting the frame type identifier includes the following steps: Read the NALU header information or slice header information under the video coding standard protocol to obtain the frame type field; When the frame type field indicates that the current data frame is an I-frame, the current data frame is determined to be the starting frame of the next data packet; The step of triggering the release of the cache for all original data frames belonging to the previous data group in the cache queue includes the following steps: Traverse the cache queue, find the original data frame whose frame type is identified as I-frame, P-frame or B-frame and whose status has been updated to sent, and call the memory reclamation interface to release the memory space occupied by the original data frame.
[0008] As a further preferred embodiment of the present invention, the predicted frame P or B frame depends on the reference data of the key frame I during decoding. By delaying the release of the buffer of the original data frame of the key frame I and its associated predicted frame, it is ensured that the complete image group can be output back in case of the second node failure to avoid decoding screen distortion.
[0009] As a further preferred embodiment of the present invention, the step of retrieving the start frame and subsequent raw data frames of the current data packet from the cache queue and outputting them in response to detecting an anomaly in the second node includes the following steps: In response to detecting at least one of heartbeat timeout, connection interruption, or response timeout, the second node is determined to be abnormal; Discard the processed data frames already received within the current data packet; The data output channel is switched from outputting the processed data frame to outputting the original data frame, and the original data frames are output sequentially starting from the start frame of the current data group.
[0010] As a further preferred embodiment of the present invention, after switching the data output channel from outputting the processed data frame to outputting the original data frame, the method further includes the following steps: Continuously send probe messages to the second node and monitor the response status of the second node; When a normal response message is received from the second node, it is determined that the second node has returned to normal. In response to the second node returning to normal, the data transmission connection with the second node is re-established, and the data output channel is switched back from outputting the original data frame to outputting the processed data frame. After switching back to the data output channel, the remaining original data frames in the buffer queue are cleared to release memory resources.
[0011] As a further preferred embodiment of the present invention, clearing the original data frames remaining in the cache queue includes the following steps: obtaining the current head pointer and tail pointer of the cache queue, traversing all nodes between the head pointer and the tail pointer, returning the original data frame memory block pointed to by the node to the system memory pool, and resetting the head pointer and tail pointer to their initial state.
[0012] As a further preferred embodiment of the present invention, the method further includes the following steps: If no anomaly is detected in the second node, monitor the current length of the cache queue; When the current length exceeds a preset threshold, the original data frame is forcibly popped from the head of the cache queue and output, while the cache of the popped original data frame is released.
[0013] As a further preferred embodiment of the present invention, the step of assigning a unique identifier to the original data frame and storing the original data frame with the unique identifier into a cache queue includes the following steps: Obtain the high-precision timestamp or incrementing sequence number of the current system and use it as the unique identifier of the original data frame; Use the unique identifier as the key and the memory address and frame state information of the original data frame as the value to construct a key-value pair; The key-value pairs are inserted into the cache queue, which is implemented using a hash table or a doubly linked list structure. The step of updating the state of the corresponding original data frame in the buffer queue in response to receiving the processed data frame includes the following steps: The unique identifier carried in the processed data frame is parsed, and a lookup with O time complexity is performed in the hash table using the key value, or a matching is performed by traversing the nodes in the doubly linked list; When a matching node is found, the frame status information in the node is updated from the pending state to the sent state, but the memory release operation is not performed until the delayed triggering condition for cache release is met.
[0014] An enhanced video lossless compression apparatus is also disclosed, which employs the aforementioned enhanced video lossless compression method, and includes: The cache management module is used to obtain the original data frame, assign a unique identifier to the original data frame, and store the original data frame with the unique identifier into the cache queue. The data transceiver module is used to send the original data frame to the second node for processing, and to receive the processed data frame with a corresponding unique identifier returned by the second node. A state control module is used to update the state of the corresponding original data frame in the buffer queue in response to receiving the processed data frame, and output the processed data frame, wherein the buffer release of the original data frame is delayed until the start frame belonging to the next data packet is received; The anomaly degradation module is used to retrieve the start frame and subsequent raw data frames of the current data packet from the cache queue and output them in response to the detection of an anomaly in the second node.
[0015] In summary, the beneficial effects of the above-described technical solutions conceived by this invention compared with the prior art include: The enhanced lossless video compression method and apparatus of the present invention buffers original data frames with unique identifiers at the sending end and delays the buffer release until the start frame of the next data packet is received. This ensures that even if a second node fails, the buffer queue always retains a complete sequence of original data frames for the current data packet. When an anomaly is detected, the system can backtrack from the start frame of the current data packet and output original data frames to replace the incomplete processed data frames. This collaborative mechanism of delayed cleanup and fault backtracking based on data packet boundaries, on the one hand, uses original data frames to fill data gaps during faults, ensuring the continuity of the data stream; on the other hand, by backtracking to the start frame of the packet for output, it ensures that the output data conforms to the coding dependency structure, avoiding decoding screen distortion caused by missing inter-frame references, thereby achieving highly reliable lossless fault-tolerant transmission. Attached Figure Description
[0016] Figure 1 This is a flowchart of the enhanced video lossless compression method in an embodiment of the present invention; Detailed Implementation To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0017] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0018] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0019] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0020] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.
[0021] Example: Please see Figure 1 In a preferred embodiment of the present invention, the enhanced video lossless compression method and apparatus are applied to a first node in a data processing system to achieve fault tolerance during data transmission between the first node and a second node.
[0022] It's just a matter of specifics, such as... Figure 1 As shown in the preferred embodiment of this application, the enhanced video lossless compression method includes the following steps: Obtain the raw data frame, assign a unique identifier to the raw data frame, and store the raw data frame with the unique identifier into the buffer queue. The original data frame is sent to the second node for processing, and the processed data frame with a corresponding unique identifier is received from the second node. In response to receiving a processed data frame, update the state of the corresponding original data frame in the buffer queue and output the processed data frame; The release of the buffer for the original data frame is delayed until the start frame belonging to the next data packet is received; In response to the detection of an anomaly in the second node, the start frame and subsequent raw data frames of the current data packet are retrieved from the cache queue and output.
[0023] Furthermore, in a preferred embodiment of this application, the buffer release of the original data frame is delayed until the start frame belonging to the next data packet is received, including the following steps: Parse the header information of the received data frame and extract the frame type identifier; When the frame type identifier indicates that the current data frame is the starting frame of the next data packet, the buffer release of all raw data frames belonging to the previous data packet in the buffer queue is triggered.
[0024] More preferably, in the preferred embodiment of this application, the data group is a group of images (GOP), the starting frame is a key frame (I-frame), and the non-starting frames in the data group are prediction frames (P-frames or B-frames). Parsing the header information of the received data frame and extracting the frame type identifier includes the following steps: Read the NALU header information or slice header information under the video coding standard protocol to obtain the frame type field; When the frame type field indicates that the current data frame is an I-frame, the current data frame is determined to be the starting frame of the next data packet; Triggering the release of the cache for all raw data frames belonging to the previous data packet in the cache queue includes the following steps: Traverse the cache queue, find the original data frame whose frame type is identified as I-frame, P-frame, or B-frame and whose status has been updated to "sent", and call the memory reclamation interface to release the memory space occupied by the original data frame.
[0025] More specifically, in a preferred embodiment of this application, the predicted frame P or B frame depends on the reference data of the key frame I during decoding. By delaying the release of the buffer of the original data frame of the key frame I and its associated predicted frame, it is ensured that the complete image group can be output back in case of a second node failure to avoid decoding screen distortion.
[0026] Furthermore, in a preferred embodiment of this application, in response to detecting an anomaly in the second node, retrieving the start frame and subsequent raw data frames of the current data packet from the cache queue and outputting them includes the following steps: In response to detecting at least one of heartbeat timeout, connection interruption, or response timeout, determine that the second node is abnormal; Discard the processed data frames already received within the current data packet; Switch the data output channel from outputting processed data frames to outputting raw data frames, and output raw data frames sequentially starting from the beginning frame of the current data group.
[0027] More preferably, in a preferred embodiment of this application, after switching the data output channel from outputting processed data frames to outputting raw data frames, the method further includes the following steps: Continuously send probe messages to the second node and listen to the response status of the second node; When a normal response message is received from the second node, it is determined that the second node has returned to normal. In response to the second node returning to normal, the data transmission connection with the second node is re-established, and the data output channel is switched from outputting raw data frames back to outputting processed data frames. After switching back to the data output channel, clear any remaining raw data frames in the buffer queue to free up memory resources.
[0028] More specifically, in a preferred embodiment of this application, clearing the remaining original data frames in the cache queue includes the following steps: obtaining the current head pointer and tail pointer of the cache queue, traversing all nodes between the head pointer and the tail pointer, returning the original data frame memory block pointed to by the node to the system memory pool, and resetting the head pointer and tail pointer to their initial state.
[0029] Furthermore, in a preferred embodiment of this application, the method further includes the following steps: If no anomaly is detected in the second node, monitor the current length of the cache queue; When the current length exceeds the preset threshold, the original data frame is forcibly popped from the head of the cache queue and output, while the cache of the popped original data frame is released.
[0030] Further preferably, in the preferred embodiment of this application, assigning a unique identifier to the original data frame and storing the original data frame with the unique identifier in the buffer queue includes the following steps: Obtain the high-precision timestamp or incrementing sequence number of the current system and use it as the unique identifier of the original data frame; Use the unique identifier as the key and the memory address and frame state information of the original data frame as the value to construct a key-value pair; Insert key-value pairs into a cache queue implemented using a hash table or a doubly linked list structure; In response to receiving a processed data frame, update the state of the corresponding original data frame in the buffer queue, including: The unique identifier carried by the parsed data frame is used to perform a lookup in the hash table using the key-value pair in O time complexity, or to match the data by traversing the nodes in the doubly linked list. When a matching node is found, the frame status information in the node is updated from pending to sent, but the memory release operation is not performed until the delayed trigger condition for cache release is met.
[0031] Furthermore, in a preferred embodiment of this application, an enhanced video lossless compression device is also disclosed, which employs the above-mentioned enhanced video lossless compression method and includes a cache management module, a data transceiver module, a status control module, and an anomaly degradation module.
[0032] The cache management module is used to acquire raw data frames, assign unique identifiers to them, and store the raw data frames with unique identifiers in the cache queue. The data transceiver module is used to send raw data frames to the second node for processing and receive the processed data frames with corresponding unique identifiers returned by the second node. The status control module is used to update the status of the corresponding raw data frame in the cache queue and output the processed data frame in response to receiving the processed data frame. The cache release of the raw data frame is delayed until the start frame of the next data packet is received. The exception degradation module is used to acquire the start frame of the current data packet and subsequent raw data frames from the cache queue and output them in response to detecting an exception in the second node.
[0033] The enhanced lossless video compression method and apparatus of this invention buffers original data frames with unique identifiers at the sending end and delays the buffer release until the start frame of the next data packet is received. This ensures that even if a second node fails, the buffer queue always retains a complete sequence of original data frames for the current data packet. When an anomaly is detected, the system can backtrack from the start frame of the current data packet and output original data frames to replace the incomplete processed data frames. This collaborative mechanism of delayed cleanup and fault backtracking based on data packet boundaries, on the one hand, uses original data frames to fill data gaps during faults, ensuring the continuity of the data stream; on the other hand, by backtracking to the start frame of the packet for output, it ensures that the output data conforms to the coding dependency structure, avoiding decoding screen distortion caused by missing inter-frame references, thereby achieving highly reliable lossless fault-tolerant transmission.
[0034] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for enhancing lossless video compression, characterized in that, Includes the following steps: Obtain the original data frame, assign a unique identifier to the original data frame, and store the original data frame with the unique identifier into the cache queue; The original data frame is sent to the second node for processing, and the processed data frame with a corresponding unique identifier is received from the second node. In response to receiving the processed data frame, the state of the corresponding original data frame in the buffer queue is updated, and the processed data frame is output. The release of the buffer for the original data frame is delayed until the start frame belonging to the next data packet is received; In response to the detection of an anomaly in the second node, the start frame and subsequent raw data frames of the current data packet are retrieved from the cache queue and output.
2. The enhanced video lossless compression method according to claim 1, characterized in that, The buffer release of the original data frame is delayed until the start frame belonging to the next data packet is received, and includes the following steps: Parse the header information of the received data frame and extract the frame type identifier; When the frame type identifier indicates that the current data frame is the starting frame of the next data packet, the cache release of all original data frames belonging to the previous data packet in the cache queue is triggered.
3. The enhanced video lossless compression method according to claim 2, characterized in that, The data group is a group of images (GOP), the starting frame is a key frame (I-frame), and the non-starting frames in the data group are prediction frames (P-frames) or B-frames. The process of parsing the header information of the received data frame and extracting the frame type identifier includes the following steps: Read the NALU header information or slice header information under the video coding standard protocol to obtain the frame type field; When the frame type field indicates that the current data frame is an I-frame, the current data frame is determined to be the starting frame of the next data packet; The step of triggering the release of the cache for all original data frames belonging to the previous data group in the cache queue includes the following steps: Traverse the cache queue, find the original data frame whose frame type is identified as I-frame, P-frame or B-frame and whose status has been updated to sent, and call the memory reclamation interface to release the memory space occupied by the original data frame.
4. The enhanced video lossless compression method according to claim 3, characterized in that, The predicted P-frame or B-frame relies on the reference data of the key frame I-frame during decoding. By delaying the release of the buffer of the original data frames of the key frame I-frame and its associated predicted frames, it is ensured that the complete image group can be output back in case of an anomaly in the second node to avoid decoding screen distortion.
5. The enhanced video lossless compression method according to any one of claims 1 to 4, characterized in that, The response to detecting an anomaly in the second node, retrieving the start frame and subsequent raw data frames of the current data packet from the cache queue and outputting them, includes the following steps: In response to detecting at least one of heartbeat timeout, connection interruption, or response timeout, the second node is determined to be abnormal; Discard the processed data frames already received within the current data packet; The data output channel is switched from outputting the processed data frame to outputting the original data frame, and the original data frames are output sequentially starting from the start frame of the current data group.
6. The enhanced video lossless compression method according to claim 5, characterized in that, After switching the data output channel from outputting the processed data frame to outputting the original data frame, the method further includes the following steps: Continuously send probe messages to the second node and monitor the response status of the second node; When a normal response message is received from the second node, it is determined that the second node has returned to normal. In response to the second node returning to normal, the data transmission connection with the second node is re-established, and the data output channel is switched back from outputting the original data frame to outputting the processed data frame. After switching back to the data output channel, the remaining original data frames in the buffer queue are cleared to release memory resources.
7. The enhanced video lossless compression method according to claim 6, characterized in that, The process of clearing the remaining original data frames in the cache queue includes the following steps: obtaining the current head pointer and tail pointer of the cache queue, traversing all nodes between the head pointer and the tail pointer, returning the original data frame memory block pointed to by the node to the system memory pool, and resetting the head pointer and tail pointer to their initial states.
8. The enhanced video lossless compression method according to any one of claims 1 to 4, characterized in that, The method further includes the following steps: If no anomaly is detected in the second node, monitor the current length of the cache queue; When the current length exceeds a preset threshold, the original data frame is forcibly popped from the head of the cache queue and output, while the cache of the popped original data frame is released.
9. The enhanced video lossless compression method according to claim 8, characterized in that, Assigning a unique identifier to the original data frame and storing the original data frame with the unique identifier in a cache queue includes the following steps: Obtain the high-precision timestamp or incrementing sequence number of the current system and use it as the unique identifier of the original data frame; Use the unique identifier as the key and the memory address and frame state information of the original data frame as the value to construct a key-value pair; The key-value pairs are inserted into the cache queue, which is implemented using a hash table or a doubly linked list structure. The step of updating the state of the corresponding original data frame in the buffer queue in response to receiving the processed data frame includes: The unique identifier carried in the processed data frame is parsed, and a lookup with O time complexity is performed in the hash table using the key value, or a matching is performed by traversing the nodes in the doubly linked list; When a matching node is found, the frame status information in the node is updated from the pending state to the sent state, but the memory release operation is not performed until the delayed triggering condition for cache release is met.
10. An enhanced lossless video compression device, characterized in that, The enhanced video lossless compression method according to any one of claims 1 to 9 includes: The cache management module is used to obtain the original data frame, assign a unique identifier to the original data frame, and store the original data frame with the unique identifier into the cache queue. The data transceiver module is used to send the original data frame to the second node for processing, and to receive the processed data frame with a corresponding unique identifier returned by the second node. A state control module is used to update the state of the corresponding original data frame in the buffer queue in response to receiving the processed data frame, and output the processed data frame, wherein the buffer release of the original data frame is delayed until the start frame belonging to the next data packet is received; The anomaly degradation module is used to retrieve the start frame and subsequent raw data frames of the current data packet from the cache queue and output them in response to the detection of an anomaly in the second node.