A simple and efficient multi-link data deduplication method
A multi-link and link technology, applied in the field of data processing, can solve problems such as inaccurate deduplication, affecting business software processing results, and incomplete data
Active Publication Date: 2022-02-25
BEIJING CHERILEAD TECH
View PDF5 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
However, based on the three-level redundant transmission, when the receiving end performs data reorganization, it will also receive duplicate fragmented data. Too many redundant fragments will occupy more memory and occupy memory garbage collection processing time.
In more extreme cases, in order to ensure the maximum reliability of transmission, the sending end of multiple links may make multiple copies of the original user data and send them to the receiving end of the multi-link device. If these data packets are sent to the service port of the receiving end Output, on the one hand, increases the burden of post-processing, on the other hand, it may also affect the results of business software processing
[0003] In the prior art, the redundant data obtained by the receiving end has the problems of incomplete data deduplication, low deduplication efficiency and inaccurate deduplication. Based on inaccurate deduplication, the reorganized data will be incomplete, resulting in system based on Error control made by incomplete data affects the security and reliability of the system
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0059] The preferred embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention.
[0060] like figure 1 As shown, the embodiment of the present invention proposes a simple and efficient multi-link data deduplication method, including steps S1-S4:
[0061] S1. Obtain several link data packets received by the receiving end:
[0062] S2. Determine and classify the types of several link data packets respectively, and determine them as a first link data packet set and a second link data packet set; the first link data packet set includes a plurality of first link A data packet, the first link data packet is link fragment data; the second link data packet set includes a plurality of second link data packets, and the second link data packet is link comp...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a simple and efficient multi-link data deduplication method, which includes: obtaining several link data packets received by the receiving end; respectively determining and classifying the types of several link data packets, and determining them as the first link data packets A set of road data packets and a second set of link data packets; the first set of link data packets includes a plurality of first link data packets, and the first link data packets are link fragment data; the second The link data packet set includes a plurality of second link data packets, and the second link data packet is link complete data; respectively judge whether the first link data packet in the first link data packet set is redundant Packets, screen out the first link data packets that are redundant packets and perform deduplication processing; respectively judge whether the second link data packets in the second link data packet set are redundant packets, and filter out redundant packets The second link data packet is deduplicated. Improved deduplication efficiency.
Description
technical field [0001] The invention relates to the technical field of data processing, in particular to a simple and efficient multi-link data deduplication method. Background technique [0002] With the continuous development of equipment redundancy technology, it is widely used in the field of communication. In the process of data communication, in order to ensure that the link can achieve the most reliable data transmission, the multi-link device will perform three-level redundant transmission when transmitting the original user data, so as to ensure the integrity of the data received by the receiving end. Avoid data incompleteness caused by packet loss during data transmission. However, based on the three-level redundant transmission, when the receiving end performs data reorganization, it will also receive duplicate fragmented data. Too many redundant fragments will occupy more memory and occupy memory garbage collection processing time. In more extreme cases, in ord...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): H04L47/31H04L47/32
CPCH04L47/31H04L47/32
Inventor 张凯郑应强刘同鹤
Owner BEIJING CHERILEAD TECH