File transmission method based on Reed-Solomon coding

Abstract

The invention provides a file transmission method based on Reed-Solomon coding. The file transmission method comprises the following steps: carrying out file partitioning and preprocessing; and carrying out improved Reed-Solomon coding on the subpackage content of the file, wherein the improved Reed-Solomon coding comprises the following steps of: selecting a corresponding coding standard according to the bit error rate of a link; performing length filling on the sub-package content of the file; splitting into a plurality of standard length sub-segments according to a coding standard, and generating independent ECC check codes; for all the ECC check codes, jointly generating an association code according to byte XOR; integrating to obtain a coded data packet, and sending the coded data packet from a sending end to a receiving end; decoding is carried out, and Reed-Solomon cooperative decoding is carried out at least according to the association code; and multi-level verification and splicing are carried out. According to the file transmission method based on Reed-Solomon coding, the problems of low file transmission reliability and poor efficiency in an unstable network environment can be solved.

Description

Technical Field

[0001] This invention relates to the field of satellite communication technology, and specifically to a file transfer method based on Reed-Solomon encoding. Background Technology

[0002] In satellite communications, file transfer is a crucial link in data exchange, and its reliability and efficiency directly affect the effectiveness of various aerospace applications.

[0003] Traditional file transfer methods rely primarily on retransmission mechanisms (such as TCP timeout retransmission) to ensure reliability, but this mechanism has significant drawbacks in satellite communication environments:

[0004] 1. Impact of high bit error rate: Satellite signals are affected by atmospheric interference, ionospheric scintillation, and other factors, resulting in a high data transmission bit error rate (typically 10). -6 Up to 10 -3 The transmission speed is much higher than that of terrestrial networks, and frequent retransmissions will lead to a significant decrease in transmission efficiency;

[0005] 2. Long latency issue: The one-way transmission latency of geostationary satellites is about 0.27 seconds, and the traditional ACK-based retransmission mechanism will lead to a lot of bandwidth waste;

[0006] 3. Link instability: Satellite links may be frequently interrupted due to obstruction, equipment failure, etc. Traditional transmission methods lack effective breakpoint resumption mechanisms, resulting in the waste of transmitted data;

[0007] 4. Bandwidth limitations: Satellite communication has limited bandwidth and is costly, requiring efficient use of bandwidth resources and reduction of redundant transmissions;

[0008] 5. Out-of-order problem: Due to differences in signal propagation paths, variations in processing delays at forwarding nodes, and channel contention, messages received by the receiver are prone to out-of-order delivery, severely impacting data block reassembly efficiency.

[0009] Existing error-correcting coding techniques, as an effective means of solving data transmission errors, can correct some errors at the receiving end without retransmission by adding redundant information to the data. Among them, RS coding is a high-performance linear block code with strong error correction capabilities, especially suitable for correcting burst errors and random errors.

[0010] However, simple error correction coding techniques cannot completely solve all the problems in file transmission. For example, when the number of errors exceeds the error correction capacity, retransmission mechanisms are still needed; interruptions that may occur during file transmission (such as network disconnection or device failure) can lead to the waste of transmitted data; in addition, the method of data block segmentation and the choice of verification mechanism will also affect transmission efficiency and reliability.

[0011] In the prior art, functions such as chunking and subcontracting, Reed-Solomon coding (hereinafter referred to as RS coding), multi-level verification, and resume from breakpoint often exist as independent modules or simple superpositions, and there are the following key pain points:

[0012] 1. The chunk parameters are fixed and cannot adapt to the dynamically fluctuating bandwidth and bit error rate of satellite links, easily resulting in the contradiction of "waste of retransmission due to overly large chunks and sharp increase in overhead due to overly small chunks";

[0013] 2. RS coding only supports the processing of fixed-length data. For large-size data packets, the coding efficiency is low, and the error correction ability of a single coding block cannot cope with sudden high bit errors;

[0014] 3. There is no linkage feedback in multi-level verification. Small errors trigger full-file retransmission, and the feedback delay is high;

[0015] 4. Resume from breakpoint adopts full-state synchronization. In a long-delay environment, the synchronization takes a long time, and there is a lack of an interruption prediction mechanism;

[0016] 5. Each module works independently, resulting in out-of-sync timing and unbalanced resource allocation, leading to a decline in the overall transmission performance.

[0017] Therefore, it is necessary to propose a new file transmission method based on Reed-Solomon coding to combine error correction coding technology with the actual needs of file transmission, taking into account reliability, efficiency, and adaptability. Summary of the Invention

[0018] The purpose of the present invention is to provide a file transmission method based on Reed-Solomon coding to solve the problems of low reliability and poor efficiency in file transmission in an unstable network environment.

[0019] To achieve the above purpose, the present invention provides a file transmission method based on Reed-Solomon coding, including:

[0020] S1: Perform file chunking and preprocessing to obtain original data packets and corresponding file subcontract contents;

[0021] S2: Perform improved Reed-Solomon coding on the file subcontract contents to obtain encoded data packets; S2 includes:

[0022] S21: Select the corresponding coding standard according to the bit error rate of the link;

[0023] S22: Perform length padding on the file subcontract contents so that the length of the file subcontract contents is an integer multiple of the standard length segment of the selected coding standard;

[0024] S23: For each file packet content, split it into multiple standard-length segments according to the encoding standard and generate independent ECC check codes;

[0025] S24: For all standard-length sub-segments, generate an association code by XORing the bytes together;

[0026] S25: Integrate the standard length sub-segment, ECC check code, and association code into a coding block, and integrate multiple coding blocks into a single encoded data packet;

[0027] S3: Send the encoded data packet from the sender to the receiver;

[0028] S4: Perform improved Reed-Solomon decoding and extract the original data packet, packet header, and message header; step S4 includes at least Reed-Solomon collaborative decoding based on the association code;

[0029] S5: Performs multi-level verification, data block concatenation, and file concatenation.

[0030] Step S1 includes:

[0031] S11: Dynamically divide and packetize the file to obtain multiple data blocks and multiple raw data packets for each data block;

[0032] S12: Add a packet header and a message header to each raw data packet to obtain the file packet content.

[0033] In step S11, the maximum length of the original data packet and the number of original data packets in each data block are dynamically adjusted based on the real-time bandwidth and bit error rate of the satellite link. In step S12, the packet header includes the block ID, packet ID, packet length, and packet checksum; the message header includes the command type, message sequence number, message body length, and message header checksum.

[0034] In step S21, the encoding standards include RS(255,223) encoding and RS(255,200) encoding. When the link bit error rate is higher than the preset bit error rate threshold, the RS(255,200) encoding standard is switched. In step S22, the original data features are used for length padding.

[0035] The preset error rate threshold includes 10. -4 The switching condition for the encoding standard is: the bit error rate is higher than 10 for 500ms. -4 Switch to RS(255,200) encoding; the bit error rate remains below 10 for 500ms. -4 Switch to RS(255,223) encoding.

[0036] Step S4 includes:

[0037] S41: The receiving end receives the encoded data packet;

[0038] S42: Check if the length of the encoded block is an integer multiple of 255 bytes and the length of the associated code. If not, the encoded data packet is determined to be invalid and return to step S41. If yes, proceed to step S43.

[0039] S43: Perform Reed-Solomon collaborative decoding based on the association code;

[0040] S43 specifically includes: the receiver independently Reed-Solomon decodes each basic codeword and makes a preliminary error judgment, the correlation code is used to deduce the theoretical value of the ECC check code of the error, the theoretical value of the ECC check code is used to obtain the basic codeword for back-off error correction, and the basic codeword for back-off error correction is subjected to secondary Reed-Solomon decoding to realize Reed-Solomon collaborative decoding;

[0041] S44: After successful decoding, extract the original data packet, message header, and packet header.

[0042] In step S3, after all encoded data packets of a data block are sent, the segmented MD5 corresponding to that data block is sent synchronously. The segmented MD5 is obtained by calculating the total MD5 of the file and splitting it into multiple segmented MD5s, with each segmented MD5 corresponding to a data block. Before sending the first encoded data packet of the file, the process also includes: the sending end sending file configuration information to the receiving end, including the file name, size, total MD5 of the file, total number of blocks, and total number of packets. Step S5 includes concatenating the original data packets to obtain data blocks, concatenating the data blocks to obtain the file, and verifying the segmented MD5 and the total MD5 of the file during this process.

[0043] Step S5 includes:

[0044] S51: Verify the message header checksum. If the verification passes, parse the message header; otherwise, discard the data packet and record the error.

[0045] S52: Verify the packet checksum. If the verification passes, the original data packet is considered correct and saved; otherwise, the original data packet is discarded and the error is recorded.

[0046] S53: The original data packets are concatenated to obtain data blocks, and the data blocks are concatenated to obtain a file. During this process, the segment MD5 and the total MD5 of the file are verified.

[0047] After step S4 and before step S5, the method further includes:

[0048] The sending end periodically sends file block transmission permission requests, and the receiving end responds to these requests by sending file block transmission permission responses. Each file block transmission permission request includes at least the current transmission block ID, and each file block transmission permission response includes at least the current transmission block ID and whether transmission is permitted, to achieve state synchronization. Alternatively, after a transmission interruption and reconnection, the sending end sends a file block transmission permission request, and the receiving end responds by sending a file block transmission permission response. Each file block transmission permission request includes at least the current transmission block ID, and each file block transmission permission response includes at least the current transmission block ID and whether transmission is permitted, to implement a breakpoint resumption mechanism, ensuring that only data blocks that were not successfully received are transmitted after reconnection.

[0049] After step S5, the process further includes: the sending end sending a file block transmission result request, and the receiving end sending a file block transmission result response based on the verification result of step S5.

[0050] The file transfer method based on Reed-Solomon encoding of this invention addresses the characteristics of high bit error rate, long latency, limited bandwidth, and unstable links in satellite communication. By combining error correction coding technology with the actual needs of file transfer, and through block packet processing, RS encoding error correction, multi-level verification, and breakpoint resumption, it minimizes the impact of data errors on file integrity while ensuring transmission efficiency. This achieves efficient and reliable file transfer applicable to various network environments, especially excelling in high bit error rate scenarios, and solves the problems of low reliability and poor efficiency of file transfer in unstable network environments in existing technologies. Attached Figure Description

[0051] Figure 1 This is an overall timing diagram of the file transfer method based on Reed-Solomon encoding of the present invention;

[0052] Figure 2 This is a message format diagram of the UDP packet in the file transfer method based on Reed-Solomon encoding of the present invention;

[0053] Figure 3 This is a flowchart of the sending end of the file transfer method based on Reed-Solomon encoding according to the present invention;

[0054] Figure 4 This is a flowchart of the receiving end of the file transfer method based on Reed-Solomon encoding according to the present invention;

[0055] Figure 5A and Figure 5B Here is a flowchart illustrating Reed-Solomon encoding and Reed-Solomon decoding;

[0056] Figure 6This is a diagram of the overall system architecture of the present invention. Detailed Implementation

[0057] The present invention will be further described below with reference to specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0058] like Figures 1-4 As shown, the file transfer method based on Reed-Solomon encoding of the present invention specifically includes the following steps:

[0059] Step S1: Perform file segmentation and preprocessing to obtain the original data packets and the corresponding file packet contents;

[0060] Step S1 is implemented using a data segmentation module, specifically including:

[0061] Step S11: Dynamically segment and packetize the file to obtain multiple data blocks and multiple raw data packets for each data block;

[0062] Specifically, based on the real-time bandwidth and bit error rate of the satellite link, the maximum length of the raw data packets (default 1024 bytes) and the number of raw data packets per data block (default 100) are dynamically adjusted. For example, if the bit error rate is higher than 10... -4 When retransmitting, the system automatically reduces the number of original data packets in each file block to 50, minimizing the cost of retransmitting individual data blocks. For example, if there are many signal errors (high error rate), the data block will contain fewer data packets, reducing waste during retransmission; if the signal is good and the bandwidth is sufficient, the data block will contain more data packets, with less additional overhead.

[0063] Step S12: Add a packet header and a message header to each raw data packet to obtain the file packet content.

[0064] like Figure 2 As shown, each file packet includes a message header (i.e., a message header), a packet header, and the original data packet as the packet content. Therefore, the file packet content can be sent as a single UDP message.

[0065] The packet header includes identification information for identification and verification, and a packet checksum (Adler32 algorithm). The identification information includes: chunk ID, packet ID, and packet length.

[0066] Chunk ID (4 bytes): Identifies the data block to which the data packet belongs, used for data block reassembly at the receiving end;

[0067] Packet ID (4 bytes): Identifies the sequence number of the data packet in its data block, used for data packet sorting at the receiving end;

[0068] Packet length (4 bytes): Records the length of the packet content, which helps the receiving end to correctly parse the data;

[0069] Packet checksum (4 bytes): The packet checksum is calculated using the Adler32 algorithm and is used by the receiving end to verify data integrity. The Adler32 algorithm can quickly detect errors during data transmission by calculating the packet checksum; its computational efficiency is high, making it suitable for scenarios with real-time requirements. The packet checksum is calculated based on the identifier information and the packet content.

[0070] The message header includes the command type, message sequence number, message body length, and message header checksum.

[0071] The UDP packet consists of several parts: The message command (2 bytes) identifies the type of data packet (e.g., data transmission, status query); the message sequence number (4 bytes) identifies the UDP packet and ensures its ordered nature (e.g., file packet content, status query requests), thus handling the request-response flow and ensuring that the request and response sequence numbers are identical to prevent message corruption; the message body length (2 bytes) is the total length of the UDP packet excluding the header, used by the receiving end to correctly parse the data packet; and the header checksum (4 bytes) is the total checksum of the UDP packet, used to verify the integrity of the header.

[0072] Step S2: Perform improved Reed-Solomon encoding on the file packet content to obtain the encoded data packet;

[0073] It should be noted that Reed-Solomon encoding encodes all sent UDP packets, including the header of the original data packet and the message header (i.e., the message header) in the UDP protocol.

[0074] Step S2 is implemented using the RS encoding module of the transmitting end, specifically including:

[0075] Step S21: Select the appropriate coding standard based on the bit error rate of the link;

[0076] In this embodiment, the encoding standards include RS(255,223) encoding and RS(255,200) encoding. The default encoding standard is RS(255,223) (i.e., 223 bytes of raw data generate a 32-byte ECC checksum, correcting up to 16 bytes of errors). When the link error rate exceeds a preset threshold, it automatically switches to RS(255,200) encoding (i.e., 200 bytes generate a 55-byte ECC checksum) to improve error correction redundancy.

[0077] In this embodiment, the preset threshold for bit error rate includes 10. -4The switching of encoding standards can be triggered by a bit error rate that remains above 10 for 500ms. -4 Switch to RS(255,200) encoding; the bit error rate remains below 10 for 500ms. -4 The encoding is switched to RS(255,223). The switching of encoding standards is automatically triggered by the central control module based on the real-time bit error rate of the link, without manual intervention.

[0078] Step S22: Pad the file packet content to make the length of the file packet content an integer multiple of the standard length segment of the selected encoding standard;

[0079] In this method, length padding is performed using original data features. In one embodiment, if the length of the file packet content is less than an integer multiple of the standard length segment of the encoding standard, the hash fragment of the first 8 bytes of the data packet is used as the original data feature for length padding. This satisfies the RS encoding standard and provides additional evidence for subsequent verification. In another embodiment, if the length of the file packet content is less than an integer multiple of the standard length segment of the encoding standard, all zeros are used as the original data feature for length padding.

[0080] Step S23: For each file packet content, split it into multiple standard-length segments according to the encoding standard and independently perform Reed-Solomon encoding to generate an independent ECC checksum;

[0081] For data packets exceeding 223 / 200 bytes, they are split into multiple standard-length segments of 223 / 200 bytes each. Each standard-length segment is independently Reed-Solomon encoded and generates an independent ECC checksum.

[0082] like Figure 5A As shown, for example, encoding 1024 bytes of data:

[0083] RS(255,223) mode: 5 standard length segments (S1-S5) each 223 bytes, each standard length segment generates 32 bytes of ECC code (E1-E5), thus forming 5 basic codewords of 255 bytes (Cᵢ=Sᵢ+Eᵢ).

[0084] RS(255,200) mode: 6 standard length segments (T1-T6) each 200 bytes, each standard length segment generates 55 bytes of ECC code (F1-F6), thus forming 6 255-byte basic codewords (Dᵢ=Tᵢ+Fᵢ).

[0085] Step S24: For the ECC check codes of all standard length segments, generate an associated code by XORing the bytes; thus, segmented encoding and collaborative error correction are realized.

[0086] The length of the association code is the same as the ECC check code (32 / 55 bytes), with no extra redundancy. That is, for 1024-byte data encoding, the generation logic of the association code is as follows: under RS(255,223) encoding, the 5 ECC check codes are XORed by bytes to generate a 32-byte association code; under RS(255,200) encoding, the 6 ECC check codes are XORed by bytes to generate a 55-byte association code.

[0087] RS(255,223) encoding (32-byte association code P) 32 ): Take 5 standard-length sub-segments of ECC checksum (E1-E5, 32 bytes each), and generate the associated code by XORing the bytes: P 32 [k] = E1[k]⊕E2[k]⊕E3[k]⊕E4[k]⊕E5[k] (k = 0-31, corresponding to 32 bytes);

[0088] RS(255,200) encoding (55-byte association code Q) 55 ): Take the ECC checksums of 6 standard-length segments (T1-T6, 55 bytes each), and generate the association code by XORing the bytes: Q 55 [k]=F1[k]⊕F2[k]⊕F3[k]⊕F4[k]⊕F5[k]⊕F6[k] (k=0-54, corresponding to 55 bytes).

[0089] Step S25: Integrate the standard length segment, ECC check code, and association code into a coding block, and integrate multiple coding blocks into a single encoded data packet.

[0090] A single coded block consists of raw data (223×N or 200×N bytes) of a single standard-length segment, an ECC checksum (32×N or 55×N bytes), and an association code (32 or 55 bytes).

[0091] Multiple coded blocks share the header of a single data packet to form the encoded data packet.

[0092] Step S3: Send the encoded data packet from the sending end to the receiving end;

[0093] In this process, the encoded data packets are sent from the sending end to the receiving end through a transmission protocol.

[0094] Step S3 is implemented using a UDP sending module and a UDP receiving module, specifically including:

[0095] Step S31: Select the UDP transport protocol for data transmission;

[0096] The UDP transport protocol is characterized by high transmission speed and low overhead, making it suitable for scenarios with high real-time requirements. Although the UDP transport protocol itself does not provide reliability guarantees, reliable transmission can be achieved on the basis of the UDP transport protocol through the error correction coding and verification mechanism in this invention.

[0097] Step S32: Send the encoded data packet according to the specified sending strategy.

[0098] The sending end sends each data packet sequentially according to the data block order, and records the status of the sent data packets. Furthermore, multiple data packets within the same data block are defined by their packet ID order, and data packets within the same data block that are not sent correctly or not sent at all are sent in order of their packet IDs.

[0099] Each encoded data packet serves as a UDP message to be sent, including a header and a data packet; the header is completed before RS encoding, giving it error correction capabilities.

[0100] After all encoded data packets for a data block have been sent, the segmented MD5 corresponding to that data block is sent synchronously. In this embodiment, the segmented MD5 is sent in the form of a file block transfer permission request. The segmented MD5 is obtained by calculating the total MD5 of the file and splitting it into multiple segmented MD5s (e.g., 200 segmented MD5s), with each segmented MD5 corresponding to a data block (corresponding to the integrity verification of a single data block).

[0101] Before sending the first encoded data packet of the file, the process includes: the sender sending file configuration information to the receiver, including the filename, size, total MD5 hash of the file, total number of blocks, and total number of packets (corresponding to the final integrity check of the entire file). After the receiver confirms receipt of the file configuration information and sends a configuration information confirmation response to the sender, the sender begins sending the encoded data packets according to the specified sending strategy.

[0102] Step S4: Perform improved Reed-Solomon decoding and extract the original data packets, packet headers, and message headers;

[0103] Step S4 is implemented using the RS decoding module at the receiving end, specifically including:

[0104] Step S41: The receiving end listens on the specified port to receive the encoded data packets;

[0105] Step S42: Check if the length of the encoded block is an integer multiple of the length of the associated code of 255 bytes. If not, the encoded data packet is determined to be invalid and return to step S41. If yes, proceed to step S43.

[0106] The association code length is 32 or 55, so the remainder can be either 32 or 55.

[0107] Step S43: Perform Reed-Solomon collaborative decoding based on the association code.

[0108] Step S43 specifically includes: the receiving end independently Reed-Solomon decodes each basic codeword and makes a preliminary error judgment, infers the theoretical value of the ECC check code of the error from the associated code, obtains the basic codeword for backtracking using the theoretical value of the ECC check code, and performs a second Reed-Solomon decoding on the basic codeword for backtracking to achieve Reed-Solomon collaborative decoding.

[0109] (1) Example of RS(255,223) mode (assuming C3 error exceeds 16 bytes)

[0110] The receiving end performs independent Reed-Solomon decoding and preliminary error assessment for each basic codeword:

[0111] Perform RS decoding on each basic codeword C1'-C5'. C1', C2', C4', and C5' are successfully decoded (errors ≤ 16 bytes), while C3' fails to decode (errors > 16 bytes). Record the ECC checksum (E1) of the successful segment. dec E2 dec E4 dec E5 dec ).

[0112] The theoretical value of the ECC check code of the base codeword from which the associated code deduces the error:

[0113] According to the association code P 32 The theoretical value of the ECC checksum of the faulty segment is calculated by reverse XOR logic: E3 theo [k]=E1 dec [k]⊕E2 dec [k]⊕E4 dec [k]⊕E5 dec [k]⊕P 32 ′[k].

[0114] Furthermore, the actual ECC check code (E3') of the basic codeword C3' is compared with the theoretical value E3 of the ECC check code. theo If the number of differing bytes is greater than 16, C3 is confirmed to be an error segment, and the error is concentrated in this basic codeword.

[0115] Using the theoretical value of ECC checksums, we obtain the basic codewords for backtracking and error correction:

[0116] Using the theoretical value E3 theo Replace the actual ECC checksum E3' in the basic codeword C3' to obtain the backtracking codeword C. 3rev (C)3rev =S3'+E3 theo At this point, E3 theo Including correct error correction information can narrow down the error range of S3' (e.g., 20-byte error → 8 bytes).

[0117] Perform secondary Reed-Solomon decoding on the base codewords for backoff correction:

[0118] For C 3rev Perform RS(255,223) secondary decoding. If the 8-byte error is less than or equal to the 16-byte error correction limit, the decoding is successful and the correct sub-segment S3 is restored.

[0119] (2) Example of RS(255,200) mode (assuming D4 error exceeds 27 bytes)

[0120] The receiver performs independent Reed-Solomon decoding and preliminary error assessment on each basic codeword: D1'-D3' and D5'-D6' are successfully decoded, while D4' fails.

[0121] The theoretical value of the ECC check code of the underlying codeword from which the associated code deduces the error: via Q 55 'Reverse push F4' theo (F4) theo =F1 dec ⊕F2 dec ⊕F3 dec ⊕F5 dec ⊕F6 dec ⊕Q 55 ');

[0122] Using the theoretical value of ECC checksums, we obtain the basic codewords for backtracking and error correction: using F4. theo Replace F4' in D4' to form D 4rev ;

[0123] Secondary Reed-Solomon decoding is performed on the base codewords for backoff correction: D 4rev The error has been reduced to 20 bytes (≤27 bytes), and decoding is successful.

[0124] The maximum error correction capability of the collaborative decoding is 24 bytes (when RS(255,200) is encoded). If the error exceeds 24 bytes, the decoding is deemed to have failed and a request is made to retransmit the data packet.

[0125] Step S44: After successful decoding, extract the original data packet (remove the original data feature padding fragments), message header and packet header (including block ID, packet ID, and packet length).

[0126] Step S5: Perform multi-level verification, data block concatenation, and file concatenation.

[0127] Step S5 is implemented using the verification module of the receiving end, and specifically includes:

[0128] Step S51: Verify the message header checksum. If the verification passes, parse the message header; otherwise, discard the data packet and record the error.

[0129] Step S52: Verify the packet checksum. If the verification passes, the original data packet is deemed correct and saved; otherwise, the original data packet is discarded and the error is recorded.

[0130] The receiving end calculates the parcel checksum of the decoded packet content and compares it with the parcel checksum recorded in the packet header. If they match, the original data packet is considered correct and saved; otherwise, the original data packet is discarded and an error is recorded.

[0131] Step S53: The original data packets are concatenated to obtain data blocks, and the data blocks are concatenated to obtain a file. During this process, the segment MD5 and the total MD5 of the file are verified to check the integrity of the data blocks and the file.

[0132] Once the receiving end receives all the original data packets for a given data block, it reassembles the original data packets according to the packet ID order to form a complete data block. After all data blocks have been received, they are reassembled according to the block ID order to obtain the complete file.

[0133] As mentioned above, the sending end splits the total MD5 of the file into segmented MD5s that are the same number of data blocks, and synchronously sends the segmented MD5 corresponding to each data block after each data block is transmitted.

[0134] After a single data block is concatenated, the segment MD5 corresponding to the data block is calculated and compared with the segment MD5 sent by the sender to verify the segment MD5, and then cached after successful verification.

[0135] After the file is concatenated, the total MD5 of the file is calculated and compared with the total MD5 of the file sent by the sender to verify the total MD5 of the file. If they match, the transmission is considered complete; otherwise, the data block is marked as faulty.

[0136] After step S4 and before step S5, the method further includes:

[0137] 1) The sending end periodically sends file block transfer permission requests, and the receiving end responds to the file block transfer permission requests by sending file block transfer permission responses. The file block transfer permission requests include at least the current transfer block ID, and the file block transfer permission responses include at least the current transfer block ID and whether transfer is allowed (to indicate whether the data block has been received completely), so as to achieve state synchronization.

[0138] Specifically, during data block transmission, the sending end periodically sends a request to the receiving end to obtain the data block's reception status (also known as a file block transmission permission request). The receiving end synchronously sends a response (i.e., a file block transmission permission response) to check whether the data block content needs updating, ensuring consistency of status information between the two parties. If the sending end does not receive a file block transmission permission response within the response time range after sending the request to obtain the data block's reception status (i.e., it finds that a certain data block has not been acknowledged for a long time), it actively retransmits the data block to avoid transmission stalls. In this embodiment, after each data block is sent, the sending end sends a file block transmission permission request for that data block.

[0139] In this embodiment, the file block transfer permission request includes a message header (12 bytes), filename (64 bytes), file MD5 (16 bytes), total number of file packets (4 bytes), total number of file blocks (4 bytes), current transfer block ID (4 bytes), and segment MD5 (16 bytes). The file block transfer permission response includes a message header (12 bytes), whether transfer is allowed (4 bytes), file information (including filename (64 bytes), file MD5 (16 bytes), total number of file packets (4 bytes), total number of file blocks (4 bytes)), current transfer block ID (4 bytes), and block transfer status (16 bytes).

[0140] The current transport block ID is the block ID of the data block that has not been completely transmitted during state synchronization or breakpoint resumption. For example, if data blocks with block IDs 1-4 have been correctly received, then the current transport block ID is 5. The "Allow Transmission" option indicates whether the data block has been completely received. If it indicates no transmission is needed, the data block has been completely received; otherwise, it indicates that the data block needs to be transmitted, and the sender will start transmitting from this block. The block transmission status identifies the data packets that have been completely transmitted within the block, and is obtained by bitwise identification. For example, EFFFF… indicates that the first packet has been completely transmitted; 0FFFFFFF indicates that the first 15 packets have been completely transmitted.

[0141] And / or; 2) After a transmission interruption and reconnection, the sending end sends a file block transmission permission request, and the receiving end responds by sending a file block transmission permission response. The file block transmission permission request includes at least the current transmission block ID, and the file block transmission permission response includes at least the current transmission block ID and whether transmission is allowed (to indicate whether the data block has been fully received), to implement a breakpoint resumption mechanism. This ensures that only data blocks that were not successfully received are transmitted after reconnection, thereby improving data transmission efficiency in the face of satellite link instability. In other words, transmission is allowed to start from a certain block ID, thus only transmitting data blocks that were not received or whose verification failed, which resulted in permission to transmit, without retransmitting already received data blocks. This incremental transmission method significantly reduces the amount of data transmitted repeatedly and improves transmission efficiency.

[0142] In this embodiment, the content of the file block transfer permission request and the file block transfer permission response is the same as above.

[0143] In this process, each block ID is used as the current transport block ID to send file block transport permission requests to the receiving end in a loop. When the receiving end replies with a file block transport permission response indicating whether transport is allowed, the transport begins from the data block corresponding to the current transport block ID.

[0144] The breakpoint resume mechanism specifically includes:

[0145] 1) Status Recording: During transmission, the receiving end records the reception status (e.g., received, not received, verification failed, etc.) of each data block using a hierarchical hash table, and determines whether transmission of each data block is permitted. The hash table uses the block ID as the key and the reception status as the value, facilitating fast querying and updating.

[0146] 2) State saving and loading: When the transmission is interrupted (such as when the network is disconnected or the device is restarted), the receiving end saves the current receiving state of each data block to a local file; when the connection is re-established, the receiving end loads the receiving state of each data block from the local file and restores the receiving state of each data block before the transmission.

[0147] 3) Incremental Transmission: After reconnection, the sending end sends a file block transmission permission request to the receiving end to request the current reception status. This file block transmission permission request includes the current transmission block ID, causing the receiving end to send a file block transmission permission response. The file block transmission permission response includes whether the data block corresponding to the current transmission block ID is allowed to be transmitted (used to indicate whether the data block has been completely received). Thus, the sending end only transmits data blocks that were not received or whose verification failed, thus allowing transmission, without needing to retransmit already received data blocks.

[0148] After step S5, the process further includes: the sending end sending a file block transmission result request, and the receiving end sending a file block transmission result response based on the verification result of step S5.

[0149] like Figure 2 As shown, in this embodiment, the file block transmission result request includes a message header (12 bytes) and file information (88 bytes); the file block transmission result response includes a message header (12 bytes) and transmission result (4 bytes). The transmission result (4 bytes) indicates whether the MD5 checksum of the entire file passes.

[0150] In this embodiment, all UDP packets involved need to be encoded using improved Reed-Solomon encoding to correct errors that occur during transmission at the receiving end, significantly reducing the need for retransmissions. These UDP packets include file block transfer permission requests, file block transfer permission responses, file block transfer result requests, file block transfer result responses, and file packet content.

[0151] The Reed-Solomon-based file transfer method of the present invention is applied to a file transfer system, such as... Figure 6 As shown, the file transfer system includes a sender and a receiver, which interact via a satellite communication link. The sender includes a file processing module, a protocol state machine, a message encoding module, an RS encoding module, and a UDP sending module, responsible for file segmentation, encoding, verification, calculation, and data transmission. The receiver includes a UDP receiving module, an RS decoding module, a message decoding module, a command processing module, and a file processing module, responsible for data reception, decoding, error correction, verification, and file merging. The file transfer system adopts a "centralized control + modular collaboration" design. The central control timing scheduler coordinates the work of each module, and a dynamic resource balancing algorithm adapts to link fluctuations, ensuring transmission reliability and efficiency.

[0152] Furthermore, the file transfer system of the present invention includes the following core modules, which work collaboratively by a central control timing scheduler:

[0153] 1. Data Segmentation Module (included in the File Processing Module): Performs file segmentation and preprocessing to obtain the original data packets and the corresponding file segment contents;

[0154] In this process, a packet header and a message header are added to each original data packet to obtain the file packet content.

[0155] 2. Checksum calculation module (included in the message encoding module): The Adler32 algorithm is used to calculate the packet checksum, and the total MD5 and segment MD5 of the file are calculated.

[0156] 3. RS Encoding Module: Used to perform improved Reed-Solomon encoding (segmented encoding + cooperative error correction) on the file packet content to obtain the encoded data packet;

[0157] The specific content of the improved Reed-Solomon encoding is as described in step S2 above.

[0158] 4. UDP sending module and UDP receiving module: These modules send the encoded data packets from the sending end to the receiving end.

[0159] In addition, the receiving end records the reception status of data blocks, and the UDP sending module and encoding module are linked in a buffered manner.

[0160] 5. RS Decoding Module: Performs improved Reed-Solomon decoding and extracts the original data packets, headers, and message headers;

[0161] Among them, the error correction capability is extended by using the correlation code between segments.

[0162] 6. Verification module (included in the message decoding module): performs multi-level verification (verifies message header checksum, verifies packet checksum, verifies segment MD5 and total file MD5), data block concatenation and file concatenation;

[0163] When verification fails, the original data packet is discarded and a retransmission is triggered.

[0164] 7. Resume interrupted download module (included in command processing module): The sending end periodically or after the transmission is interrupted and reconnected sends a file block transmission permission request, and the receiving end responds to the file block transmission permission request by sending a file block transmission permission response.

[0165] 8. Central Control Module: Includes a central control timing scheduler (coordinates the encoding, transmission, and decoding rhythm of the core modules to avoid buffer overflow) and a dynamic resource balancing algorithm (configures the encoding standard, the length of the verification information, and the frequency of state synchronization based on bandwidth), and handles abnormal situations (decoding failure, verification failure).

[0166] For example, when bandwidth is low, the RS (255,223) encoding standard is preferred to save redundancy. When bandwidth is less than 1Mbps, the length of the checksum information is reduced (segmented MD5 is compressed from 16 bytes to 8 bytes) to prioritize data transmission.

[0167] The file transfer method based on Reed-Solomon encoding of this invention addresses the characteristics of high bit error rate, long latency, limited bandwidth, and unstable links in satellite communication. By combining error correction coding technology with the actual needs of file transfer, and through block packet processing, RS encoding error correction, multi-level verification, and breakpoint resumption, it minimizes the impact of data errors on file integrity while ensuring transmission efficiency. This achieves efficient and reliable file transfer applicable to various network environments, especially excelling in high bit error rate scenarios, and solves the problems of low reliability and poor efficiency of file transfer in unstable network environments in existing technologies.

[0168] Compared with the prior art, the present invention has the following beneficial effects:

[0169] High anti-interference and dynamic adaptation capabilities: Employing RS coding technology, it can correct errors occurring during transmission at the receiving end, significantly reducing retransmission requirements. The improved RS coding, through collaborative error correction, increases the maximum error correction capability of a single coding block to 24 / 35 bytes. Combined with dynamic coding standard adjustments, the decoding success rate is improved by 30% in high bit error rate (1%) environments, and the retransmission rate is reduced from 15% to below 5%. It is particularly suitable for high bit error rate network environments in satellite communication.

[0170] Long latency adaptation and efficiency improvement: The hierarchical hash table state synchronization reduces the breakpoint reconnection time from 2-3 seconds to less than 100ms. The central control timing scheduling avoids data backlog between modules, reduces the buffer overflow rate to below 0.5%, and improves transmission efficiency by 6 times. Through the optimized timeout retransmission mechanism and periodic state synchronization strategy, the long latency problem of satellite communication is overcome, and the inefficiency caused by the traditional ACK mechanism is avoided.

[0171] High-efficiency bandwidth utilization: Dynamic block partitioning and resource balancing algorithms reduce the amount of data retransmitted per block by 40%, segmented MD5 reduces the amount of data retransmitted due to file-level errors by 80%, and the data transmission rate remains stable at 450kbps at 1Mbps bandwidth; combined with error correction coding and selective retransmission mechanisms, redundant data transmission in the satellite link is reduced, improving the utilization of limited bandwidth.

[0172] Out-of-order and tamper protection: Dual ID association coding improves the accuracy of out-of-order correction to 99.9%, and Adler32 check improves the data tamper detection rate by 30%. It can effectively identify and handle common out-of-order phenomena in satellite links, ensure the correct reassembly of data blocks, and meet the high reliability requirements of satellite transmission.

[0173] Resume interruption support: In response to the instability of satellite links, a comprehensive resume interruption mechanism has been designed to avoid data retransmission waste after link interruption, which significantly improves transmission efficiency.

[0174] Multi-layered integrity assurance: Through a multi-layered verification mechanism of Adler32 checksum and MD5 checksum, the integrity of data at each level from data packet to complete file is ensured, meeting the high reliability requirements of satellite transmission.

[0175] Example 1: Complete File Transfer Process

[0176] This embodiment uses the transfer of a 50MB file "version.bin" as an example to explain the file transfer process of the present invention in detail.

[0177] Step S0: Initialize the sending end;

[0178] The sending end receives the path to the file to be transmitted, "version.bin", and the receiving end's IP address, and verifies the existence and readability of the file.

[0179] Initialize the network socket and establish a UDP communication link with the receiving end.

[0180] Initialize the improved RS encoder (default RS(255,223)), hierarchical hash table (to record transmission status), and central control timing scheduler;

[0181] Calculate the total MD5 (checksum) of the file and split it into 200 segment MD5s (corresponding to 200 data blocks).

[0182] Step S1: Perform file segmentation and preprocessing to obtain the original data packets and the corresponding file packet contents;

[0183] The calculated file size is 50MB (i.e., 52,428,800 bytes).

[0184] If the maximum length of each data packet is set to 1024 bytes and each block contains 256 data packets, then the maximum data size of each data block is 256 × 1024 = 262144 bytes (256KB).

[0185] Total number of packets: 52,428,800 ÷ 1024 = 51,200 data packets.

[0186] Calculate the total number of blocks: 51200 ÷ 256 = 200 data blocks.

[0187] The total MD5 hash of the file is calculated using the MD5 algorithm and stored in the file information.

[0188] Step S2: Perform improved Reed-Solomon encoding on the file packet content to obtain the encoded data packet;

[0189] The file is divided into 200 data blocks, each containing 256 data packets (the last block may contain fewer than 256 packets, depending on the actual file size).

[0190] Calculate the Adler32 checksum for the sub-contents of each data packet and add it to the packet header.

[0191] Each data packet is RS encoded in 223-byte segments to generate an encoded block containing a 32-byte ECC checksum and an association code, which is then concatenated.

[0192] Step S3: Send the encoded data packet from the sending end to the receiving end;

[0193] The sending end sends file configuration information (including file name, size, total checksum, total number of blocks, total number of packets, etc.) to the receiving end.

[0194] After the receiving end confirms the file configuration information, the sending end begins to send the encoded data packets in blocks.

[0195] For each data packet sent, its sending status is recorded;

[0196] After each data block is sent, a file block transfer permission request is sent to the receiving end (to obtain the number of file blocks and packets that the receiving end has received and saved, for example, 20 blocks + 30 packets have been received). If no confirmation is received from the receiving end after a preset time, the data packet corresponding to the file block transfer permission request is resent.

[0197] Step S4: Perform improved Reed-Solomon decoding and extract the original data packets, packet headers, and message headers;

[0198] The receiving end listens on a specified port, receives data packets, and parses the packet header information.

[0199] RS decoding is performed on the encoded block to correct transmission errors; if the errors exceed the error correction capability, it is considered a reception failure.

[0200] Verify the Adler32 checksum of the decoded data, and save the packet content if it passes the verification.

[0201] Step S5: Perform multi-level verification, data block concatenation, and file concatenation.

[0202] After receiving all data packets, the receiving end concatenates the data according to the block ID and packet ID to form a complete file.

[0203] Calculate the MD5 checksum of the concatenated file and compare it with the total checksum transmitted by the sender to confirm file integrity.

[0204] If the verification passes, a successful transmission is reported to the sender; otherwise, an error data block is identified and a retransmission is requested.

[0205] Example 2: Resuming interrupted downloads

[0206] This embodiment simulates the process of resuming transmission through a breakpoint resumption mechanism after a network interruption during transmission. The remaining steps of the file transfer method of this invention are consistent with the first embodiment, except that, after step S4 and before step S5, the following step is also included:

[0207] 1) Status recording: During transmission, the receiving end records the reception status of each data block (such as received, not received, verification failed, etc.) through a hierarchical hash table, and obtains whether each data block is allowed to be transmitted.

[0208] 2) State saving and loading: When transmission is interrupted (such as network disconnection or device restart), the receiving end saves the current receiving state of each data block to a local file in advance based on link quality prediction; when reconnected, the receiving end loads the receiving state of each data block from the local file and restores the receiving state of each data block before transmission.

[0209] Suppose that when the file is being transferred to the 100th block, the network suddenly stops. At this point, the receiving end has correctly received the first 99 blocks of data, and the 100th block of data is partially lost.

[0210] When the receiving end detects a transmission interruption, it saves the current receiving state (received block IDs, unreceived block IDs, etc.) to the local file "resume.state" using the breakpoint resume module.

[0211] After the network is restored, the sending and receiving ends re-establish their connection. The receiving end loads the locally saved "resume.state" file to restore the receiving status information.

[0212] 3) Incremental transmission: After reconnection, the sender sends a file block transmission permission request to the receiver to request the current reception status. The file block transmission permission request includes the current transmission block ID, which causes the receiver to send a file block transmission permission response. The file block transmission permission response includes whether the data block corresponding to the current transmission block ID is allowed to be transmitted (used to indicate whether the data block has been received completely).

[0213] After reconnection, the sending end sends a file block transmission permission request to the receiving end to request the current reception status. The receiving end reports that it has received the first 99 blocks of data, and the 100th block of data needs to be retransmitted.

[0214] Therefore, the sending end only transmits the 100th data block, without needing to retransmit the first 99 blocks. The receiving end receives and processes these data blocks, verifies their integrity, and then concatenates them.

[0215] After step S5 (i.e., transmission completed and verification completed), the following is also included:

[0216] The sending end sends a file block transmission result request, and the receiving end sends a file block transmission result response based on the verification result of step S5, thereby sending feedback to the sending end that the transmission was successful.

[0217] Example 3: Transmission optimization under high bit error rate environment

[0218] This embodiment simulates the process by which the system improves transmission reliability through Reed-Solomon coding in high bit error rate environments such as satellite communication.

[0219] 1. Environment Setup:

[0220] Assume the error rate of the network environment is 1%, meaning that there may be 1 byte error in every 100 bytes of data.

[0221] The sending end transmits a 10MB file named "satellite_data.bin" containing a large amount of critical data.

[0222] The sending end detected that the bit error rate exceeded the threshold, adjusted the RS encoding standard to RS(255,200), and increased the ECC check code length to 55 bytes;

[0223] 2. Encoding and Transmission:

[0224] Divide the 10MB file into blocks (each block contains 256 1024-byte data packets), split each data packet into 5 sub-segments for encoding, and add inter-segment association codes.

[0225] During transmission, 30% of data packets encountered errors (single block error ≤ 24 bytes).

[0226] 3. Decoding and Error Correction:

[0227] The receiving end decodes each coded block, successfully corrects errors that occurred during transmission, and recovers the original data.

[0228] Only a few data packets failed to be decoded due to errors exceeding 24 bytes, and the receiving end requested retransmission of these data packets.

[0229] 4. Result Comparison:

[0230] After adopting the method of the present invention, the retransmission rate of file transfer is reduced from 30% of the traditional method to less than 5%, and the transmission efficiency is improved by about 6 times.

[0231] The file passed final verification, its integrity was guaranteed, and it met the transmission requirements under high error rate environments.

[0232] This invention improves upon the correlation technologies of segmentation and packetization, RS encoding, multi-level verification, and breakpoint resumption, and combines them with a central control timing scheduling and dynamic resource balancing mechanism to solve the problem that simple integration of existing technologies cannot adapt to the characteristics of satellite communication. This solution addresses the high error rate, long latency, limited bandwidth, and unstable links of satellite communication, achieving efficient and reliable file transmission. Future development could further optimize the encoding algorithm efficiency, combine it with encryption technology for secure transmission, or introduce AI algorithms to dynamically adjust transmission strategies to adapt to more complex network environments. This invention provides new ideas and methods for reliable file transmission technology in satellite communication, possessing significant theoretical and practical application value.

[0233] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of the invention. Various variations can be made to the above embodiments of the present invention. All simple and equivalent changes and modifications made in accordance with the claims and description of this application fall within the protection scope of the claims of this patent. All aspects not described in detail in this invention are conventional technical content.

Claims

1. A file transfer method based on Reed-Solomon encoding, characterized in that, include: Step S1: Perform file segmentation and preprocessing to obtain the original data packets and the corresponding file packet contents; Step S2: Perform improved Reed-Solomon encoding on the file packet content to obtain the encoded data packet; Step S2 includes: Step S21: Select the appropriate coding standard based on the bit error rate of the link; Step S22: Pad the file packet content to make the length of the file packet content an integer multiple of the standard length segment of the selected encoding standard; Step S23: For each file packet content, split it into multiple standard-length segments according to the encoding standard and generate independent ECC check codes; Step S24: For the ECC check codes of all standard length segments, generate an association code by XORing the bytes together; Step S25: Integrate the standard length segment, ECC check code, and association code into a coding block, and integrate multiple coding blocks into a single encoded data packet; Step S3: Send the encoded data packet from the sending end to the receiving end; Step S4: Perform improved Reed-Solomon decoding and extract the original data packet, packet header, and message header; Step S4 includes at least Reed-Solomon collaborative decoding based on the association code; Step S5: Perform multi-level verification, data block concatenation, and file concatenation.

2. The file transfer method according to claim 1, characterized in that, Step S1 includes: Step S11: Dynamically segment and packetize the file to obtain multiple data blocks and multiple raw data packets for each data block; Step S12: Add a packet header and a message header to each raw data packet to obtain the file packet content.

3. The file transfer method according to claim 2, characterized in that, In step S11, the maximum length of the original data packet and the number of original data packets for each data block are dynamically adjusted based on the real-time bandwidth and bit error rate of the satellite link. In step S12, the packet header includes a block ID, a packet ID, a packet length, and a packet checksum; The message header includes the command type, message sequence number, message body length, and message header checksum.

4. The file transfer method according to claim 2, characterized in that, In step S21, the encoding standard includes RS(255,223) encoding and RS(255,200) encoding. When the link bit error rate is higher than the preset bit error rate threshold, the encoding standard is switched to RS(255,200). In step S22, the original data features are used for length padding.

5. The file transfer method according to claim 4, characterized in that, The preset error rate threshold includes 10. -4 The switching condition for the encoding standard is: the bit error rate is higher than 10 for 500ms. -4 Switch to RS(255,200) encoding; The bit error rate remains below 10 for 500ms. -4 Switch to RS(255,223) encoding.

6. The file transfer method according to claim 1, characterized in that, Step S4 includes: Step S41: The receiving end receives the encoded data packet; Step S42: Check if the length of the encoded block is an integer multiple of the length of the associated code of 255 bytes. If not, the encoded data packet is determined to be invalid and return to step S41. If yes, proceed to step S43. Step S43: Perform Reed-Solomon collaborative decoding based on the association code; Step S43 specifically includes: the receiving end independently Reed-Solomon decodes each basic codeword and makes a preliminary error judgment, the associated code is used to deduce the theoretical value of the ECC check code of the error, the theoretical value of the ECC check code is used to obtain the basic codeword for back-off error correction, and the basic codeword for back-off error correction is subjected to secondary Reed-Solomon decoding to realize Reed-Solomon collaborative decoding; Step S44: After successful decoding, extract the original data packet, message header, and packet header.

7. The file transfer method according to claim 1, characterized in that, In step S3, after all the encoded data packets of a data block are sent, the segment MD5 corresponding to that data block is sent synchronously. The segment MD5 is obtained by calculating the total MD5 of the file and splitting it into multiple segment MD5s, with each segment MD5 corresponding to a data block. Before the first encoded data packet of the file is sent, the following steps are also included: the sender sends file configuration information to the receiver, including file name, size, total MD5 hash of the file, total number of blocks, and total number of packets; Step S5 includes concatenating the original data packets to obtain data blocks, concatenating the data blocks to obtain a file, and verifying the segment MD5 and the total MD5 of the file during this process.

8. The file transfer method according to claim 7, characterized in that, Step S5 includes: Step S51: Verify the message header checksum. If the verification passes, parse the message header; otherwise, discard the data packet and record the error. Step S52: Verify the packet checksum. If the verification passes, the original data packet is deemed correct and saved; otherwise, the original data packet is discarded and the error is recorded. Step S53: The original data packets are concatenated to obtain data blocks, and the data blocks are concatenated to obtain a file. During this process, the segment MD5 and the total MD5 of the file are verified.

9. The file transfer method according to claim 1, characterized in that, After step S4 and before step S5, the method further includes: The sending end periodically sends file block transfer permission requests, and the receiving end responds by sending a file block transfer permission response. The file block transfer permission request includes at least the current transfer block ID, and the file block transfer permission response includes at least the current transfer block ID and whether transfer is permitted, to achieve state synchronization; and / or After a transmission interruption and reconnection, the sending end sends a file block transmission permission request. The receiving end responds to the file block transmission permission request by sending a file block transmission permission response. The file block transmission permission request includes at least the current transmission block ID, and the file block transmission permission response includes at least the current transmission block ID and whether transmission is allowed, so as to implement the breakpoint resume mechanism, so that only the data blocks that were not successfully received are transmitted after reconnection.

10. The file transfer method according to claim 1, characterized in that, After step S5, the process further includes: the sending end sending a file block transmission result request, and the receiving end sending a file block transmission result response based on the verification result of step S5.

Images