Single-guide client-based large file chunk transmission and loss retransmission method and system

Abstract

The application belongs to the technical field of network security isolation transmission, and particularly relates to a large file fragmentation transmission and loss retransmission method and system based on a single guide client, which comprises the following steps: a sending end cuts a large file into fragments with sequential numbering, stores the fragment information persistently, and distributes the fragment information to multiple front-end servers through load balancing for concurrent transmission; a receiving end receives the fragments through multiple back-end servers, stores the fragments in a non-sequential temporary cache according to sequence, and stores the recorded fragments persistently; the receiving end monitors the idle time, determines packet loss and generates a list of fragments that have not been stored on disk when the idle time exceeds a threshold, and feeds back the list to the sending end according to a one-way or two-way network topology; and the sending end retransmits only the lost fragments, and resumes the broken point continuous transmission according to the persistent record when restarted.

Description

Technical Field

[0001] This invention belongs to the field of network security isolation transmission technology, and particularly relates to a method and system for large file fragment transmission and retransmission based on a single-channel client. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Currently, in the field of network security isolation transmission, fragmented transmission and retransmission technologies have been developed to a certain extent, and corresponding solutions exist for file transmission in unidirectional isolation environments. For example, one file transmission method based on a unidirectional isolation device involves the sending end dividing the file into fragments of a fixed size, transmitting them sequentially to the receiving end via a unidirectional channel, and then the receiving end reassembling the file according to the fragment sequence number.

[0004] Meanwhile, existing retransmission technologies mainly include two types: passive retransmission based on timeout and active retransmission based on packet loss feedback. The former automatically triggers the retransmission of a fragment when the fragment upload times out or fails; the latter requires the receiving end to extract the length of the data to be transmitted from the option field of the first data packet and actively send a packet loss message based on the receiving waiting time to drive the sending end to retransmit.

[0005] However, the existing technical solutions mentioned above still have the following shortcomings: First, the overhead of breakpoint state management is large, the server needs to maintain a large number of fragment states, and the memory consumption is too high under high concurrency, making it difficult to support real-time tracking of massive fragments; Second, the atomicity of the merge operation is insufficient. If the service crashes during the fragment merging process, it may lead to half-file residue or data inconsistency, affecting the reliability of transmission; In addition, load balancing usually distributes files in turn, and fragments of the same file are sequentially allocated to the same device, causing the bandwidth of a single device to become the transmission bottleneck. For a single ultra-large file of more than 4TB, the cross-network transmission time is difficult to compress effectively, resulting in poor transmission timeliness. Summary of the Invention

[0006] To overcome the shortcomings of the existing technologies, this invention provides a method and system for large file fragment transmission and retransmission based on a single-client architecture. It aims to improve the cross-network timeliness of large files through fragment load balancing and concurrent transmission, achieve precise breakpoint resumption through idle timeout packet loss judgment and retransmission of lost fragments only, and adapt to unidirectional and bidirectional network topologies. This solves the problems of low transmission efficiency, high retransmission overhead and coarse granularity of breakpoint resumption in the existing technologies.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: Firstly, a method for transmitting and retransmitting fragmented ultra-large files based on a single-channel client is disclosed, including: The sending end splits files exceeding the threshold into multiple sequentially numbered fragments, which are then transmitted concurrently through multiple front-end servers in a load-balanced mode. The receiving end receives the fragments in a load-balanced mode through multiple back-end servers, creates temporary files and cache records, and writes the received fragments sequentially to the temporary file if their numbers match the expected numbers; otherwise, they are stored in the cache. When the number of fragments accumulated in the cache reaches a set threshold, the fragments in the cache are sequentially retrieved from the smallest number and written to the temporary file. During transmission, the receiving end monitors the idle time of receiving fragments. When the idle time exceeds a preset threshold, it determines that packet loss has occurred, generates a packet loss record containing fragment information that has not been written, and feeds it back to the sending end. The sending end retransmits the corresponding lost fragments according to the packet loss record.

[0008] Furthermore, the sending end persists the metadata information of each fragment in the local database. The metadata information includes at least: the oversized file identifier, the fragment number, the fragment size, the starting offset of the fragment in the original file, and the current sending status of the fragment.

[0009] Furthermore, when the receiving end first receives a fragment with an oversized file identifier, it creates an associated temporary file and a cache queue, and initializes the expected number.

[0010] Furthermore, when the receiving end receives a fragment belonging to the same super-large file identifier, if the fragment number is equal to the expected number, it writes it to a temporary file and updates the expected number; otherwise, it stores it in the cache.

[0011] Furthermore, when retrieving the cached fragments sequentially from the smallest number and writing them to the temporary file, the process includes: obtaining the current smallest numbered fragment in the cache, calculating its corresponding write offset in the original file based on the fragment number and the preset fragment size, moving the temporary file write pointer to the offset position, writing the fragments in the cache sequentially according to their numbers, and restoring the write pointer to the sequential write position corresponding to the current expected number after writing is completed.

[0012] Furthermore, the receiving end maintains a receiving timestamp for each ultra-large file identifier and monitors idle time. When the threshold is exceeded, packet loss is determined, and a list of fragments that have not been written to disk is generated by comparing the records already written to disk and encapsulating them as packet loss information.

[0013] Furthermore, after receiving the packet loss information, the sending end retransmits the lost fragments according to the network transmission topology.

[0014] Secondly, a system for transmitting and retransmitting large files in chunks based on a single-channel client is disclosed, including: The sending module is used to split files exceeding the threshold into multiple sequentially numbered fragments and transmit them concurrently through multiple front-end servers in a load-balanced mode. The receiving module is used to receive the fragments in a load-balanced mode through multiple back-end servers, and create temporary files and cache records; the receiving module is also used to: write the received fragments sequentially to a temporary file if their numbers match the expected numbers, otherwise store them in a cache; when the number of fragments accumulated in the cache reaches a set threshold, retrieve the fragments in the cache sequentially starting from the smallest number and write them to a temporary file. The packet loss detection module, located at the receiving end, is used to monitor the idle time of receiving fragments during transmission. When the idle time exceeds a preset threshold, it determines that packet loss has occurred, generates a packet loss record containing fragment information that has not been written, and feeds it back to the sending end. The retransmission module, located at the sending end, is used to retransmit the corresponding lost fragments according to the packet loss record.

[0015] Thirdly, a computer device is disclosed, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the steps of the method described above.

[0016] Fourthly, a computer-readable storage medium is disclosed having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0017] The above one or more technical solutions have the following beneficial effects: This invention divides extremely large files into ordered data blocks, and the receiving end assembles and writes them to disk according to the data block number sequence. This makes file transmission no longer limited by the continuous available storage space of the sending or receiving end, realizing the "theoretically unlimited" file transmission capability. It effectively solves the problem that extremely large files cannot be transmitted due to insufficient disk space and memory overflow in the traditional transmission mode.

[0018] This invention introduces a segmentation-level load balancing transmission mechanism on the basis of segmentation, which distributes different segments of the same ultra-large file to a cluster of multiple devices for concurrent transmission. This makes the transmission of a single ultra-large file no longer limited by the bandwidth limit of a single device. At the same time, by sharing the transmission load among multiple devices, the system stability and network transmission capacity utilization are effectively improved, and the cross-network transmission time of ultra-large files is greatly reduced.

[0019] This invention maintains a receiving timestamp at the receiving end and automatically determines packet loss based on idle timeout. Combined with the persistent recording of fragment information already written to disk in the local database at the receiving end, the sending end only needs to retransmit the lost fragments instead of retransmitting the entire file. This avoids the resource waste of global retransmission due to the failure of a single fragment in traditional solutions and achieves true breakpoint resume, significantly improving the transmission success rate and transmission efficiency in weak network environments.

[0020] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of the method for transmitting and retransmitting large files in fragments based on a single-channel client according to Embodiment 1 of the present invention; Figure 2 This is a module diagram of a single-channel client-based ultra-large file fragment transmission and loss retransmission system according to Embodiment 2 of the present invention. Detailed Implementation

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] Example 1 like Figure 1 As shown, this embodiment discloses a method for large file fragment transmission and retransmission based on a single-channel client, including: Step 1: The sending end splits files exceeding the threshold into multiple sequentially numbered fragments, which are then transmitted concurrently through multiple front-end servers in a load-balanced mode. Specifically: After the sending end initiates the file transfer task, it obtains the file size of the file to be transferred. If the file size does not exceed the preset large file threshold, it is processed according to the normal small file transfer process; if the file size exceeds or equals the preset large file threshold, the file is identified as a large file and transferred in a fragmented mode.

[0027] After entering the fragmentation mode, the sending end performs a splitting operation on the ultra-large file according to the preset fragment size, generating multiple fragments. Each fragment is assigned a sequential number starting from 1, i.e., a fragment sequence number, used to identify its order in the original file. At the same time, a globally unique ultra-large file identifier is generated for all fragments belonging to the same ultra-large file, used to distinguish fragments of different ultra-large files. The fragment sequence number and the ultra-large file identifier together constitute a globally unique index for each fragment.

[0028] After the data is split, the sending end persists the metadata information of each fragment to a local database. The metadata information includes: the identifier of the oversized file, the fragment sequence number, the fragment size, the starting offset of the fragment in the original file, and the current sending status of the fragment. The persisted information is used to support breakpoint resumption after the sending end restarts abnormally, as well as accurate retransmission of subsequently lost fragments.

[0029] The sending end is pre-configured and load-balanced, establishing a connection pool with multiple front-end servers. These front-end servers are deployed in an untrusted network domain, forming a front-end server cluster.

[0030] The sending end distributes multiple fragments of the same large file concurrently to different server nodes in the front-end server cluster through a load balancing mode. Specifically, the load balancing mode distributes the fragment tasks to each front-end server in a cyclical manner according to the order of the fragment tasks. During each polling, servers whose tasks have not yet been completed are skipped to avoid overloading a single node while other nodes are idle, thereby maximizing resource utilization and optimizing system performance.

[0031] Through the aforementioned concurrent distribution mechanism, different fragments of the same ultra-large file can be transmitted simultaneously through multiple front-end servers, thereby breaking through the bandwidth limitation of a single device and significantly improving the timeliness of cross-network transmission of ultra-large files.

[0032] After receiving the data fragments, each front-end server can adopt one of the following two processing methods based on the current network conditions and service load: First, temporarily store the fragments on local storage media and forward them again when the network is less busy; second, directly transmit the fragments to the trusted network domain via a one-way import device cluster. The one-way import device cluster consists of two or more one-way import devices, and its clustered deployment achieves secure isolation between networks and one-way data transmission.

[0033] After the front-end server completes the fragment forwarding, it backs up the information of the forwarded large files locally and persistently records the backup logs to provide a basis for data recovery in case of subsequent link failures. After being transmitted through a one-way import device, the fragments arrive at multiple back-end servers deployed in a trusted network domain.

[0034] Step 2: The receiving end receives the fragments through multiple back-end servers in a load-balanced mode, and creates temporary files and cache records; When the receiver starts up, it modifies and loads the receiver configuration file, enables load balancing mode, and establishes a connection pool with multiple back-end servers. These multiple back-end servers are deployed in a trusted network domain, forming a back-end server cluster, which is connected to the front-end server cluster in an untrusted network domain via a one-way import device.

[0035] The adjustable parameters in the receiver configuration file include the large file data packet buffer threshold and the receive idle timeout. The large file data packet buffer threshold is used to control the maximum number of fragments temporarily stored in the receiver's buffer queue to avoid memory overflow. The receive idle timeout is used for subsequent packet loss judgment, and its specific value can be dynamically set according to network quality.

[0036] Step 3: For the received fragments, if their numbers match the expected numbers, they are written sequentially to a temporary file; otherwise, they are stored in a cache. When the number of fragments accumulated in the cache reaches a set threshold, the fragments in the cache are sequentially retrieved from the smallest number and written to a temporary file. After receiving the fragmented data packets through the backend server in load balancing mode, the receiving end first identifies the file type to which the fragmented data packet belongs. If the fragmented data packet carries an oversized file identifier, it is determined to be an oversized file fragmented data packet and is placed in the oversized file data packet queue for processing; if it is a normal small file data packet, it is directly written to disk according to the normal process.

[0037] When the disk write thread processes a fragment belonging to a certain very large file identifier for the first time, it performs the following initialization operations: 1) Create a temporary file in the storage medium associated with the identifier of the oversized file, which will be used to eventually assemble the complete file; 2) Create a cache queue corresponding to the identifier of the large file to temporarily store fragments that have not yet met the sequential write conditions; 3) Initialize the expected fragment number to be received, setting the initial value to 1; 4) Determine if the current fragment number is 1: If yes, immediately write the fragment data to the beginning of the temporary file and update the expected number to 2; if not, store the fragment in the cache queue, keep the expected number 1 unchanged, and wait for the fragment with number 1 to arrive before triggering sequential writing.

[0038] When the disk write thread subsequently receives another fragment belonging to the same large file identifier, and the cached record corresponding to that large file identifier already exists, the following judgment logic is executed: 1) If the number of the current data packet is equal to the expected number of the current record, then the data packet is immediately appended to the current write pointer position of the temporary file. After writing is completed, the write pointer is moved forward by the length of the data packet, and the expected number is updated to the current expected number + 1. 2) If the number of the current fragment is not equal to the expected number of the current record, the fragment is stored in the cache queue, the expected number remains unchanged, and it is written sequentially together after the missing preceding fragment arrives.

[0039] The receiving end monitors the number of fragments temporarily stored in the cache queue in real time. When the number of fragments in the cache queue reaches the preset cache threshold, the following disk write operation is performed: the smallest numbered fragment of the current record is retrieved from the cache queue, and its corresponding write offset in the original file is calculated based on the fragment number and the preset fragment size. The offset is the byte position calculated from the beginning of the file. The offset of the nth fragment is equal to n minus one multiplied by the fragment size, that is, offset=(n-1)*fragment size. Then the write pointer of the temporary file is moved to the offset position.

[0040] Next, starting from the smallest numbered fragment, the fragments in the cache are written to the corresponding positions in the temporary file in ascending order of their numbers. After the above cached data is written, the write pointer of the temporary file is restored to the original writing position, that is, the writing position corresponding to the current expected number, so that the fragments that arrive in order can be directly appended.

[0041] Step 4: During transmission, the receiving end monitors the idle time of the received fragments. When the idle time exceeds a preset threshold, packet loss is determined, a packet loss record containing fragment information that was not written is generated, and fed back to the sending end. Specifically: After each fragment is completely written to disk in a temporary file, the receiving end immediately persists the disk write information of that fragment in the local database. The disk write information includes the oversized file identifier, the fragment number that has been written to disk, and the disk write completion timestamp; Through the aforementioned persistent records, the receiving end can obtain in real time the set of fragments that have been written to disk for any very large file, providing a data foundation for packet loss detection and breakpoint resumption.

[0042] During transmission, the receiving end maintains a corresponding reception timestamp for each large file identifier being received. Whenever a fragment belonging to that large file identifier is received, regardless of whether the fragment arrives in sequence or out of order, the reception timestamp is immediately updated to the current system time.

[0043] The receiving end simultaneously starts an independent idle monitoring thread to periodically calculate the difference between the current system time and the received timestamp, which is used as the idle time for receiving the very large file.

[0044] During transmission, when the idle monitoring thread detects that the receiving idle time corresponding to a certain large file identifier exceeds the preset receiving idle timeout threshold, the receiving end determines that the large file has been fragmented and lost in the current transmission link.

[0045] The preset receive idle timeout threshold can be flexibly adjusted in the receiver configuration file according to the network environment to adapt to the packet loss detection sensitivity requirements under different network quality conditions.

[0046] Upon detecting packet loss, the receiving end immediately queries its local database to retrieve the set of disk-written fragment IDs corresponding to the large file identifier. Then, it compares the total number of fragments of the large file with the set of disk-written fragment IDs to calculate a list of fragment IDs that have not yet been written to disk. This list of fragment IDs is then encapsulated into a packet loss information file.

[0047] The receiving end uploads the generated packet loss information file through a backend server and executes different feedback paths according to the network transmission topology, ultimately feeding back the packet loss information to the sending end.

[0048] After each segment of the file is persisted, the receiving end checks whether all segments of the large file have been written to disk. If the number of segments written to disk equals the total number of segments in the large file, then all segments are considered to have been successfully received and written to disk. At this point, the file handle of the temporary file is closed to ensure that the data is completely flushed to the storage medium, and the temporary file is renamed to the final target file name. Simultaneously, all persistent records related to the large file identifier in the local database are deleted, including segment write-to-disk records and cached status records, to free up storage resources.

[0049] Step 5: The sending end retransmits the corresponding lost fragments according to the packet loss record.

[0050] After the receiving end generates a packet loss information file, it uses different feedback paths to transmit the packet loss information to the sending end, depending on the network transmission topology. The sending end then performs precise retransmission of the lost fragments accordingly. Specifically: The steps for packet loss feedback and retransmission in a unidirectional transmission environment include: a. During transmission, the receiving end refreshes the receiving timestamp corresponding to the large file identifier to the current system time each time it receives a fragment that belongs to the large file identifier.

[0051] The difference between the received timestamp and the current system time is defined as the receiving idle time. When the receiving idle time exceeds the preset receiving idle timeout threshold in the configuration file, the receiving end determines that the large file has been lost.

[0052] b. After determining packet loss, the receiving end immediately queries the local database to obtain the set of fragment numbers that have been written to disk corresponding to the large file identifier, calculates the list of fragment numbers that have not yet been written to disk, writes the fragment information that has not been written to disk into the packet loss information file, and uploads it to the backend server.

[0053] c. After the receiving end initially uploads the packet loss information file, if it subsequently receives data packets belonging to the same large file and completes disk write-to-disk, the set of fragments already written to disk will change. When the receiving idle time exceeds the preset threshold again, the receiving end recalculates the list of fragment numbers that have not yet been written to disk and compares it with the packet loss record uploaded last time.

[0054] If the packet loss record changes, meaning that some previously lost fragments have successfully arrived and been written to disk, the receiving end will re-upload the latest packet loss information file to the backend server; if the packet loss record remains unchanged, it will not be re-uploaded. The backend server synchronously receives and updates the latest packet loss information for this large file.

[0055] d. When the backend server receives the packet loss information file, it displays the current status of the large file transfer on the large file transfer details page, including three statuses: completed, in progress, and packet loss. In the packet loss status, a packet loss information download function is provided, and the packet loss information file can be manually downloaded and stored.

[0056] The front-end server management interface provides a function to import packet loss information files on the details page of very large files. Users can manually download the packet loss information file from the back-end management interface and then import the file into the front-end server through the front-end management interface.

[0057] The front-end server parses the packet loss information file, extracting the identifier of the oversized file and the user information to which it belongs. When the sending user is online, the front-end server immediately pushes the packet loss information to that online user.

[0058] e. After receiving the packet loss information pushed by the front-end server, the sending end parses the list of fragment numbers that have not been written to disk. Based on the list of lost fragment numbers, the sending end reads the corresponding fragment data from local storage and only retransmits the lost fragments of the large file to the front-end server, instead of retransmitting the entire large file.

[0059] f. After receiving the retransmitted fragment from the sender, the receiving end first determines whether the temporary file corresponding to the oversized file identifier still exists in the local storage medium: (a) If a temporary file exists, continue to determine whether the number of the current retransmitted fragment is equal to the current expected receive number; if yes, immediately write the fragment data to the temporary file; if not, store the fragment in the buffer queue, and the program logic is consistent with the processing method during normal fragment transmission. (b) If the temporary file does not exist, the retransmission is deemed to have failed. In this case, it indicates that the temporary file of the large file that has not yet been written to disk has been manually deleted, and the receiving end refuses to accept the retransmission fragment to ensure data consistency.

[0060] The steps for packet loss feedback and retransmission in a bidirectional transmission environment include: a. When the receiving end detects that the receiving idle time exceeds the preset threshold, it determines that there is packet loss, obtains the fragment information that has not been written to disk, generates a packet loss information file, and uploads it to the backend server.

[0061] b. After receiving the packet loss information file uploaded by the receiving end, the back-end server automatically pushes the packet loss information file to the front-end server without waiting for manual intervention, since the back-end server and the front-end server support bidirectional communication.

[0062] After receiving the packet loss information file, the front-end server parses the user information within it. If the sending user is online, the server immediately pushes the packet loss information to that online user. The entire push process requires no manual intervention, achieving automatic feedback of packet loss information.

[0063] c. After receiving the packet loss information, the sending end parses the list of fragment numbers that have not been written to disk and only retransmits the lost fragments to the front-end server.

[0064] d. After receiving the retransmission data packet, the receiving end checks whether the temporary file exists. If it exists, it is processed according to the normal disk write logic; if it does not exist, the retransmission is deemed to have failed.

[0065] e. After the sending end completes the retransmission of all lost fragments, the receiving end writes the retransmitted fragments to a temporary file according to the normal disk write process and updates the corresponding fragment disk write record in the local database. When all fragments of the large file have been written to disk, the temporary file closing and renaming operation is triggered, completing the transmission of the entire large file.

[0066] If there are still fragments that have not been written to disk after retransmission, and some retransmitted fragments are lost again due to network interruption, the receiving end will continue to trigger idle timeout packet loss judgment and repeat the above retransmission process until all fragments are written to disk or the preset maximum number of retransmissions is reached, and then an error will be reported and the process will be exited.

[0067] In addition, if the receiving end experiences an abnormal restart or service restart during operation, it will first scan the disk write information in the local database after startup. For large files with fragmented disk write records but not yet marked as complete (i.e., there are still unwritten fragments), the receiving end will then read all the disk-written fragment numbers corresponding to the large file identifier in the database, thereby reconstructing the set of disk-written fragments.

[0068] Then, based on the fragment number already written to disk, the next expected number to be received is calculated, and the buffer queue state is rebuilt accordingly. At the same time, the corresponding temporary file is reopened, and the file write pointer is positioned at the sequential write position. After that, the normal receiving state is restored, and the system continues to wait for the arrival of subsequent fragments or trigger packet loss retransmission.

[0069] Through the aforementioned persistence and recovery mechanisms, the receiving end can seamlessly resume the receiving progress of extremely large files after an abnormal restart, without having to retransmit the entire file, thus truly realizing the function of resuming interrupted transmissions.

[0070] Example 2 Based on the method described in Embodiment 1, the purpose of this embodiment is to provide a system for transmitting and retransmitting large files in fragments using a single-channel client, including: The sending module is used to split files exceeding the threshold into multiple sequentially numbered fragments and transmit them concurrently through multiple front-end servers in a load-balanced mode. The receiving module is used to receive the fragments in a load-balanced mode through multiple back-end servers, and create temporary files and cache records; the receiving module is also used to: write the received fragments sequentially to a temporary file if their numbers match the expected numbers, otherwise store them in a cache; when the number of fragments accumulated in the cache reaches a set threshold, retrieve the fragments in the cache sequentially starting from the smallest number and write them to a temporary file. The packet loss detection module, located at the receiving end, is used to monitor the idle time of receiving fragments during transmission. When the idle time exceeds a preset threshold, it determines that packet loss has occurred, generates a packet loss record containing fragment information that has not been written, and feeds it back to the sending end. The retransmission module, located at the sending end, is used to retransmit the corresponding lost fragments according to the packet loss record.

[0071] Example 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0072] Example 4 The purpose of this embodiment is to provide a computer-readable storage medium.

[0073] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0074] The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0075] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0076] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for transmitting and retransmitting large files in fragments based on a single-client architecture, characterized in that: include: The sending end splits files exceeding the threshold into multiple sequentially numbered fragments, which are then transmitted concurrently through multiple front-end servers in a load-balanced mode. The receiving end receives the fragments in a load-balanced mode through multiple back-end servers, creates temporary files and cache records, and writes the received fragments sequentially to the temporary file if their numbers match the expected numbers; otherwise, they are stored in the cache. When the number of fragments accumulated in the cache reaches a set threshold, the fragments in the cache are sequentially retrieved from the smallest number and written to the temporary file. During transmission, the receiving end monitors the idle time of receiving fragments. When the idle time exceeds a preset threshold, it determines that packet loss has occurred, generates a packet loss record containing fragment information that has not been written, and feeds it back to the sending end. The sending end retransmits the corresponding lost fragments based on the packet loss record.

2. The method for large file fragment transmission and retransmission based on a single-client architecture as described in claim 1, characterized in that, The sending end persists the metadata information of each fragment in the local database. The metadata information includes at least: the oversized file identifier, the fragment number, the fragment size, the starting offset of the fragment in the original file, and the current sending status of the fragment.

3. The method for large file fragment transmission and retransmission based on a single-client architecture as described in claim 1, characterized in that, When the receiving end first receives a fragment with a very large file identifier, it creates an associated temporary file and a cache queue, and initializes the expected number.

4. The method for large file fragment transmission and retransmission based on a single-client architecture as described in claim 1, characterized in that, When the receiving end receives a fragment belonging to the same super-large file identifier, if the fragment number is equal to the expected number, it writes it to a temporary file and updates the expected number; otherwise, it stores it in the cache.

5. The method for large file fragment transmission and retransmission based on a single-client architecture as described in claim 1, characterized in that, When retrieving and writing the cached fragments sequentially from the smallest number to the temporary file, the process includes: obtaining the current smallest numbered fragment in the cache, calculating its corresponding write offset in the original file based on the fragment number and the preset fragment size, moving the temporary file write pointer to the offset position, writing the fragments in the cache sequentially according to their numbers, and restoring the write pointer to the sequential write position corresponding to the current expected number after writing is completed.

6. The method for large file fragment transmission and retransmission based on a single-client architecture as described in claim 1, characterized in that, The receiving end maintains a receiving timestamp for each ultra-large file identifier and monitors idle time. When the threshold is exceeded, packet loss is determined. The received file is compared with the records that have been written to disk to generate a list of fragments that have not been written to disk, and then encapsulated as packet loss information.

7. The method for large file fragment transmission and retransmission based on a single-client architecture as described in claim 1, characterized in that, After receiving the packet loss information, the sending end retransmits the lost fragments according to the network transmission topology.

8. A system for transmitting and retransmitting large files in fragments based on a single-client architecture, characterized in that: include: The sending module is used to split files exceeding the threshold into multiple sequentially numbered fragments and transmit them concurrently through multiple front-end servers in a load-balanced mode. The receiving module is used to receive the fragments in a load-balanced mode through multiple back-end servers, and create temporary files and cache records; the receiving module is also used to: write the received fragments sequentially to a temporary file if their numbers match the expected numbers, otherwise store them in a cache; when the number of fragments accumulated in the cache reaches a set threshold, retrieve the fragments in the cache sequentially starting from the smallest number and write them to a temporary file. The packet loss detection module, located at the receiving end, is used to monitor the idle time of receiving fragments during transmission. When the idle time exceeds a preset threshold, it determines that packet loss has occurred, generates a packet loss record containing fragment information that has not been written, and feeds it back to the sending end. The retransmission module, located at the sending end, is used to retransmit the corresponding lost fragments according to the packet loss record.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the single-channel client-based ultra-large file fragmentation transmission and loss retransmission method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the steps of the single-channel client-based ultra-large file fragmentation transmission and loss retransmission method as described in any one of claims 1-7.

Images