A ship-shore data disordering synchronization method and device, electronic equipment and storage medium

Abstract

This invention discloses a method, apparatus, electronic device, and storage medium for out-of-order synchronization of ship-to-shore data. By initiating a data synchronization request to the shore end, the compressed information of the compressed package to be synchronized is sent to the shore end, enabling the shore end to perform data synchronization detection based on the compressed information and obtain a full bitmap of the data synchronization status. Based on the full bitmap, breakpoint detection is performed on data blocks to obtain breakpoint data. If the number of breakpoint data exceeds a certain threshold, the breakpoint data is resent to the shore end; otherwise, the data blocks after the target block are sequentially synchronized to the shore end. The block reception status at the shore end is periodically acquired, and the full bitmap is updated based on the block reception status. The process returns to the step of breakingpoint detection on data blocks based on the full bitmap to obtain breakpoint data, initiating a new round of data synchronization iterations until all data blocks of the compressed package are synchronized. This invention can efficiently achieve ship-to-shore data synchronization and can be widely applied in the field of data processing technology.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, electronic device, and storage medium for out-of-order synchronization of ship-to-shore data. Background Technology

[0002] Current ship-to-shore data synchronization technologies primarily rely on TCP (Transmission Control Protocol). This approach aligns with the data synchronization strategy between two points on land, where a TCP communication channel is first established between the two points, and then synchronization data is sent through this channel. However, existing technologies have a significant problem: because data is sent sequentially and retransmitted, much data that has already been successfully received by the receiving end is still retransmitted. This repeated transmission results in a large amount of retransmitted data, thus affecting the overall transmission speed. Therefore, current technologies still require further improvements and optimizations to enhance transmission efficiency. Summary of the Invention

[0003] This invention aims to at least partially address the limitations of related technologies. To this end, this invention proposes a method, apparatus, electronic device, and storage medium for out-of-order ship-to-shore data synchronization, which can efficiently achieve ship-to-shore data synchronization.

[0004] On one hand, embodiments of the present invention provide a ship-to-shore out-of-order data synchronization method, applied to the ship's end, the method comprising the following steps:

[0005] The data synchronization request initiated to the shore sends the compression information of the compressed package to be synchronized to the shore, so that the shore can perform data synchronization detection based on the compression information and obtain a full bitmap of the data synchronization status of the shore.

[0006] The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes synchronization complete, synchronization in progress, and waiting for synchronization.

[0007] Breakpoint detection is performed on data blocks based on the full bitmap to obtain breakpoint data; if the number of breakpoint data is greater than a first number, the breakpoint data is resent to the shore end; otherwise, the data blocks after the target block are sequentially synchronized to the shore end.

[0008] Among them, the synchronization status of breakpoint data is "synchronizing", and breakpoint data indicates that there is a data block whose synchronization status is "completed" after it; the target block indicates the data block with the largest sequence number whose synchronization status is "completed".

[0009] Periodically obtain the segmented reception status from the shore end, and update the full bitmap based on the segmented reception status;

[0010] Return to the step of performing breakpoint detection on the data blocks based on the full bitmap, obtain the breakpoint data, and start a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

[0011] Optionally, periodically obtain the segment reception status at the shore end, including at least one of the following steps:

[0012] When the number of data blocks received by the shore end in a data synchronization iteration is greater than the second number, the shore end's block reception status is obtained in response to the shore end's reply.

[0013] Periodically send block verification requests and heartbeat requests to the shore end so that the shore end can respond to the block verification requests and / or heartbeat requests with block reception status, thereby periodically obtaining the block reception status of the shore end.

[0014] Optionally, the block reception status includes a bitmap start ID and incremental bitmap information. The bitmap start ID corresponds to the largest sequence number among all data blocks that have completed data synchronization; the bitmap start ID represents the sequence number corresponding to the starting bitmap, the synchronization status of the starting bitmap is completed, and the starting bitmap represents that there is no preceding data block with the largest sequence number that is in the synchronization state; updating the full bitmap based on the block reception status includes the following steps:

[0015] Update the synchronization status of the starting bitmap and all preceding data blocks to complete synchronization;

[0016] The incremental bitmap information includes the synchronization status of a preset number of data blocks following the data block corresponding to the starting bitmap.

[0017] Based on the incremental bitmap information, the synchronization status of a preset number of data blocks following the starting bitmap is updated.

[0018] Among them, the status update includes updating the synchronization status of the data blocks to be transmitted in the next round of data synchronization iteration from waiting for synchronization to synchronization.

[0019] On the other hand, embodiments of the present invention provide a ship-to-shore out-of-order data synchronization device, applied at the ship's end, comprising:

[0020] The first module is used to initiate a data synchronization request to the shore end, sending the compression information of the compressed package to be synchronized to the shore end so that the shore end can perform data synchronization detection based on the compression information and obtain a full bitmap of the data synchronization status of the shore end.

[0021] The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes completed synchronization, in the process of synchronization, and waiting for synchronization.

[0022] The second module is used to detect breakpoints in data blocks based on the full bitmap and obtain breakpoint data; if the number of breakpoint data is greater than the first number, the breakpoint data is resent to the shore end; otherwise, the data blocks after the target block are sequentially synchronized to the shore end.

[0023] Among them, the synchronization status of breakpoint data is "synchronizing", and breakpoint data indicates that there is a data block whose synchronization status is "completed" after it; the target block indicates the data block with the largest sequence number whose synchronization status is "completed".

[0024] The third module is used to periodically obtain the segmented reception status of the shore end and update the full bitmap based on the segmented reception status;

[0025] The fourth module is used to return to the execution of the second module and perform a new round of data synchronization iterations until all data blocks of the compressed package are synchronized.

[0026] On the other hand, embodiments of the present invention provide a ship-to-shore out-of-order data synchronization method, applied to the shore end. This method interacts with the aforementioned ship-to-shore out-of-order data synchronization method, and includes the following steps:

[0027] In response to the data synchronization request from the ship, obtain the compression information of the compressed package to be synchronized; perform data synchronization detection based on the compression information to obtain a full bitmap of the data synchronization status; and send the full bitmap to the ship.

[0028] The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes synchronization complete, synchronization in progress, and waiting for synchronization.

[0029] The out-of-order data blocks sent by the ship are received and stored sequentially based on the sequence number;

[0030] Based on the storage status of data blocks, the system periodically replies to the ship's end with the status of block reception, so that the ship's end can update the full bitmap based on the status of block reception.

[0031] Return to the step of executing the out-of-order data chunks sent by the receiving ship, and start a new round of data synchronization iterations until all data chunks of the compressed package are synchronized.

[0032] Optionally, the ship periodically sends block verification requests and / or heartbeat requests to the shore, and periodically replies to the ship regarding block reception status, including at least one of the following steps:

[0033] When the number of data blocks received in a data synchronization iteration is greater than the second number, the block reception status is replied to the ship end.

[0034] In response to the block verification requests and / or heartbeat requests periodically sent by the ship, reply to the ship with the block reception status.

[0035] Optionally, the ship periodically sends a block verification request to the shore, and the method further includes the following steps:

[0036] In response to the block verification request from the ship, obtain the first verification value of the data block sent by the ship;

[0037] The second check value of the received data block is obtained by processing it with a preset message digest algorithm;

[0038] Data consistency is verified based on the first and second check values ​​corresponding to each synchronized data block.

[0039] When the data consistency check fails, a retransmission request for the corresponding data block is sent to the ship, so that the ship responds to the retransmission request and retransmits the corresponding data block.

[0040] On the other hand, embodiments of the present invention provide a ship-to-shore out-of-order data synchronization device, which interacts with the aforementioned ship-to-shore out-of-order data synchronization device and includes:

[0041] The fifth module is used to respond to the data synchronization request from the ship end, obtain the compression information of the compressed package to be synchronized; perform data synchronization detection based on the compression information to obtain a full bitmap of the data synchronization status; and send the full bitmap to the ship end.

[0042] The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes synchronization complete, synchronization in progress, and waiting for synchronization.

[0043] The sixth module is used to receive out-of-order data blocks sent by the ship and store the received data blocks sequentially based on the sequence number.

[0044] The seventh module is used to periodically reply to the ship terminal with the block reception status based on the storage status of data blocks, so that the ship terminal can update the full bitmap based on the block reception status.

[0045] The eighth module is used to return to the execution of the sixth module and perform a new round of data synchronization iterations until all data blocks of the compressed package have been synchronized.

[0046] Optionally, the ship periodically sends a block verification request to the shore, and the device also includes:

[0047] The ninth module is used to respond to the block verification request from the ship and obtain the first verification value of the data block sent by the ship.

[0048] The tenth module is used to obtain the second check value of the received data block through a preset message digest algorithm;

[0049] Module 11 is used to perform data consistency verification based on the first and second check values ​​corresponding to each synchronized data block;

[0050] The twelfth module is used to send a retransmission request for the corresponding data block to the ship when the data consistency check fails, so that the ship can respond to the retransmission request and retransmit the corresponding data block.

[0051] On the other hand, embodiments of the present invention provide an electronic device, including: a processor and a memory; the memory is used to store a program; the processor executes the program to implement the above-described ship-shore data out-of-order synchronization method.

[0052] On the other hand, embodiments of the present invention provide a computer storage medium storing a processor-executable program, which, when executed by a processor, is used to implement the above-described ship-shore data out-of-order synchronization method.

[0053] This invention, through a data synchronization request initiated to the shore end, sends the compression information of the compressed package to be synchronized to the shore end, enabling the shore end to perform data synchronization detection based on the compression information and obtain a full bitmap of the data synchronization status. The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes completed synchronization, in progress synchronization, and waiting for synchronization. Breakpoint detection is performed on the data blocks based on the full bitmap to obtain breakpoint data. If the number of breakpoint data exceeds a certain threshold, the breakpoint data is resent to the shore end; otherwise, the data blocks after the target block are sequentially synchronized to the shore end. The synchronization status of breakpoint data is "in progress synchronization," indicating that there are subsequent data blocks with a synchronized status of "completed synchronization." The target block represents the data block with the largest sequence number that has a synchronized status of "completed synchronization." The block reception status at the shore end is periodically obtained, and the full bitmap is updated based on the block reception status. The process then returns to the previous step of performing breakpoint detection on the data blocks based on the full bitmap to obtain breakpoint data, initiating a new round of data synchronization iterations until all data blocks of the compressed package are synchronized. The embodiments of the present invention include at least the following beneficial effects:

[0054] Improve synchronization efficiency: By sending compressed information of the compressed package to the shore end, the shore end can perform data synchronization detection based on this information, thereby reducing unnecessary data transmission and improving synchronization efficiency.

[0055] Breakpoint resume capability: The solution performs breakpoint detection on data blocks. When the number of breakpoint data exceeds the first limit, only the breakpoint data is retransmitted to the shore end, avoiding the repeated transmission of a large amount of synchronized data and further improving synchronization efficiency.

[0056] Intelligent synchronization strategy: The synchronization strategy is dynamically adjusted based on the data block synchronization status in the full bitmap. For data blocks that have already been synchronized, they are no longer transmitted repeatedly. Instead, the data blocks after the target block are sequentially synchronized to the shore end, realizing an intelligent synchronization process.

[0057] Real-time updates and iterations: The system periodically acquires the segmented reception status from the shore end and updates the full bitmap accordingly. Then, it returns to execute the breakpoint detection step and begins a new round of data synchronization iterations. This real-time update and iteration mechanism ensures the continuity and accuracy of the synchronization process.

[0058] Comprehensive synchronization status monitoring: The full bitmap contains the sequence number and synchronization status of each data block in the compressed package, making the monitoring of the synchronization process more comprehensive and detailed, which helps to identify and solve problems in a timely manner.

[0059] In summary, this technical solution significantly improves the performance and reliability of data synchronization from ship to shore by enhancing synchronization efficiency, enabling breakpoint resume, employing intelligent synchronization strategies, real-time updates and iterations, and providing comprehensive synchronization status monitoring. Attached Figure Description

[0060] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0061] Figure 1 This is a schematic diagram of an implementation environment for out-of-order synchronization of ship-to-shore data provided in an embodiment of the present invention.

[0062] Figure 2 A flowchart illustrating a method for out-of-order synchronization of ship-to-shore data applied to a ship's end, provided in an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of the full bitmap principle for breakpoint data synchronization interaction provided in an embodiment of the present invention;

[0064] Figure 4 A flowchart illustrating a method for out-of-order synchronization of ship-to-shore data applied to the shore end, provided in an embodiment of the present invention;

[0065] Figure 5 This is a schematic diagram illustrating the principle and process of conventional ship-to-shore data synchronization and interaction.

[0066] Figure 6 This is a schematic diagram of an extended process for a shore-based out-of-order data synchronization method provided in an embodiment of the present invention.

[0067] Figure 7A schematic diagram illustrating the overall business process principle of the ship-shore out-of-order data synchronization method provided in this embodiment of the invention;

[0068] Figure 8 A schematic diagram of a ship-to-shore out-of-order data synchronization device applied to the ship's end, provided in an embodiment of the present invention;

[0069] Figure 9 A schematic diagram of a ship-shore out-of-order data synchronization device applied to the shore end, provided in an embodiment of the present invention;

[0070] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0072] It should be noted that although functional modules are divided in the system diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100," "second / S200," etc., in the specification, claims, and the aforementioned figures are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0073] In this invention, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this invention can be combined with other embodiments.

[0074] It is understood that the ship-to-shore out-of-order data synchronization method provided in this embodiment of the invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various types of terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal can be a smartphone, tablet, laptop, or desktop computer, but it is not limited to these.

[0075] To facilitate understanding of the technical solution of this invention, the technical terms that may appear in the embodiments of this invention will first be explained:

[0076] VSAT Satellite Communication (Very Small Aperture Terminal): Because VSAT originated from traditional satellite communication systems, it is also called a satellite small data station or personal earth station. The "small" here refers to the small antenna aperture of the small station equipment in the VSAT system, which is usually 0.3m to 1.4m. The equipment is compact, solid-state, intelligent, inexpensive, easy to install, has low requirements for the operating environment, and is not limited by the terrestrial network, making networking flexible.

[0077] A drilling vessel is a ship specifically designed for drilling operations on seabed geological structures, primarily used for marine geological exploration. With the deepening implementation of the national maritime strategy, my country's drilling vessels are increasingly appearing in international oceans such as the Pacific Ocean. At this time, the only means of communication between the ships and shore-based bases is satellite.

[0078] The Transmission Control Protocol (TCP) is a connection-oriented, reliable, byte-stream-based transport layer communication protocol. TCP is designed to adapt to layered protocol hierarchies that support multiple network applications. Pairs of processes in host computers connected to different but interconnected computer networks rely on TCP for reliable communication services. TCP assumes it can obtain simple, potentially unreliable datagram services from lower-level protocols. In principle, TCP should be able to operate over a variety of communication systems, from hardwired connections to packet-switched or circuit-switched networks.

[0079] The User Datagram Protocol (UDP) is based on a connectionless, unreliable datagram transmission mechanism. It only provides source port, destination port, data length, checksum, and encapsulation to achieve fast and simple end-to-end communication. It does not guarantee ordered arrival, reliable transmission, or flow control of data. It is suitable for applications that are latency-sensitive, can tolerate packet loss, and have self-correcting capabilities, such as real-time audio and video streaming, online games, network broadcasting, and DNS queries.

[0080] LZMA Compression Algorithm: In practical compression algorithms, LZMA (Lempel-Ziv-Markov chain Algorithm) is generally considered to offer extremely high compression ratios. Proposed in 1998, it is an improved version of the LZ77 algorithm used to implement the 7-Zip file format (.7z). LZMA uses a chained compression method and applies a modified LZ77 algorithm at the bit level to achieve superior compression efficiency, especially when processing highly repetitive data such as text and source code.

[0081] LZMA2 compression algorithm: As a successor to LZMA, LZMA2 further improves compression performance and multi-core processor support while maintaining a high compression ratio. In practical applications, such as using 7-Zip software, choosing the LZMA2 compression method often yields very high compression ratios.

[0082] MD5 (Message-Digest Algorithm 5): It is a widely used cryptographic hash function designed to convert information of arbitrary length into a fixed-length hash value. This process is one-way, meaning it is very difficult to deduce the original information from the hash value. The core significance of MD5 lies in providing a data verification mechanism. By calculating the MD5 value of data, it is possible to efficiently compare whether the data is consistent without directly comparing the data itself. It is implemented through complex mathematical operations, and even small changes in the data can lead to significant differences in the calculated MD5 value.

[0083] like Figure 1 The diagram shown is a schematic representation of an implementation environment provided by an embodiment of the invention. (Refer to...) Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected via a network, either wirelessly or via a wired connection, to complete data transmission and exchange.

[0084] Server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0085] Additionally, server 101 can also be a node server in a blockchain network. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms.

[0086] Terminal 102 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. Terminal 102 and server 101 can be directly or indirectly connected via wired or wireless communication, and this embodiment of the invention does not impose any limitations.

[0087] Exemplary based on Figure 1 The implementation environment shown in this embodiment of the invention provides a ship-to-shore out-of-order data synchronization method. The following description uses the application of this ship-to-shore out-of-order data synchronization method in server 101 as an example. It can be understood that this ship-to-shore out-of-order data synchronization method can also be applied to terminal 102.

[0088] Reference Figure 2 , Figure 2 This is a flowchart illustrating a ship-to-shore out-of-order data synchronization method applied to a server, provided in an embodiment of the present invention. The executing entity of this ship-to-shore out-of-order data synchronization method can be any of the aforementioned computer devices (including servers or terminals). (Refer to...) Figure 2 This method, applied to the ship's bow, includes the following steps:

[0089] S100: Initiate a data synchronization request to the shore end, and send the compression information of the compressed package to be synchronized to the shore end so that the shore end can perform data synchronization detection based on the compression information and obtain a full bitmap of the data synchronization status of the shore end.

[0090] The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes synchronization complete, synchronization in progress, and waiting for synchronization.

[0091] For example, in some specific implementations, the ship sends a data synchronization request, which is then forwarded to the shore via satellite. This request mainly includes detailed information about the compressed data package being transmitted, such as the file name, file size, file MD5 hash, file creation time, last modification time, and so on. This information is primarily used to determine whether interrupted transmission needs to be resumed.

[0092] When the shore end receives a data synchronization request, it will check the files that have been transferred on the shore end. The main judgment process is as follows:

[0093] 1) Verify the file name and check if the file exists on the other end. If it does not exist, it indicates that this is the first time the file has been transmitted. If it still does not exist, continue comparing the file's MD5 hash.

[0094] 2) Verify the MD5 hash of the files. Check if the MD5 hashes of the files on the other end and the target file match. If they don't match, the target file has been changed and needs to be retransmitted. If they match, continue comparing file sizes.

[0095] 3) Verify the bitmap information and file size. If they match, the file has been transferred before, and you can choose to resume the transfer. If they don't match, the target file has changed, and you need to retransmit it. This step is necessary because the file is stored in a cache before successful transfer, and the cache may be modified or cleared.

[0096] 4) Verify the bitmap information. If all data packets have been received, the data synchronization is successful; otherwise, retransmission or resuming interrupted transmission is required. If the data has been successfully transmitted, check the final verification flag on the shore. If verification is complete, reply to the shipside that synchronization is successful; otherwise, reply that final verification is required.

[0097] Once the inspection is complete, the monitored information will be directly fed back to the ship, mainly including file information already existing on the shore (filename, MD5, file bitmap, file size, last modification time, transmission progress, etc.). Note that the bitmap here is the full bitmap, meaning the entire bitmap needs to be synchronized to the ship.

[0098] It should be noted that in some embodiments, the method may further include a step of locally compressing the files to be transmitted before data synchronization; for example, in some specific embodiments, this can be implemented as follows:

[0099] There are usually many files to be transmitted on board, mainly test results from technical experiments, in formats including tables, text, and images, with fewer high-definition videos. These data files need to be compressed before transmission. This paper uses the open-source LZMA2 compression algorithm, which has a very high compression ratio. Compared to other compression algorithms, the compression speed is slightly slower, but considering satellite bandwidth, a high compression ratio is still necessary. Shipboard data transmission management regulations discourage the transmission of high-definition video. This is because current videos are usually in compressed formats, and even with further compression using LZMA2, the file compression ratio will not be very high. Furthermore, video consumes a significant amount of bandwidth, affecting the transmission of other files. Therefore, high-definition video files are usually synchronized by direct copying after the ship docks.

[0100] Normally, multiple files await synchronization. These files are sorted by their last modified time and divided into multiple groups for compression. This article uses 100MB as a baseline for grouping, meaning the sum of the file sizes in each group is approximately 100MB. If a single file exceeds 100MB, it is grouped separately. Each group is then compressed, and the compressed file name is the MD5 hash of all file names, resulting in a series of compressed files. This grouping standard can be customized, but exceeding 200MB is not recommended, as larger compressed files increase the file transfer failure rate. These compressed files are transmitted synchronously one by one; this invention synchronizes data on a per-compressed-file basis.

[0101] S200: Based on the full bitmap, perform breakpoint detection on the data blocks to obtain breakpoint data; if the number of breakpoint data is greater than the first number, resend the breakpoint data to the shore end; otherwise, synchronize the data blocks after the target block to the shore end in sequence.

[0102] Among them, the synchronization status of breakpoint data is "synchronizing", and breakpoint data indicates that there is a data block whose synchronization status is "completed" after it; the target block indicates the data block with the largest sequence number whose synchronization status is "completed".

[0103] For example, in some implementations, based on the file transfer bitmap, if it is determined that the file has not yet been successfully synchronized, then out-of-order data chunks are prepared for transmission. (See reference...) Figure 3 The data status is divided into three states: synchronizing, waiting for synchronization, and synchronization completed. The sending strategy prioritizes sending interrupted data, followed by sending data sequentially one by one, using a strategy of a maximum fixed number of data to be synchronized.

[0104] Because it involves out-of-order transmission and reception, and relies on bitmaps to confirm data reception, the bitmap may contain portions that were not received if data packets are lost. (See reference...) Figure 3Segments 2, 4, and 6 are examples of packet loss retransmissions. When a packet is not received by the receiving end, it indicates potential network congestion. Therefore, the packet is retransmitted. This approach aims to ensure all data is sent as quickly as possible, thus improving transmission speed.

[0105] When sending data packets, these "breakpoint" cases are first counted and prioritized for transmission, with a maximum packet size of WindowSize1 (the window size can be customized). If the breakpoint data exceeds WindowSize1, then the breakpoint data is sent primarily; this retransmission is called retransmission. This is done to facilitate periodic data verification.

[0106] When the number of breakpoint data is less than or equal to During this process, sequential data is sent using a fixed window that moves continuously, sending a maximum of WindowSize2 data blocks at a time (the window size can be customized, but must be at least 1, with a recommended range of 1-6). Data blocks in synchronization can also be a maximum of WindowSize2. Whenever a transmission progress response is received, based on the data reception status on the shore end, the status of data blocks is marked (some are changed from "in synchronization" to "synchronization complete," and some are changed from "waiting for synchronization" to "in synchronization"). Then, the sending window is moved, and preparations are made to send new data blocks.

[0107] S300: Periodically obtain the segmented reception status of the shore end, and update the full bitmap based on the segmented reception status;

[0108] It should be noted that, in some embodiments, periodically obtaining the segment reception status at the shore end may include at least one of the following steps:

[0109] When the number of data blocks received by the shore end in a data synchronization iteration is greater than the second number, the shore end's block reception status is obtained in response to the shore end's reply.

[0110] Periodically send block verification requests and heartbeat requests to the shore end so that the shore end can respond to the block verification requests and / or heartbeat requests with block reception status, thereby periodically obtaining the block reception status of the shore end.

[0111] For example, in some specific implementations, the shore end receives a large number of data packets sequentially and stores them in sequence. However, the shore end does not send a reply after receiving each data block, as this would reduce data transmission efficiency. The shore end will immediately reply with the data block reception status if the following three conditions are met:

[0112] Case 1: When received When (if not divisible, round up) valid data blocks are received, a data block reception status is immediately replied to. Valid data blocks do not include duplicate packets. For example, if WindowsSize2 is 3, and the data reception order is data packet 1 (block 5), data packet 2 (block 5), data packet 3 (block 6), and data packet 4 (block 6), when data packet 3 is received, that is, two valid data blocks (blocks 5 and 6) are received, and a data block reception status is replied to. Data packets 2 and 4 are duplicate data packets and will be discarded. This shows that, typically, the number of data retransmissions on the ship side is mostly [number missing]. indivual.

[0113] Case 2: Upon receiving a data segmentation verification request, immediately reply with the data segmentation reception status.

[0114] Case 3: Upon receiving a heartbeat request, immediately reply with the data chunk reception status.

[0115] In summary, if any of Case 1, Case 2, or Case 3 is met, a response will be proactively sent out, detailing the received data chunks, including the file ID, bitmap start ID (also known as the start chunk ID), and incremental bitmap information. Because the bitmap itself can be very large, the entire bitmap is not transmitted here; rather, incremental bitmap information is transmitted. The bitmap is generally transmitted in chunks of size `BitMapSize` (this can be customized, but is typically 1KB, or 1024 chunks). Based on the bitmap start ID, the starting range of the bitmap can be determined as [bitmap start ID, bitmap start ID + BitMapSize]. This can be understood as the latest received bitmap chunks of `BitMapSize`. The bitmap start ID is dynamically calculated, meaning that data packets in the range [0, start chunk ID) have already been synchronized, and the currently synchronized data packets are those in the range [bitmap start ID, bitmap start ID + BitMapSize). Incremental bitmaps can reduce transmission bandwidth pressure and improve transmission speed.

[0116] It should also be noted that the block reception status includes the bitmap start ID and incremental bitmap information. The bitmap start ID corresponds to the largest sequence number among all data blocks that have completed data synchronization; the bitmap start ID represents the sequence number corresponding to the start bitmap, the synchronization status of the start bitmap is completed, and the start bitmap represents the largest sequence number of data blocks that are not synchronized yet. In some embodiments, updating the full bitmap based on the block reception status may include the following steps: updating the synchronization status of the start bitmap and all data blocks preceding it to completed synchronization; wherein, the incremental bitmap information includes the synchronization status of a preset number of data blocks following the data block corresponding to the start bitmap; based on the incremental bitmap information, updating the synchronization status of the preset number of data blocks following the start bitmap; wherein, the status update includes updating the synchronization status of the data blocks to be transmitted in the next round of data synchronization iteration from waiting for synchronization to synchronize.

[0117] In some embodiments, the method may further include transmission start / stop control for data synchronization based on the timeliness of the sent heartbeat request and the shore-side heartbeat response to the heartbeat request. Specifically, this can be implemented as follows:

[0118] The ship periodically sends heartbeat requests, including information such as the heartbeat SeqID and file ID. Each heartbeat increments the SeqID by 1. The sending frequency (SendFrequency) can be customized, typically once per second (ideally between 500ms and 1 second). The main reasons for setting up heartbeat requests are as follows:

[0119] 1) Determine network status. The ship can determine the network status by responding to heartbeat requests. If no heartbeat response is received multiple times, it indicates that the network quality is particularly poor. In this case, the sending progress will be adjusted or even stopped.

[0120] 2) Improve transmission speed. Upon receiving a heartbeat, the shore end immediately updates the bitmap information, and the ship end sends data based on the latest bitmap. Because the heartbeat is periodic, this mechanism effectively avoids the extreme situation where the ship end can never obtain the latest bitmap due to the loss of other data packets, thus accelerating the movement of the ship end's sending window and improving transmission speed.

[0121] Upon receiving a heartbeat request, the shore end processes it and replies to the ship end with information such as heartbeat SeqID, file ID, bitmap start ID, and incremental bitmap information. The ship end needs to determine network quality based on the heartbeat response. This invention determines whether data is lost based on bitmap information, so it does not require a data loss assessment.

[0122] Network quality determination: Since the heartbeat packet is very simple, the time consumed by the shore end to send and the ship end to receive and process is almost negligible. The network quality can be directly reflected by the round-trip of the heartbeat. If no heartbeat reply packet is received within a continuous period of M * SendFrequency (M can be customized, usually 3), that is, M heartbeat packets have no replies, it can be determined that the network quality is extremely poor at this time, and all data transmissions will be immediately stopped, but the sending frequency of the heartbeat packet remains unchanged.

[0123] If a heartbeat reply is received, assume the received heartbeat reply is SeqID1 and the latest sent heartbeat is SeqID2. If SeqID2 - SeqID1 >= M, it means the network quality is extremely poor, resulting in a long round-trip time for the heartbeat. At this time, data transmission is still stopped. If SeqID2 - SeqID1 < M, it means the network quality is acceptable, and at this time, data packets can be resumed or continued to be sent. Generally speaking, the heartbeat is a fallback method for the ship end to determine the network. It can not only improve the transmission speed but also enable breakpoint resumption in a timely manner after the network returns to normal.

[0124] S400. Return to the step of performing breakpoint detection on the data chunks based on the full bitmap to obtain breakpoint data, and conduct a new round of data synchronization iteration until all data chunks of the compressed package are synchronized.

[0125] Refer to Figure 4 , Figure 4 FIG. is a flowchart of the ship-shore data out-of-order synchronization method applied to a server provided by an embodiment of the present invention, and the execution subject of the ship-shore data out-of-order synchronization method can be any of the foregoing computer devices (including a server or a terminal). Refer to Figure 4 This method is applied to the shore end and includes the following steps:

[0126] T100. In response to the data synchronization request from the ship end, obtain the compression information of the compression package to be synchronized; perform data synchronization detection based on the compression information to obtain a full bitmap of the data synchronization situation; send the full bitmap to the ship end;

[0127] Among them, the full bitmap includes the sequence number and synchronization status of each data chunk of the compression package; the synchronization status includes completed synchronization, in-synchronization, and waiting for synchronization;

[0128] It should be noted that the specific implementation manner and application principle of this step correspond to those of the method in the first aspect described above, and will not be elaborated here.

[0129] T200. Receive the data chunks sent by the ship end out of order, and store the received data chunks in sequence based on the sequence number;

[0130] For example, in some specific implementations, the shore end receives multiple UDP packets, which include three parts of information: file ID, chunk ID, and data chunk. The file ID determines which file the data belongs to, and the chunk ID determines the chunk position, which is its position in the bitmap. Considering network factors and retransmission, the packets received by the shore end may not arrive in sequence, but the shore end stores them in chunk ID order. Those not yet received are considered "breakpoint" data. When the bitmap is synchronized, the shore end retransmits the "breakpoint" data. Simultaneously, the shore end may receive packets with the same file ID and chunk ID multiple times. If these packets have already been saved, the duplicate packets are discarded, as they may be retransmitted packets.

[0131] refer to Figure 5 The normal data transmission process involves repeated data transmission from the ship and reception at the shore. The shore periodically verifies the data and replies to the ship. When data loss is confirmed, the "breakpoint" data is retransmitted first. Once the "breakpoint" data is almost completely transmitted, new data is transmitted sequentially again until all data is successfully received.

[0132] T300, based on the storage status of data blocks, periodically replies to the ship end with the block reception status, so that the ship end can update the full bitmap based on the block reception status;

[0133] It should be noted that the ship end periodically sends block verification requests and / or heartbeat requests to the shore end. In some embodiments, the ship end periodically replies to the ship end with the block reception status, including at least one of the following steps: when the number of data blocks received in a round of data synchronization iteration is greater than a second number, replying to the ship end with the block reception status; responding to the block verification requests and / or heartbeat requests periodically sent by the ship end with the block reception status.

[0134] It should be noted that the specific implementation method and application principle of this step are consistent with the principle of the method in the first aspect mentioned above, and will not be repeated here.

[0135] In some embodiments, the ship periodically sends block verification requests to the shore, such as... Figure 6 As shown, the method may further include the following steps: P100, in response to the block verification request from the ship end, obtain the first verification value of the data block sent by the ship end; P200, obtain the second verification value of the received data block through a preset message digest algorithm; P300, perform data consistency verification based on the first and second verification values ​​corresponding to each synchronized data block; P400, when the data consistency verification fails, send a retransmission request for the corresponding data block to the ship end, so that the ship end responds to the retransmission request and retransmits the corresponding data block.

[0136] For example, in some specific implementations, upon receiving a verification request, the data is retrieved based on the start block verification ID and the end block verification ID to perform an MD5 checksum and obtain the MD5 checksum value. Assume the MD5 checksum calculated on the shore is A1, and the MD5 checksum calculated on the ship is A2.

[0137] If A1 and A2 are consistent, then reply with a checksum that matches and reply with the latest incremental bitmap.

[0138] If A1 and A2 are inconsistent, it indicates that the data may have been tampered with. In this case, the data [starting block check ID, ending block check ID] is cleared, the corresponding bitmap state is reset to waiting for synchronization, and the latest bitmap on the ship side is updated (i.e., the latest starting block ID is set to the starting block check ID). Then, a response is sent to confirm the check inconsistency, along with the latest incremental bitmap. This is one of the core points of this invention: data that fails block check can be actively retransmitted.

[0139] If the verification matches, update the latest starting verification block ID to the ending verification block ID, and update the latest bitmap.

[0140] If the verification fails, the data (starting block verification ID, ending block verification ID) is cleared, and the ship's bitmap is directly overwritten with the latest bitmap, preparing for the "breakpoint" data retransmission stage. Based on experience, this situation is rare.

[0141] It's important to note here that only in cases of data inconsistency will the incremental bitmap from the shore be used to directly overwrite the bitmap information from the ship. In all other cases, the bitmap is synchronized based on the largest and latest starting block ID.

[0142] In some embodiments, the method may further include the following step: after all data blocks have been synchronized, perform an overall verification of the synchronized data. Specifically, this can be achieved as follows:

[0143] Based on the ship-side bitmap, once the ship-side determines that all data has been received, it indicates successful data synchronization. At this point, a verification request will be sent, which mainly contains information about the currently transmitted file (filename, file ID, MD5, file size, last modified time, etc.).

[0144] Upon receiving the verification request, the receiving end recalculates the file's MD5 hash and compares it with the requested MD5 hash. If they match, the file transfer was successful. If they don't match, it indicates that some data packets may have been tampered with. In most cases, the MD5 hashes will match.

[0145] T400: Return to the step of receiving out-of-order data blocks sent by the ship end, and perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

[0146] To explain in detail the principle of the technical solution of the present invention, the overall process of the present invention will be described below with reference to some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and should not be regarded as a limitation of the present invention.

[0147] First, it should be noted that the current mainstream solution for ship-to-shore data synchronization is mainly based on TCP, which is consistent with the approach of data synchronization between two points on land. That is, a TCP communication channel is established between the two points, and then synchronization data is sent.

[0148] Existing working approach: Currently, data synchronization from ship to shore is divided into three stages: metadata preparation, data synchronization, and final verification. The transmission characteristics are sequential fragmented transmission, sequential breakpoint resumption, making full use of idle bandwidth and improving transmission speed.

[0149] However, the current mainstream solutions have the following drawbacks:

[0150] 1) Slow speed: Currently, data from ship to shore is mainly transmitted via satellite. Satellite bandwidth is relatively low, resulting in slow data synchronization speed.

[0151] 2) Poor experience: Considering the low satellite bandwidth and the fact that data synchronization and shipboard technicians all need to be connected to the internet, bandwidth contention will occur, resulting in a poor internet experience for users.

[0152] 3) Low synchronization success rate: Currently, data synchronization is mainly based on TCP for transmission. The integrity of data transmission is theoretically guaranteed. However, considering the low bandwidth of satellites, data transmission often fails halfway through and eventually fails.

[0153] Disadvantages of existing work plans:

[0154] 1) Excessive retransmission bandwidth: Because existing work is sent and retransmitted sequentially, much of the already received data will still be retransmitted, resulting in a large amount of retransmitted data and the transmission speed still needs to be improved.

[0155] 2) Low synchronization success rate: Data synchronization may fail after reaching 80% or more, due to network issues causing continuous retransmissions.

[0156] Data is easily tampered with: With only a final verification mechanism, intermediate data can be easily tampered with, leading to the failure of final data synchronization.

[0157] In view of this, the present invention proposes a method for out-of-order synchronization of ship-to-shore data, referring to... Figure 7 The specific steps are as follows:

[0158] Step 1: Local Data Compression. There are usually many files to be transmitted on board, mainly test results from technical experiments, including tables, text, and images, with fewer high-definition videos. These data files need to be compressed before transmission. This article uses the open-source LZMA2 compression algorithm, which has a very high compression ratio. Compared to other compression algorithms, the compression speed is slightly slower, but considering satellite bandwidth, a high compression ratio is still necessary. Shipboard data transmission management regulations discourage the transmission of high-definition video. This is because current videos are usually in compressed formats, and even with further compression using LZMA2, the file compression ratio will not be very high. Furthermore, video consumes a significant amount of bandwidth, affecting the transmission of other files. Therefore, high-definition video files are usually synchronized by direct copying after the ship docks.

[0159] Normally, multiple files await synchronization. These files are sorted by their last modified time and divided into multiple groups for compression. This article uses 100MB as a baseline for grouping, meaning the sum of the file sizes in each group is approximately 100MB. If a single file exceeds 100MB, it is grouped separately. Each group is then compressed, and the compressed file name is the MD5 hash of all file names, resulting in a series of compressed files. This grouping standard can be customized, but exceeding 200MB is not recommended, as larger compressed files increase the file transfer failure rate. These compressed files are transmitted synchronously one by one; this invention synchronizes data on a per-compressed-file basis.

[0160] In most cases, text file compression rates can reach approximately 60%, meaning the compressed text file size is about 40% of the original size. With 100MB blocks per file, each compressed file would be around 40MB. In real-world scenarios, compression rates can even reach 70%.

[0161] Regarding grouping, let's take an example. Suppose that files A (80MB), B (17MB), C (60MB), D (60MB), E (70MB), F (80MB), and G (170MB) are already sorted by their last modification time, then the grouping would be as follows:

[0162] 1) A (80M in size) and B (17M in size) are grouped together because A+B=97M, which is around 100M. The compressed file name is MD5(AB).

[0163] 2) C (60M in size) and D (60M in size) are a group. C is 60M, which is 40M away from 100M. However, C+D=120M, which is only 20M away from 100M. Therefore, C and D are a group. The compressed file name is MD5(CD).

[0164] 3) E (70M in size) is a separate group. E is 70M, which is 30M away from 100M. However, E+F=150M, which is 50M away from 100M. Therefore, E is a separate group, and the compressed file name is MD5(E).

[0165] 4) F (size 80M) is grouped separately for the same reason as E;

[0166] 5) G (size 170M) is a separate group;

[0167] In summary, the data was divided into 5 groups, and each group was compressed into a package using LZMA2.

[0168] Step 2: Initiate a ship-side data synchronization request. The ship sends a data synchronization request, which is relayed to the shore via satellite. This request mainly includes detailed information about the compressed data package being transmitted, such as the filename, file size, file MD5 hash, file creation time, last modification time, etc. This information is primarily used to determine whether interrupted transmission needs to be resumed.

[0169] Step 3: Check data synchronization status. When the shore end receives a data synchronization request, it will check the files already transferred on the shore. The main process is as follows:

[0170] 1) Verify the file name and check if the file exists on the other end. If it does not exist, it indicates that this is the first time the file has been transmitted. If it still does not exist, continue comparing the file's MD5 hash.

[0171] 2) Verify the MD5 hash of the files. Check if the MD5 hashes of the files on the other end and the target file match. If they don't match, the target file has been changed and needs to be retransmitted. If they match, continue comparing file sizes.

[0172] 3) Verify the bitmap information and file size. If they match, the file has been transferred before, and you can choose to resume the transfer. If they don't match, the target file has changed, and you need to retransmit it. This step is necessary because the file is stored in a cache before successful transfer, and the cache may be modified or cleared.

[0173] 4) Verify the bitmap information. If all data packets have been received, the data synchronization is successful; otherwise, retransmission or resuming interrupted transmission is required. If the data has been successfully transmitted, check the final verification flag on the shore. If verification is complete, reply to the shipside that synchronization is successful; otherwise, reply that final verification is required.

[0174] The bitmap information is used to indicate the status of the received file on the other end. Assuming the compressed file is 50MB and each data packet is approximately 1KB using UDP, then there are a total of 50MB / 1KB = 51200 bits needed to represent the status of each packet (0 for not received, 1 for received). Therefore, the bitmap size is 51200 bits, requiring 51200 / 8 = 6.25KB to represent it. The bitmap is primarily used for data retransmission, reducing redundant bandwidth and improving transmission speed.

[0175] Step 4: Shore-side data synchronization status feedback. After the check is completed, the monitored information will be directly fed back to the ship, mainly including the file information already existing on the shore (filename, MD5, file bitmap, file size, last modification time, transmission progress, etc.). Note that the bitmap here is the full bitmap, meaning the entire bitmap needs to be synchronized to the ship.

[0176] Step 5: Send data chunks out of order. Based on the file transfer bitmap, if it is determined that all data packets have been received, a final data verification is required. The ship checks whether the final verification of the file has been completed. If the file has completed the final verification, the entire synchronous transmission process ends. If the file has not yet undergone final verification, proceed to step 13 for data verification.

[0177] Based on the file transfer bitmap, if it is determined that the file synchronization has not yet succeeded, then out-of-order data chunks are prepared for transmission. (Reference) Figure 3 The data status is divided into three states: synchronizing, waiting for synchronization, and synchronization completed. The sending strategy prioritizes sending interrupted data, followed by sending data sequentially one by one, using a strategy of a maximum fixed number of data to be synchronized.

[0178] Because it involves out-of-order transmission and reception, and relies on bitmaps to confirm data reception, the bitmap may contain portions that were not received if data packets are lost. (See reference...) Figure 3 Segments 2, 4, and 6 are examples of packet loss retransmissions. When a segment is not received by the receiving end, it indicates potential network congestion, so the segment is retransmitted. This approach aims to ensure all data is sent as quickly as possible, improving transmission speed. The rules for determining whether the receiving end has received a data block will be explained in step 12 and will not be elaborated here.

[0179] When sending data packets, these "breakpoint" cases are first counted and prioritized for transmission, with a maximum packet size of WindowSize1 (the window size can be customized). If the breakpoint data exceeds WindowSize1, then the breakpoint data is sent primarily; this retransmission is called retransmission. This is done to facilitate periodic data verification.

[0180] When the number of breakpoint data is less than or equal to During this process, sequential data is sent using a fixed window that moves continuously, sending a maximum of WindowSize2 data blocks at a time (the window size can be customized, but must be at least 1, with a recommended range of 1-6). Data blocks in synchronization can also be a maximum of WindowSize2. Whenever a transmission progress response is received, based on the data reception status on the shore end, the status of data blocks is marked (some are changed from "in synchronization" to "synchronization complete," and some are changed from "waiting for synchronization" to "in synchronization"). Then, the sending window is moved, and preparations are made to send new data blocks.

[0181] For example, see reference Figure 2 Assuming WindowSize1 is 2 and WindowSize2 is 3, then segments 2, 4, and 6 are of the type requiring retransmission after packet loss. Segments 2 and 4 need to be retransmitted first. Once segments 2 and 4 are synchronized, only segment 6 remains for retransmission after packet loss, meaning there is only one data point left to retransmit. At this point, transmission resumes sequentially from segment 9 onwards, i.e., segments 9, 10, and 11 are sent. Prioritizing retransmission of lost data ensures that received data has no "breakpoints," laying a solid foundation for subsequent periodic segmentation and verification of data.

[0182] Step 6: Receive data chunks out of order and store them sequentially. The shore end will receive multiple UDP packets, including three parts: file ID, chunk ID, and data chunk. The file ID determines which file the data belongs to, and the chunk ID determines the chunk position, which is its position in the bitmap. Considering network factors and retransmission, the data packets received by the shore end may not arrive in order, but the shore end will store them in chunk ID order. Those not yet received are considered "breakpoint" data. When the bitmap is synchronized, the ship end will retransmit the "breakpoint" data. Simultaneously, the shore end may receive data packets with the same file ID and chunk ID multiple times. If these packets have already been saved, they are discarded, as they may be retransmitted packets.

[0183] Whenever a data packet is received, the bitmap is updated, and the bitmap is periodically synchronized according to rules, which are described in step 8.

[0184] refer to Figure 5 The normal data transmission process involves repeated data transmission from the ship and reception at the shore. The shore periodically verifies the data and replies to the ship. When data loss is confirmed, the "breakpoint" data is retransmitted first. Once the "breakpoint" data is almost completely transmitted, new data is transmitted sequentially again until all data is successfully received.

[0185] Step 7: Periodically reply with data block reception status. The shore end will receive a large number of data packets one after another and store them sequentially. However, the shore end does not send a reply after receiving each data block, as this would reduce data transmission efficiency. The shore end will immediately reply with data block reception status when the following three conditions are met:

[0186] Case 1: When received When (if not divisible, round up) valid data blocks are received, a data block reception status is immediately replied to. Valid data blocks do not include duplicate packets. For example, if WindowsSize2 is 3, and the data reception order is data packet 1 (block 5), data packet 2 (block 5), data packet 3 (block 6), and data packet 4 (block 6), when data packet 3 is received, that is, two valid data blocks (blocks 5 and 6) are received, and a data block reception status is replied to. Data packets 2 and 4 are duplicate data packets and will be discarded. This shows that, typically, the number of data retransmissions on the ship side is mostly [number missing]. indivual.

[0187] Case 2: Upon receiving a data segmentation verification request, immediately reply with the data segmentation reception status.

[0188] Case 3: Upon receiving a heartbeat request, immediately reply with the data chunk reception status.

[0189] In summary, if any of Case 1, Case 2, or Case 3 is met, a response will be proactively sent out, detailing the received data chunks, including the file ID, bitmap start ID (also known as the start chunk ID), and incremental bitmap information. Because the bitmap itself can be very large, the entire bitmap is not transmitted here; rather, incremental bitmap information is transmitted. The bitmap is generally transmitted in chunks of size `BitMapSize` (this can be customized, but is typically 1KB, or 1024 chunks). Based on the bitmap start ID, the starting range of the bitmap can be determined as [bitmap start ID, bitmap start ID + BitMapSize]. This can be understood as the latest received bitmap chunks of `BitMapSize`. The bitmap start ID is dynamically calculated, meaning that data packets in the range [0, start chunk ID) have already been synchronized, and the currently synchronized data packets are those in the range [bitmap start ID, bitmap start ID + BitMapSize). Incremental bitmaps can reduce transmission bandwidth pressure and improve transmission speed.

[0190] Here's an example: Suppose the compressed file is 50MB, and each data packet is transmitted using the UDP protocol, with each packet being approximately 1KB. Then, there are a total of 50MB / 1KB = 50,000 bits needed to represent the status of each packet (0 for not received, 1 for received). This results in a bitmap size of 50,000 / 8 = 6250 = 6.1KB. A 6.1KB bitmap is quite large, so we choose to transmit incremental bitmaps. Assuming the starting ID is 0 and the BitMapSize is 1024, the bitmap for receiving data in chunks will be [0, 1024].

[0191] Regarding the "breakpoint" retransmission strategy in step 5, there is a point that needs discussion. The shore end will send data chunks according to the fixed rules of Case 1, Case 2, and Case 3. When the ship end receives the latest bitmap information, it will proactively check the "breakpoint" data status and stop sending current data, prioritizing the transmission of the "breakpoint" data. When multiple incremental bitmap messages are received, the largest starting bitmap ID is used as the latest progress (an exception is the special case of data verification failure, in which case the ship end bitmap will be directly overwritten, and the system will enter a large breakpoint retransmission state).

[0192] Step 8: Periodically send block verification requests. When the ship receives the incremental bitmap, it will synchronize the bitmap, mainly performing the following operations.

[0193] 1. Check the data block of synchronization [0, starting block ID), set the status to synchronization complete, synchronize and update the bitmap on the ship side.

[0194] 2. Based on the incremental bitmap [starting block ID, bitmap start ID + 1K), directly overwrite, synchronize and update the bitmap on the ship side.

[0195] Meanwhile, the client may receive incremental bitmap information multiple times. In this case, the largest starting block ID is used as the latest bitmap, and other incremental bitmap information is discarded. After the client updates to the latest bitmap, it will periodically send data verification requests. The client will send verification requests based on the latest starting verification block ID and the data verification size VerifSize (which can be customized, with a default value of 100). Let's assume the latest starting block ID is LastRecvID and the latest starting verification block ID is LastVerifID.

[0196] Calculate BatchSize using the following formula.

[0197]

[0198] If BatchSize is 0, it means that the latest received data is less than VerifSize, and no verification request is sent. If BatchSize > 0, the shore side will calculate the MD5 checksum of [LastVerifID, LastVerifID + BatchSize * VerifSize) and then send a data verification request for the range [LastVerifID, LastVerifID + BatchSize * VerifSize).

[0199] If LastRecvID is the last ID in the bitmap, meaning all data synchronization is complete, then a data verification request within the range [LastVerifID, LastRecvID] is issued.

[0200] The data verification request packet contains the following information: file ID, start block verification ID, end block verification ID, and MD5 checksum.

[0201] Here's an example: if LastRecvID is 1027, LastVerifID is 200, and VerifSize is 100, then batchSize is 8, meaning that a block ID verification in the range [200, 1000) is issued.

[0202] Here's an explanation of how to set the VerifSize. Currently, the bandwidth between the ship and shore is around 2.5MB, or 2500KB. Considering bandwidth usage by other components on the ship, we'll assume a data synchronization bandwidth of 1KB. Setting VerifSize to 100 (each packet is 1KB, so 100 packets equals 100 seconds) means performing a data verification approximately every 100 seconds. VerifSize should be set based on the current satellite bandwidth and shouldn't be too large; based on current satellite bandwidth, it's recommended not to exceed 500.

[0203] Step 9: Verify the received data. Upon receiving a verification request, the data is retrieved based on the start block verification ID and end block verification ID, and an MD5 checksum is obtained. Assume the MD5 checksum calculated on shore is A1, and the MD5 checksum calculated on ship is A2.

[0204] If A1 and A2 are consistent, then reply with a checksum that matches and reply with the latest incremental bitmap.

[0205] If A1 and A2 are inconsistent, it indicates that the data may have been tampered with. In this case, the data [starting block check ID, ending block check ID] is cleared, the corresponding bitmap state is reset to waiting for synchronization, and the latest bitmap on the ship side is updated (i.e., the latest starting block ID is set to the starting block check ID). Then, a response is sent to confirm the check inconsistency, along with the latest incremental bitmap. This is one of the core points of this invention: data that fails block check can be actively retransmitted.

[0206] Step 10: Reply with data verification status and selectively retransmit.

[0207] If the verification matches, update the latest starting verification block ID to the ending verification block ID, and update the latest bitmap.

[0208] If the verification fails, the data (starting block verification ID, ending block verification ID) is cleared, and the ship's bitmap is directly overwritten with the latest bitmap, preparing for the "breakpoint" data retransmission stage. Based on experience, this situation is rare.

[0209] It's important to note here that only in cases of data inconsistency will the incremental bitmap from the shore be used to directly overwrite the bitmap information from the ship. In all other cases, the bitmap is synchronized based on the largest and latest starting block ID.

[0210] Step 11: Send a heartbeat request. The ship will periodically send heartbeat requests, including information such as the heartbeat SeqID and file ID. Each time a heartbeat is sent, SeqID = SeaID + 1. The sending frequency (SendFrequency) can be customized, generally once per second (recommended between 500ms and 1 second). The main reasons for setting a heartbeat request are as follows:

[0211] 1) Determine network status. The ship can determine the network status by responding to heartbeat requests. If no heartbeat response is received multiple times, it indicates that the network quality is particularly poor. In this case, the sending progress will be adjusted or even stopped.

[0212] 2) Improve transmission speed. Upon receiving a heartbeat, the shore end immediately updates the bitmap information, and the ship end sends data based on the latest bitmap. Because the heartbeat is periodic, this mechanism effectively avoids the extreme situation where the ship end can never obtain the latest bitmap due to the loss of other data packets, thus accelerating the movement of the ship end's sending window and improving transmission speed.

[0213] Step 12: Heartbeat Response. Upon receiving a heartbeat request, the shore end processes it and responds to the ship end with information such as the heartbeat SeqID, file ID, bitmap start ID, and incremental bitmap information. The ship end needs to determine network quality based on the heartbeat response. This invention determines whether data is lost based on bitmap information, so it does not require a data loss assessment.

[0214] Network quality determination: Since the heartbeat packet is very simple, the time consumed by the shore end to send and the ship end to receive and process is almost negligible. The network quality can be directly reflected by the heartbeat going and coming back. If no heartbeat reply packet is received for a continuous period of M * SendFrequency (M can be customized, usually 3), that is, M heartbeat packets have no reply, it can be determined that the network quality is extremely poor at this time, and all data transmissions will be immediately stopped, but the sending frequency of the heartbeat packet remains unchanged.

[0215] If a heartbeat reply is received, assume the received heartbeat reply is SeqID1 and the latest sent heartbeat is SeqID2. If SeqID2 - SeqID1 >= M, it means the network quality is extremely poor, resulting in a very long heartbeat round-trip time. At this time, data transmission is still stopped. If SeqID2 - SeqID1 < M, it means the network quality is acceptable, and at this time, data packet transmission can be resumed or continued. Generally speaking, the heartbeat is a fallback method for the ship end to determine the network. It can not only improve the transmission speed but also enable breakpoint resumption in a timely manner after the network returns to normal.

[0216] Here is an example. Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 1, receive heartbeat reply 1, send heartbeat 2, receive heartbeat reply 2, send heartbeat 3, receive heartbeat reply 3. In this case of normal data network, the general network delay on land is generally 30ms to 50ms. Considering satellite transmission, the ship-shore network delay is estimated to be 200ms, that is, the reply of heartbeat 1 can be received after 200ms. So the above belongs to the normal network situation.

[0217] Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 1, send heartbeat 2, receive heartbeat reply 1, receive heartbeat reply 2, send heartbeat 3, receive heartbeat reply 3. This situation belongs to a slightly worse network, but data transmission can still continue.

[0218] Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 1, send heartbeat 2, send heartbeat 3, send heartbeat 4. At this time, it belongs to the situation where no reply is received for M consecutive heartbeats, indicating that the network quality is very poor, and all data transmissions will be immediately stopped.

[0219] Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 4, receive heartbeat reply 1. At this time, although a heartbeat reply is received, this reply is from a long time ago, and all data transmissions will still be stopped.

[0220] Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 4, receive heartbeat reply 1, receive heartbeat reply 4. At this time, it indicates that the network situation has improved, and data transmission will be resumed.

[0221] Step 13: Data synchronization successful, send verification request. Based on the ship-side bitmap, the ship-side determines that data synchronization is successful after all data has been received. At this time, a verification request will be sent, which mainly contains information about the currently transmitted file (filename, file ID, MD5, file size, last modified time, etc.).

[0222] Step 14: Local Data Verification. Upon receiving the verification request, the shore end recalculates the file's MD5 hash and compares it with the requested MD5 hash. If they match, the file transfer was successful. If they don't match, it indicates that some data packets may have been tampered with. In most cases, the MD5 hashes will match.

[0223] Step 15: Reply with data verification results. The shore end replies with MD5 verification results, mainly including information about the currently transmitted file (filename, file ID, MD5, file size, last modification time, etc.). After receiving the reply, the shore end records the transmission details, marks the file as successfully synchronized, and notifies the user. If the MD5 matches, the user is notified that the file transmission was successful; otherwise, the user is notified that intermediate data has been tampered with and resynchronization is required (data will not be reset or cleared here; the user needs to actively retransmit). Simply put, data that fails in block verification can be actively retransmitted, but if it reaches the final verification step, data that fails verification will not be actively retransmitted; the user will be notified and asked to choose to retransmit.

[0224] Once steps 13 through 15 are completed, the file will be marked as synchronized, and another file will be transferred next. One extreme case needs explanation: if the file has just been successfully transferred and then the network suddenly drops, the ship will not receive a data verification response for a long time. In this case, the file is still in a synchronized state. Once the network is restored, the ship will resend the verification request until a verification response is received, at which point the file transfer process is truly complete.

[0225] In summary, the entire transmission process consists of three stages:

[0226] 1) Preparation stage, including steps 1 to 4. This stage mainly involves clarifying the information of the file to be transmitted on the shore and synchronizing the latest bitmap. It belongs to the data preparation stage before transmission.

[0227] 2) Synchronization phase, including steps 5 to 12. This phase mainly involves synchronizing data, including data synchronization and bitmap synchronization. It dynamically adjusts the sending rhythm through heartbeat and block verification mechanisms to improve file transfer efficiency.

[0228] 3) Verification stage, including steps 13 to 15, mainly involves the final data verification.

[0229] Compared with the prior art, the present invention has at least the following beneficial effects:

[0230] 1) Introduce a first-level bitmap mechanism to control the data sending process, reduce data retransmission bandwidth, and improve transmission speed.

[0231] 2) Implement out-of-order transmission based on a fixed window transmission strategy to make full use of idle bandwidth and improve transmission speed.

[0232] 3) Intermediate data is periodically verified and supports breakpoint resume, ensuring data integrity throughout the entire process from the start of data download to completion.

[0233] On the other hand, such as Figure 8 As shown, this embodiment of the invention provides a ship-to-shore out-of-order data synchronization device 800, applied at the ship's end, which may include:

[0234] The first module 801 is used to initiate a data synchronization request to the shore end, and send the compression information of the compressed package to be synchronized to the shore end so that the shore end can perform data synchronization detection based on the compression information and obtain a full bitmap of the data synchronization status of the shore end.

[0235] The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes completed synchronization, in the process of synchronization, and waiting for synchronization.

[0236] The second module 802 is used to perform breakpoint detection on data blocks based on the full bitmap and obtain breakpoint data; if the number of breakpoint data is greater than the first number, the breakpoint data is resent to the shore end; otherwise, the data blocks after the target block are sequentially synchronized to the shore end.

[0237] Among them, the synchronization status of breakpoint data is "synchronizing", and breakpoint data indicates that there is a data block whose synchronization status is "completed" after it; the target block indicates the data block with the largest sequence number whose synchronization status is "completed".

[0238] The third module 803 is used to periodically obtain the segmented reception status of the shore end and update the full bitmap based on the segmented reception status;

[0239] The fourth module, 804, is used to return to the execution of the second module and perform a new round of data synchronization iterations until all data blocks of the compressed package are synchronized.

[0240] On the other hand, such as Figure 9 As shown, this embodiment of the invention provides a ship-to-shore out-of-order data synchronization device 900, applied at the shore end, which may include:

[0241] The fifth module 901 is used to respond to the data synchronization request from the ship end, obtain the compression information of the compressed package to be synchronized; perform data synchronization detection based on the compression information to obtain a full bitmap of the data synchronization status; and send the full bitmap to the ship end.

[0242] The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes completed synchronization, in the process of synchronization, and waiting for synchronization.

[0243] The sixth module 902 is used to receive out-of-order data blocks sent by the ship and to store the received data blocks sequentially based on the sequence number.

[0244] Module 7, 903, is used to periodically reply to the ship terminal with the block reception status based on the storage status of data blocks, so that the ship terminal can update the full bitmap based on the block reception status.

[0245] Module 8, 904, is used to return to Module 6 for a new round of data synchronization iterations until all data blocks in the compressed package are synchronized.

[0246] In some embodiments, the ship periodically sends block verification requests to the shore, and the ship-shore out-of-order data synchronization device applied to the shore may further include:

[0247] The ninth module is used to respond to the block verification request from the ship and obtain the first verification value of the data block sent by the ship.

[0248] The tenth module is used to obtain the second check value of the received data block through a preset message digest algorithm;

[0249] Module 11 is used to perform data consistency verification based on the first and second check values ​​corresponding to each synchronized data block;

[0250] The twelfth module is used to send a retransmission request for the corresponding data block to the ship when the data consistency check fails, so that the ship can respond to the retransmission request and retransmit the corresponding data block.

[0251] The content of the method embodiments of the present invention is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0252] On the other hand, embodiments of the present invention also provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method for predicting the bottom boundary of the hydrate stability domain. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0253] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0254] like Figure 10 As shown, Figure 10 The hardware structure of an electronic device 1000 according to another embodiment is illustrated. The electronic device 1000 includes:

[0255] The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0256] The memory 1002 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1002 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 to execute the network node population optimization method of the embodiments of this invention.

[0257] Input / output interface 1003 is used to implement information input and output;

[0258] The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0259] Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004);

[0260] The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0261] The electronic device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0262] The content of the method embodiments of the present invention is applicable to the embodiments of the present electronic device. The specific functions implemented by the embodiments of the present electronic device are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0263] Another aspect of this invention provides a computer-readable storage medium storing a program that is executed by a processor to implement the aforementioned method.

[0264] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD to ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein computer-readable program code is carried. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0265] The content of the method embodiments of the present invention is applicable to the computer-readable storage medium embodiments. The specific functions implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.

[0266] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0267] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0268] It should be noted that although several modules for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0269] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0270] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented in this invention. Alternative embodiments are contemplated, in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0271] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0272] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0273] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution means, apparatus, or device (such as a computer-based device, a processor-including device, or other means that can fetch and execute instructions from, or in conjunction with, an instruction execution means, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution means, apparatus, or device.

[0274] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0275] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0276] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0277] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0278] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for out-of-order synchronization of ship-to-shore data, characterized in that, Applied to the end of a ship, the method includes the following steps: A data synchronization request is initiated to the shore end, and the compression information of the compressed package to be synchronized is sent to the shore end so that the shore end can perform data synchronization detection based on the compression information and obtain a full bitmap of the data synchronization status of the shore end. The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes synchronization complete, synchronization in progress, and waiting for synchronization. Based on the full bitmap, breakpoint detection is performed on the data blocks to obtain breakpoint data; if the number of breakpoint data is greater than a first number, the breakpoint data is resent to the shore end; otherwise, the data blocks after the target block are sequentially synchronized to the shore end. Wherein, the synchronization status of the breakpoint data is "in synchronization", and the breakpoint data indicates that there is a data block whose synchronization status is "completed synchronization"; the target block indicates the data block with the largest sequence number whose synchronization status is "completed synchronization". Periodically obtain the segmented reception status of the shore end, and update the full bitmap based on the segmented reception status; Return to the step of performing breakpoint detection on the data blocks based on the full bitmap and obtaining breakpoint data, and perform a new round of data synchronization iteration until all the data blocks of the compressed package are synchronized.

2. The ship-to-shore out-of-order data synchronization method according to claim 1, characterized in that, The process of periodically acquiring the segmented reception status at the shore end... Includes at least one of the following steps: When the number of data blocks received by the shore end in a round of data synchronization iteration is greater than the second number, the shore end's block reception status is obtained in response to the shore end's reply; Periodically send block verification requests and / or heartbeat requests to the shore end, so that the shore end responds to the block verification requests and / or the heartbeat requests with the block reception status, thereby periodically obtaining the block reception status of the shore end.

3. The ship-to-shore out-of-order data synchronization method according to claim 1, characterized in that, The block reception status includes a bitmap start ID and incremental bitmap information. The bitmap start ID corresponds to the largest sequence number among all data blocks that have completed data synchronization. The bitmap start ID represents the sequence number corresponding to the starting bitmap, the synchronization status of the starting bitmap is "synchronization complete," and the starting bitmap indicates that there is no preceding data block with the largest sequence number in the synchronization status. Updating the full bitmap based on the block reception status includes the following steps: Update the synchronization status of the starting bitmap and all preceding data blocks to "synchronization complete"; The incremental bitmap information includes the synchronization status of a preset number of data blocks following the data block corresponding to the starting bitmap; Based on the incremental bitmap information, the synchronization status of the preset number of data blocks following the starting bitmap is updated. The status update includes updating the synchronization status of the data block to be transmitted in the next round of data synchronization iteration from "waiting for synchronization" to "synchronizing".

4. A ship-to-shore out-of-order data synchronization device, characterized in that, Applied to the ship's end, including: The first module is used to initiate a data synchronization request to the shore end, and send the compression information of the compressed package to be synchronized to the shore end, so that the shore end can perform data synchronization detection based on the compression information and obtain a full bitmap of the data synchronization status of the shore end. The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes synchronization complete, synchronization in progress, and waiting for synchronization. The second module is used to perform breakpoint detection on the data blocks based on the full bitmap to obtain breakpoint data; when the number of breakpoint data is greater than a first number, the breakpoint data is resent to the shore end; otherwise, the data blocks after the target block are sequentially synchronized to the shore end. Wherein, the synchronization status of the breakpoint data is "in synchronization", and the breakpoint data indicates that there is a data block whose synchronization status is "completed synchronization"; the target block indicates the data block with the largest sequence number whose synchronization status is "completed synchronization". The third module is used to periodically obtain the segmented reception status of the shore end and update the full bitmap based on the segmented reception status; The fourth module is used to return to the execution of the second module and perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

5. A method for out-of-order synchronization of ship-to-shore data, characterized in that, Applied to shore-based applications, the method interacts with the ship-to-shore out-of-order data synchronization method as described in any one of claims 1 to 3, and the method includes the following steps: In response to a data synchronization request from the ship, the compression information of the compressed package to be synchronized is obtained; based on the compression information, data synchronization detection is performed to obtain a full bitmap of the data synchronization status; and the full bitmap is sent to the ship. The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes synchronization complete, synchronization in progress, and waiting for synchronization. The ship receives the data blocks sent out of order and stores them sequentially based on the sequence number. Based on the storage status of the data blocks, the ship terminal periodically replies with the block reception status, so that the ship terminal updates the full bitmap based on the block reception status. Return to the step of receiving the out-of-order data blocks sent by the ship end, and perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

6. The ship-to-shore out-of-order data synchronization method according to claim 5, characterized in that, The ship can periodically send block verification requests and / or heartbeat requests to the shore, and periodically reply to the ship with block reception status. Includes at least one of the following steps: When the number of data blocks received in a data synchronization iteration is greater than the second number, the receiving status of the data blocks is replied to the ship end. In response to the block verification request and / or the heartbeat request periodically sent by the ship end, reply to the ship end with the block reception status.

7. The ship-to-shore out-of-order data synchronization method according to claim 5, characterized in that, The ship can periodically send block verification requests to the shore. The method also includes the following steps: In response to the block verification request from the ship, the first verification value of the data block sent by the ship is obtained from the ship. The second check value of the received data block is obtained by processing it through a preset message digest algorithm; Data consistency verification is performed based on the first and second check values ​​corresponding to each synchronized data block; If the data consistency check fails, a retransmission request for the corresponding data block is sent to the ship terminal, so that the ship terminal responds to the retransmission request and retransmits the corresponding data block.

8. A ship-to-shore out-of-order data synchronization device, characterized in that, Applied to the shore, the device interacts with the ship-to-shore data out-of-order synchronization device as described in claim 4, including: The fifth module is used to respond to the data synchronization request from the ship end, obtain the compression information of the compressed package to be synchronized; perform data synchronization detection based on the compression information to obtain a full bitmap of the data synchronization status; and send the full bitmap to the ship end. The full bitmap includes the sequence number and synchronization status of each data block in the compressed package; the synchronization status includes synchronization complete, synchronization in progress, and waiting for synchronization. The sixth module is used to receive the data blocks sent by the ship in out-of-order order, and to store the received data blocks in order based on the sequence number; The seventh module is used to periodically reply to the ship terminal with the block reception status based on the storage status of the data blocks, so that the ship terminal can update the full bitmap of the ship terminal based on the block reception status. The eighth module is used to return to the execution of the sixth module and perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

9. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 3 or 5 to 7.

10. A computer storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to implement the method as described in any one of claims 1 to 3 or 5 to 7.

Images